Continuous delivery raises the bar for performance testing
Competition between digital businesses is fierce. Application performance issues can be catastrophic, ruining customer experience and brand loyalty. Traditional performance testing requires days or even weeks to complete, and involves a lot of manual effort. With the mass adoption of continuous delivery, DevOps, chaos engineering and SRE, such delays are no longer acceptable. Performance testing should be fast and performed continuously to let the development team fix issues early in the release process, so that the context of the issues are fresh and the costs are minimal.
Pioneers in continuous performance testing
Grid Dynamics has been working in continuous performance testing for years, well before it became a popular addition to the testing lexicon. This practice developed from our expertise in building mission-critical applications, our longstanding tradition of automating everything, and our use of agile best practices. Our architects and engineers have a lot of experience in continuous performance testing, having implemented it with many clients in different industries and platforms over the years. In 2011, we even released our own continuous performance testing tool called Jagger.
Approach to continuous performance testing
Identify KPIs and design tests for them
The first important step of continuous performance testing is to find the most critical KPIs and determine what levels of performance are required for them. Once KPIs and test targets are known, the design of performance tests is similar to the design of functional tests. Some of the important questions that need to be answered involve test data management and dependency management. In some cases, testing with synthetic data and service virtualization may bring the biggest benefit, while in others, using production data and real dependencies makes more sense.
Automate everything
Let developers and testers find defects immediately so that they can be fixed quickly and cheaply. Do this correctly by:
Augment results analysis
A couple techniques can be used to automate the analysis of results:
Continuous performance testing is often performed on smaller environment to save costs. In order to predict absolute numbers, the correlation of metrics between production and the performance lab should be learned based on the past results.
Key benefits
Meet performance targets
Accurately forecast how much load your applications can securely handle.
Catch performance issues early
Ensure that developers can continuously test the performance of development applications.
Integrate with CI/CD pipeline
Remove human error and accelerate development by adding performance testing to the continuous delivery pipeline.
Automatically analyze results
Use statistics and machine learning to compare the new results with previous ones, and stop the pipeline if needed.
Save on environment costs
Execute continuous performance testing on small environments to perform relative analysis.
Avoid costly performance failures
Prevent performance failures during peak loads that will antagonize customers and slash online conversions.
High level reference architecture
Technology stack
Open source continuous performance testing platform
Load testing tool and measures performance
Load and performance testing tool
Open source continuous performance testing platform
Load testing tool and measures performance
Load and performance testing tool
Open source continuous performance testing platform
Load testing tool and measures performance
Load and performance testing tool
Engagement model
We prefer a hands-on approach in our engagements from day one. With the help of clients, our architects and engineers can identify challenging applications and services which would benefit the most from continuous performance testing. We then begin our assessments by looking at the application requirements, performance KPIs and existing methods of testing.
After we understand the current situation in depth, we make point improvements to bring the most value to our clients with the least amount of work, and in the shortest period of time. Once these show value on a small scale, we move to long-term improvements in the process and tooling on a larger scale, adopt continuous performance techniques across the board, and educate client teams to spread knowledge and know-how within their organization.
Read more
Get in touch
We'd love to hear from you. Please provide us with your preferred contact method so we can be sure to reach you.
Thank you for getting in touch with Grid Dynamics!
Your inquiry will be directed to the appropriate team and we will get back to you as soon as possible.
Something went wrong...
There are possible difficulties with connection or other issues.
Please try again after some time.