Home Solutions Cloud & DevOps Continuous performance testing

Continuous delivery raises the bar for performance testing

Competition between digital businesses is fierce. Application performance issues can be catastrophic, ruining customer experience and brand loyalty. Traditional performance testing requires days or even weeks to complete, and involves a lot of manual effort. With the mass adoption of continuous delivery, DevOps, chaos engineering and SRE, such delays are no longer acceptable. Performance testing should be fast and performed continuously to let the development team fix issues early in the release process, so that the context of the issues are fresh and the costs are minimal.

Pioneers in continuous performance testing

Grid Dynamics has been working in continuous performance testing for years, well before it became a popular addition to the testing lexicon. This practice developed from our expertise in building mission-critical applications, our longstanding tradition of automating everything, and our use of agile best practices. Our architects and engineers have a lot of experience in continuous performance testing, having implemented it with many clients in different industries and platforms over the years. In 2011, we even released our own continuous performance testing tool called Jagger.

a web developer with a cube icon

Approach to continuous performance testing

Identify KPIs and design tests for them

The first important step of continuous performance testing is to find the most critical KPIs and determine what levels of performance are required for them. Once KPIs and test targets are known, the design of performance tests is similar to the design of functional tests. Some of the important questions that need to be answered involve test data management and dependency management. In some cases, testing with synthetic data and service virtualization may bring the biggest benefit, while in others, using production data and real dependencies makes more sense.

Automate everything

Let developers and testers find defects immediately so that they can be fixed quickly and cheaply. Do this correctly by:

  • Creating a distributed test environment with the required dependencies and test data.
  • Implementing load scenarios and collecting performance metrics from infrastructure and applications.
  • Building statistical and machine learning models for data correlation and results interpretation.
  • Creating pass/fail thresholds and delivering results via reports or data visualization.
  • Creating a distributed test environment with the required dependencies and test data.
  • Implementing load scenarios and collecting performance metrics from infrastructure and applications.
  • Building statistical and machine learning models for data correlation and results interpretation.
  • Creating pass/fail thresholds and delivering results via reports or data visualization.

Augment results analysis

A couple techniques can be used to automate the analysis of results:

  • Set KPI thresholds based on observed performance metrics.
  • Learn KPIs with machine learning and find anomalies if the new results differ too much from the old ones.
  • Set KPI thresholds based on observed performance metrics.
  • Learn KPIs with machine learning and find anomalies if the new results differ too much from the old ones.

Continuous performance testing is often performed on smaller environment to save costs. In order to predict absolute numbers, the correlation of metrics between production and the performance lab should be learned based on the past results.

Key benefits

an orange circle and a gray icon

Meet performance targets

Accurately forecast how much load your applications can securely handle.

an orange circle and two thin arrows

Catch performance issues early

Ensure that developers can continuously test the performance of development applications.

an orange circle and icon

Integrate with CI/CD pipeline

Remove human error and accelerate development by adding performance testing to the continuous delivery pipeline.

an orange circle and arrow icon

Automatically analyze results

Use statistics and machine learning to compare the new results with previous ones, and stop the pipeline if needed.

an orange circle and a monitor

Save on environment costs

Execute continuous performance testing on small environments to perform relative analysis.

an orange circle and a hand

Avoid costly performance failures

Prevent performance failures during peak loads that will antagonize customers and slash online conversions.

High level reference architecture

High level reference architecture

Technology stack

Logo of Jagger

Open source continuous performance testing platform

Logo of JMeter

Load testing tool and measures performance

Logo of Gatling

Load and performance testing tool

Logo of Jagger

Open source continuous performance testing platform

Logo of JMeter

Load testing tool and measures performance

Logo of Gatling

Load and performance testing tool

Logo of Jagger

Open source continuous performance testing platform

Logo of JMeter

Load testing tool and measures performance

Logo of Gatling

Load and performance testing tool

Engagement model

We prefer a hands-on approach in our engagements from day one. With the help of clients, our architects and engineers can identify challenging applications and services which would benefit the most from continuous performance testing. We then begin our assessments by looking at the application requirements, performance KPIs and existing methods of testing.

After we understand the current situation in depth, we make point improvements to bring the most value to our clients with the least amount of work, and in the shortest period of time. Once these show value on a small scale, we move to long-term improvements in the process and tooling on a larger scale, adopt continuous performance techniques across the board, and educate client teams to spread knowledge and know-how within their organization.

Read more

Get in touch

Let's connect! How can we reach you?

    Invalid phone format
    Submitting
    Continuous performance testing

    Thank you!

    It is very important to be in touch with you.
    We will get back to you soon. Have a great day!

    check

    Something went wrong...

    There are possible difficulties with connection or other issues.
    Please try again after some time.

    Retry