Search performance is critical to keeping low costs and high quality
Business problem
When improving digital search engines, firms need to invest equally in product discovery features and search performance efforts so that search speeds are fast and costs are kept under control. Whenever firms add new product discovery features, there are new types of data, business signals and sophisticated queries for search engines to handle. Eventually, these changes can negatively affect response times, indexing duration and overall search data staleness. These accumulated problems will slow search, upset customers and increase search infrastructure costs. Therefore, companies need to control new features and ensure that they are properly supported by search performance engineering.
Experts in search engines performance
Grid Dynamics started in 2006 with the goal of making mission-critical systems scalable. Since then, performance and scalability engineering have been at the core of our company's DNA. As Solr and Lucene contributors, we have a deep understanding of search engine internals, and possess years of experience tuning the performance of search systems. For over 5 years, the search solutions that we have built for our retail customers have survived Black Friday traffic storms with flying colors, without a single outage or breach of SLAs. We have helped numerous customers solve their search performance issues. We improved response times, online conversion rates from search, and indexing speeds.
Our approach to search performance
Taking a top-down approach
When it comes to performance engineering, we prefer to use a top-down methodology by starting our analysis with high level system metrics, then drilling down to understand the true root cause of the issues.
We ask ourselves three guiding questions:
- What computational resources are bottlenecking and preventing applications from going faster? We use deep system-level monitoring to find the true culprit.
- Where exactly does the bottleneck happen? We use sampling and profiling to identify hotspots and resource congestions down to the individual subsystems of the search engine.
- How can the bottleneck be removed? This is where our deep understanding of the internal workings of the search engine helps. It is often possible to slightly change the index structure, rewrite a query or tune caches to improve performance dramatically.
Search performance engineering methodology overview
Areas of performance optimization
Search index structure
Indexing performance
Query performance
