A paradigm shift in retail search
Over the last two decades, retail search has made quite a journey from its roots in keyword matching full text search engines to modern sophisticated search platforms. Customers of the new AI age expect retail search solutions not just to match their keywords, but to understand the meaning of their queries and, ultimately, their shopping intent. Relevant, timely and personalized search results make product discovery experience frictionless and delightful, with direct impact on conversion rates and revenue per session.
Striving to achieve relevant results, modern search solutions employ a wide array of techniques such as semantic query parsing, query expansion with domain-specific knowledge graphs and intent classification. Those techniques help to compensate for the shortcomings of a fundamental keyword matching approach. Still, there is a lot of room for improvement.
Currently, deep learning-based natural language processing capabilities are evolving by leaps and bounds. Modern state-of-the-art NLP models enable the alternative approach to retail search, based on deep, “semantic” modeling of catalog data and customer’s shopping behavior. The next generation of semantic vector search engines is rapidly maturing, achieving relevant, timely and personalized results that closely match customer buying intent.
Semantic vector search in a nutshell
The main idea of semantic vector search is to represent both products and queries as a semantic vectors in the multidimensional semantic vector space. Products and queries have to be mapped to vectors in such a way that similar products and queries close by meaning would be clustered together.
This is achieved by training a deep learning model based on all available catalog data and customer engagement history mined from the clickstream. The model takes into account all available data about the products, such as attributes, images, descriptions and reviews, prices and promotions to find the best possible vector representation.
This makes it very simple to answer search requests; all you need to do is map the query to a semantic vector space and find all the products in the immediate neighborhood.
Semantic vector search provides for a powerful self-learning product discovery system which can be trained towards explicit business goals, such as click-through ratio or order conversion.
How semantic vector search improves the relevance of search results
Search engineers always deal with two key phenomena of the human language: polysemy and synonymy. Polysemy means that the same word may represent different concepts, like the word “table” in “table game” vs “accent table”. Synonymy means that the same concept can be represented in many ways by different words, like “Big Apple”, “NYC” and “New York” which all refer to the same city.
Semantic vector models support basic synonymy with the help of NLP models which are pre-trained on a large volume of e-commerce data. On top of that, terms used by the customers can be associated with relevant products when the semantic search model is trained on customer engagement data. When it comes to polysemic queries, semantic vector search easily resolves ambiguity of alternative meanings based on customer behavior.
Semantic vector search can directly tackle thematic searches, when customers are vaguely or subjectively referring to a particular class of products, such as “chicky dress” or “luxurious watch”.
Symptom searches, when customers are referring to a problem rather than a product that solves it, can be also easily supported by semantic vector search based on analysis of customer shopping history.