Applied AI & AI in business /

Beyond the hype: A brief look at semantic search

April 7/4 min read

In a time where all of humanity’s knowledge is at our fingertips, and business internal documents compile to an ever-increasing unstructured mess, the ability to ask the right questions and knowing how to skim through results to find an answer has become the kung-fu of our time. In today's competitive environment, businesses can't afford to miss out on data to drive corporate success. But the way we go about finding and retrieving relevant information has been extremely inefficient, and still is. Enterprise search engines with semantic capabilities are the new kids on the block. Here is why you need to look into it.

What would happen if your business could find accurate and relevant information within documents from various sources within minutes, rather than hours or days? The answer to this question has led to the emergence of a branch in natural language processing (NLP) called semantic search and is currently one of the most promising innovations within the field of AI. 

According to the Gartner Hype Cycle (2021) semantic search is – in the midst of a hyped and evolutionary AI market with inflated expectations that dilute businesses' decision-making – the most commercially viable AI innovation to date. And it’s now on the verge of mass adoption. That being said, what is it all about? 

Gartner Hype Cycle, 2021

What is semantic search?

Semantic search is a specific application of natural language processing (NLP) that utilizes semantic similarity to accurately surface relevant information with minimal human intervention. More intuitively, we can think of semantic search as retrieving information based on the underlying meaning of a query, instead of simply matching keywords. As a result, semantic search delivers results that not only include the exact words in the query, but also synonyms and metaphors. Consequently, it’s a shift from string matching to concept matching. This enables us to not only write queries in a more flexible and intuitive way, but also to retrieve information that was previously impossible through a keyword search. 

Example:

As an illustration, words like “practitioner”, “physician” and “medical doctor” mean the same thing but do not share any similar strings. Similarly, the word “GOAT” can refer to both an animal as well as the “greatest of all time”, depending on the context and intent of the query. 

In a keyword search setting, unable to capture these variations, our search results would consequently be inefficient and retrieve a small portion or irrelevant information. In semantic search, however, we are able to capture the context and enhance the accuracy dramatically. Here’s a simple depiction:

For a more technical description of how text similarity, the underlying technique of semantic search, works in detail, see this blog post: Search text by Semantic similarity

How can semantic search engines help enterprises?

As semantic search engines are on the verge of revolutionizing search strategies by retrieving more accurate and relevant results, the implications on businesses are profound. For example:

  • Retrieve relevant results faster
    As semantic search becomes better, your blackbelt in “Google-fu” will successively lose its edge on your CV. But this is good news for businesses. We all know the staggering hours we spent scrolling through documents only to receive nothing of value. Accumulated across businesses, this is a real source of inefficiency. Organizations that work with unstructured or unorganized data can benefit profoundly from semantic search as it matches concepts rather than keywords, and allows you to retrieve relevant information across multiple sources at the same time such as email, social media, documents.

  • Better decision-making
    To be a data driven organization is key in today’s competitive environment. But as organizations oftentimes have accumulated massive volumes of unstructured data, relevant information is difficult to find using keyword-search. With the ability to retrieve a higher degree of relevant information and thus utilize new sources of insights, businesses are able to make better strategic decisions.

  • Better UX
    As semantic search understands the intent behind a search query and retrieves results that are conceptually relevant, the user experience of finding information will drastically enhance. Expanding this to multimodal capabilities, enabling dynamic search through text to retrieve images – one can imagine the implications for an industry like retail.

  • SEO

The emergence of semantic search does not only carry implications for business internal operations, but have also affected the field of SEO. Ever since Google  started to adopt semantic search in their search engine, the game of SEO has drastically transformed. SEO professionals have had to shift focus from creating pages and content for every single keyword, to creating content around a certain topic. Today, more comprehensive content that answers questions around a certain topic or concept, are the pages that relieves the most traffic. 

To get an intuitive feel of what semantic search is all about, Peltarion has created an application that allows you to state a question (in any language) and receive answers to similar questions from the Google Natural Question database.*  This type of application is for example utilized by many customer services in need of rapidly and dynamically retrieving answers to already resolved questions. Although this might not be how you will use semantic search in your business, it allows you to get a feel for the fundamental implications it has on the future of search queries. 

Try it out here!

* If there exists a similar question in the database! Google's Natural Question database contains 300 000 questions posed by real people. Although an impressive amount of questions, expect some setbacks if you aim to cheat at Trivial Pursuit. This application is for illustrative purposes only. 

  • Jacob Sjöberg

    Jacob Sjöberg

    Growth Representative

    Jacob is currently working with questions regarding growth at Peltarion. From a perspective of business and management, he explores how organizations can become successful in their AI efforts and maintain competitiveness in a new era of machine learning.

02/ More on Business & Applied AI