Building intelligent market research surveys with text similarity search


Founded in 1975 Headquarters in Paris

AI method

Text similarity



Dealing with data is part of daily life for the global market research company Ipsos. And for a company already skilled in data processes, the potential with adding AI is tremendous. So we decided to team up to bring their domain expertise and our AI experience together, to see what good we could create. We quickly realized there are so many areas to bring value to with AI, but one of them is “Question Library Similarity Search”. And we’re currently working our way through more!

02/ How can AI help in market research?

Drawing conclusions from data is really the core in market research, so the step to using AI to automate is fairly small. If you have datasets of images or text, you can easily use them to classify different factors such as geography, demography, ethnicity, and connect them with habits, opinions, and much more.

The majority of data that market research companies possess is text. Text data that can be classified into different categories, text data where correlations are found with other text data, and text data with a past which makes it possible to predict the future. All of which AI can help, to make things go faster and be more accurate.

03/ How did we come up with the right ideas?

Bringing the right people together is key as always. In this case, we brought experts from Ipsos together with data scientists at Peltarion, and people with business minds from both sides. This way, we could brainstorm our way to the best and most valuable use case ideas for Ipsos in specific.  

To come up with a few ideas may sound easy, but finding the right use case ideas that are timely, commercially viable, and practically doable, is tricky but crucial to succeeding with AI.

04/ How we did the Do-It-Yourself survey builder

After a few discovery workshops together we came up with the project “Question Library Similarity Search”

Asking questions could be a really simple task, but knowing how to professionally build up a survey with the “right” questions is a whole scientific field of its own... For instance - you can ask “How old are you?” and you get the answer “42”. But if you read the answer two years later it’s no longer correct. Instead, asking “What year were you born?” with the answer ”1978”, could be a much more valuable answer to keep for the records.

Same question in two different ways, where the latter will remain useful also in a few years...

For the Question Library Similarity Search, we used the questions from 90’000 projects, resulting in som 600’000 raw questions as input and used state of the art NLP (Natural Language Processing) techniques to provide users with similar questions, to present good alternatives to choose from.

The value for Ipsos? By systemizing the process of finding new alternatives for questions with help of AI, Ipsos can raise the quality of how their research is done, and maintain a high and consistent standard across the globe. And - in many cases save a massive amount of time. All of which will give Ipsos a better competitor advantage.

If you’re curious to hear more about our project, here’s a good video of Andrei Postoaca, CEO Ipsos Digital, and Anders Arpteg, our Head of Research, talking about "Similarity Assessment Using Bert in Market Research - Ipsos Digital", at Re Work’s Deep Learning Summit in London. And - take the chance to create your own survey and try out Ipsos.Digital FastFacts with the trained model implemented!

BERT encoder from Peltarion Platform, helping to solve the problem

05/ Starting your own AI journey!

As mentioned, the possibilities to automate market research with AI are endless ish. The advice we can share after our initial and ongoing projects together - Ipsos & Peltarion - is to start. Really start. Once you’ve done one project with a clear problem statement, with data available, and with a curious mind, you will see it’s not that hard to start with AI. The roadblocks you lift from your first project will make it easier for the next one, and then the next one, and next... So go ahead and start!

Want to replicate this use case in your business? Get in touch

06/ Try for yourself