Applied AI & AI in business /

What is NLP and how can I make use of it in my business?

October 21 2020/5 min read

Making sense of NLP

NLP, or Natural Language Processing, is a field within AI aiming to enable computers to handle human communications (spoken or written).

It’s NLP that lets you say “Assistant, uh… raise the volume, like, a lil’ bit?” instead of “Computer, set parameter: volume to value: 11” and still get the expected result.

To make this possible, language needs to be encoded in a set of rules that a computer can understand and follow, which is the goal of NLP. 

This is an extremely difficult task. We usually take language for granted, but it’s actually a very complex construct. Here are just some examples of its intricacies: 

And to make matters even more complicated, we need to add our our own failings on top of all these language complexities, because we’re far from being perfect in using language correctly:

Sample misspellings of ‘Philadelphia’. Originally found here.

Needless to say, there are a mind bogglingly large number of possibilities of how words, word order, punctuation and many more aspects of a language which we haven’t mentioned here, can be used and somehow they need to be captured in rules for a computer to be able to process language correctly.

Traditionally the field of NLP has relied on complex sets of cleverly hand-crafted rules to, at least partly, model the language structure. More recently, it has made use of deep learning techniques for the same purpose, which have revolutionized the field. 

The latest techniques allow computers to come up with a way to build a model by themselves only by looking at text data examples, rather than being preprogrammed with a set of hand-crafted rules. Computers analyze billions of sentences gathered from multiple sources (e.g. novels, newspapers, chat messages, emails, etc.) and automatically look for patterns in the words and word / sentences structures. 

By repeatedly doing this, the computer begins to recognize patterns in the language and develops a model of its structure. Once it has built this model, next time it comes across a new sentence it should be able to say a lot about it. We’ll go more in detail about what this means in the next section.

What can / can’t  NLP do?

Before we explore the opportunities of NLP, let’s first dive deeper into what NLP models can and can’t do. 

Let’s start with the latter as it’s going to be shorter. Computers using NLP techniques don’t read or know anything about the significance of the text they process. In other words, NLP models (or anything AI for that matter) don’t have any cognitive abilities. They can’t think, reason, deduce, infer, accumulate knowledge or any other ability that you might associate with intelligence. All they do is look for patterns and relationships in the text data.

Now, you might be thinking that NLP models are probably not that useful then, after all, to extract valuable information from text data one needs to have an understanding about what the text is actually saying. Well, NLP models can do that… sort of.

If this sounds confusing, we don’t blame you. Here’s our attempt at clarifying things.

Modern NLP models analyze sequences of words:

This allows a model to start finding patterns in them, for example:

  • The word -the- can appear multiple times in a sentence and be preceded / followed by different words each time
  • The sequence order the-cat-didn’t-eat occurs often, while the sequence eat-cat-the-didn’t appears very seldomly if at all
  • The words eat, food and hungry often appear very close to each other in a sequence
  • The sequence didn’t-eat-food-wasn’t-fresh appears more often than the sequence eat-food-wasn’t-fresh
  • The word -it- often appears in places where sequences like the cat or the food usually also appear. Thus, it is interchangeable with the cat, the food and other similar sequences
  • etc.

By uncovering countless patterns like this and doing it over and over again across billions of sentences, modern models are able to build up an internal representation of how sequences of words are constructed depending on the words immediately preceding and following a given sequence. This means that they are able to correctly understand sequences of words in different contexts.

This is very powerful! It allows modern NLP models to “understand”, for example, that the word bank can be used in two completely different contexts:

The model doesn’t really know what bank means, but it knows that the word bank is often associated with two distinct groups of words which rarely are used in the same sequence.

As we’ll see more in depth in the next section, the fact that it can make this distinction means that, for example, we can confidently ask a model to go through a large number of text / documents and ask it to return only the instances where the documents are talking about banks in a financial context and not in the context of a river bank. 

This is how a modern NLP model is able to find useful information in text data and why we said earlier that they can (sort of) understand the context of what the text is saying.

An this is just one example of what these models can do, there are many more tasks they can perform: 

Let’s explore some of these in more detail in the article: The opportunities of NLP for business.

  • Reynaldo Boulogne

    Reynaldo Boulogne

    With over 15 years of experience, Reynaldo has worked within the intersection of business and technology across multiple sectors, most recently at Klarna and Spotify. He is passionate about innovation, leadership, and building things from scratch. Reynaldo is also a former Vice-chairman of the Stockholm based AI forum, Stockholm AI.

02/ More on Business & Applied AI