Classify customer complaints with sentiment analysis
Use natural language processing and deploy a working model
No one likes to think about customer complaints. You want to spend all your time trying to build a product so good that you won’t receive any.
… but when they eventually do trickle in, it’s good to know right away what the customer complains about so you can get to the bottom of it.
- Beginner. No prior data science experience is needed.
- 15 minutes to download the data and build the AI model.
You will learn to
- Build a classification model
- Deploy your model
- Work with natural language processing (NLP)
You can watch how Björn goes through everything from downloading a dataset and building the AI model to finally calling the API in this step-by-step demo.
Create a project
Let’s begin! First, click New project and name it, , e.g. Consumer Complaints, so you know what kind of project it is.
After creating the project, you will be taken to the Datasets view, where you can import data. So time to get the data.
Download the data from CFPB
You will use data from the CFPB (Consumer Financial Protection Bureau). The database is huge and includes complaints about consumer financial products and services that the CFPB has sent to companies for response.
Go to this page and look through the terms and conditions so that you know what you agree to.
In this CFPB page, you’ll see that we’ve added filters so the downloaded file only includes complaints with the following conditions:
From New York State
Click on Export data that appears a bit down on the screen, just above the map view that is made up by squares.
In the pop-up dialog, select these two boxes:
Filtered dataset box.
Click Export data. This will download the dataset to your computer.
Upload and process the data
Upload the data
In the Peltarion Platform Datasets view, click Upload files and upload the dataset file. Name the dataset, e.g., Complaints, and click Done.
Process the data
Click the Table tab, and you’ll see the individual records of the CSV file.
Find the column named Consumer complaint narrative and click the wrench icon next to it.
Change the label to narrative. This will make it easier to refer to this column in later steps.
Click Use in new experiment.
Create an experiment and build a model
Now, it’s time to build and train the model. This model will be able to predict what product the consumer is referring to by looking at the customers’ written complaints. You will use the English BERT block that includes the whole BERT network. The pretrained block allows you to use a massive network with weights that are pre-trained to understand the text. You don’t have to build it yourself. Major win!
Start by giving an Experiment name to the experiment, e.g., Experiment 1.
The Experiment wizard has a few steps. In the first step, the correct subsets are selected by default, so Click Next.
In the next step, select:
In the Input(s) column, check only narrative
In the Target column, select Product
Set Problem type to Single-label text classification.
Edit settings and start training
The selected BERT block appears on the Modeling canvas. You now want to update the settings and train our model.
Go to the Settings tab to the left of the canvas.
Check Early stopping.
This will make sure that the training is stopped when the performance of the model no longer improves, making training faster.
Update Epochs to 50.
One epoch is when the model has seen all the examples once. Since you have early stopping, you can have any high number here since the training will automatically stop.
Now you are ready to train the model. click Run in the upper-right corner.
Training will take some time, so now will be a good time to take a little break.
Deploy the model
In this case, the results you get from the default settings are pretty good. If you want to try to tweak it further, you can try setting up a few more experiments to see if there are ways to improve it - otherwise, we’re ready to set up a deployment.
In the Evaluation view click Create deployment. The Create deployment dialog will appear.
Select which Experiment and Checkpoint that you want to use (usually you would choose the one with the best overall performance). Name the deployment.
Click the Enable button to enable your deployment. As soon as your deployment is enabled, you can start requesting predictions.
When you submit a request to the deployed model, you have to send all the Input features. The response will contain the predicted Output feature for each submitted example.
To send queries via the API, use the URL and the Token required to allow the deployment to respond with predictions.
Now your AI model is built, and you’re ready to go! The next thing you want to do is to use the deployment in your business: