Automate your content moderation with AI

By leveraging modern artificial intelligence methods, companies can automate and speed-up their content moderation; a method that is also called AI moderation. AI moderation can be used for automatically analyzing and classifying potentially malicious content as a part of the full moderation process. Thus, increasing speed and effectiveness of the operators managing the moderation.

... does your content moderation feel like an overwhelming task?

02/ The problem

As companies are expanding their presence on various social media and community platforms, incoming poor quality content, spam, scam, inappropriate content, etc. is an increasing risk and a new reality for many. If negative content persists and is not continuously monitored and handled, it can affect customers negatively and increase the likelihood of potential customers turning to competitors.

03/ The opportunity for deep learning

Deep learning is excellent at classifying all kinds of user-generated content (from text to image, video and even sound!) and is, therefore, a great technique for automated content moderation.

Independent on what type of user-generated content you are working with, deep learning enables you as a company to classify incoming content in three different ways: 1) Binary classification can be used to classify between two classes, the most straightforward is spam or not spam. 2) Multiclass lets you define your own type of categorization, for example different kinds of unwanted content. 3) Multi-label classification can be used if the content needs to be classified with one or several labels.

All of these options open up the possibility of shaping your automated content moderation process as you need it.

04/ Platform model to use

Text classification, Sentiment Analysis, Image classification, and Video classification can all be used for content moderation.

05/ How does the model work?

Depending on the data type that you are using, the model will work a bit differently. For example, text classification will use a pre-trained natural language processing model trained on many texts in over a hundred languages to learn how languages are constructed. The model will then use the training data provided for the content moderation task to find the differences between the sentences to understand what words and combination of words that can be used to differentiate between the classes.

Image classification and video classification models consist of several deep learning layers. The first layers learn to recognize more fundamental and simple patterns, as edges and borders. The last layers combine the information from the first layers to learn how to recognize objects, such as animals or humans.

06/ Data requirements

As the models used for content moderation are supervised models, it is necessary to have both data in the format that is going to be moderated and labels for each data point that corresponds to the class we want the model to learn. How many necessary data points are hard to predict because it depends on the domain and how hard the classes are to differentiate. But at least a hundred examples per class is a good start.

07/ Model performance & success

The model performance can be visualized and analyzed in the Evaluation view in the platform after training your first model. One important aspect of classification models is deciding if it is better that the model flags spam that is not spam, also called false positives sample, or if it is better to flag no spam when it is spam, called false-negative samples. This is a balance between how sensitive the model should be and can be fine-tuned on the platform.

For these types of questions, you can always use the Peltarion support if you are not entirely certain on how you want to proceed.

08/ Where to learn more?

If you want to learn more about building a content moderation AI for your data, follow this tutorial to learn about image classification or this tutorial to learn about text classification in several languages.

And if you need help along the way with your data, we at Peltarion are always here to help.