Multi-label image classification / cheat sheet

Target audience: Data scientists and developers

Problem formulation

Use this cheat sheet:
If your input data consists of a set of images (.jpg or .png), where each image can be said to contain or not contain multiple attributes. That is, the model tries to decide for each class whether the example belongs to that class or not.
This is called multi-label classification.

Example use cases

Please note that data sets, models and other content, including open source software, (collectively referred to as "Content") provided and/or suggested by Peltarion for use in the Platform, may be subject to separate third party terms of use or license terms. You are solely responsible for complying with the applicable terms. Peltarion makes no representations or warranties about Content and specifically disclaim all responsibility for any liability, loss, or risk, which is incurred as a consequence, directly or indirectly, of the use or application of any of the Content.

Smiling face PA2
Label: Smiling, Side part haircut, Outdoors

Images of faces annotated with information such as whether the subject is wearing eyeglasses, is smiling, or has a certain hairstyle. Here’s an example.

Lubeck river church PA2
Label: Sky, Water, Church

Images of landscapes, annotated with information such as whether they contain water, people, mountains, etc.

100 Days Of Sunshine 1 cut square PA1
Label: Happy, Dreamy, Piano, Vocals

Predicting the moods of a song. Yes, this can be done by looking at spectrograms. Check out this tutorial, for instance.

Data preparation

Data input requirements

Prepare a zip file containing all your images and a corresponding index.csv.
The Peltarion Platform supports .jpg or .png.

index csv multi PA1

Structure of index.csv

You need to structure the data so that each of the classes/labels is represented with an integer in its column in the index.csv file. Each row contains a 1 if the label is present in the image and a 0 if it is not.

image Label_1 Label_2 Label_3 Label_4











Create a feature set in the Datasets view

Once you’ve uploaded the dataset to the platform, create a Feature set with all the label columns. Use this Feature set as the target in the your deep learning model.


Try the smallest depth model first

The Experiment wizard will select the most appropriate model for your data and use case.

If you want to try other models, begin with smaller models first, since it will be faster to train and may already be complex enough to model your data well.
If the results are not good enough, you can move towards increasingly deep models in later experiments.

Changes in the Modeling view

Make sure that the last Dense block in the Modeling view has the same number of Nodes as the number of different labels that your data might have.

Set the Activation to Sigmoid. We choose sigmoid because it allows the model to output a number between 0 and 1 for each label independently. This number indicates the probability that the corresponding attribute is present in the image.

Make sure that the Loss in the Target block is Binary crossentropy.

Binary crossentropy setup
Was this page helpful?