Multi-label image classification / cheat sheet

Use this cheat sheet

If your input data consists of a set of images, where each image can contain or not contain multiple labels.

Example use cases

Note
Disclaimer
Please note that data sets, models and other content, including open source software, (collectively referred to as "Content") provided and/or suggested by Peltarion for use in the Platform, may be subject to separate third party terms of use or license terms. You are solely responsible for complying with the applicable terms. Peltarion makes no representations or warranties about Content and specifically disclaim all responsibility for any liability, loss, or risk, which is incurred as a consequence, directly or indirectly, of the use or application of any of the Content.

Lubeck river church PA2
Label: Sky, Water, Church

Images of landscapes, annotated with information such as whether they contain water, people, mountains, etc.

100 Days Of Sunshine 1 cut square PA1
Label: Happy, Dreamy, Piano, Vocals

Predicting the moods of a song. Yes, this can be done by looking at spectrograms. Check out this tutorial, for instance.

Data preparation

Zip with index.csv including file paths

The zip file must include a file named index.csv.
The index.csv must include paths to the files in the zip.

Example:

Zip file structure containing images and an index file.

Structure of index.csv

Each of the features (also called labels) must be represented by a column in the index.csv file. All features used as target must be binary. That is, either the image contains the feature or not.

Example:
Each row contains a 1 if the label is present in the image and a 0 if it is not. The 1 is the Positive class.

image 1 2 3 4 5 6

images/image_0.jpg

0

0

1

1

0

1

images/image_1.jpg

1

0

0

1

0

1

Create a feature set in the Datasets view

Once you’ve uploaded the dataset to the platform, create a Feature set with all the label columns.
Use this Feature set as the target in your deep learning model.

Experiment wizard

  • Dataset tab
    Make sure your multi-label dataset is selected.

  • Inputs / target tab

    • The image feature should be set as Input feature, since you want to classify images.

    • The feature set with all labels as Target feature

  • Problem type tab
    Select Multi-label image classification as problem type.

  • Click Create.

The Experiment wizard will select the most appropriate model for your data and use case.

Inspect in the Modeling view

The experiment is done and ready to be trained. All settings have been pre-populated by the platform, for example:

  • The last Dense block in the Modeling view has the same number of Nodes as the number of different labels that your data might have.

  • The Activation to Sigmoid.
    We choose sigmoid because it allows the model to output a number between 0 and 1 for each label independently. This number indicates the probability that the corresponding attribute is present in the image.

  • The Loss in the Target block is Binary crossentropy.

Binary crossentropy setup

Run experiment

Click Run in the top right corner to start the training.

Run button

Evaluate experiment

In the Evaluation view, you can see how the loss gets lower for each epoch (when the complete training set has run through the model one time).

Predictions inspection

By defatult the predictions from the validation subset at the best epoch are shown.

Select the class Label that you want to inspect. The prediction table, the Confusion matrix, and the ROC Curve will show the performance of the model on the selected Label.

The threshold value allows you to control how the errors made by the model distribute between false positive and false negative.

Deploy experiment

To deploy your experiment click Create deployment.

Create deployment button
  1. Select experiment and checkpoint of your trained model to test it for predictions or enable for business product calls.
    Both Best epoch and Last epoch for each trained experiment are available for deployment.

  2. Click the Enable button to deploy the experiment.

Enable button
Was this page helpful?
YesNo