Target audience: Data scientists and Developers
In this tutorial, you will use a pretrained snippet in a classification model designed to detect different types of car damages. The number of images in the input data is small relative to the number of classes and has a significant amount of variation. This makes it challenging to create a well-performing model based on this dataset alone. As you will see, transfer learning can help to mitigate these problems.
The first layer in a convolutional neural network is designed to detect lower level features such as curves and edges. The output from this and subsequent convolutional layers will recognize increasingly higher level features that together represent the object to be classified, let's say dogs, clothes or fruits.
If you have previously created a classifier model for a set of objects in one domain, chances are that the learned weights in its first layers are also applicable to other objects in different domains. This is because the low-level geometric features are shared among all the classes in the different models.
One common approach in deep learning is to take a model that has been trained on a very large dataset, e.g., Imagenet, freeze the weights and then feed the output to a set of fully connected layers, we call them Dense blocks. Once you have trained the Dense blocks, often referred to as the "top" in a CNN, you can continue to train the model in iterations. In each of these iterations, you will "unfreeze" the weights for a group of 2D Convolution blocks while gradually lowering the learning rate. To continue to train a model on the platform, you will just duplicate an experiment and include the weights from the best epoch in the last experiment.
When you create an experiment on the Peltarion platform, you can make use of the learned weights from previous experiments, or you can use a pretrained snippet such as the VGG16 feature extractor. This will reduce the training time, and thereby GPU hour costs. It also makes it possible to improve the performance of models that are trained on small datasets.
The dataset that you will use in this experiment contains approximately 1,500 unique RGB images with the dimensions 224 x 224 pixels, and is split into a training- and a validation subset. The underrepresented classes in the training subset have been upsampled in the preprocessing in order to reduce bias. This means that the index file (index.csv) has duplicate entries that are linking to the same image file. The total number of entries in the index file is approximately 3,800.
Each image belongs to one of the following classes:
Below are sample images from the various classes in the dataset. Note that the unknown class contains images of cars that are in an either pristine or wrecked condition.
Each collected image represents one car with one specific type of damage. This means that the dataset can be used to solve a single-label classification problem.
If you want to learn how the raw data was processed to create the dataset used in this tutorial, you can dive deeper here.
First, create a project and name it so you know what kind of project it is. Naming is important.
A project combines all of the steps in solving a problem, from pre-processing of datasets to model building, evaluation, and deployment. Using projects makes it easy to collaborate with others.
The subset column, containing a T or a V, indicates if the row should be used for training or validation. The split between training and validation data is approximately 80% and 20%. This column was created during the preprocessing of the raw data.
Even though it is possible to use the default subsets created by the platform when you upload the data, it is more advantageous to create a conditional split based on the subset column. For this dataset, there is no separate labeled test subset, in case you want to analyze the performance of the deployed model outside the platform. Instead, you can compare the model predictions with the ground truth provided with the predefined validation subset.
Delete the default the subsets by clicking the Options menu (...) then Delete.
Click New subset and fill it out appropriately for each selection:
Select the training subset that you have created in Normalize on subset.
You’ve now created a dataset ready to be used in the platform. Click Save version and navigate to the Modeling view.
Let's start training the Dense blocks of the model for ten epochs with the Adam optimizer. Everything should be set up correctly so just click Run.
Go to the Evaluation view. Since the model solves a classification problem, a confusion matrix is displayed. The top-left to bottom-right diagonal shows correct predictions. Everything outside this diagonal are errors.
Note that metrics are based on the validation subset which only consists of 20% of the original dataset.
Click the dropdown next to Cells and select Percentage. The normalized values that are now displayed correspond to the recall for each class.
The recall values clearly indicate that the model has learned the features in the images, but there is still room for improvement. So far, you have only trained Dense blocks of the model. By training some of the 2D convolution blocks in the snippet, it should be possible to get better results.
Go to the Evaluation view. You should see a general improvement in all the performance metrics, including the training loss and the recall values for each class.
If you are familiar with Python, you can analyze the model predictions on the validation subset using this Jupyter Notebook.
Using transfer learning, you have saved hours of training time and created a better performing than would have been possible, had you trained the model from scratch.
In this tutorial, only three blocks of the Feature extractor blocks were set to trainable. For other datasets, training more blocks in the Feature extractor snippet in a longer chain of iterations will continue to provide meaningful improvements to the performance of the model. There are are also cases when training the entire model in a single iteration will give you good results.