Detecting defects in mass produced parts

Use image classification to solve real business problems

This tutorial will show you how to develop an AI-model that uses images to detect production faults quickly and efficiently, an important task in the manufacturing industry.

Once it has been trained, this model could be implemented straight into a production line to automatically indicate which parts to sell and which to scrap.

Person- Target audience: Beginners
Clock- Estimated time: Setup - 10 minutes. Training - 30 minutes
Spaceship- Tutorial type: Learn AI & platform
Bell- Problem type: Image classification

You will learn to
Peltarion logo- Solve a binary image classification problem. Detect defective products using images.
Peltarion logo- Analyze the performance of your model.
Peltarion logo- Test the deployment of your model.

Create a project

Let’s begin! First, click New project and name it so you know what kind of project it is.

New project button

Import the data

After creating the project, you will be taken to the Datasets view, where you can import data.

Click the Import free datasets button.

Import free datasets button

This will import the dataset in your project, and you can now edit it.

The data - Images of mass produced parts

Look for the Defects in metal casting dataset in the list.

The goal of this tutorial is to detect defects in submersible pump impellers. These are mass-produced metal components and the defects featured in this dataset are typical of many manufacturing processes.

The Defects in metal casting dataset features 1330 grayscale images of impellers with shape 256 x 256 pixels. The images have been labeled as either defective or non-defective.

Defect detection parts
Figure 1. Images of impellers. Non-defective to the left and defective to the right.

If you agree with the license, click Accept and import. This will import the dataset to your project, and you will see the dataset’s details where you can edit features and subsets.

Accept and import button

Preprocess the data

Use the default data subsets

By default, your data is split into three subsets:

  • Training (80%)
    This is the data that the model uses during training to learn how to classify the images.

  • Validation (10%)
    The validation set is used to evaluate the model, after each epoch, on data that it has not yet seen. This is used to identify the version of the model with the best performance.

  • Test (10%)
    Not used in this tutorial.

Set binary encoding

This problem is an example of binary image classification. This means that the model assigns each photo one of only two possible classes, true or false depending on whether or not the part is defective. Therefore, we will use Binary encoding.

Click the spanner icon to check if the default settings of the Defective feature are appropriate:

  • The Encoding should be Binary

  • The Positive class should be true

Binary encoding

Binary encoding will allow more precise evaluation options at the end of the experiment.

We are now ready to begin building the model. Click Use in new experiment.

Build the model

You will find the Experiment wizard, which takes you through the following steps:

  • Dataset tab
    Double-check that you are using the latest dataset version and that you have the appropriate subsets selected as training and validation subsets.

  • Input(s)/Target tab
    Make sure that your input feature is the Image and that your Target feature is the Defective.

  • Problem type tab
    Select the Single-label image classification.

This completes the Experiment wizard. Click Create to continue to the Modeling View.

Run the experiment

It is now time to run the model. The default settings work well for this experiment, but feel free to play around with optimization settings as much as you would like to see how it impacts performance.

Once you click Run, the model will take a while so have a break and check back later to see how well it is doing.

Run button

Evaluate the experiment

In the Evaluation view, you will find several ways of looking at how your model is performing.

Binary accuracy and recall

In the Evaluation view, there are various classification loss metrics.

For this particular situation, we are interested in binary accuracy and recall:

  • Binary accuracy measures the proportion of total predictions that are correct.

  • Recall measures the proportion of actual positives (defective parts) that were marked as defective.

The reason for looking at both of these metrics is that while it is important to have a high accuracy overall, in this case, false negatives are more harmful than false positives. This is the difference between accidentally discarding a non-defective part and accidentally shipping a defective one.

Confusion matrix and ROC curve

To inspect the predictions of your model, select the subset and checkpoint (epoch) you wish inspect and click Predict. By defatult the confusion matrix shows the predictions for the validation subset in the best epoch.

The confusion matrix is a good way of understanding exactly how your model has performed It shows you how many predictions fall into each possibility (true positive, true negative, false positive, and false negative).

The Threshold slider shows a sensitivity to making a positive prediction. In this case, because false negatives are much more harmful than false positives, we use a low threshold value.

confusion matrix

Another method which is specific to binary classification is the ROC curve. This is powerful because it lets you analyze its performance on the positive cases.

ROC curve

Deploy your model

The last step in this process is to deploy your model and see how it performs in the real world. To do this, download and unzip the test data - You will find 5 defective and 5 non-defective images that were removed from the other dataset. This means that they are completely unseen by the model so far.

In the Evaluation view click Create deployment. The Create deployment dialog will appear.

Create deployment button
  1. Select your desired Experiment and your best epoch as your Checkpoint.

  2. Click the Enable button to deploy the experiment.

Test how it works using our web app

To test your model, you can use either use the Open web app button or use the web app we made specifically for this case below.

Test with Peltarion web app

Open web app button

Tutorial recap

Congratulations, you have completed the defect detection tutorial! In this experiment, you have learned how to solve a binary image classification problem that can be implemented in real value-adding applications.

It is easy to imagine how this example can be efficiently integrated into a production line to automatically discard any defective parts. To read more about defect detection, read the in-depth blog post.

Next steps

Multi-label classification

In this tutorial, either the part was defect or not, a binary problem. But what if you had multiple features that could exist in one image, a multi-label classification problem.

We suggest that you continue with our tutorial Build your own music critic. You will solve a multi-label image classification problem for content-based automatic music tagging.

Test with Bubble made app

  1. Copy the URL and Token and paste into the corresponding fields in app.

  2. Click the image icon to upload one of the test images. Once you have tried it, you can upload another image by clicking the image again.

This web app was built using the Peltarion plugin on a no-code web development service called Bubble. To learn how to build app like this yourself, follow the Create a no-code AI app tutorial.

Was this page helpful?
Yes No