Denoising images

Reconstructing images with an autoencoder

This tutorial will show you how to build a model for unsupervised learning using an autoencoder. Unsupervised in this context means that the input data has not been labeled, classified or categorized.

An autoencoder encodes a dense representation of the input data and then decodes it to reconstruct the input.

Person - Target group: Intermediate users

You will learn to
Peltarion logo - Build a custom model, block by block.
Peltarion logo - Deploy and test an AI model.

Preread: Before following this tutorial, it is strongly recommended that you complete the tutorial Deploy an operational AI model if you have not done that already.


The problem — Denoising images

You will use the images in the MNIST dataset to train your model. These images will be used both as input and target in the model.

Autoencoder reconstructing images
Figure 1. Autoencoder reconstructing images

An interesting property of autoencoders is that they will typically denoise input images that have artificial noise added to them, even though they were never trained to do this specifically. The cause of this behavior is that the dense representation learned during training will not contain information about the random noise, and the images will be reconstructed without it. You will use this property to realize the image denoising use case.

Send noisy images to the deployed model to test it

Once the model is deployed, you can send it images that have been modified to contain some random pixel noise. This noise is not part of the learned representation of the data and will be filtered out in the model predictions.


The data

The original MNIST dataset consists of small, 28 x 28 pixel images of handwritten numbers that are annotated with a label indicating the correct number.


Create a project for denoising images

First, create a project and name it, so you know what kind of project it is.

Once created, you can click on the project name in the Project options menu at any time to view the description, running experiments, and the amount of computing resources that you have spent on the project.

New project button

Add the grayscale MNIST dataset to the platform

  1. Navigate to the Datasets view and click URL Import.

  2. Copy the URL below:
    https://storage.googleapis.com/bucket-8732/ImageDenoiser/MNISTbundle.zip

  3. Paste the copied URL and click Import. The zip includes the complete MNIST training dataset.

  4. Name the dataset MNIST and click Done.

Rename the image-file column to image

Later in this tutorial you will have the option to evaluate the model off-platform using a Jupyter notebook. The dash(-) character is not a valid symbol in Python, so it is recommended that you rename the image-file column to image.

  1. To do this, click the wrenchwrench.

  2. Then change the name to image.

Alternatively, you can change the name of an input or output feature in the Deployment view, after you have completed an experiment.

You’ve now created a dataset ready to be used in the platform. Click Use in new experiment.

Create an experiment

Design a deep learning autoencoder

The model that you will build is based directly on the example provided in this Keras autoencoder tutorial.

Create a new experiment

  1. Name the experiment and make sure that the correct dataset is selected in the Experiment wizard.

  2. Click Create custom experiment.

Build the autoencoder

In the Build tab click on the following blocks to build an autoencoder on the Modeling canvas.

Inspector

First part of the model

Block Parameter

Add an Input block.

Set Feature to image.

Encoder

Expand the 2D heading. Change the parameter noted.

Block Parameter

Add a 2D Convolution block.

Change Filters to 16.

Add a Max pooling 2D block.

Change Horizontal stride to 2.
Change Vertical stride to 2.

Add a 2D Convolution block.

Change Filters to 8.

Add a Max pooling 2D block.

Change Horizontal stride to 2.
Change Vertical stride to 2.

Add a 2D Convolution block.

Change Filters to 8.

Add a Max pooling 2D block.

Change Horizontal stride to 2.
Change Vertical stride to 2.

Decoder

Block Parameter

Add a 2D Convolution block.

Change Filters to 8.

Add a 2D Upsampling block.

-

Add a 2D Convolution block.

Change Filters to 8.

Add a 2D Upsampling block.

-

Add a 2D Convolution block.

Change Filters to 16.
Change Padding to Valid.

Add a 2D Upsampling block.

-

Finalize the model

Block Parameter

Add a 2D Convolution block.

Change Filters to 1.
Change Padding to Same.
Change Activation to Sigmoid.

Add a Target block.

Set Feature to image.


Run experiment

You can train the model with the default values in the Settings tab, so just click Run.

Run button

Analyze experiment

In the Evaluation view you can watch the model train. You can see how the loss is getting lower epoch by epoch.

Navigate to the Predictions inspection tab. Here you can see that the model is quite good at removing noice.
What’s good really depends on your use case. Sometimes you need really good results. Sometimes you just need the model to give you good enough results.


Test if your autoencoder can remove pixel noise

Deploy the trained model

In the Evaluation view click Create deployment.

Create deployment button

The Create deployment popup will appear.

  1. Select your last Experiment and the Checkpoint for best epoch.

  2. Click Enable to deploy the experiment.

Enable button

Alternative 1 — Single predictions with CURL

You can use the CURL command to test model on a handful of input images.

tut10 7
Figure 2. Deployment view - Input examples (Curl)
  1. Download and unzip this test dataset. The images in this dataset contain random noise.

  2. Open a terminal and change to the directory that contains the noisy image files.

  3. In the Deployment view, click Copy to clipboard next to the Input example.

  4. Update the curl example so that the image parameter references one of the test files.
    Example:

curl -X POST \
-F "image=@1.png" \
-u "<Token>" \
<URLl>
  1. Run the CURL command in the terminal. The output will be a Base64 encoded string.
    Remember to keep the @.

  2. To view the output, copy all characters between the double quotes in the output and paste into an online Base64 decoder tool, e.g., onlinejpgtools.

  3. Save the reconstructed image

Compare input and reconstructed image

Compare the reconstructed image with the input image. Are they similar and has the model successfully removed the noise?

Alternative 2 — Analyzing the model output in Python

If you are familiar with Python, you can analyze the model predictions on the test dataset using this Jupyter notebook. To run the notebook you must either save or clone the entire GitHub repository or save the file in raw format with the .ipynb extension.

  1. Download and unzip this test dataset without random noise (it will be added in the notebook).

  2. Start the Jupyter Notebook:

$ jupyter notebook image_denoising_analysis.ipynb
  1. Install Sidekick and any other required Python packages.

  2. Update:

    • The path to your test dataset

    • URL

    • Token for your deployment.

  3. Run the notebook.

Compare input and reconstructed image

Compare the reconstructed image with the input image. Are they similar and has the model successfully removed the noise?

Notebook output when we run it
Figure 3. Notebook output when we run it

Tutorial recap

You have learned how to create an autoencoder, a type of unsupervised neural network. The model is trained to reconstruct images of handwritten numbers. In this process, it will filter out pixel noise that was not present in the training data.


Alternative solutions to image denoising

The autoencoder approach to image denoising has the advantage that it does not require access to both noisy images and clean images that represent the ground truth. However, if you want to create a model that is optimized for noise reduction only, supervised learning with, e.g., a link:U-Net or Tiramisu^s architecture will give better results.


Other applications for autoencoders

Other examples of practical datavision applications of autoencoders for include:

  • Computer vision tasks such as black and white image coloring and low-light image enhancement

  • Recommendation systems — predicting user preferences

  • Anomaly detection for manufacturing, maintenance, medical applications etc.

Anomaly detection example
Figure 4. Anomaly detection example - Reconstruction loss (error) will be high when an input sample is dissimilar from the data that was used in training.
Was this page helpful?
YesNo