Find similar images of fruits

Use image similarity to find out what fruit you (probably) got.

Imagine you want to buy a nice new kind of fruit. You’re looking at all the fruits available, and you think; So many new kinds of fruit! What fruit is that?
That’s when you happily reach for your deployed image similarity fruit model.

Person - Target audience: Beginners
Clock - Estimated time: 15 minutes

You will learn
Peltarion logo - Build and deploy a model for image similarity
Peltarion logo - Use the output from a specific block in the model

Why image similarity search?
Image similarity is a way to quantify how similar 2 images are.
Similarity search is faster than pixel by pixel comparison since the data becomes so compressed. Another advantage is that the image will be compared with each other on multiple levels and not just by their individual pixel values.

Similar cases

- What spare part is that?
- Where is my image used online?
- Where can I buy a similar bag?
- Do I have a lookalike?
- Totally looks like who?

Create a project

First, click New project to create a project and name it, so you know what kind of project it is. Naming is important.

New project button

Add the Fruit 360 dataset to the platform

After creating the project, you will be taken to the Datasets view.

Click Data library and look for the Fruit 360 - tutorial data dataset in the list.
Click on it to get more information.

The Fruit 360 dataset

This dataset consists of over 50000 photos of 100 types of fruits, photographed in color, 100x100 pixels under various angles.

Fruit images from the dataset
Figure 1. Fruit images from the dataset

If you agree with the license, click Accept and import. This will import the dataset in your project, and you will be taken to the dataset’s details where you can edit features and subsets.

Save the dataset

The dataset is automatically set up, so you just have to click Save version.

Then click Use in new experiment to open up the Experiment wizard.

Use in new experiment button

Build model with the Experiment wizard

  • Dataset tab
    Make sure the training and validation subsets are selected.

  • Inputs /target tab
    Select images as Input and fruit_class as Target. (This will help the model learn not only what the fruits look like, but also what they actually are).

  • Snippet tab
    Select Problem type Image similarity.
    Select the EfficientNet BO embedding snippet.

  • Weights tab
    Keep the weights from ImageNet and make sure the Weights trainable check-box is selected.
    Finally, click Create and you will find the model in the Modeling view.

Change to only 1 epoch

Before you can run the model you need to do a small change. Navigate to the Settings tab and set the number of epochs to 1. This means that all the images in the dataset will pass through the model once.

Why only 1 epoch?

Since the model is pretrained on ImageNet, it knows many common visual patterns and understands a wide variety of pictures. Fine-tuning for only 1 epoch makes them more familiar with the type of image in your dataset without forgetting everything they know. You can read more about catastrophic forgetting in our Knowledge center article.

So to run it for 1 epoch is good, then the model recognizes the new classes a little better but doesn’t put them above its pre-learned state.

Click Run and wait until the experiment has finished 1 epoch.

Run button

How does image similarity search work?

With image similarity, you want to compare a new image with all the images you have in your dataset to find the most similar images in the dataset.

  1. First you convert each image in the dataset into a vector with a deep learning model (in our case EfficientNet).
    All images’ vectors are saved into a large index.

  2. To find similar images to a new image, you transform that image into a vector with the same deep learning model, then you compare that vector with every vector in the index to pick the most similar ones.

How image similarity search works

Deploy model

Navigate to the Deployment view (you can skip evaluation this time) and click New deployment.

  1. Name the deployment and select Similarity search.
    Similiarity search deploy switch

  2. Make sure all settings are what you want them to be:

    • Dataset — The fruit dataset

    • Dataset version#1

    • Feature used for indexfruit_class

    • Experiment — The experiment you want to deploy

    • CheckpointEpoch 1 since you only trained for 1 epoch

    • Output featureImage embedding

  3. Click Create.

  4. Click Enable, and you will build an index that includes all image-vectors.
    Enable button

Your experiment is now ready to be called via the deployment API or try it with our API tester.

Test deployment

We’ve made it super easy for you to test the deployment. Click on Test deployment, and you will be directed to our Image similarity - API tester.

All info from your deployment is prepopulated (token, URL, Input parameter).

You just need an image to test. Try to download this image of some kind of fruit or take your own photo.

Test fruit test image

Add the photo to the bottom left corner and then click the Play button.

Image similarity test app

Tutorial recap

  • You’ve added a dataset to your project and built a model based on the pretrained snippet EfficientNet.

  • You’ve run the model for only one epoch to get the output from the model. This output is an index of all images in the dataset.

  • You’ve deployed your model and tested if it could tell you what fruit it was.

Further reading

You can dive deeper into what image similarity search is by reading this blog-article by Romain.

Test image similarity deployment with Postman

For expert users:
If you’re an expert user, we’ve created an easy walk-through how to use Postman to call your deployed model.

Image similarity result
Was this page helpful?