Find similar images of fruits
Use image similarity to find out what fruit you (probably) got.
Imagine you want to buy a nice new kind of fruit.
You’re looking at all the fruits available, and you think; So many new kinds of fruit!
What fruit is that?
That’s when you happily reach for your deployed image similarity fruit model.
- Target audience: Beginners
- Tutorial type: Get started tutorial
- Estimated time: 15 minutes
- Problem type: Image similarity
You will learn to
- Build and deploy a model for image similarity
- Use the output from a specific block in the model
Why image similarity search?
Image similarity is a way to quantify how similar 2 images are.
Similarity search is faster than pixel by pixel comparison since the data becomes so compressed. Another advantage is that the image will be compared with each other on multiple levels and not just by their individual pixel values.
- What spare part is that?
Create a project
First, click New project to create a project and name it, so you know what kind of project it is. Naming is important.
Add dataset to the platform
After creating the project, you will be taken to the Datasets view.
Click the Import free datasets button.
Look for the Fruits 360 - tutorial data dataset in the list.
Click on it to get more information.
The Fruits 360 dataset
This dataset consists of over 50000 photos of 100 types of fruits, photographed in color, 100x100 pixels under various angles.
If you agree with the license, click Accept and import. This will import the dataset in your project, and you will be taken to the dataset’s details where you can edit features and subsets.
Build model with the Experiment wizard
This will import the dataset in your project. You can now edit it but you don’t have to since the dataset is automatically set up. You just have to click Use in new experiment to open up the Experiment wizard.
Make sure that the Fruit 360 dataset is selected.
Inputs / target tab
Select image as Input and fruit_class as Target.
(This will help the model learn not only what the fruits look like, but also what they actually are).
Problem type tab
Select Problem type Image similarity.
Finally, click Create and you will find the model in the Modeling view.
Run the model
Click Run and wait until the experiment has finished 1 epoch, this means that all the images in the dataset will pass through the model once.
In the Evaluation view, click Create deployment (you can skip evaluation this time).
Name the deployment and select Similarity search.
Make sure all settings are what you want them to be:
Experiment — The experiment you want to deploy.
Checkpoint — Epoch 1 since you only trained for 1 epoch.
Embedding block — Image embedding. These embeddings will be stored in the index used for similarity search.
Output features - Both fruit_class and image.
Click Create and you will build an index that includes all image-vectors. This may take some time.
Click Enable when the index is done.
Your experiment is now ready to be called via the deployment API or by deploying your own deployment web app straight from the deployment page.
Test with deployment web app
We’ve made it super easy for you to test the deployment. Click on Open web app, and you will be directed to the Deployment web app.
You just need an image to test. Try to download this image of some kind of fruit or take your own photo.
Add the photo and then click Get your result.
You’ve added a dataset to your project and built a model based on the pretrained block EfficientNet.
You’ve run the model for only one epoch to get the output from the model. This output is an index of all images in the dataset.
You’ve deployed your model and tested if it could tell you what fruit it was.
Learn how to improve image similarity
In the tutorial Product recommendations with image similarity, you will learn how to create a better customer experience with targeted product recommendations using image similarity.
Get started with image classification
We suggest that you start your deep dive into the Peltarion Platform with the Car damage assessment tutorial. It’ll walk you through the workflow at a higher level than this tutorial.
How does image similarity search work?
With image similarity, you want to compare a new image with all the images you have in your dataset to find the most similar images in the dataset.
First you convert each image in the dataset into a vector with a deep learning model (in our case EfficientNet).
All images’ vectors are saved into a large index.
To find similar images to a new image, you transform that image into a vector with the same deep learning model, then you compare that vector with every vector in the index to pick the most similar ones.
You can dive deeper into what image similarity search is by reading this blog-article by Romain.