AI targeting tumors & protecting patients to fight cancer


Radiotherapy company working with data scientists at Peltarion, Sweden

AI method




Two out of five people will be diagnosed with cancer at some point in their lives. Every year, we lose nine million people to this disease. We cannot stop cancer just yet. But what we can do is help doctors do their very best to help patients in a better, more effective, more efficient way.

One of the main treatments for cancer is radiotherapy, which uses x-rays to destroy cancer cells. The good thing with radiotherapy is that it kills the cancer cells for good. The bad thing is that the x-rays may also kill normal cells. Normal cells have a good chance of recovery, but they don’t always recover.

Radiotherapy has other drawbacks, too. It takes time to recover from the treatment. It may harm fertility. And it’s harmful to the patient’s sex life, at the very least.[1]  The more optimized the treatment can be, the more doctors can help patients with fewer negative effects.

Deep learning can be applied to tumor detection via segmentation masks. These are masks used to separate bad cells from good ones in an image, pixel by pixel. To be most effective, the ideal mask should be fitted as closely to just the bad cells as possible. Creating such an accurate, detailed mask requires a lot of expertise, for which training is difficult. It’s also a very time-consuming task for humans. 

Deep learning, on the other hand, is brilliant at analyzing images and can be applied to the creation of segmentation masks for any type of tumor. In a project testing this theory on brain tumor segmentation masks, data scientists at Peltarion worked together with a leading radiotherapy company to create deep learning models on the Peltarion Platform.

03/ Tumor locations captured on images

The first step was to prepare the data. In this case, we had a limited amount of input, so we had to be creative. 

We took 3D body scans and sliced them up into 2D layers, creating flat images of the area around the tumor. In this way, we could translate existing 3D data to use on the Peltarion Platform. To have more data to train and validate on, we used image augmentation to increase the dataset. We rotated the images +/- 10°, we zoomed 90 - 110%, and we flipped them horizontally.

Creating a trustworthy and correct dataset is crucial for making successful AI models, because eventually, performance will be impacted by the input. Any potential problem or mistake will only escalate along the way, so this step was very important for the project.

04/ Creating models and tweaking to improve

As a base, when building our model on the Peltarion Platform, we used a U-NET architecture developed by Ronneberger et al[2]  which is commonly used for segmentation. We also built a Tiramisu architecture[3] where blocks of the U-NET were changed to DenseNet blocks. 

In the dataset, we could see that only a minority of the image pixels were cancer cell pixels (thankfully). To be precise, only 2.4%. As these minority pixels are really important, we chose to add weight to make this data relatively more important. In this way, the few cancer pixels in the images got a more significant effect on the loss function, and the model got a better chance to train on important input.

05/ What results did we get to?

The main objective of the project was to find a way to optimize the segmentation masks. With help from the platform, we could use data, build models and find ways to be quicker — and also more precise — in the tumor detection process. 

Improved precision translates to more normal cells saved and quicker patient recovery. The whole process also saves the doctors time, giving them more time to take care of more patients and prioritized tasks.

This segmentation process could be used in any kind of healthcare diagnosis — pathology, oncology, ophthalmology, cardiology.

06/ More possibilities for AI in radiotherapy

Creating precise segmentation masks is one way of using AI in radiotherapy. Another area of AI worth understanding and exploring is personalization. As of today, most radiotherapy is a generic task. With help from AI and personal medical history data, the treatment could be made more ideal for a specific patient by adjusted the dose of the treatment acutely to the patient.

07/ Advice for the AI journey

The leap to using AI for healthcare is a profound one, but that doesn’t mean it has to be daunting. Look at it this way: choosing not to use AI risks lives.

Top tips from Peltarion:

  1. Look at the advantages AI can bring in healthcare and dare to start working with it. Not using AI is also a choice — where you say “no” to the chance of saving lives.
  2. Be innovative and try to apply the same methodology from one area into another.
  3. Find creative ways to translate and augment your data.


  1. U-Net: Convolutional Networks for Biomedical Image Segmentation — Olaf Ronneberger,Philipp Fischer & Thomas Brox (2015)
  2. The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation v3 — Simon Jégou, Michal Drozdzal, David Vazquez, Adriana Romero & Yoshua Bengio (2017)
  3. What is Radiotherapy? —
Want to replicate this use case in your business? Get in touch

08/ Try for yourself

09/ Get more insight