This is part 2 of a 2 part series. You can find part 1 here.
From idea to AI powered applications, the challenges (part 2)
In this series, we’re exploring the challenges behind making deep learning based applications.
In the previous article in this series, we discussed the model development lifecycle challenges that come with creating a model aimed for a real-life application.
In this article, we’re going to look at the challenges of serving and integrating your model into your app, or in other words: how do you actually use your model in an application.
Model serving and integration
For the purpose of this discussion, we will be looking at the challenges that come with creating deep learning powered desktop / mobile apps and web apps.
Desktop / mobile apps
If you set out to create a standalone desktop / mobile app that will be powered by a model that you’ve trained, one of the issues that you’ll run into is that most deep learning frameworks are not using the same programming language and development tools.
Most of the deep learning model development frameworks/tools use the Python programming language and libraries such as TensorFlow, Keras, PyTorch, MXNet, or CNTK, If you were planning to develop your application in another language and tools like Android Studio, you have a problem.
To get around this, some of the popular deep learning frameworks offer ports or APIs of their library such as TensorFlow Lite in Java. , However, as you might guess, these libraries are not 100 % feature complete and do not keep up with the latest version of the original libraries...
In other cases, the community has stepped up to the challenge and has either created bindings or even implemented the whole deep learning framework in a language of choice (say a Tensorflow implementation in C#). But as with many non-official repos, they tend to suffer from not having all the functionality of the original framework, either because they haven’t been ported yet or because the contributors are having a hard time keeping up with the changes to the original.
And this is only the software side of things. When we start considering running applications that have a deep learning model at their core on different kinds of hardware (e.g. think off all the possible kinds of mobile phone hardware out there) things get even more complicated and technical and it might be necessary to convert trained models into a format suitable for the device’s processing and memory capabilities.
To avoid running into the problems above an alternative would be to move the application/the model to the web.
Unfortunately, there is no free lunch and this option comes with its own challenges:
- Web development - stating the obvious here: to move your app or model to the web you will need to have experience with backends, frontends, protocols, etc. If this is not part of your current skill set, you would have to spend some time learning how to do this, e.g. how to set up Kubernetes and separate TensorFlow containers that can interact with the rest of the system
- Choosing your web server framework - One particular backend task that will be crucial for your application will be choosing your web server framework. There are several alternatives available and figuring out which one to use and how it will interact with your application can require a lot of time to research.
- Setting up and managing your model deployment infrastructure - Whether you opt for an IaaS or PaaS you will need to have the necessary skills (and time) to set up and manage the cloud infrastructure that you want to use to run your model or application on.
This list is rather short, but looks can be deceiving as each one of these points is days, if not weeks of research and work to resolve. For a more comprehensive list of software engineering challenges with deep learning applications, see this scientific article published by the Research team at Peltarion.
Like in the previous article, we are aware of these problems and wanted to make the life of our users easier. That’s why we build the Peltarion platform to make things as easy as possible:
- One-click deployment - Deploy/rollback your trained models into production directly from the platform with one click. No configuration or extra work required. We make sure you have all the deployment infrastructure you need.
- Rest-API deployment - Integrate your model and model predictions into your own applications and automate interactions with the platform with a few lines of code. The platform allows you / your application to access your trained model via Rest-API.
- Scalability and availability - Run any volume of predictions, thanks to autoscaled resources allocated to your application.
- No technology hassles - The zero configuration, no-code environment of the platform means not having to deal with dependency hell or stitching together multiple libraries. The single integrated environment allows you to develop, deploy, maintain, and track your deep learning models.
A more humble list when compared to the one in the first article, but by no means a lesser list of features.
But again don’t take our word for it. We encourage you to try the whole end-to-end development process on the platform for yourself. It’s free!
This is part 2 of a 2 part series. If you’ve read this far, you’re probably going to like part 1 where I will be discussing what it takes to use a model that you’ve trained in an application. See you there!