Recently, I got the opportunity to speak at the initiative seminar on AI organized by the Chalmers AI Research Centre (CHAIR). The key message in my presentation (see video here) was that working with artificial intelligence (AI), more specifically machine and deep learning (ML/DL), constitutes a major software engineering challenge that is severely underestimated by companies that start to experiment with machine and deep learning.
Machine and deep learning: Experimentation stage
Although I have discussed some of the challenges in a previous blog post, we have continued to conduct research in this area and have collected additional data concerning the specific challenges. In addition, we have developed a model that captures how companies typically evolve in their adoption of AI/ML/DL. In the figure below, the steps that companies typically evolve through are illustrated. In this blog series, I intend to discuss the challenges associated with each step, with supporting research from an article that was recently accepted for publication in the proceedings of the XP 2019 conference.
As the figure illustrates, the first step that most companies engage in is experimentation and prototyping. In this case, the work on machine and deep learning models is conducted purely in-house and without any connection to the products and services that the company offers to its customers. The work with basically any ML/DL approach follows the process shown in the figure below. In the simplest view, here are four stages: assemble datasets (or data pipes), create models, train and evaluate and deployment. There are two iterative processes represented by the inner loop and the outer loop. The inner loop is concerned with the typical activity of creating a model, training it, evaluating it and then tweaking the model with the intent of increasing accuracy and reducing error rates. The outer loop illustrates the periodic or continuous retraining of models based on the most recent data and the subsequent continuous deployment of models into operation.