As most of you know, Sweden is a late starter in the adoption of the latest AI techniques. However, a number of initiatives have recently been undertaken to accelerate AI adoption in Sweden. One such government-backed initiative, AI Innovation of Sweden, aims to gain learnings and insights from other countries and their AI journeys. Canada, and especially Montreal, has been recognized as one of the top AI-hubs in the world. With great help by Lindholmen Science Park and Montreal International, a Swedish delegation was put together, and a trip to Montreal was organized in November 2018.
The Montreal AI Ecosystem: Learnings from the Swedish Delegation
Participating was the Swedish delegation, together with representatives from Montreal International. In the picture below, from left to right we have Mats Nordlund (Zenuity), Martin Svensson (Lindholmen Science Park), myself from Peltarion, Justin Schneiderman (Gothenburg University), Anders Wennberg (Chalmers) and Jan Kilhamn (Västra Götalandsregionen). The front row consists of Jenny Almkvist (Västra Götalandsregionen), Adel Daoud (Gothenburg University), Maria Vedechkina (Montreal International) and Eric Kucharsky (Montreal International).
We had a very intensive schedule, with three days fully booked with meetings. The first day focused on meetings with various institutions, including the Innovation, Science and Economic Development of Canada, and research institutions such as IVADO and MILA. Here are some interesting facts and learnings from these sessions:
- The impact Yoshua Bengio has had on Montreal cannot be overstated. He is the only one of the three deep learning pioneers, a.k.a the Canadian Mafia, that has remained in academia. The other two, Yann LeCun and Geoffrey Hinton, were recruited into Facebook and Google Brain respectively.
- Bengio is also the founder of MILA, a research institute that focuses on fundamental research in AI. Very interesting listening to Myriam Côté (Director AI for Humanity), talking in no uncertain terms about the organization’s excellence, how they can choose and pick the best people, how partnering companies need to accept “no strings attached” and her mission to promote ethical and socially responsible usage of AI.
- When asked about intellectual property (IP), it was interesting to hear the experts’ views of patents and publications. In short, they claim there is no problem with respect to IP, as the AI field is moving so fast that they become obsolete anyway once published.
- To conduct proper research, work needs to be performed independently of product development and associated customer deliveries. Both are important but need to be carried out separately.
- IVADO is an institute for data science acting as a broker between academia and industry, which is very similar to the goals of Lindholmen Science Park and RISE. They provide the following services: 1) define research project 2) find suitable professor 3) help with funding 4) find an intern to do the work and 5) do the work.
- The Canadian federal government identified five AI “superclusters,” i.e., a set of important applications of AI. They will split government funding of up to $1 B. Montreal has awarded one of these superclusters to Scale.AI, which focus on boosting supply chain technologies with AI.
- Even though research institutions such as IVADO and MILA perform impressive fundamental research in AI, the need for applied AI should not be understated. When hearing from CRIM, an applied AI and IoT company with 50 percent funding from the government, it was impressive to learn about their work making real use of AI and industry 4.0 technology.
The second day focused on technology companies and accelerators. Here are interesting notes from some of the meetings.
- Stradigi.AI’s mission is to help enterprises transform into AI-powered solutions. They have two main teams: a solutions team and a research lab team (rather similar to us at Peltarion). The need is clear; the latest AI technologies have great potential, but we need better tools to enable all companies (not just the technology giants) to successfully take advantage of AI.
- TandemLaunch is an incubator providing co-location space for startups. They provide initial funding of around $100k, and if successful, startups continue with seed funding. The need to help turn research work into commercial applications is clear, and incubators like this are essential to making that happen.
- One successful example of a startup going through TandemLaunch is Algolux. They build intelligent and autonomous cameras, i.e., cameras that provide functionality such as object detection and lane tracking out of the box. It was very interesting to hear about their upcoming plans to support multiple cameras in to further improve the accuracy and safety of self-driving cars.
- Imagia makes use of AI for precision medicine, not only for radiology but also for endoscopy. With AI, they make it possible to predict, in real-time, if a polyp is benign or malignant They can even predict Alzheimer's disease with non-invasive pictures of the eye’s retina.
- The highlight of the day, if not the whole week, was the meeting with DeepMind. Most other meetings started with a presentation about the company/institution, but that was not the case for DeepMind. Doina Precup, Research Team Lead at DeepMind Montreal, simply sat down and said, “Ask me whatever you want.”Here are some highlights from that discussion:
- DeepMind was founded in London but has now expanded to Montreal, Edmonton and Paris. The Montreal office now has around 20 people and will continue to grow, with 70% PhDs.
- They apply an “open publication model” without any patents, meaning that they openly publish their work in academic publications and code in GitHub repositories.
- DeepMind is organized in a matrix structure, not unsimilar to how Spotify organizes with squads and chapters. Teams focus on topics and “efforts” cut across teams horizontally.
- Tooling includes using TensorFlow, PyTorch, Notebooks and Colaboratory. Interestingly, they do not have shared tooling for experiment management, but they do have internal project management tools that Doina, unfortunately, could not give any details about.
- They have a very clear research focus, putting none of their work into production. Successful projects such as WaveNet were initially researched in DeepMind and then handed over to engineering teams to make the project sufficiently performant to be implemented in Google Home.
- When asked about the importance of having research teams, Donia pointed out that research is not only important for long-term innovation but also impacts the ability to hire top talent.
- Deep learning has revolutionized AI in recent years, and according to Donia, the next hot topics will be in reinforcement learning (glad to hear since that is my own specialty) and AutoML.
The last day was similar to day 2, with a focus on accelerators and tech companies. At this point, the entire team was rather exhausted, but I did capture some interesting highlights:
- The co-founder of Techstars, David Brown, gave a very impressive talk about their progress over the last 12 years helping founders build technology companies around the world. 1600 companies have passed their program, 80% are still in business, 10% have performed an exit and 10% have failed.
- They have a clearly defined three-month program consisting of 1) mentor madness in the first month, with lots of conflicting advice to get the arrow pointed in the right direction, 2) starting to “fire the arrow” and talk to customers and 3) building the story in preparation for the “demo day” at the end of the three-month period.
- Another accelerator named District 3 provides free office space and utilities for startups thanks to government funding. Startups that show clear and dedicated commitment receive not only office space but also mentorship programs and coaching to help innovative minds achieve true impact.
- The last stop was the rather familiar company Ericsson, which operates in 180 countries around the world, with hundreds of billions SEK in sales. They are one of the top ten R&D companies in Canada and have been active there for 65 years. Lots of learnings from speaking with them. Here are some highlights:
- Their innovation program, called Encqor, works as a catalyst to drive innovation in 5G use cases. They also have a “garage” where people, both external and employees, are invited to their space to experiment with future possibilities of 5G, AI and IoT.
- Future possibilities for 5G are impressive. For example, they spoke of Hydro, a company that with 5G installed inside mines, that will be able to remotely operate the mining operations.
- Their tagline, “from copper wire to artificial intelligence,” clearly denotes their direction going forward. Because Montreal is one of the top AI hubs in the world, it’s a very important ecosystem for them to invest in.
Sweden has produced many success stories over the years with technology companies including Spotify, Skype, King, Truecaller and Minecraft. However, we have a lot to learn when it comes to adopting the latest AI techniques. Canada has a proven success story with the Canadian AI mafia of Yoshua Bengio, Yann LeCun and Geoffrey Hinton. Companies are investing heavily in the Montreal and Toronto areas, to a large extent due to the impressive research advances made by these three people. According to DeepMind, with great research, the rest will follow.
The Canadian government has also taken many actions to promote AI adoption by the industry. For example, it was interesting to hear IVADO talking about special tax credits companies can claim for scientific research work. The investments made in Canada, on a federal, regional and local level, are substantial. This includes both investments in research institutions such as MILA and IVADO, the five superclusters of AI and accelerators such as District 3 and TandemLaunch.
The full ecosystem of AI in Montreal is impressive, as visualized in the picture below. An important concept is “anchor companies,” i.e. companies such as MILA and Ericsson that fuel inspiration to people and other companies. We have that in Sweden as well, e.g., Spotify, King and Ericsson could demonstrate strong leadership in driving the adoption of the latest AI techniques.
Towards a token-free future in NLP
Improving multilingual models for low-resource languages
Extending BERT-like Language Models for Multiple Languages and Longer Documents
Predicting Bitcoin prices with AI
You need not have deep pockets or talented engineers to predict Bitcoin prices. Just use Python to download publicly available data and the Peltarion platform to train an AI on it. How hard can it be? As it turns out, it is not hard at all.
Search text by semantic similarity