AIfES is an AI/ML framework written in C for even the smallest microcontrollers
In the last few years there have been more and more solutions for running machine learning (ML) on microcontrollers. Some of the most popular are scaled down versions of frameworks designed for servers. But those are only suitable for fairly powerful MCUs.
This is now a thing of the past. With the open-source solution AIfES (Artificial Intelligence for Embedded Systems) from the Fraunhofer Institute for Microelectronic Circuits and Systems (IMS) it’s possible to run, and even train, artificial neural networks (ANN) on almost any hardware, including the 8-bit Arduino Uno.
Note: The Fraunhofer IMS is part of the Fraunhofer Society, which currently operates more than 80 research facilities, including 75 institutes, at over 40 locations all over Germany.
Arduino AI Framework Library Is Here
AIfES is a standalone AI framework completely written in C. It allows on-device training without the need for a PC and can be used on almost any hardware. Even on the 8-bit controller of the Arduino Uno, an ANN can be trained pretty quickly. Personal, customized devices can be created that adapt to the task at hand by way of training. AIfES , with Arduino’s support, is now available for the Arduino IDE using the Arduino Library Manager.
This technology allows the development of small, self-learning, battery-powered devices that can be independent of a cloud or another device. Sensor data can be processed where it’s generated, directly on the device. Training data can be recorded by the device and used for training.
AIfES is comparable and compatible with well-known Python ML frameworks. This includes TensorFlow, Keras or PyTorch, while being significantly reduced in functionality. In the current version, feedforward neural networks (FNN) are supported, which can be configured completely freely. Also, common activation functions like ReLU, sigmoid or softmax are already integrated. A full implementation of Convolutional Neural Networks (ConvNet) will follow soon.
The model development is also based on the Python frameworks, so you can find your way around directly. The common algorithms for training, such as the gradient descent optimizer (SGD) or the adam optimizer, are also included.
Import ANNs From Other ML Frameworks
It’s possible to import a previously trained ANN from another ML framework. Only the network structure and weights are needed to map an ANN and train it further. A Keras example is included in the library, and it’s not necessary to port the model to TensorFlow Lite. The weights of an ANN in AIfES can also be read out, saved or sent directly from the device to be used on another device. Different ANNs can be loaded at runtime by reconfiguring the model and loading new weights. The weight sets can be stored in the cloud to make them available to other devices. The applications are endless.
All algorithms are optimized for use on resource-limited embedded systems. Even the required memory area for the ANN can be specified by the developer.
AIfES has been developed in a modular way to be able to exchange different components of the algorithms. A matrix multiplication, for instance. This allows the easy use of hardware accelerators of the different processor families. For example, the CMSIS DSP properties of the ARM Cortex controllers can already be used to compute or train deep neural networks particularly efficiently.
The Fraunhofer IMS has been using AIfES in AI research and development for years. Internally it’s been used as a development tool for customer-specific AI solutions until a level of development was reached where it could be used as a stand-alone product.
Interesting demonstrators have been created in that time. It’s been part of many public projects, and has already been integrated into future products together with companies from across the industry. As demos, an extremely compact handwriting recognition system was realized on an Arduino Uno. There’s a wireless current sensor for condition monitoring, and a gesture recognition system.
Pre-Trained and Adaptable AIfES Gesture Recognition
In gesture recognition, the model was pre-trained and the user performs the gestures as the developer trained them. This is the conventional way, but not very intuitive and the user has to adapt to the hardware. The new demonstrator uses the Arduino Nano 33 BLE Sense, which adapts to the user instead.
The desired gestures can be trained directly in the system without the need for a PC. A first example of product development within industry is the project “noKat.” A neural optical camera tracker for the detection of approaching persons, where a particularly compact human recognition system was realized using AIfES.
Due to their size, deep neural networks (DNN) are problematic for the implementation on a microcontroller. For this reason, Fraunhofer IMS has been researching techniques to realize particularly compact ML solutions for years. Using special feature extraction techniques the number of inputs can be reduced significantly, so only the relevant features are used.
AIfES is offered as a dual license model. For private projects or developers of free open-source software under the GNU General Public License (GPL) version 3, AIfES can be used free of charge. If AIfES is to be combined and distributed with commercially licensed software, or if the AIfES source code for commercially licensed software is not to be distributed under the GNU GPL V3, a commercial license agreement must be concluded with the Fraunhofer IMS. If you’re interested, contact the AIfES team.
AIfES for Arduino includes examples that get you started right away. Besides inference and training, it also shows how to import an ANN from Keras. The Arduino fruit identification project has been re-implemented in AIfES for the Nano 33 BLE Sense. Now the colors of three objects are trained directly on the device, without the need for a PC.
Download AIfES directly from the Arduino library manager or check out the AIfES for Arduino GitHub repository.
We’re super excited about this technology at Arduino, and we can’t wait to see what the community is going to build with it! Make sure you let us know your thoughts, and about any applications you create, over on the forums.