Research projects aim to bring AI to the physical world
Researchers are proving that bringing AI to IoT technology on the edge has the potential to identify trends and respond to them in new ways that were, until recently, out of reach.
To that end, IoT sensors that accomplish on-board AI machine learning are being tested for applications from wildfire detection to insect identification—in fact, a sensor has recognized the wingbeats of a mosquito to help prevent the spread of diseases such as malaria or dengue in one global research project.
The IoT system, tested in 2023, discharges mosquito repellent when an anopheles mosquito is detected in a room. It accomplishes this by classifying raw sounds from mosquito wingbeats to identify the mosquitos and their proximity.
Opportunities Abound
Research like this may open opportunities for IoT systems that are lower cost, and less power hungry, according to researchers at a consortium of universities.
Participants from Columbia University, the Abdul Salam international Center for theoretical physics (ICTP) Universidad federal de Itajuba (UNIFEI) and Harvard University among others have established TinyML4D Ledu learning open education initiative (TinyMLedu)—a consortium with the objective to make educational resources for embedding machine learning available worldwide. The Tiny ML4D academic network launched in 2021.
The goal of the Tiny ML4D research team is to educate—making embedded machine learning (ML) education scalable globally through open source curriculum and introductory workshops. The team hopes that education can empower developers and researchers worldwide to build locally relevant AI solutions on low-cost hardware.
Reducing the Flow of Data
Already commercial developers, as well as the university teams, have been working on enabling IoT devices to process AI data, helping to reduce the flow of data that needs to be transmitted to a server. With compression techniques, developers are enabling AI computing on low cost, low energy RFID and IoT devices.
There are a variety of such compression techniques that can be applied, including quantization classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT), Knowledge Distillation, Pruning, Low-rank Factorization, Fast-Conv and Selective Attention networks.
Internet of Intelligence
Marcelo Rovai, engineering professor at UNIFEI said embedded machine learning on low power devices enables a variety of applications that could potentially solve real world problems.
With what he calls the Internet of the physical world, IoT proliferation has reached the point in which cloud processing costs are too high for many applications, preventing them from scaling to the volumes of devices needed are ahead.
RFIDJournal previously published Examining The Interdependence of AI and RFID as part of our coverage about what AI means for the RFID and IoT industries.
“We are getting so much data and need to put this data to use to gain insight. But the problem is demand for connectivity and power,” said Rovai. For the sensor using IoT technology, “if I’m very close to where the data is generated, I need less power,” said Rovai, adding the data is also more secure.
The Uses of Sensors
An example of such onboard AI can be found in every pocket or purse. Smartphones accomplish facial recognition to authorize a recognized user to access the phone, while the data needed to detect that individual’s face can be stored in the phone itself, as opposed to the cloud.
Taking that effort and applying it to the world of low cost IoT devices, the goal is to train a chip of a sensor—from a temperature datalogging device to a RTLS locator—to identify an action as well as learn from data over time. That can mean identifying optical information or vibration, temperature or other conditions, and then determining the appropriate response without support from the cloud.
“Until now, such models have relied on the information available on the Internet, [while] objects such as wearable devices can be generating data that will inform an entirely different model,” said Rovai. Currently, developers are enabling AI algorithms to be stored in very small devices that use milliwatts of energy.
Understanding Conditions in Shipments
One demonstrable example is in logistics, where a global transportation company can gain relevant information about shipments based on the AI analysis of sensors deployed on containers. If the shipping company is tracking loaded containers traveling overseas, edge-based AI can help enable the measure and access of important information remotely.
“Let’s say I want to monitor containers all over the world. I want to see how the movement of those containers is being handled. I can train a small device with an accelerometer and gyroscope for example,” said Rovai.
The device can then use embedded machine learning (TinyML) to identify vertical movement, say a forklift transporting the container, or understand when something catastrophic has occurred, such as the container rolling.
Training Devices to Identify Events
A company could train AI-based TinyML devices to identify four or five states based on thousands of data points from the sensor’s own measurements. When the container comes within range of an IoT or RFID receiver, that sensor can then send specific information, based on AI, so that large volumes of information captured by the sensors does not have to be processed in the cloud.
“I can send a small packet of data with all that information rather than raw data,” said Rovai.
Other potential use cases in the shipping industry might be detecting that a container has been opened in the wrong location, at which time the sensor could identify the problem and send relevant alerts as trained.
Such systems are in use now for medical applications, in which wearable devices detect changes such as echo cardiogram (ECG) heartbeat measurements. The wearable can train themselves over time to detect when a potential heart problem is happening.
Agriculture and Remote Deployments
The TinyML academic team is now researching how such edge-based IoT could benefit communities or people in remote locations.
One pilot is an application detecting a specific kind of ant that is destructive to crops. To capture data from the field, the team is experimenting with a tractor that has an IoT-based, optical sensor on-board. The tractor drives through a field or vineyard as the sensor identifies conditions in the crops that could indicate the presence of the ant species of concern.
Such a solution can use LoRaWAN technology, with low-energy, long-range transmissions from the optical sensor to the cloud, via a gateway, providing alerts if the ant is present.
In Scandinavia, another project aims at tracking energy transmission lines with small condition monitoring devices connected to towers that capture temperature, vibration and other information to detect problems in the tower or potentially a fire.
If such a situation is detected and identified, the devices can then send an alert via LoRaWAN. Helicopters in remote areas such as snowy mountains can similarly use a wireless device with TinyML to detect wildfires.
“We saw an opportunity to do the same in Africa for wildlife detection or to protect animals such as elephants,” said Rovai. In this instance, drones could be sent over forested areas to capture data, accomplish analytics on board, and send relevant data as a result if a problem is detected.
Reducing Environmental Impact
AI on the Edge is being enabled by specific MCUs built into sensors.
Renesas Electronics, for example, recently released its single-chip RZ/V2H MPU that is designed to enable engineers to process AI applications at edge AI devices. It accomplishes this with AI acceleration technology said Daryl Khoo, the company’s Embedded Processing 1st Business Division VP. The accelerator speeds up the processing to enhance AI computing as well as image processing algorithms.
Edge-based machine learning systems also decrease the environmental impact of IoT installations, Rovai says. Using AI in the cloud stresses data centers and requires additional energy. Simply asking your smart speaker a question, Rovai pointed out, relies on energy and water to manage the data in the cloud.
“What we’re working on today is to take those models and shrink them to one-bit so that in the future we’ll see the same device that required a large amount of energy accomplishing AI on the Edge,” said Rovai.
Looking ahead, he predicted, “we’ll see some amazing things—increasing the power of devices, the power of their processing, using less and less energy. The models are becoming smaller as well as more powerful.”
Key Takeaways:
- AI on the Edge could make IoT solutions more scalable, less costly and reduce environmental impact.
- Some technology companies are releasing IoT sensor MCUs capable of AI independent of the Internet.