The era of sensor fusion is coming soon

Posted by Pansy Chan on July 12th, 2019

Over time, sensors have evolved from simple analog and mechanical structures to chip-based digital devices that can be connected to a machine to monitor its health and environmental conditions. Similarly, sensor fusion (multiple sensors work together to solve problems) can be combined with a variety of other technologies to create something new and refreshing.

The idea of ​​using computing devices to classify, combine, and draw data from multiple sensors dates back to the 1950s, but it is difficult to implement. Around 1960, mathematicians developed an algorithm that attempted to draw a conclusion based on the input of multiple sensors by means of a machine and filtered out invalid data introduced by noise or other sources. Soon after, the military decided to adopt this technology. By processing multiple source inputs and comparing them with stored data, the military was able to better track and identify potential air targets and even calculate the exact location. With the advent of more advanced computers and sensors, the technology is also evolving, but there are still some overly complex problems that cost a lot.

Potential application

Microprocessors were once called a solution when they were first introduced, and sensor fusion can be considered as such. If you can monitor multiple sensors in some way, analyze the data in real time, and issue simple instructions or control an action, the application that sensor fusion can achieve will go far beyond your thoughts! Here are just a few simple examples:

Health monitoring – including fitness, patient monitoring and research

Elderly health monitoring - health monitoring to reduce staff burden

Automotive, transportation systems - monitoring and controlling transport efficiency and safety

Public safety – more accurate identification of potentially dangerous situations compared to simple fire and safety systems

Entertainment - games, including controllers and virtual reality headsets

Weather—Intelligent Weather Station not only predicts weather conditions, but also controls the system to prepare for storms (such as closing storm shutters, closing valves, etc.)

HVAC/Air Quality Monitoring - Intelligent control of indoor temperature, humidity, air quality, system maintenance, etc.

Although these functions have existed for many years, the system is revolutionary innovation by detecting multiple sensors and drawing intelligent conclusions and even taking action.

Like other electronics, some IC vendors have begun to study the heavy task of technology convergence. At present, various digital sensors can be effectively connected through off-the-shelf sensor fusion and sensor hub chips or other means, and there is no need to develop algorithms by themselves.

The methods used by these IC vendors may vary. Some will modify existing product lines, and some will develop new products for solving sensor fusion problems, but ultimately they will be implemented through specialized control chips, such as MCUs and sensor hubs. Or sensor fusion processor. Now, this technology has been successfully applied to smartphones, motion monitors and other consumer products.

New generation smartphones such as Apple and Samsung can combine multiple powerful sensors, including three-axis magnetometers, three-axis accelerometers, and three-axis gyroscopes, even without external connections. This combination is commonly referred to as 9-DoF, which is nine degrees of freedom.

In most cases, these features in your phone should be "always on." If these sensor data are all handled by the mobile phone's central microcontroller, battery life will be greatly reduced. Conversely, using an efficient dedicated microcontroller chip as a sensor hub to process data can dramatically reduce power consumption. NXP's ARM M3 series MCU is an example. EETimes reported that chip analysis company Chipworks survey results show that Apple's M7 processor uses NXP's customized version of the M3 chip to monitor the various sensors in the iPhone5S, including gyroscopes, accelerometers and compasses. The Samsung mobile phone uses Atmel's 8-bit AVR MCU.

With powerful board-level sensing technology, applications can use the 9-DoF feature in their mobile phones to monitor health and motion, or combine GPS and external data to provide users with more information. It is now also possible to add external data to those multi-function sensors via Bluetooth communication, and is almost unlimited. The goal of chipmakers is to allow engineers to design systems that provide real-time sensory data that provides the required context-sensitive information with low power consumption and long battery life. In addition to smartphones, highly optimized solutions are also available for a wide range of applications including tablets, ultrabooks, IoT devices, gaming consoles, healthcare, environmental monitoring and wearables.

Development boards that can help engineers easily experience this sensing technology, such as Atmel's ATAVRSBIN2, are now available on the market. Atmel integrates fusion sensors into many products and calls them "complete sensor ecosystems." Atmel realized that simultaneously analyzing and merging multiple sensor data is not an independent task. To solve this complex problem, the company works with a number of well-known sensor manufacturers and sensor fusion experts to provide a complete, easy-to-implement sensor center.

The current trend is to package three or more MEMS sensors with MCUs, such as STMicroelectronics' LIS331EB, which integrates a high-precision three-axis digital accelerometer with a microcontroller in a 3 x 3 x 1mm package. The microcontroller uses an ultra-low-power ARM Cortex-M0 with 64K bytes of flash, 128K bytes of RAM, embedded timers, two I2Cs (master/slave) and one SPI (master/slave). The LIS331EB can also process external sensor data, including nine external sensors including gyroscopes, magnetometers and pressure sensors. STMicroelectronics' iNEMO Engine Sensor Fusion Software Suite uses an adaptive prediction and filtering algorithm to process (or fuse) complex information from multiple sensors. The LIS331EB is used as a sensor hub that fuses all inputs together through the iNEMO engine software.

Freescale also introduced a family of products that package microcontrollers and sensors together. Their FXLC95000 Xtrinsic motion sensing platform integrates a MEMS accelerometer and a 32-bit ColdFire microcontroller. Similar to the STMicroelectronics device described above, the FXLC95000 can manage both internal and external sensor data. Freescale is the first company to offer an integrated sensing hub microcontroller that can be set up based on specific applications and algorithms. This family of devices manages up to 16 sensor inputs, separating calibration, compensation, and sensing functions from the application processor, but with the use of Freescale's own or third-party drivers.

The Freescale Xtrinsic FXLC9500 32-bit MCU sensor fusion hub with accelerometer provides a scalable, autonomous, high-precision multi-sensor hub solution that provides local computing and sensor management in an open architecture

Manufacturers such as Bosch, Fairchild, Honeywell, Microchip, and TI have also made significant achievements in this area.