Over the last decade we’ve seen a massive growth in the wearables market, and diversification of the applications and markets and what they can track about the individual. We’ve seen the move from simple MEMS-based step counters, through different types of sensors and integration into clothing, (as my colleague was predicting back in 2015) through incorporation of more medical measurements such as ECG. At the recent CES we saw, as usual, masses of wearables. So what will come next?
If you could have your health and performance and sleep tracked and get recommendations, without having to wear something – how would that change your world?
It’s like the theramin pictured - an electronic musical instrument being played without physical contact. At CES there were lots of wearable wearables, but also a few making contactless measurements:
- The CES innovation award honoree Binah.ai app uses any camera-equipped device: "The contactless, video-based-only solution removes the need for wearables and provides medical-grade vital signs measurements such as heart rate, heart rate variability (HRV), oxygen saturation, respiration, temperature, mental stress, blood pressure"
Dornob.com reports on two non-contact monitoring systems:
- Vayyar: "we have a 3D imaging sensor that can look at you without actually having your picture," which detects falls.
- Platform House HED-Net demonstrated a system: "install a sensor recessed onto [the] ceiling. By monitoring vital data such as heart rate, perspiration rate, and blood pressure and using our proprietary algorithm, we [can] detect the onset of acute diseases.”
My colleague Jaquie Finn, from our medical business, delivered one of the keynotes of the CES Digital Health Summit, The Next Horizon for Passive Remote Monitoring Transcends. She put forward the idea that a non-contact monitoring system would be a game changer in hospitals. Today nurses make periodic, intrusive measurements, such as taking temperature and blood pressure, even in the middle of the night. What if these measurements didn’t need to disturb the patient? What if they were continuous, so as soon as a patient's condition changed, it could be detected? And all without anything contacting the patient?
To help bring alive what could be possible and help explore possible applications we have put together an early technology demonstrator to tell the story of how you could use non-contact monitoring to permit early diagnosis of sepsis, which is a major killer in hospital patients. My colleague has told the details of the sepsis medical story. Here’s how the system works:
The system was developed to measure heart rate, respiration rate, body temperature, and an alertness estimate, using off-the-shelf components, together with some advanced machine learning algorithms. We used a machine vision camera, thermal imaging camera, and depth (time-of-flight) camera. Machine learning and sensor fusion algorithms use this information to estimate body parameters. Facial tracking algorithms locate where to estimate the different parameters and allow estimating a surrogate for consciousness, measuring eyes open and eyes blinking. An area on the forehead is identified to measure body temperature using the thermal camera. The heart rate is estimated from the color camera, looking for subtle changes of color that happen each beat with flushing from the pulsing blood. The body form is reconstructed from the depth camera, which allows measuring the respiration rate. Together, these estimates give a snapshot of the physiological state of the body. Trended over time, they permit early detection of a degradation of the individual's health state.
What is interesting about this project is the ability to combine the output of readily-available sensors, derive information about underlying physiological states, and then create actionable insights, using sensor fusion and machine learning.
The application to in-hospital monitoring was reported above, but there are many other exciting application areas for this non-contact monitoring approach:
- Home medicine - At home, by collecting trends over time, they could provide early warning of oncoming shifts in health
- Automotive / driver safety - In a car or cab of a truck, or even cockpit of an airplane, to measure the alertness and health of the person at the wheel
- Baby care - In a baby’s crib, to detect distress
- Construction - At a construction site, to monitor health of workers
- Sports - At a fitness club, to monitor a workout or detect a safety-related event
- Consumer trials - In consumer trials to gather information about product usage / experiential sensing
Thus, I believe the future of wearables is - not wearable!