A new study shows how unlocking a regular phone could eventually support passive heart rate tracking, using facial video and deep learning to make cardiovascular monitoring more accessible, while raising important questions about accuracy, privacy, and real-world clinical use.
Research: Passive heart rate monitoring during smartphone use in daily life. Image credit: Have a nice day Photo/Shutterstock
Smartphones will soon automatically track heart health while people use their smartphones normally, without the need for smartwatches, fitness trackers, or intentional heart rate checks. In a large study published in the journal natureresearchers created a passive heart rate monitoring (PHRM) system that uses a cell phone’s front camera to detect changes in blood flow in a person’s face.
The system then uses deep learning algorithms to analyze these changes and estimate the individual’s heart rate (HR) and resting heart rate (RHR). Validated in over 160,000 videos, the system performed well in laboratory settings and everyday situations across skin tones, addressing the limitations of previous systems. Since smartphones are already widely used, this technology could make heart health tracking more accessible to many people, especially those who cannot afford smart wearable devices.
RHR refers to the number of heartbeats per minute when the body is at rest. This is an important measure of cardiovascular health. Changes in RHR may indicate an increased long-term risk of heart disease. Currently, measuring RHR over time typically requires a fitness band or smartwatch. However, not everyone uses these devices regularly. On the other hand, smartphones are widely used all over the world. Being able to measure RHR over the phone could make heart tracking easier, more convenient, and more affordable for people from different socio-economic groups.
Scientists have previously used remote photoplethysmography (rPPG) technology to measure RHR using smartphone cameras. However, these studies included only small numbers of participants and tested the technology in controlled conditions. The system was also less accurate for people with darker skin tones. As a result, it remains unclear whether this technology can reliably measure RHR during daily smartphone use in diverse races and ethnicities around the world.
About research
In the current study, researchers developed and validated the PHRM technology using data from several laboratory studies and real-world studies conducted between 2020 and 2024. The system records a short 8-second video of a user’s face when they unlock their phone. An AI algorithm then analyzes these videos to estimate the person’s HR and RHR.
The team first trained the system to recognize HR patterns using 192,353 video recordings from 485 people. They then tested the system using another set of 162,546 recordings collected from 211 different individuals of different ages, genders, body sizes, and skin colors. This diverse participant pool helped researchers assess whether the technology would work across a wide range of users. The researchers intentionally included more people with dark skin to address previous limitations in measurement accuracy.
The researchers assessed skin tone using spectrocolorimeter measurements and the Fitzpatrick skin type classification in the laboratory study, whereas participants in the free-living study self-reported their skin tone using the Monk Skin Tone (MST) scale.
An AI algorithm verified the quality of the video input. Confidence-based gates were then used to reject low-quality measurements, such as recordings affected by poor signal quality, inadequate lighting, or excessive motion. The system then combined multiple measurements taken throughout the day to calculate a person’s RHR.
Finally, the team evaluated the accuracy of the new PHRM system by comparing smartphone measurements with electrocardiographic recordings of heart rate and wearable-derived RHR estimates. They also investigated whether telephone-based RHR correlated with established indicators of cardiovascular health.
result
The PHRM system accurately measured HR both in controlled clinical tests and in routine daily use. It also produced accurate estimates of RHR. When the researchers compared smartphone measurements to measurements obtained from baseline ECG recordings, they observed similar results. For valid HR measurements, the average absolute error percentage was below the industry threshold of 10 percent for all three skin color groups.
The PHRM system met industry accuracy standards for consumer heart rate monitors. Compared to wearable HR trackers, PHRM had an overall mean absolute error (MAE) of 5 beats per minute (bpm) per day below the prespecified goal. This system performed better than 15 models using rPPG technology.
However, the system did not produce usable measurements from all videos, and the rate of valid video level measurements was lower for the darkest skin color group in free-living conditions.
For RHR, participant-level accuracy for the darkest-skinned group initially fell short of the 5 bpm goal, but performance improved after day 3 as the system’s filtering algorithm converged.
A key finding was that elevated smartphone-derived RHR was associated with known cardiovascular risk markers, such as higher BMI and lower cardiorespiratory fitness. Therefore, rather than demonstrating that the system can diagnose heart disease, the smartphone measurements confirmed the physiological validity of the approach.
Smartphone-based measurements were also more consistent on a daily basis than traditional one-time RHR checks. If this approach is further validated, it may be possible to monitor changes in cardiovascular health during daily smartphone use.
conclusion
The findings suggest that individuals could eventually be able to track their heart health simply by using their smartphones throughout the day. When people unlock their phones at different times, the phone can take multiple measurements and assess changes in heart rate patterns without having to wear a specific tracker or fitness device. Further work could optimize performance by reducing battery usage and testing on people with different heart conditions.
The researchers also emphasized that because the system involves passive facial video capture, actual use would require explicit informed consent, strong privacy protections, and secure on-device processing.
By making their AI models and research datasets publicly available, researchers hope to advance privacy-friendly heart rate monitoring technology that can expand access to cardiovascular monitoring through the smartphones people already use every day.
The study was conducted by researchers at Google Research and the University of Washington with funding from Alphabet or an Alphabet subsidiary, and the authors reported on Alphabet’s employment, potential stock ownership, and related patent filings.
Click here to download your PDF copy.
