The results of a new artificial intelligence (AI)-based tool developed by a group of physicians, scientists, and engineers at Mount Sinai, published in Lancet’s eClinicalMedicine, could pave the way for a scalable, minimally invasive approach to continuous neurologic monitoring in NICUs. This approach would offer vital real-time insights into infant health that were previously unattainable.
Using video feeds of newborns in the neonatal intensive care unit (NICU), the team trained a deep learning pose-recognition algorithm to precisely track their movements and pinpoint important neurologic metrics.
In the United States, over 300,000 newborns are admitted to NICUs annually. Since infant alertness reflects the integrity of the entire central nervous system, it is regarded as the most sensitive component of the neurologic examination. In NICUs, neurologic decline can occur suddenly and have disastrous results.
Neurotelemetry has been challenging to implement in most NICUs despite decades of work in electroencephalography (EEG) and specialized neuro-NICUs. This is in contrast to cardiorespiratory telemetry, which continuously monitors babies' heart and lung function in the NICU. Periodically, neurologic status is assessed through imprecise physical examinations that might overlook subacute changes.
The Mount Sinai team postulated that neurologic changes in the NICU could be predicted using a computer vision technique to monitor infant movement. Robotics and athletics have been transformed by “Pose AI”, a machine-learning technique that uses video data to track anatomic landmarks.
More than 16,938,000 seconds of video footage from a diverse group of 115 infants in the NICU at Mount Sinai Hospital undergoing continuous video EEG monitoring were used by the Mount Sinai team to train an AI algorithm.
They showed that Pose AI can precisely track infant landmarks using video data. Using anatomic landmarks from the video data, they then made highly accurate predictions about two crucial conditions—sedation and cerebral dysfunction.
Although many neonatal intensive care units contain video cameras, to date they do not apply deep learning to monitor patients. Our study shows that applying an AI algorithm to cameras that continuously monitor infants in the NICU is an effective way to detect neurologic changes early, potentially allowing for faster interventions and better outcomes.
Felix Richter, MD, PhD, Study Senior Author and Instructor, Newborn Medicine, Department of Pediatrics, Mount Sinai Hospital
The research team was taken aback by how well Pose AI performed in various lighting scenarios (day versus night versus phototherapy-treated babies) and from various viewpoints. The correlation between their Pose AI movement index and gestational and postnatal ages caught them off guard.
Dr. Richter added, “It is important to note that this approach does not replace the physician and nursing assessments that are critical in the NICU. Rather, it augments these by providing a continuous readout that can then be acted on in a given clinical context. We envision a future system where cameras continuously monitor infants in the NICU, with AI providing a neuro-telemetry strip similar to heart rate or respiratory monitoring, with alert for changes in sedation levels or cerebral dysfunction. Clinicians could review videos and AI-generated insights when needed, offering an intuitive and easily interpretable tool for bedside care.”
The team pointed out the study’s limitations, including the fact that the AI models were trained using data gathered at a single institution. This means that video data from other institutions and cameras must be used to assess the algorithm and neurologic predictions.
The research team intends to conduct clinical trials to evaluate the technology’s effect on care and test it in more NICUs. They are extending its use to adult populations and investigating its potential for use in other neurological conditions.
At Mount Sinai, we are committed to ensuring that new artificial intelligence possibilities are investigated and leveraged to advance care for our patients. AI tools are already advancing clinical care across the Mount Sinai Health System, including by shortening length of stay, reducing hospital readmissions, aiding in cancer diagnostics and therapeutic targeting, and delivering real-time care to patients based on physiological data generated from wearables, to name a few. We are excited to now be bringing this non-invasive, safe, and effective AI tool into the NICU to improve outcomes for our smallest, most fragile patients.
Girish N. Nadkarni MD, MPH, Director, Mount Sinai Clinical Intelligence Center