A research group from INESC TEC and the University of Munich, along with Carnegie Mellon Portugal (CMU Portugal) Ph.D. student Tamás Karácsony, assessed an advanced solution to categorize seizures, which are the main symptom of epilepsy, with the use of 3D videos and infrared radar.
Image from an MRI simulator from INESC TEC Neuroengineering and Advanced Human Sensing Laboratory. Image Credits: INESC TEC.
The findings of this work coordinated by Tamás Karácsony’s supervisor and CMU Portugal Scientific Director, João Paulo Cunha, a researcher at INESC TEC and professor at FEUP were recently published in Nature Scientific Reports.
Although there is an extensive range of video materials available on seizure categorization, studies on the subject are yet rare; and approaches for programmed, AI-supported solutions are even rarer.
This new work shows a novel approach that is the first to consider near-real-time categorization from two-second samples, thereby presenting the probability of a system to support diagnosis and monitoring process (depending on recognition of action) that utilizes deep learning. This technique enables classifying between frontal and temporal lobes seizures (the two most common classes of epilepsy), or non-epileptic events.
Epilepsy, a chronic neurological disease, impacts 1% of the world population—where seizures are one of the key symptoms—whose semiology is important to diagnose possible events. The analysis of a seizure is generally done at epilepsy monitoring units (EMUs) by specialized healthcare professionals using 2D video-EEG (electroencephalogram).
During clinical diagnosis, the clinicians use these videos to visually recognize movements of interests defined by movement features (semiology).
Tamás Karácsony, Study First Author and Researcher, INESC TEC
Tamás Karácsony is also a CMU Portugal Ph.D. Student at the Faculty of Engineering of the University of Porto (FEUP).
The semiology assessment, however, is restricted by a high inter-rater variability among said professionals, and although being potential, the automatic and semi-automatic methods employing computer vision are still based on substantial “human in the loop” effort.
“A patient is usually monitored for several days, which has to be fully reviewed afterwards for the seizures. This requires a lot of time and effort from the clinical staff,” the researcher added.
The research group has made a deep learning-based method for the near real-time and automatic categorization of epileptic seizures to overcome this.
We present a new contribution inspired by the way experts analyze the semiology of seizures, taking into account not only the presence of specific movements of interest in different parts of the patients’ bodies, but also their dynamics and their biomechanical aspects, such as speed or acceleration patterns, or range of motion.
Tamás Karácsony, Study First Author and Researcher, INESC TEC
The research group turned to the largest 3D video-EEG database in the world and retrieved videos of 115 seizures, creating a semi-specialized and programmed pre-processing algorithm to eliminate unwanted environments from the videos.
Two image cropping methods are integrated in practical terms: depth and Mask R-CNN, offering a clean scenario and refining the retrieval of applicable information from the videos available, reducing unrelated variations, and upgrading the seizure distinguishing process.
As an additional clarification about the process utilized, Tamás detailed “Our solution uses an action recognition approach with an intelligent 3D cropping of the scene to remove unrelated information, such as clinicians moving around the patients. By removing it, our method significantly improved classification performance.”
According to him, this study has also showcased the practicality of the action-recognition method to categorize two epilepsy classes and the non-epileptic class, using just two seconds of samples, thereby making it suitable for near-real-time monitoring. Furthermore, the proposed solution can be employed in other 3D video datasets for monitoring and analysis of seizures.
Hence, the approach aids two purposes in translating this knowledge to better treatment and diagnosis: “it can be used for monitoring and alarms—which can alarm staff; or, if the approach is transferred to an ambulatory setting, a caregiver, when a seizure is ongoing, resulting in a faster response, which might decrease associated risks and Sudden Unexpected Death in Epilepsy (SUDEP). Without a near real-time approach this would not be feasible,” stated Tamás Karácsony.
More studies are required before implementing this system in clinical practice. However, the system is believed to be advantageous for the clinics, clinicians, and patients in the long run. “With automated diagnosis support, the clinicians have to spend less time reviewing the videos, thus can treat more patients, hopefully make better decisions, which reduces associated costs (material and health) for clinics and society,” he concludes.
Tamás Karácsony CMU Portugal/ Fundação para a Ciência e a Tecnologia (FCT) fellowship partially funded this study. Tamás, an ML researcher at INESCTEC, holds an MSc in Biomedical Engineering from the Technical University of Denmark and an MSc in Mechatronics from the Budapest University of Technology and Economics. Action recognition, computer vision, neuroengineering, and biomedical applications of ML are his key research areas.
“Interpretable DL Based Clinical MoCap for Epileptic Seizure Classification” is his preliminary thesis, and he will be spending his research tenure at CMU under Fernando De la Torre at Carnegie Mellon’s School of Computer Science.
Journal Reference:
Karácsony. K., et al (2022) Novel 3D video action recognition deep learning approach for near real time epileptic seizure classification. Scientific Reports. doi.org/10.1038/s41598-022-23133-9.