In a recent study published in the journal iScience, researchers introduced a novel approach to enhance brain tumor surgery using deep learning-based hyperspectral image correction and unmixing. This method improves the accuracy and reliability of hyperspectral imaging (HSI) for fluorescence-guided tumor removal. The goal was to address the limitations in existing imaging techniques, offering a framework to enhance image quality and precision in identifying tumor boundaries during surgery.
Deep Learning Based Hyperspectral Image Correction and Unmixing for Brain Tumor Surgery. Study: Image Credit: khunkornStudio/Shutterstock.com
Hyperspectral Imaging in Neurosurgery
HSI is an advanced imaging technology that captures a wide spectrum of light from each pixel in an image. This capability allows for the identification of various materials based on their spectral signatures. While HSI is used in fields like agriculture and environmental monitoring, it has significant potential in medical fields, especially in fluorescence-guided surgeries to distinguish tumors from healthy tissues.
For example, fluorescence-guided surgery using 5-aminolevulinic acid (5-ALA) allows the visualization of malignant gliomas. This compound is absorbed by tumor cells and is metabolized into protoporphyrin IX (PpIX), which fluoresces under specific wavelengths of light. However, traditional fluorescence data analysis often struggles with complexities due to tissue heterogeneity and optical artifacts. Therefore, there is a need for novel approaches to enhance the reliability and effectiveness of fluorescence-guided imaging in neurosurgery.
Deep Learning for Hyperspectral Imaging
In this paper, the authors used two deep-learning models to correct and unmix hyperspectral images captured during brain tumor surgeries. The first model, the Attenuation Correction and Unmixing Network (ACU-Net) is a supervised deep-learning architecture designed to process fluorescence spectra and estimate PpIX concentrations.
The second model, Attenuation Correction and Unmixing by a Spectrally-informed Autoencoder (ACU-SA) uses a semi-supervised approach to leverage labeled and unlabeled data. Both models are based on a convolutional neural network (CNN) structure, equipped to handle complex, high-dimensional data typical of HSI in neurosurgery.
The researchers conducted experiments on a large dataset of hyperspectral images from 184 patients, including 891 fluorescence HSI data cubes covering 12 tumor types. These datasets represented all four World Health Organization (WHO) grades and included isocitrate dehydrogenase (IDH) mutant and wild-type samples. Furthermore, training was performed on phantom and pig brain homogenate (PBH) data with known PpIX concentrations to assess the performance of each model.
The ACU-Net model integrates residual connections and convolutional layers to enhance feature extraction, aiming to minimize variance between predicted and actual fluorescence spectra for improved PpIX quantification. In contrast, the ACU-SA model leverages a Siamese architecture to condition the network on known endmember spectra, which helps unmix fluorescence data while also allowing the integration of unlabeled human data.
Impact of Using Deep Learning
The study indicated that both the ACU-Net and ACU-SA models significantly improved the accuracy of PpIX concentration estimation compared to traditional imaging methods in brain tumor surgery. The ACU-Net model achieved Pearson correlation coefficients of 0.997 for phantom data and 0.990 for pig-brain data.
These values represent the close match between known and computed PpIX concentrations. In comparison, traditional methods, such as dual-band normalization followed by non-negative least squares (NNLS) unmixing, yielded lower correlation coefficients of 0.93 and 0.82, respectively.
The semi-supervised ACU-SA model also showed promising performance. It achieved correlation coefficients of 0.98 for phantom data and 0.91 for pig-brain data, suggesting its potential for generalizing to human data. Importantly, the ACU-SA model demonstrated a 36% reduction in false-positive rates for PpIX detection in human samples. This reduction is valuable for minimizing the risk of removing healthy tissue during surgery.
Additionally, the deep learning models exhibited enhanced robustness against common challenges in hyperspectral imaging, such as artifacts and variations in fluorescence signals. The authors highlighted that the ACU-Net model not only improved quantitative outcomes but also provided better visual quality in PpIX concentration maps.
These outcomes indicated that deep learning-based approaches effectively addressed the limitations of traditional imaging techniques and provided a more reliable tool for intraoperative decision-making. Furthermore, both the presented models offered faster processing times, making them suitable for real-time use in surgery.
Key Applications
This research has significant implications for neurosurgery. The enhanced capabilities of the ACU-Net and ACU-SA models can significantly improve the accuracy of tumor detection, supporting more effective surgical interventions.
Beyond brain surgery, these deep learning techniques could also be adapted for oncology, where accurate tumor margin assessment is crucial for treating different types of cancers. They could also improve diagnostic imaging technology, enhancing patient outcomes.
Conclusion and Future Scopes
In summary, deep learning proved effective in enhancing the accuracy of HSI for medical applications, particularly brain tumor surgery. By addressing optical and geometric variations in fluorescence signals, these models significantly improve tumor margin detection accuracy during brain surgeries. These findings not only support more effective surgical interventions but also suggest that deep learning can advance diagnostic imaging technologies.
Future work should focus on expanding the dataset, incorporating more fluorophores, and further optimizing these models to enhance clinical utility and applicability. Integrating these imaging advancements could transform surgical practices and improve patient care.
Journal Reference
Black, D., & et al. Deep Learning Based Hyperspectral Image Correction and Unmixing for Brain Tumor Surgery. iScience, 2024, 111273. DOI: 10.1016/j.isci.2024.111273, https://www.sciencedirect.com/science/article/pii/S2589004224024982
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.