Exploring multimodal data fusion and augmentation strategies for imbalanced datasets for intraoperative aneurysm occlusion prognosis

Purpose: This study investigates the efficacy of a multimodal machine learning framework in predicting treatment outcomes for intracranial aneurysms (IAs), focusing on how hemodynamic alterations captured through angiographic parametric imaging are fused with detailed patient information and disease morphology from digital subtraction angiography (DSA). The research explores various data augmentation strategies to manage dataset imbalances, aiming to enhance the model's accuracy. By integrating deep learning with this comprehensive, fused dataset, the study seeks to significantly improve the precision and predictability of prognostic assessments for IAs. Materials and Methods: Data from 478 patients with endovascularly treated IAs and DSA imaging were curated for this study. We utilized a multimodal approach, integrating quantitative and categorical data—ranging from angiographic parameters at the aneurysm dome to patient demographics like age, gender, and race. Separate deep neural networks were trained for each data type. Pre-decision layers from these networks were concatenated and used as inputs to a final network designed to predict treatment outcomes. Additionally, different multimodal architectures were tested for their ability to process and combine these data types effectively. To manage data imbalances, strategies such as Synthetic Minority Oversampling Technique (SMOTE) were implemented. The model's performance was evaluated through a 20-fold Monte Carlo cross-validation, focusing on metrics including the area under the receiver operating characteristic curve. Results: Our study indicates that augmenting multimodal predictive models for intracranial aneurysm outcomes with techniques including SMOTE enhances model performance. Late fusion models showed notable percentage improvements in predictive accuracy. The effectiveness of other augmentation methods varied, suggesting that the choice of technique is important for optimizing results. Conclusions: This study validates the effectiveness of a multimodal machine learning framework in improving the accuracy of predicting treatment outcomes for intracranial aneurysms, with data augmentation techniques substantially enhancing model performance.