Modality-AGnostic image Cascade (MAGIC) for multi-modality cardiac substructure segmentation

Purpose: Cardiac substructure delineation is emerging in treatment planning to minimize the risk of radiation-induced heart disease. Deep learning offers efficient methods to reduce contouring burden but currently lacks generalizability across different modalities and overlapping structures. This work introduces and validates a Modality-AGnostic Image Cascade (MAGIC) deep-learning pipeline for comprehensive and multi-modal cardiac substructure segmentation. Materials and methods: MAGIC is implemented through replicated encoding and decoding branches of an nnU-Net backbone to handle multi-modality inputs and overlapping labels. First benchmarked on the multi-modality whole-heart segmentation (MMWHS) dataset including cardiac CT-angiography (CCTA) and MR modalities, twenty cardiac substructures (heart, chambers, great vessels (GVs), valves, coronary arteries (CAs), and conduction nodes) from clinical simulation CT (Sim-CT), low-field MR-Linac, and cardiac CT-angiography (CCTA) modalities were delineated to train semi-supervised (n = 151), validate (n = 15), and test (n = 30) MAGIC. For comparison, fourteen single-modality comparison models (two MMWHS modalities and four subgroups across three clinical modalities) were trained. Methods were evaluated for efficiency and against reference contours through the Dice similarity coefficient (DSC) and two-tailed Wilcoxon Signed-Rank test (p < 0.05). Results: Average MMWHS DSC scores across CCTA and MR inputs were 0.88 ± 0.08 and 0.87 ± 0.04 respectively with significant improvement (p < 0.05) over unimodal baselines. Average 20-structure DSC scores were 0.75 ± 0.16 (heart, 0.96 ± 0.01; chambers, 0.89 ± 0.05; GVs, 0.81 ± 0.09; CAs, 0.60 ± 0.13; valves, 0.70 ± 0.18; nodes, 0.66 ± 0.12) for Sim-CT, 0.68 ± 0.21 (heart, 0.94 ± 0.01; chambers, 0.87 ± 0.05; GVs, 0.72 ± 0.18; CAs, 0.50 ± 0.18; valves, 0.62 ± 0.16; nodes, 0.52 ± 0.16) for MR-Linac, and 0.80 ± 0.16 (heart, 0.95 ± 0.01; chambers, 0.93 ± 0.04; GVs, 0.84 ± 0.06; CAs, 0.77 ± 0.12; valves, 0.68 ± 0.23; nodes, 0.72 ± 0.11) for CCTA. Furthermore, > 80% and > 70% reductions in training time and parameters were achieved, respectively. Conclusions: MAGIC offers an efficient, lightweight solution capable of segmenting multiple image modalities and overlapping structures in a single model without compromising segmentation accuracy.