ROI-informed cross-modal supervised contrastive learning for multimodal depression classification using rs-fMRI and T1-weighted MRI

  • Byeon, Jinmo; 
  • Lee, Hakjin; 
  • Jang, MyeongGyun; 
  • Yoon, Sujung; 
  • Lee, Hwa Min
Citations

WEB OF SCIENCE

0
Citations

SCOPUS

0

초록

Neuroimaging-based classification of depression-related symptom and diagnostic groups remains challenging because of clinical and neurobiological heterogeneity. Structural MRI and resting-state functional MRI provide complementary information, but most multimodal approaches rely on global fusion and do not incorporate region-level (ROI) information into cross-modal alignment. We propose a two-stage multimodal framework for depression classification based on ROI-informed cross-modal supervised contrastive learning (RCSCL), which does not require explicit ROI-to-ROI correspondence. Utilizing the Schaefer 200 atlas, we extracted ROI tokens using a MedicalNet-pretrained 3D ResNet-18 for structural MRI and a graph attention network for resting-state fMRI. The tokens were aggregated with attention pooling and the resulting modality-level representations were aligned in a shared embedding space using class-conditional supervised contrastive learning. Global features were then fused using self-attention, and the classification model was optimized using focal loss to address class imbalance. The framework was evaluated on an in-house single-center dataset of 500 subjects and the publicly available SRPBS multi-site MDD dataset. In the primary attention-based implementation, RCSCL significantly increased AUC/AUPRC from 0.805/0.326 to 0.855/0.410 in the in-house dataset and from 0.736/0.704 to 0.766/0.744 on SRPBS, numerically exceeding those of all evaluated fusion baselines without RCSCL; extending RCSCL to these backbones produced further but backbone-dependent gains. Hierarchical visualization analyses identified default mode, limbic, and prefrontal regions as recurrently salient, consistent with prior depression neuroimaging findings, although these attention-derived importance values are model-internal indices. These results suggest that ROI-informed multimodal contrastive learning improves class-discriminative representation learning for multimodal depression classification.

키워드

Multimodal neuroimaging; Resting-state fMRI; T1-weighted MRI; Supervised contrastive learning; Structure-function coupling; LOW-FREQUENCY FLUCTUATION; NEURAL-NETWORKS; AMPLITUDE
제목
ROI-informed cross-modal supervised contrastive learning for multimodal depression classification using rs-fMRI and T1-weighted MRI
저자
Byeon, Jinmo; Lee, Hakjin; Jang, MyeongGyun; Yoon, Sujung; Lee, Hwa Min
DOI
10.1016/j.media.2026.104350
발행일
2027-01
유형
Article
저널명
Medical Image Analysis
권
115