Multimodal integration of neuroimaging and genetic data for the diagnosis of mood disorders based on computer vision models
- Authors
- Lee, Seungeun; Cho, Yongwon; Ji, Yuyoung; Jeon, Minhyek; Kim, Aram; Ham, Byung - Joo; Joo, Yoonjung Yoonie
- Issue Date
- Apr-2024
- Publisher
- Pergamon Press Ltd.
- Keywords
- Computer vision models; Imaging genetics; Mood disorders; Psychiatric diagnosis; Precision psychiatry; Vision transformer
- Citation
- Journal of Psychiatric Research, v.172, pp 144 - 155
- Pages
- 12
- Indexed
- SCIE
SSCI
SCOPUS
- Journal Title
- Journal of Psychiatric Research
- Volume
- 172
- Start Page
- 144
- End Page
- 155
- URI
- https://scholarworks.korea.ac.kr/kumedicine/handle/2021.sw.kumedicine/65985
- DOI
- 10.1016/j.jpsychires.2024.02.036
- ISSN
- 0022-3956
1879-1379
- Abstract
- Mood disorders, particularly major depressive disorder (MDD) and bipolar disorder (BD), are often underdiagnosed, leading to substantial morbidity. Harnessing the potential of emerging methodologies, we propose a novel multimodal fusion approach that integrates patient -oriented brain structural magnetic resonance imaging (sMRI) scans with DNA whole-exome sequencing (WES) data. Multimodal data fusion aims to improve the detection of mood disorders by employing established deep-learning architectures for computer vision and machine-learning strategies. We analyzed brain imaging genetic data of 321 East Asian individuals, including 147 patients with MDD, 78 patients with BD, and 96 healthy controls. We developed and evaluated six fusion models by leveraging common computer vision models in image classification: Vision Transformer (ViT), Inception-V3, and ResNet50, in conjunction with advanced machine-learning techniques (XGBoost and LightGBM) known for high-dimensional data analysis. Model validation was performed using a 10-fold crossvalidation. Our ViT circle plus XGBoost fusion model with MRI scans, genomic Single Nucleotide polymorphism (SNP) data, and unweighted polygenic risk score (PRS) outperformed baseline models, achieving an incremental area under the curve (AUC) of 0.2162 (32.03% increase) and 0.0675 (+8.19%) and incremental accuracy of 0.1455 (+25.14%) and 0.0849 (+13.28%) compared to SNP-only and image-only baseline models, respectively. Our findings highlight the opportunity to refine mood disorder diagnostics by demonstrating the transformative potential of integrating diverse, yet complementary, data modalities and methodologies.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - 4. Research institute > Institute of Human Behavior and Genetics > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.