Stable Multimodal Medical AI Developed to Function Even with Incomplete Test Results or Clinical Records

Professor Park Yurang's Team at Yonsei University College of Medicine Predicts Mortality Risk Without Imputation
Performance Gap with Conventional AI Widens as Data Becomes Scarcer

A multimodal medical artificial intelligence (AI) system has been developed by a Korean research team that can reliably assess patient risk in real-world clinical settings, even when laboratory results or clinical records are incomplete.

The research team led by Professor Park Yurang and researcher Hong Jaesung from the Department of Biomedical Systems Informatics at Yonsei University College of Medicine announced on September 16 that they have developed a multimodal medical AI framework called 'ETF-UML.' This framework can predict the risk of patient mortality using only the available data, even when certain test results or clinical records are missing.

Multimodal AI technology analyzes different types of data together, such as basic patient information, laboratory results, and clinicians' records. However, in real-world medical settings like emergency rooms or intensive care units, it is often challenging to collect all relevant data simultaneously. For example, blood tests and vital signs vary depending on the timing and frequency of measurement, and certain clinical records, such as discharge summaries, may not exist in the early stages of hospitalization.

Traditional multimodal AI models are often trained on the assumption that all data will be available, resulting in decreased performance if any data is missing. To address this, some models artificially fill in missing data through "imputation" methods, but this can distort a patient's true condition and medical judgment if unverified information is estimated and inserted.

The research team focused on a method that utilizes only the available data, without generating or imputing missing values. ETF-UML independently learns structured data such as basic patient information and diagnostic or prescription records; time-series data such as blood test results and vital signs; and clinical text written by healthcare professionals. The framework is designed so that these diverse data types share a common decision-making standard.

The team verified the performance of ETF-UML using three mortality-prediction tasks with the internationally recognized intensive care databases MIMIC-IV and eICU. For MIMIC-IV, 190,027 records were used for 30-day mortality prediction and 164,723 records for 90-day mortality prediction; for eICU, 119,852 records were used for 10-day in-hospital mortality prediction. As a result, ETF-UML showed higher AUROC (predictive accuracy) across all three tasks compared to existing models, including single-modality models, integrated learning fusion models, and models using missing data handling techniques such as GRAPE, SMIL, SHASPEC, and MUSE.

ETF-UML recorded the highest AUROC compared to existing models such as the 'single modality model', 'integrated learning fusion model', and missing data handling techniques like 'GRAPE, SMIL, SHASPEC, and MUSE' across all three evaluations.

The strengths of ETF-UML became even more pronounced when data was scarce. Even when all structured data such as basic patient information was available, but the observation rate of the remaining two types of data was reduced to 30%, ETF-UML still outperformed all existing top-performing models in every scenario. In the eICU dataset, the performance gap was as large as 5.34 points. When using only two data types, ETF-UML still surpassed the existing models by 4.4 to 10.7 points in multiple settings.

ETF-UML also required fewer computational resources for AI operation. Its maximum used graphic memory was just 0.44GB, about one-tenth that of MUSE, a comparative multimodal AI model, which required 4.70GB. The total amount of computation required was also reduced by more than half.

The research team explained that these findings are significant for applying multimodal AI in real medical environments, where it is difficult to have complete patient data. Professor Park Yurang stated, "We designed the framework so that all types of data share a common decision criterion, making reliable predictions possible even with only the available information. Through further studies, we will expand this framework to include imaging data and a broader range of clinical tasks, and aim to develop an AI system applicable in actual clinical practice."

This research was published in the international journal Information Fusion (Impact Factor 17.4).

This content was produced with the assistance of AI translation services.