MedCT-BERT: Multimodal Mortality Prediction Using Medical ConvTransformer-BERT Model.

In the Intensive Care Unit (ICU), mortality prediction tasks primarily rely on clinical records consisting of patients’ clinical time series data and physicians’ diagnostic opinions. However, due to the irregularity of time series data and clinical records, existing medical multimodal models focus on generating complete and regular data to address this issue, neglecting the impact of generated modality data on multimodal feature fusion. Inaccurate generation of modality data may lead to noise. To overcome this problem, we propose a novel medical multimodal model named MedCT-BERT in this paper. Specifically, the existing models can address the irregularities in time series data but are limited to handling fixed time series data. We optimize their hyperparameters for better performance. By doing so, our model can process and generate complete regular imputed values in parallel for time series data with varying missing values and sequence lengths. To iteratively refine the generated imputed values, we introduce a time series feature correlation information to reduce noise in multimodal data fusion. We conduct experiments on the MIMIC-III dataset with MedCT-BERT and validate the effectiveness of the model in mortality prediction tasks.