M-US-EMRs: A Multi-modal Data Fusion Method of Ultrasonic Images and Electronic Medical Records Used for Screening of Coronary Heart Disease

Coronary heart disease (CHD) is the leading cause of death and disease burden in China and world-wide. The accurate screening for CHD can be of great value to guiding and facilitating the treatment. Traditional methods, such as computed tomography (CT) and coronary computed tomography angiography (CCTA), are costly and harmful to humans. To address these issues, we proposed a multi-modal data fusion method of ultrasonic (US) images and electronic medical records (EMRs) named M-US-EMRs model to automatically screen CHD. Comparing to traditional methods, this model features cheap, harmless to humans, easy to implement and independent of doctors. The experiment result shows that, M-US-EMRs model reached an overall classification AUC of 79.19 $$\%$$ . Furthermore, the interpretable models that we proposed lead to an increased trust from the people who use them. Our project developed effective tools with good performance for CHD screening among a Chinese population that will help to improve the primary prevention and management of cardiovascular disease. Our method lays the groundwork for using automated interpretation to support serial patient tracking and scalable analysis of millions of ultrasonic images and EMRs archived within healthcare systems.