MultiCOVID: a multi modal Deep Learning approach for COVID-19 diagnosis

The rapid spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) led to a global overextension of healthcare. Both Chest X-rays (CXR) and blood test have been demonstrated to have predictive value on Coronavirus Disease 2019 (COVID-19) diagnosis on different prevalence scenarios. With the objective of improving and accelerating the diagnosis of COVID-19, a multi modal prediction algorithm (MultiCOVID) based on CXR and blood test was developed, to discriminate between COVID-19, Heart Failure (HF) and Non-Covid Pneumonia (NCP) and healthy (Control) patients. This retrospective single-center study includes CXR and blood test obtained between January 2017 and May 2020. Multi modal prediction models were generated using opensource DL algorithms. Performance of the MultiCOVID algorithm was compared with interpretations from five experienced thoracic radiologists on 300 random test images using the McNemar-Bowker test. A total of 8578 samples from 6123 patients (mean age 66 +/- 18 years of standard deviation, 3523 men) were evaluated across datasets. For the entire test set, the overall accuracy of MultiCOVID was 84%, with a mean AUC of 0.92 (0.89-0.94). For 300 random test images, overall accuracy of MultiCOVID was significantly higher (69.6%) compared with individual radiologists (range, 43.7%- 58.7%) and the consensus of all five radiologists (59.3%, P<.001). Overall, we have developed a multimodal deep learning algorithm, MultiCOVID, that discriminates among COVID-19, heart failure, non-covid pneumonia and healthy patients using both CXR and blood test with a significantly better performance than experienced thoracic radiologists. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This study did not receive any funding ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethics committee of Parc Salut Mar PSMAR) Consortium gave ethical approval for this work (2020/9199/I) Ethics committee of Parc Salut Mar (PSMAR) Consortium waived ethical approval for this work (2020/9199/I) I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors * DL : deep learning CXR : chest X-rays AUC : area under the receiver operating characteristic curve COVID-19 : coronavirus disease 2019 RT-PCR : reverse-transcription polymerase chain reaction SARS-CoV-2 : severe acute respiratory syndrome coronavirus 2 HF : heart failure NCP : non-covid pneumonia