Multi-task multi-modal learning for joint diagnosis and prognosis of human cancers.

•Consider the inherent correlation between diagnosis and prognosis tasks and propose a novel multi-task multi-modal learning framework for joint diagnosis and prognosis of human cancer.•Integrate histopathological image and genomic data for the diagnosis and prognosis of human cancers.•Conduct experiments on three cancer cohorts from the TCGA database that can validate the effectiveness of the proposed method.•In-depth explanation of the selected multi-modal biomarkers.