A bstract Dynamic distribution shifts caused by evolving diseases and demographic changes require domain-incremental adaptation of clinical deep learning models. However, this process of adaptation is often accompanied by catastrophic forgetting , and even the most sophisticated methods are not good enough for clinical applications. This paper studies incremental learning from the perspective of mode connections , that is, the low-loss paths connecting the minimisers of neural architectures (modes or trained weights) in the parameter space. The paper argues for learning the low-loss paths originating from an existing mode and exploring the learned paths to find an acceptable mode for the new domain. The learned paths, and hence the new domain mode, are a function of the existing mode. As a result, unlike traditional incremental learning, the proposed approach is able to exploit information from a deployed model without changing its weights. Pre-COVID and COVID-19 data collected in Oxford University hospitals are used as a case study to demonstrate the need for domain-incremental learning and the advantages of the proposed approach.

This article proposes a multiple self-sensing gripper (MssGripper) driven by the shape memory alloy (SMA) and empowered by machine learning algorithms. The MssGripper can identify objects without external sensors. A single SMA wire can drive the gripper for self-sensing accurately. This article confirms the resistance of the SMA can reflect the phase transition and can be used for displacement and force prediction, as well as for object stiffness prediction. Through machine learning, a backpropagation neural network and long-short-term-memory (LSTM) are used to establish multiple self-sensing models for prediction. The robustness experimental results show that the self-sensing models based on LSTM have higher prediction accuracy. The average root-mean-square errors of displacement prediction and force prediction are 0.063 mm and 0.236 N, respectively, and the stiffness prediction error is less than 9.4%. Moreover, the accuracy of the classifier in stiffness identification is 97.2%. The MssGripper can accurately predict the displacement, force, and stiffness and identify objects such as springs, rubber bars and steel bars. The establishment of the models expands the novel idea of gripper sensing, which is beneficial to promoting miniaturization and compactness.

Tetanus is a life-threatening infectious disease, which is still common in low- and middle-income countries, including in Vietnam. This disease is characterized by muscle spasm and in severe cases is complicated by autonomic dysfunction. Ideally continuous vital sign monitoring using bedside monitors allows the prompt detection of the onset of autonomic nervous system dysfunction or avoiding rapid deterioration. Detection can be improved using heart rate variability analysis from ECG signals. Recently, characteristic ECG and heart rate variability features have been shown to be of value in classifying tetanus severity. However, conventional manual analysis of ECG is time-consuming. The traditional convolutional neural network (CNN) has limitations in extracting the global context information, due to its fixed-sized kernel filters. In this work, we propose a novel hybrid CNN-Transformer model to automatically classify tetanus severity using tetanus monitoring from low-cost wearable sensors. This model can capture the local features from the CNN and the global features from the Transformer. The time series imaging - spectrogram - is transformed from one-dimensional ECG signal and input to the proposed model. The CNN-Transformer model outperforms state-of-the-art methods in tetanus classification, achieves results with a F1 score of $\mathbf {0.82\pm 0.03}$ , precision of $\mathbf {0.94\pm 0.03}$ , recall of $\mathbf {0.73\pm 0.07}$ , specificity of $\mathbf {0.97\pm 0.02}$ , accuracy of $\mathbf {0.88\pm 0.01}$ and AUC of $\mathbf {0.85\pm 0.03}$ . In addition, we found that Random Forest with enough manually selected features can be comparable with the proposed CNN-Transformer model.

Deep neural networks have been integrated into the whole clinical decision procedure which can improve the efficiency of diagnosis and alleviate the heavy workload of physicians. Since most neural networks are supervised, their performance heavily depends on the volume and quality of available labels. However, few such labels exist for rare diseases (e.g., new pandemics). Here we report a medical multimodal large language model (Med-MLLM) for radiograph representation learning, which can learn broad medical knowledge (e.g., image understanding, text semantics, and clinical phenotypes) from unlabelled data. As a result, when encountering a rare disease, our Med-MLLM can be rapidly deployed and easily adapted to them with limited labels. Furthermore, our model supports medical data across visual modality (e.g., chest X-ray and CT) and textual modality (e.g., medical report and free-text clinical note); therefore, it can be used for clinical tasks that involve both visual and textual data. We demonstrate the effectiveness of our Med-MLLM by showing how it would perform using the COVID-19 pandemic "in replay". In the retrospective setting, we test the model on the early COVID-19 datasets; and in the prospective setting, we test the model on the new variant COVID-19-Omicron. The experiments are conducted on 1) three kinds of input data; 2) three kinds of downstream tasks, including disease reporting, diagnosis, and prognosis; 3) five COVID-19 datasets; and 4) three different languages, including English, Chinese, and Spanish. All experiments show that our model can make accurate and robust COVID-19 decision-support with little labelled data.

Abstract Deep neural networks have been integrated into the whole clinical decision procedure which can improve the efficiency of diagnosis and alleviate the heavy workload of physicians. Typical applications include 1) medical report generation, 2) disease classification, and 3) survival prediction. Since most neural networks are supervised, their quality heavily depends on the volume and quality of available labels. However, for novel diseases, e.g., new pandemics or new variants, there are few existing labels. In addition, the acquisition of new pandemic cases to collect sufficient labels for training is time-consuming and is typically unavailable at the early stage. To prepare neural networks for the next pandemic, in this paper, we propose a large language model - Unsupervised Learning from Unlabelled Medical Images and Text (ULUMIT) framework, which can learn broad medical knowledge (e.g., image understanding, text semantics, and clinical phenotypes) from unlabelled data. As a result, when encountering new pandemics, our framework can be rapidly deployed and easily adapted to them with extremely limited labels. Furthermore, ULUMIT supports medical data across visual modality (e.g., chest X-ray and CT) and textual modality (e.g., medical report and free-text clinical note); therefore, it can be used for any clinical task that involves both visual and textual medical data. We demonstrate the effectiveness of our ULUMIT by showing how it would perform using the COVID-19 pandemic ``in replay''. In particular, in the retrospective setting, we test the model on the early COVID-19 datasets; and in the prospective setting, we test the model on the new variant COVID-19-Omicron. The experiments are conducted on 1) three kinds of input medical data, image-only, text-only, and image-text; 2) three kinds of downstream tasks, medical reporting, diagnosis, and prognosis; 3) five public COVID-19 datasets; and 4) three different languages, i.e., English, Chinese, and Spanish. All experiments consistently show that our framework can make accurate and robust COVID-19 decision-support tasks with little labelled data (such as considering information from only one patient), providing an impact on medical data analysis during the early stage of the next pandemic. Besides COVID-19, our framework can be applied to identify 14 common thorax diseases and tuberculosis across five additional public datasets, demonstrating its robustness in generalization and transferability. In brief, our framework achieves state-of-the-art performances on ten datasets.

Antimicrobial resistance (AMR) poses a threat to global public health. To mitigate the impacts of AMR, it is important to identify the molecular mechanisms of AMR and thereby determine optimal therapy as early as possible. Conventional machine learning-based drug-resistance analyses assume genetic variations to be homogeneous, thus not distinguishing between coding and intergenic sequences. In this study, we represent genetic data from Mycobacterium tuberculosis as a graph, and then adopt a deep graph learning method-heterogeneous graph attention network ('HGAT-AMR')-to predict anti-tuberculosis (TB) drug resistance. The HGAT-AMR model is able to accommodate incomplete phenotypic profiles, as well as provide 'attention scores' of genes and single nucleotide polymorphisms (SNPs) both at a population level and for individual samples. These scores encode the inputs, which the model is 'paying attention to' in making its drug resistance predictions. The results show that the proposed model generated the best area under the receiver operating characteristic (AUROC) for isoniazid and rifampicin (98.53 and 99.10%), the best sensitivity for three first-line drugs (94.91% for isoniazid, 96.60% for ethambutol and 90.63% for pyrazinamide), and maintained performance when the data were associated with incomplete phenotypes (i.e. for those isolates for which phenotypic data for some drugs were missing). We also demonstrate that the model successfully identifies genes and SNPs associated with drug resistance, mitigating the impact of resistance profile while considering particular drug resistance, which is consistent with domain knowledge.