CLIP and BiomedCLIP are examples of vision-language foundation models and offer strong cross-modal embeddings; however, they are not optimized for fine-grained medical retrieval tasks, such as retrieving clinically relevant radiology reports using chest X-ray (CXR) image queries. To address this shortcoming, we propose a multi-task learning framework to fine-tune BiomedCLIP and evaluate improvements to CXR image-text retrieval. Using BiomedCLIP as the backbone, we incorporate a lightweight MLP projector head trained with a multi-task composite loss function that includes: (1) a binary cross-entropy loss to distinguish normal from abnormal CXR studies, (2) a supervised contrastive loss to reinforce intra-class consistency, and (3) a CLIP loss to maintain cross-modal alignment. Experimental results demonstrate that the fine-tuned model achieves more balanced and clinically meaningful performance across both image-to-text and text-to-image retrieval tasks compared to the pretrained BiomedCLIP and general-purpose CLIP models. Furthermore, t-SNE visualizations reveal clearer semantic clustering of normal and abnormal cases, demonstrating the model's enhanced diagnostic sensitivity. These findings highlight the value of domain-adaptive, multi-task learning for advancing cross-modal retrieval in biomedical applications.

Objective Despite recent advancements in the development of foundation models and multimodal (MM) architectures in dermatology, their translation to clinical practice remains limited by the scarcity of large-scale multimodal (MM) datasets, as most publicly available resources are small, unimodal, and lack expressive clinical text. This paper investigates strategies to synthesize and exploit clinical notes paired with dermatological images to effectively train a MM architecture, focusing on solutions to limit the inherently hallucinated contents introduced by Large Language Models (LLMs) and to identify conditions under which synthetic clinical notes can be reliably leveraged. Methods The paper proposes a MM architecture trained on real dermatological images paired with LLM-synthesized clinical notes. We systematically evaluate different note generation strategies, including metadata-guided prompting, alignment of image representations with specific keywords, sentence-level filtering of clinical notes, network architectural designs. Experiments involve 16,000 image-note couples collected from six public datasets for model training and over 37,000 images from fifteen public datasets as external data for generalization assessment. Performance is assessed on cross-modal retrieval and zero-shot learning tasks to quantify robustness and generalization. Results Results show that metadata inclusion into the prompts reduces the hallucinations within LLM outputs, providing more reliable notes. The resulting MM model trained with these notes show superior performance on multiple downstream tasks. Conclusion Synthesized clinical notes can be paired with real dermatology images under specific conditions, providing a valuable resource to develop foundation models that can help reduce the dermatologists’ workload.

[This corrects the article DOI: 10.3389/frai.2026.1874287.].

PURPOSEThis study sought to evaluate the accuracy of artificial intelligence (AI)-based interpretation of digital surface images of skin lesions to diagnose Kaposi sarcoma (KS) among dark-skinned patients in Uganda.METHODSPatients were evaluated at skin biopsy services in Uganda because of clinical suspicion of KS. In a cross-sectional design, 482 consecutive participants were enrolled. Lesions were photographed using a digital camera, and punch biopsies were obtained. Histopathologic interpretation was considered the gold standard. Using training (approximately 70% of images) and validation (approximately 10% of images) sets, we developed a prediction model using a rule-based combination of You Only Look Once version 5 and 8 object detection classifiers. We determined sensitivity, specificity, and positive and negative predictive values of the AI-based prediction model in a test set (approximately 20% of images) and compared these with the accuracy of a dermatologist's visual interpretation of images.RESULTSFour hundred seventy-two participants (1,385 images) were evaluable. Of these, 36% was female; the median age was 34 years; and 94% had HIV, 332 had KS, and 140 had no KS by histopathology. In the test set, the AI-derived prediction model achieved 89% sensitivity (85%-94%) and 51% specificity (40%-61%) for diagnosing KS; the positive predictive value was 81% (75%-86%), and the negative predictive value was 67% (55%-78%). The area under the receiver operating characteristic curve was 0.72. A dermatologist evaluating the same images, with emphasis on sensitivity, achieved a sensitivity of 93% (89%-96%) and a specificity of 19% (11%-28%).CONCLUSIONAmong dark-skinned patients in Uganda with lesions suspicious for KS, evaluation of digital surface images by an AI-based prediction model produced moderate accuracy for diagnosing KS. While currently inadequate for clinical use, this inaugural assessment is sufficiently promising to justify future evaluation of larger data sets and evolving technologies.

Multimodal learning using images and associated clinical text offers richer semantic supervision for medical AI. However, models trained with synthetic reports risk hallucination, and conventional multimodal tuberculosis (TB) systems are impractical because they require text at inference. In realworld screening workflows, particularly in low-resource settings or during triage, radiology reports are often unavailable or delayed. Computer-aided detection systems for chest X-rays (CXRs) are considered a potential solution. In this context, this study proposes a method that uses clinically grounded text as privileged information during training to improve a binary CXR classifier, while enabling image-only TB prediction at deployment. Frontal CXRs from Shenzhen (internal train/validation/test), Montgomery County, TBX11K, and NIAID TB Portals (external tests) were lung-cropped using a YOLOv8s detector and resized to 224 × 224. For Shenzhen, de-identified metadata and brief clinical notes were converted into structured reports encoding population type, TB status, laterality, lobar involvement, and adjunct findings; a parallel model used raw notes. A VGG-11 vision encoder and frozen CXR BERT text encoder were co-trained in a shared 256-dimensional space using image classification, cosine similarity, and supervised contrastive alignment losses. At inference, the text branch was removed, yielding an image only classifier regularized through multimodal supervision. Multimodal training with report supervision consistently improved image-only predictions, with structured report outperforming raw notes. Across internal and external cohorts, performance gains were reflected in higher balanced accuracy, Matthews correlation coefficient, and area under the curve. UMAP embeddings showed clearer class separation, and Grad CAM maps demonstrated improved localization of TB-relevant lesions.

Multimodal (MM) algorithms and foundation models have shown strong potential for automated skin lesion analysis in dermatology, however, their translation to clinical practice is hindered by the limited availability of large, high-quality image-text datasets. Most public dermatology datasets are small, unimodal, and paired with heterogeneous labels and metadata, restricting effective model development. Large Language Models (LLMs) provide an opportunity to synthesize clinical notes from existing unimodal datasets, but their application is limited by hallucinations that can hinder MM training. This paper investigates and evaluates strategies to generate reliable clinical notes using existing LLMs and metadata paired with real images, aiming to mitigate hallucinations and improve performance on downstream tasks. We train a MM architecture on real dermatology images paired with synthesized notes and evaluate it across 15 datasets (6 internal, 9 external) on cross-modal retrieval and zero-shot learning tasks. Results demonstrate improved robustness and generalization compared to state-of-the-art medical foundation models, under specific synthesis conditions.

We advance the state of the art in AI-driven oral cancer screening approaches through a new automatic method for analysis of non-image clinical data organized in a tabular format. In this work, we distinguish oral cancer from precancer (which is its direct precursor) and improve the classification performance by mitigating the issue of clinical data imbalance. Specifically, our work includes two main components: classification using the tabular data, and tabular data synthesis for data balancing and improving classification. We use deep learning techniques in both components and conduct extensive experiments by creating multiple data sets and evaluating several models. Our work indicates that the tabular clinical information can play an important role for identifying oral cancer from precancer using deep learning techniques. The Youden index of our approach is ~0.74, the balanced accuracy is ~0.83, and sensitivity is ~0.90. Our work demonstrates that using synthetic data to balance the tabular data sets is a promising approach for improving the classification performance, and the improvement is statistically significant (p-value < 0.05) with respect to the evaluation metrics: sensitivity, Youden index, F1 score, balanced accuracy, and Matthews correlation coefficient (MCC). Broadly, the encouraging results and insights obtained through our work can help motivate new research utilizing clinical tabular data toward improving clinical AI prediction tasks.

IntroductionLateral chest X-rays (CXRs) are very important for detecting tuberculosis (TB) in infants and children, particularly for assessing TB-related lymphadenopathy and intrathoracic structures that are obscured in frontal projections. Although deep learning (DL)–based artificial intelligence (AI) has advanced CXR analysis, lateral projection imaging remains largely unexplored. Lung field detection is a critical first step in such pipelines, enabling DL models to focus on the relevant anatomy and improving downstream tasks such as disease detection, classification, and clinical decision support.MethodsOur DL-based model, called Pediatric Lateral lUng deTection with yOlo (PLUTO), enables lateral lung field detection not only in pediatric CXRs, but also demonstrates cross-domain generalizability to adult lateral CXRs. Related to prior work in this emerging area, PLUTO advances the field through a systematic age-stratified evaluation framework and an explicit assessment of cross-domain transfer to adult images. PLUTO uses a YOLO11s detector backbone selected after evaluating multiple state-of-the-art YOLO11 variants through five-fold cross-validation on age-stratified pediatric CXRs. The test cohort includes internal pediatric hold-out data and external pediatric and adult CXRs.ResultsPLUTO achieved strong performance, with mAP@[0.5:0.95] scores of 0.8816 ± 0.0061 (internal pediatric) and 0.8898 ± 0.0084 (external pediatric), and demonstrated preliminary cross-domain generalizability to adult images. However, confirmation at a larger scale remains an important direction for future work. PLUTO also improved zero-shot lateral lung field segmentation performance.DiscussionThe PLUTO model provides a valuable resource for anatomically grounded AI in lateral pediatric TB imaging and will enhance research in pulmonary TB and related diseases.

Oral cancer has one of the lowest five-year survival rates among major cancer types. Therefore, early detection is crucial for histopathological confirmation. State-of-the-art methods reported in the literature largely analyze images only for oral cancer prediction. The use of deep learning networks for related tabular medical data remains unexplored for oral cancer and understudied in general. As part of our multimodal AI/ML approach toward reliable prediction of candidate lesions to biopsy, we describe our work in deep learning approaches on a fielded clinical structured text data in spreadsheet format (tabular data) on a subset comprising 1791 patients drawn from a large ongoing oral cancer study to classify patients with a cancerous lesion from those with a precancerous lesion (i.e., direct precursor to cancer). We compare two tabular deep learning methods and one conventional algorithm for the predictive data analysis. The experimental results on a hold-out test set demonstrate a promising performance for all models (Youden index > 0.6 and AUC > 0.9). In addition, we examine and analyze the interpretability of models. All models indicate that lesion characteristics are crucial predictive features. The insights and results obtained from this work would be valuable to the research community in application of AI/ML to biomedicine.

Multimodal (MM) learning is emerging as a promising paradigm in biomedical artificial intelligence (AI) applications, integrating complementary modality, which highlight different aspects of patient health. The scarcity of large heterogeneous biomedical MM data has restrained the development of robust models for medical AI applications. In the dermatology domain, for instance, skin lesion datasets typically include only images linked to minimal metadata describing the condition, thereby limiting the benefits of MM data integration for reliable and generalizable predictions. Recent advances in Large Language Models (LLMs) enable the synthesis of textual description of image findings, potentially allowing the combination of image and text representations. However, LLMs are not specifically trained for use in the medical domain, and their naive inclusion has raised concerns about the risk of hallucinations in clinically relevant contexts. This work investigates strategies for generating synthetic textual clinical notes, in terms of prompt design and medical metadata inclusion, and evaluates their impact on MM architectures toward enhancing performance in classification and cross-modal retrieval tasks. Experiments across several heterogeneous dermatology datasets demonstrate that synthetic clinical notes not only enhance classification performance, particularly under domain shift, but also unlock cross-modal retrieval capabilities, a downstream task that is not explicitly optimized during training.

Hallucination in deep learning (DL) classification, where DL models yield confidently erroneous predictions remains a pressing concern. This study investigates whether binary classifiers are truly learning disease-specific features when distinguishing overlapping radiological presentations among pneumonia subtypes on chest X-ray (CXR) images. Specifically, we evaluate if uncertainty measure is a valuable tool in classifying signs of different pathogen-specific subtypes of pneumonia. We evaluated two binary classifiers to classify bacterial pneumonia and viral pneumonia, respectively, from normal CXRs. A third classifier explored the ability to distinguish bacterial from viral pneumonia presentation to highlight our concern regarding the observed hallucinations in the former cases. Our comprehensive analysis computes the Matthews Correlation Coefficient and prediction entropy metrics on a pediatric CXR dataset and reveals that the normal/bacterial and normal/viral classifiers consistently and confidently misclassify the unseen pneumonia subtype to their respective disease class. These findings expose a critical limitation concerning the tendency of binary classifiers to hallucinate by relying on general pneumonia indicators rather than pathogenspecific patterns, thereby challenging their utility in clinical workflows.

Chest radiographs are a vital tool for identifying pathological changes within the thoracic cavity. Artificial intelligence (AI) and machine learning (ML) driven screening or diagnostic applications require accurate detection of anatomical structures within the Chest X-ray (CXR) image. The You Only Look Once (YOLO) object detection models have recently gained prominence for their efficacy in detecting anatomical structures in medical images. However, state-of-the-art results using it are typically for single anatomical organ detection. Advanced image analysis would benefit from simultaneous detection more than one anatomical organ. In this work we propose a multi-organ detection technique through two recent YOLO versions and their sub-variants. We evaluate their effectiveness in detecting lung and heart regions in CXRs simultaneously. We used the JSRT CXR dataset for internal training, validation, and testing. Further, the generalizability of the models is evaluated using two external test sets, viz., the Montgomery CXR dataset and a subset of the RSNA CXR dataset against available annotations therein. Our evaluation demonstrates that YOLOv9 models notably outperform YOLOv8 variants. We demonstrated further improvements in detection performance through ensemble approaches.

Importance:Advanced-stage disease at the time of diagnosis, with resultant high mortality, is among the most urgent issues for HIV-related Kaposi sarcoma (KS) in sub-Saharan Africa. Lack of access to skilled clinical personnel and histopathologic technology in the region contribute to diagnostic delays and advanced stage at diagnosis. Accordingly, new paradigms for KS diagnosis are needed. Objective:To evaluate the accuracy of artificial intelligence (AI)-based interpretation of digital surface images of skin lesions to diagnose KS among dark-skinned patients in Uganda. Design:Cross-sectional study of consecutive participants referred to skin biopsy services in Uganda because of clinical suspicion of KS. Lesions were photographed using a digital camera, and punch biopsies were obtained. Histopathologic interpretation was considered the gold standard. Using training (∼70% of images) and validation (∼10% of images) sets, we developed a prediction model using a rule-based combination of YOLO (You Only Look Once) version 5 and 8 object detection classifiers. Setting:Free-of-charge skin biopsy services. Participants:Consecutive sample of 482 individuals were evaluated due to clinical suspicion of KS. Main Outcomes:Sensitivity, specificity, positive and negative predictive value (with accompanying 95% confidence intervals) of the AI-based prediction model in a test set (∼20% of images). The accuracy of a dermatologist's visual interpretation of images was also described. Results:472 participants (1385 images) were evaluable. Of these, 36% were female, median age was 34 years, and 94% had HIV; 332 had KS, and 140 had no KS by histopathology. In the test set, the AI-derived prediction model achieved 89% sensitivity (85%-94%) and 51% specificity (40%-61%) for diagnosing KS; positive predictive value was 81% (75%-86%) and negative predictive value was 67% (55%-78%). A dermatologist evaluating the same images, with emphasis on sensitivity, achieved sensitivity of 93% (89%-96%) and specificity of 19% (11%-28%). Conclusions and Relevance:Among dark-skinned patients in Uganda with skin lesions suspicious for KS, evaluation of digital surface images by an AI-based prediction model produced moderate accuracy for diagnosing KS. While currently inadequate for clinical use, this inaugural assessment is sufficiently promising to justify evaluation of larger datasets and evolving technologies to determine if accuracy can be improved. Key Points:Question: Can an artificial intelligence (AI)-based prediction model be developed from digital images to accurately distinguish Kaposi sarcoma (KS) from non-KS in dark-skinned patients?Findings: Evaluation of digital images of skin lesions from patients in Uganda by an AI-based prediction model produced moderate accuracy for diagnosing KS.Meaning: In sub-Saharan Africa, where incidence and mortality from KS is high and delayed diagnosis is common due to limited specialized personnel and technical supplies, AI-based prediction models built on digital images taken of suspicious lesions may someday hasten KS diagnoses.

Biomedical data are inherently multimodal, capturing complementary aspects of a patient condition. Deep learning (DL) algorithms that integrate multiple biomedical modalities can significantly improve clinical decision-making, especially in domains where collecting data is not simple and data are highly heterogeneous. However, developing effective and reliable multimodal DL methods remains challenging, requiring large training datasets with paired samples from modalities of interest. An increasing number of de-identifed biomedical datasets are publicly accessible, though they still tend to be unimodal. For example, several publicly available skin lesion datasets aid automated dermatology clinical decision-making. Still, they lack annotated reports paired with the images, thereby limiting the advance and use of multimodal DL algorithms. This work presents a strategy exploiting real and synthesized data in a multimodal architecture that encodes fine-grained text representations within image embeddings to create a robust representation of skin lesion data. Large language models (LLMs) are used to synthesize textual descriptions from image metadata that are subsequently paired with the original skin lesion images and used for model development. The architecture is evaluated on the classification of skin lesion images, considering nine internal and external data sources. The proposed multimodal representation outperforms the unimodal one on the classification of skin lesion images, achieving superior performance in every tested dataset.

Artificial intelligence (AI) is making significant advances toward becoming a well-established and promise-bearing technology in various medical domains such as screening, diagnostics, and biopharma research. However, its state remains relatively nascent in surgery and surgical therapeutics. This presents an opportunity for leveraging ongoing rapid advances in AI technology and the increasing availability of large, diverse datasets to pave the way for their use in these domains. Expanding the use of AI to include various processes in surgeryrelated workflows could provide several benefits, such as greater assurance for reduced errors, better assistance to surgeons, and overall improved patient outcomes. To encourage further research in surgical AI, this article summarizes the state-of-theart in AI assurance in various aspects of a patient's timeline when undergoing head and neck surgeries, including diagnostics, preoperative considerations, intraoperative guidance, and postoperative and outcome predictions. The work aims to highlight gaps in the state-of-the-art and identify opportunities for the computer-based medical systems community to encourage future research and development on the subject.

IntroductionDeep learning (DL) has significantly advanced medical image classification. However, it often relies on transfer learning (TL) from models pretrained on large, generic non-medical image datasets like ImageNet. Conversely, medical images possess unique visual characteristics that such general models may not adequately capture.MethodsThis study examines the effectiveness of modality-specific pretext learning strengthened by image denoising and deblurring in enhancing the classification of pediatric chest X-ray (CXR) images into those exhibiting no findings, i.e., normal lungs, or with cardiopulmonary disease manifestations. Specifically, we use a VGG-16-Sharp-U-Net architecture and leverage its encoder in conjunction with a classification head to distinguish normal from abnormal pediatric CXR findings. We benchmark this performance against the traditional TL approach, viz., the VGG-16 model pretrained only on ImageNet. Measures used for performance evaluation are balanced accuracy, sensitivity, specificity, F-score, Matthew’s Correlation Coefficient (MCC), Kappa statistic, and Youden’s index.ResultsOur findings reveal that models developed from CXR modality-specific pretext encoders substantially outperform the ImageNet-only pretrained model, viz., Baseline, and achieve significantly higher sensitivity (p < 0.05) with marked improvements in balanced accuracy, F-score, MCC, Kappa statistic, and Youden’s index. A novel attention-based fuzzy ensemble of the pretext-learned models further improves performance across these metrics (Balanced accuracy: 0.6376; Sensitivity: 0.4991; F-score: 0.5102; MCC: 0.2783; Kappa: 0.2782, and Youden’s index:0.2751), compared to Baseline (Balanced accuracy: 0.5654; Sensitivity: 0.1983; F-score: 0.2977; MCC: 0.1998; Kappa: 0.1599, and Youden’s index:0.1327).DiscussionThe superior results of CXR modality-specific pretext learning and their ensemble underscore its potential as a viable alternative to conventional ImageNet pretraining for medical image classification. Results from this study promote further exploration of medical modality-specific TL techniques in the development of DL models for various medical imaging applications.

Novel screening and diagnostic tests based on artificial intelligence (AI) image recognition algorithms are proliferating. Some initial reports claim outstanding accuracy followed by disappointing lack of confirmation, including our own early work on cervical screening. This is a presentation of lessons learned, organized as a conceptual step-by-step approach to bridge the gap between the creation of an AI algorithm and clinical efficacy. The first fundamental principle is specifying rigorously what the algorithm is designed to identify and what the test is intended to measure (eg, screening, diagnostic, or prognostic). Second, designing the AI algorithm to minimize the most clinically important errors. For example, many equivocal cervical images cannot yet be labeled because the borderline between cases and controls is blurred. To avoid a misclassified case-control dichotomy, we have isolated the equivocal cases and formally included an intermediate, indeterminate class (severity order of classes: case>indeterminate>control). The third principle is evaluating AI algorithms like any other test, using clinical epidemiologic criteria. Repeatability of the algorithm at the borderline, for indeterminate images, has proven extremely informative. Distinguishing between internal and external validation is also essential. Linking the AI algorithm results to clinical risk estimation is the fourth principle. Absolute risk (not relative) is the critical metric for translating a test result into clinical use. Finally, generating risk-based guidelines for clinical use that match local resources and priorities is the last principle in our approach. We are particularly interested in applications to lower-resource settings to address health disparities. We note that similar principles apply to other domains of AI-based image analysis for medical diagnostic testing.

Resting state functional Magnetic Resonance Imaging (rs-fMRI) is used to obtain spontaneous activation within the human brain in the absence of specific tasks. Analysis of the rs-fMRI data required spatially and functionally homogenous parcellation of the whole brain based on underlying temporal fluctuations. Commonly used parcellation schemes have a tradeoff between intra-cluster functional similarity and alignment with anatomical regions. In this article, we present a clustering scheme for rs-fMRI data that obtains spatially and functionally homogenous clusters. Results show that the proposed multistage approach can identify various brain networks. Moreover, the functional homogeneity of the clusters is shown to be better than those found with functional atlas and simple k-means clusters. The spatial homogeneity is shown to be better than Independent Component Analysis (ICA), and simple k-means clusters.

Deep learning (DL) has demonstrated its innate capacity to independently learn hierarchical features from complex and multi-dimensional data. A common understanding is that its performance scales up with the amount of training data. Another data attribute is the inherent variety. It follows, therefore, that semantic redundancy, which is the presence of similar or repetitive information, would tend to lower performance and limit generalizability to unseen data. In medical imaging data, semantic redundancy can occur due to the presence of multiple images that have highly similar presentations for the disease of interest. Further, the common use of augmentation methods to generate variety in DL training may be limiting performance when applied to semantically redundant data. We propose an entropy-based sample scoring approach to identify and remove semantically redundant training data. We demonstrate using the publicly available NIH chest X-ray dataset that the model trained on the resulting informative subset of training data significantly outperforms the model trained on the full training set, during both internal (recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs 0.2589, p<0.05). Our findings emphasize the importance of information-oriented training sample selection as opposed to the conventional practice of using all available training data.

Background:Large vision models (LVM) pretrained by large datasets have demonstrated their enormous capacity to understand visual patterns and capture semantic information from images. We proposed a novel method of knowledge domain adaptation with pretrained LVM for a low-cost artificial intelligence (AI) model to quantify the severity of SARS-CoV-2 pneumonia based on frontal chest X-ray (CXR) images. Methods:Our method used the pretrained LVMs as the primary feature extractor and self-supervised contrastive learning for domain adaptation. An encoder with a 2048-dimensional feature vector output was first trained by self-supervised learning for knowledge domain adaptation. Then a multi-layer perceptron (MLP) was trained for the final severity prediction. A dataset with 2599 CXR images was used for model training and evaluation. Results:The model based on the pretrained vision transformer (ViT) and self-supervised learning achieved the best performance in cross validation, with mean squared error (MSE) of 23.83 (95 % CI 22.67-25.00) and mean absolute error (MAE) of 3.64 (95 % CI 3.54-3.73). Its prediction correlation has the R 2 of 0.81 (95 % CI 0.79-0.82) and Spearman ρ of 0.80 (95 % CI 0.77-0.81), which are comparable to the current state-of-the-art (SOTA) methods trained by much larger CXR datasets. Conclusion:The proposed new method has achieved the SOTA performance to quantify the severity of SARS-CoV-2 pneumonia at a significantly lower cost. The method can be extended to other infectious disease detection or quantification to expedite the application of AI in medical research.