The flexibility of generative artificial intelligence (AI), and specifically large language models (LLMs), holds promise to transform healthcare by offering new tools to improve patient care and enhance clinician efficiency. However, this flexibility introduces new challenges, as LLM applications might inadvertently be used in ways that do not align with the intended purpose of the manufacturer, raising patient safety concerns and liability issues. We have identified a regulatory gap whereby LLM applications that could be repurposed, malfunction or be inadvertently misused for medical purposes fall outside the scope of the Medical Device Regulation (MDR) because they lack a medical intended purpose, and outside meaningful AI Act safety obligations because they do not qualify as high-risk systems, yet still present patient safety risks without adequate regulatory guidance. In this manuscript, we aim to address this regulatory gap for LLM applications in healthcare by identifying these ambiguous grey-area applications and providing four recommendations—validation, mitigation, compliance and alignment—focused on prevention and risk mitigation rather than reclassification as medical devices. These recommendations are intended to advance the responsible implementation of LLM applications in healthcare while ensuring patient safety.
Ambient artificial intelligence scribes (ambient scribes) are increasingly used across healthcare systems to support clinical documentation. Implementation strategies and associated research have been mainly concerned with the ability to accurately and efficiently produce clinical notes. Some concerns have been raised, such as potential hallucinations or omissions that require editorial work of clinicians, but this research has not moved beyond the technical performance of ambient scribes. We sought to synthesise emerging evidence and perspectives on ambient scribes from a sociotechnical perspective, focusing on how these systems interact with existing clinical work practices and organisational processes.We searched Embase, PubMed and Scopus to identify relevant literature and included 27 articles. We used a sociotechnical lens to synthesise research and map core themes against implementation domains from the Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability (NASSS) framework.We identified few empirical studies examining the risks of ambient scribes, and even fewer investigating their implementation and adoption in practice. Existing research has focused predominantly on their potential to reduce administrative burden and improve efficiency. However, ambient scribes do more than automate note-taking. They actively shape clinical workflows and patient-clinician interactions and change the nature and function of clinical documentation. This suggests that there has been a de facto acceptance by policymakers and practitioners of the value proposition without critically examining the underlying assumptions and broader consequences. Ambient scribes must be understood as complex interventions rather than just productivity tools. There is therefore an urgent need for contextual validation and longitudinal evaluation.
Digital health interventions (DHIs) are rapidly expanding across low- and middle-income countries (LMICs), driven by their potential to address persistent health system constraints, workforce shortages and health inequities. However, the evaluation of these interventions has not kept pace with their adoption. Traditional evidence-generation approaches, particularly those centred on rigid, resource-intensive study designs, are often poorly aligned with the iterative nature of digital technologies and the realities of resource-constrained settings. This misalignment can exacerbate existing system pressures, slow validation and contribute to an enduring evidence-implementation gap that limits sustainable adoption and scale. This narrative review synthesises peer-reviewed biomedical literature, grey literature, institutional reports and applied case studies published between 2015 and 2025 to examine how evidence for DHIs is generated in LMICs. The analysis is organised across three interrelated domains: contextual and stakeholder dynamics shaping evaluations, methodological innovations suited to LMIC realities and top-down policy and regulatory mechanisms influencing adoption and scale. Drawing on this synthesis, the review argues for a staged, life-cycle-based approach to evidence generation, in which evaluation methods and evidence strategies evolve as the intervention matures and the implementation context evolves. This approach emphasises pragmatic, context-sensitive strategies that balance scientific rigour with feasibility. This review aims to provide decision-oriented guidance for innovators, implementers, policy-makers and regulators seeking to generate appropriate, actionable and policy-relevant evidence to support the adoption, integration and scale of DHIs in LMIC health systems.
