Large Language Models (LLMs) are increasingly being used for medical advice by patients and healthcare providers. These models capture knowledge from their training data, which consists of vast medical corpora. However, they lack the ability to use this knowledge to causally reason about the underlying physiological processes. Moreover, they are unable to deal with uncertainty, generating responses that are confidently presented yet factually incorrect. Acting on such factually incorrect medical advice can be dangerous. Mitigating these risks requires rethinking the role of LLMs in medicine. In this work, we present an evaluation scheme for LLMs in three roles: direct clinical decision support, exact medical knowledge base, and approximate medical knowledge base. We evaluate six LLMs on two clinical studies, in obstetrics and pediatric critical care, respectively. Our results indicate that LLMs are much better suited to the approximate knowledge base role. Based on these observations, we request caution when directly employing LLMs in safety-critical domains such as medicine.
Image explanation has been one of the key research interests in the Deep Learning field. Throughout the years, several approaches have been adopted to explain an input image fed by the user. From detecting an object in a given image to explaining it in human understandable sentence, to having a conversation describing the image, this problem has seen an immense change throughout the years, However, the existing works have been often found to (a) hallucinate objects that do not exist in the image and/or (b) lack identifying the complete set of objects present in the image. In this paper, we propose a novel approach to mitigate these drawbacks of inconsistency and incompleteness of the objects recognized during the image explanation. To enable this, we propose an interpretable framework that can be plugged atop diverse image explaining frameworks including Image Captioning, Visual Question Answering (VQA) and Prompt-based AI using LLMs, thereby enhancing their explanation capabilities by rectifying the incorrect or missing objects. We further measure the efficacy of the rectified explanations generated through our proposed approaches leveraging object based precision metrics, and showcase the improvements in the inconsistency and completeness of image explanations. Quantitatively, the proposed framework is able to improve the explanations over the baseline architectures of Image Captioning (improving the completeness by 81.81% and inconsistency by 37.10%), Visual Question Answering(average of 9.6% and 37.10% in completeness and inconsistency respectively) and Prompt-based AI model (0.01% and 5.2% for completeness and inconsistency respectively) surpassing the current state-of-the-art by a substantial margin.
