Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language understanding tasks. While these models often produce linguistically coherent output, they often suffer from hallucinations, generating descriptions that are factually inconsistent with the visual content, potentially leading to adverse consequences. Therefore, the assessment of hallucinations in MLLM has become increasingly crucial in the model development process. Contemporary methodologies predominantly depend on external LLM evaluators, which are themselves susceptible to hallucinations and may present challenges in terms of domain adaptation. In this study, we propose the hypothesis that hallucination manifests as measurable irregularities within the internal layer dynamics of MLLMs, not merely due to distributional shifts but also in the context of layer-wise analysis of specific assumptions. By incorporating such modifications, HalluShift++ broadens the efficacy of hallucination detection from text-based large language models (LLMs) to encompass multimodal scenarios. Our codebase is available at https://github.com/C0mRD/HalluShift_Plus.
Traditional supervised 3D medical image segmentation models need voxel-level annotations, which require huge human effort, time, and cost. Semi-supervised learning (SSL) addresses this limitation of supervised learning by facilitating learning with a limited annotated and larger amount of unannotated training samples. However, state-of-the-art SSL models still struggle to fully exploit the potential of learning from unannotated samples. To facilitate effective learning from unannotated data, we introduce LLM-SegNet, which exploits a large language model (LLM) to integrate task-specific knowledge into our co-training framework. This knowledge aids the model in comprehensively understanding the features of the region of interest (ROI), ultimately leading to more efficient segmentation. Additionally, to further reduce erroneous segmentation, we propose a Unified Segmentation loss function. This loss function reduces erroneous segmentation by not only prioritizing regions where the model is confident in predicting between foreground or background pixels but also effectively addressing areas where the model lacks high confidence in predictions. Experiments on publicly available Left Atrium, Pancreas-CT, and Brats-19 datasets demonstrate the superior performance of LLM-SegNet compared to the state-of-the-art. Furthermore, we conducted several ablation studies to demonstrate the effectiveness of various modules and loss functions leveraged by LLM-SegNet.
The detection of change in mild cognitive impairment (MCI) is vital between one visit and another visit for the early-stage detection of Alzheimer's disease (AD). It is associated with changes in the structural and functional aspects. Diagnosis of dementia progression is a challenging but essential step for the planning of early treatment. This paper proposes the cellular automata (CA) framework to identify the changes when the disease progresses from one visit to another. An updated rule using the conduction function of anisotropic diffusion has been proposed to assess the earliest signs of dementia manifestation and progression. The progression of dementia disease in multi-visit subjects for healthy and unhealthy patients has been analyzed by examining the entropy from evolved 2D CA images. The subjects below the reference line in healthy cases lead to the prognosis of the disease, whereas subjects above the reference line in unhealthy subjects are in improved condition. The proposed CA architecture shows the transitions and progressive patterns, thereby offering a method for tracking the evolution of dementia over time.
Ensuring optimal crop protection is essential for sustainable agriculture. Effective pest management and detection are crucial for safeguarding paddy yields and minimizing losses. Traditional pest detection methods rely on visual inspection by human experts, which is time-consuming, labor-intensive, and prone to errors. Recent advancements in computer vision and deep learning have enabled automated paddy pest detection systems, offering more efficient and accurate alternatives. However, applying these technologies in real-world agricultural settings, especially in resource-constrained regions, remains challenging. To address this gap, we curated the TPD-20(Tamil Nadu Paddy Pest Dataset-20) dataset, providing researchers with a resource for developing accurate and reliable pest detection models tailored to local agricultural contexts. The TPD-20 dataset comprises 20,000 high-resolution images of paddy plants annotated with bounding boxes indicating the presence of various pests. This dataset represents a diverse range of pest species and paddy plant conditions, capturing the variability of realworld scenarios. We propose a modified Squeeze-and-Excitation (SE) based YOLO model, designed to enhance detection accuracy and robustness by integrating SE modules into the YOLO architecture. These modules selectively emphasize informative features while suppressing less relevant ones, leading to improved feature representation and model performance. Our model achieved a mean average precision (mAP) of 81.2 % and average classification accuracy of 93.8 %, demonstrating the effectiveness of our approach. This study highlights the potential of leveraging computer vision and deep learning for automated pest detection, significantly advancing crop protection efforts. We emphasize the novelty of our approach, which integrates state-of-the-art techniques tailored for paddy pest management.
With the rapid development of modern generative models, the need for an automated synthetic image detection process has never been greater. Recent works in the field of synthetic image detection focus on improving out-of-distribution (OoD) classification performance and robustness to common image pre-processing techniques. However, in this work, we intend to explore the nature of an intricate counter-forensic attack, i.e., the reconstruction of real images with Diffusion Model autoencoders, which could be used to adversely affect the performance of modern synthetic image detection algorithms. We present a variety of experiments to study the nature of this counter-forensic attack and use the inferences from these experiments to develop multiple algorithms to detect such reconstructed images while attempting to detect real and purely synthetic images accurately. To do so, we make use of trained classifiers that can detect real images, autoencoder-reconstructed images, and purely synthetic images. Furthermore, we combine these techniques to build a novel ensemble algorithm that competes with state-of-the-art (SoTA) algorithms in the 'Real vs. Fake' image detection task, while detecting autoencoder reconstructed images accurately, attaining an accuracy of 99.2% in the multiclass setting.
