Multi-object tracking (MOT) faces challenges in handling occlusions, feature degradation, and non-rigid motion. Existing methods relying on appearance-based re-identification (Re-ID) often struggle under occlusion, leading to frequent identity switches, while traditional motion models fail in dynamic scenarios. To address these issues, we propose an improved MOT framework integrating Score-based Gallery Management (SGM) to retain reliable Re-ID embeddings and Optical Flow-based Motion Compensation (OMC) to refine motion predictions. Our method achieves state-of-the-art performance on MOT20 and SportsMOT and exhibits competitive results on MOT17 and DanceTrack, demonstrating improved identity retention and tracking robustness in complex environments.
As video data is increasingly consumed by machines rather than solely by humans, there is a growing demand for new compression methods that efficiently accommodate this shift. Since object information extracted from videos is crucial for machine consumption, analyzing the similarity between adjacent pictures based on tracked object information can help identify motion-based redundancy which can then be utilized for video compression. In this paper, we perform object tracking on input video and analyze the similarity between adjacent pictures based on the movement of tracked objects. After classifying the pictures based on their similarity, highly redundant pictures within each group are aggressively resampled in the temporal domain to improve compression efficiency while maintaining machine performance. We propose a novel picture grouping method to cluster similar adjacent pictures and describe the process of similarity assessment. We evaluated the compression efficiency of the proposed object tracking-based adaptive temporal resampling through performance evaluation experiments, achieving BD-mAP improvements in object detection of 1.29%, 0.47%, and 2.44% for Random Access (RA), Low-Delay (LD), and All Intra (AI) modes, respectively, and achieving BD-MOTA improvements in object tracking of 0.02%, 0.61%, and 6.86% for RA, LD, and AI modes, respectively.
Diffusion models have recently shown great potential in object detection and instance segmentation, yet their application to scene text detection, with its unique challenges such as instance variability and subjective human annotations, remains unexplored. In this paper, we propose DRR (Diffusion ReRanking), a method that adapts diffusion-based instance segmentation for scene text detection. Traditional instance segmentation often relies on classification scores for ranking, potentially overlooking the accuracy of bounding boxes and mask quality. DRR addresses this by incorporating two networks: a diffusion network, trained with a combination of projection loss and pairwise loss in the mask branch to produce more precise and tightly-bound segmentations, and a reranking network, which refines the results by evaluating bounding box accuracy and mask quality. Extensive experiments demonstrate the effectiveness of DRR, achieving a precision of 86.7%, recall of 81.7%, and an F-measure of 84.1% on CTW1500, highlighting DRR’s potential to advance scene text detection.
Edge computing is evolving faster and has greater potential to bring a lot of transformation into the agricultural sector. It helps the farming industry to increase crop yield and production with its scientific methods and technological developments. When the farmer knows about the right amount of nutrients, soil requirements, weather updates, and water necessities at the right time, there will be a gradual increase in the crop yield. Edge infrastructure would process data and make faster decisions than other technologies. Edge infrastructure allows faster processing and decision-making. Our research aims to present an efficient edge-computing architecture for sustainable farming in smart agriculture. The objective of the proposed work is to provide a competent and cost-effective computing model for remote farmers with bandwidth limitations. The proposed edge computing-based Agro-economic model can transform conventional agriculture practices by improving speed, cost, efficiency, and latency with better data processing and decision-making. We have also reduced data risks by reducing infrastructure and operational expenditure and costs. This model provides an optimal solution for sustainable farming by addressing the challenges of current technologies and enabling a smarter approach.
Malignant melanoma is a lethal form of skin cancer, where early and precise diagnosis is essential for improving survival. While convolutional neural networks (CNNs) have shown promise in dermoscopic image analysis, they often neglect patient metadata that clinicians routinely consider. Vision-language models (VLMs) provide a unified architecture for multimodal understanding; however, their utility in clinical domains remains limited due to domain mismatch and lack of adaptation. To address these limitations, we propose a retrieval-augmented framework that integrates pre-trained VLMs with a multimodal vector database of dermoscopic images and patient metadata. For each input, the system retrieves semantically similar cases and inserts them into structured prompts, which enable few-shot classification without fine-tuning. Our method outperforms image-based, text-based, and embedding-fusion baselines by yielding higher F1-scores and reducing both false positives and false negatives. Our findings support retrieval-augmented prompting as a reliable and scalable approach to clinical decision support in melanoma diagnosis.
The research introduces a new reversible data hiding (RDH) technique using image interpolation, which enhances visual quality and embedding capacity while ensuring robust security and minimal distortion. The encryption phase segmented secret messages into different lengths and encoded them into code pairs before embedding them within the cover image. This method outperforms similar interpolation RDH algorithms, offering low computing costs and improved security.
Most deep neural networks are vulnerable to adversarial attacks, making it crucial to develop effective defenses against them. Adversarial examples are created by applying slight perturbations either to the entire image or specific regions, depending on attack methods. By contrast, benign examples typically lead models to focus attention on contiguous regions of the image during classification. Therefore, adversarial examples can be detected effectively by leveraging attention values. First, we conduct simple experiment to investigate how the model’s attention responds to adversarial perturbations. Second, we derive features by calculating the changes in attention values across ViT layers for adversarial and benign examples. These attention-derived features are then fed into the detection model, ResNet-50, to learn the differences in the activation locations between both examples. Third, we demonstrate the effectiveness of our approach by thorough experiments on various attack methods, datasets, and target models. The experiments prove the effectiveness of our approach, achieving superior performance compared to baseline methods on the ImageNet dataset across ViT-B and ViT-L. Our findings underscore the potential of utilizing attention-derived features as powerful indicators for enhancing the resilience of ViT models against adversarial perturbations.
Sensor networks play a crucial role in smart cities, with applications such as remote environmental monitoring and target tracking. A major challenge in data collection is the corruption of measurement data by unwanted noise, necessitating the development of effective data denoising methods. In this paper, the sensor network is modeled as a graph consisting of nodes and edges, allowing the application of graph filtering techniques from graph signal processing to address the denoising problem. The proposed approach involves several key steps: First, the transfer matrix of the graph filter is expressed as a linear combination of Gegenbauer polynomials, which generalize Legendre and Chebyshev polynomials. Second, the filter coefficients are determined by minimizing the integral weighted least squares error between actual and ideal spectral responses. Third, design examples of frequency-selective graph filters are presented. Fourth, an implementation structure for the designed filter is derived using the recurrence relation of Gegenbauer polynomials. Finally, the effectiveness of the proposed method is demonstrated through its application to temperature data denoising in the sensor networks at USA and Taiwan.
This paper presents a task-adaptive open-set detection framework that preserves zero-shot performance while incorporating task-specific adaptations for enhanced visual understanding. Our method integrates a frozen zero-shot detector with a learnable, task-specific adaptor module, and employs a token-level conditional inference mechanism using prompt-based feature masking. This approach selectively combines features from the pre-trained zero-shot model and the adapted module within a single forward pass, allowing both general and task-specific representations to contribute effectively. Unlike conventional full fine-tuning that transforms an open-set detector into a closed-set detector, our design maintains the inherent open-set capabilities, thereby mitigating overfitting to task-specific biases. Experimental results on the IHP and VFP290K datasets demonstrate that our method outperforms existing techniques in fallen person detection, underscoring its robustness and practical applicability.
With the rapid rise of memecoins such as Dogecoin ($DOGE) in the cryptocurrency market, their price volatility has become highly dependent on discussions and trends across social media platforms. The memecoin market is heavily influenced by celebrity endorsements, community sentiment, and short-term events. Hence, monitoring and understanding this sentiment is crucial but challenged by data volume, noise (missing modalities), and domain-specific visual/textual language (memes, slang). We propose MMS-CLIP, a robust multimodal market sentiment analysis framework leveraging the Contrastive Language-Image Pre-Training (CLIP) model enhanced with a novel Mask Token mechanism to improve complementarity between modalities to handle real-world data incompleteness. MMS-CLIP outperforms existing baseline models on the real-world social media dataset MVSA-Single. To facilitate research into the specific challenges of meme-driven sentiment, we introduce SESAME-1k, a novel dataset comprising annotated data from Reddit meme coin posts, capturing the unique signal characteristics of this domain. The code is available at https://github.com/MO7YW4NG/MMS-CLIP.
This paper presents a proactive pedestrian risk assessment system for traffic environments that integrates trajectory prediction with situation classification. Unlike end-to-end approaches that function as black boxes when predicting pedestrians' crossing intentions, our system employs trajectory forecasting combined with ground region classification of predicted paths. The proposed methodology first predicts future pedestrian trajectories using an attention-based recurrent neural network, then classifies the predicted situation using accumulated segmentation maps to assess potential pedestrian risk. Experimental evaluations demonstrate that our system outperforms black box approaches across multiple evaluation metrics. We also present validation results using real-world surveillance footage captured in urban environments, demonstrating the system’s real-time capability and practical applicability for integration into smart city infrastructure.
Adversarial patch attacks on visible person detectors pose significant risks to recognition and safety. To counter this, we propose a defense method that integrates thermal imaging with RGB images to exploit differences in thermal characteristics between adversarial patches and humans. Our approach segments thermal images using selective search to generate mask sets and apply them to RGB images for attack detection and mitigation. To evaluate the proposed method, we created a physical attack dataset by conducting a hiding attack using printed adversarial patches against a person detector. This dataset enabled a quantitative assessment of our method. Experimental results demonstrate that our approach effectively defends against adversarial patches while maintaining performance on clean images, validating its robustness in practical scenarios.
Action recognition and localization in videos pose significant challenges, as they require identifying temporal boundaries and classifying actions within long video sequences. This paper introduces an anchor-free, single-stage framework that predicts both the start and end times of actions while simultaneously classifying the actions, framing the task as a sequence labeling problem. The proposed model utilizes an encoder-decoder architecture with LSTM projections and 1D convolutions to capture rich temporal dependencies. This is followed by hierarchical feature pyramid generation, which is refined using Pyramidal Pooling Aggregation (PPA) and enhanced through a Progressive Feature Enhancement Unit (PFEU) with dilated convolutions to preserve the contextual relationships present in complex video frames. We also propose a novel Temporal Granularity Convolution (TGC) Layer, which refines temporal features. The TGC Layer captures various temporal details using a multi-branch structure consisting of Fine-Grained and Coarse-Grained Temporal Branches, designed to efficiently handle different temporal granularities. Finally, a decoder utilizes these multi-scale features to predict action instances at multiple levels of granularity. The model simultaneously integrates classification and regression heads to predict action labels and temporal boundaries, achieving improved localization and recognition. We have demonstrated the effectiveness of the proposed scheme, using mean average precision (mAP) across various Intersection over Union (IoU) thresholds on the Thumos14, ActivityNet-1.3, and MultiTHUMOS datasets over multiple state-of-the-art (SOTA) methods.
The detection of abusive language in online spaces is crucial for ensuring a safe digital environment, particularly for low-resource languages like Mizo. Mizo, a tonal Tibeto-Burman language spoken primarily in Mizoram, India, poses significant computational challenges due to its phonetic complexity and limited linguistic resources. This research presents a method based on machine learning for abusive comment detection in Mizo, addressing the lack of annotated datasets and specialized NLP tools. A structured pipeline involving data collection, preprocessing, feature engineering, and model evaluation was implemented. Our study compares the effectiveness of several conventional machine learning methods, such as Random Forest, Support Vector Machines (SVM), Logistic Regression, and XGBoost, against transformer-based models such as Multilingual BERT(mBERT) and MizBERT. According to experimental data, MizBERT achieves the highest accuracy and F1-score, outperforming all other models by a substantial margin. This work contributes to the development of computational tools for Mizo NLP, laying a foundation for automated moderation systems and fostering digital inclusivity for Mizo-Speaking communities.
Deep learning models are becoming more general and robust by the day. Specifically, image foundation models have recently shown exponential growth. In this work, we introduce a way to exploit this growth in the field of video classification. The basic idea here is that if we have a good understanding of space, we should not require complicated spatio-temporal processing. We introduce Attention Map (AM) flow, a way to identify the location of local changes between two frames in a video, without adding additional parameters specifically for it. We utilise adapters, which have been growing in popularity in the field of parameterefficient transfer learning. These help us incorporate AM flow in a pretrained image model without the need of finetuning it. With just these changes and minimal temporal processing, an image model is able to achieve state-of-the- art results on popular action recognition datasets with low training time and requiring minimal pretraining. This work explores the theory behind this idea and the intricacies involved. Through relevant experiments, we show the efficacy of this method and discuss various ideas to take this work forward. We use kinetics-400, something-something v2 and Toyota smarthome datasets and achieve state-of-the-art or comparable results. We also show that video models suffer from extensive pretraining on multiple datasets and a large training time, but our work answers these problems. actionrecognition transformers image-to-video-models
Identifying abusive content in online social networks (OSNs) continues to be a considerable issue due to linguistic complexity, code-mixing, and the existence of implicit abuse. This article presents a transformer-based hybrid model that combines contextual embeddings from transformer models with convolutional neural networks (CNNs) for pattern recognition and subsequently employs an attention mechanism to improve contextual interpretation. The model proficiently captures both word-level and character-level features, overcoming the drawbacks of current techniques that frequently break with implicit and multilingual abuse. An F1-score of 96.7%, an accuracy of 95.73%, a precision of 94.5%, a recall of 96.8%, and detailed tests on benchmark datasets show that the proposed model works very well. A comparison with cutting-edge methodologies further proves the model’s efficacy in minimizing false positives and false negatives. The findings underscore the effectiveness of the suggested method in detecting abusive content across diverse contexts. This article enhances the safety of digital platforms by offering a dependable and scalable approach for the real-time detection of abusive content.
The significance of object detection in unmanned aerial vehicles (UAVs) has been drastically increased in various applications, e.g., traffic management, agriculture, military affairs, etc. The you-only-look-once version 8s (YOLOv8s) model is widely adopted in real-time object detection tasks due to its speed and high performance. However, challenges remain when dealing with complex real-world scenarios such as dense scenes with multiple small targets. To address the issue of false or missed detection of small targets in UAV-captured scenes, we improve the YOLOv8s model by incorporating (i) a small target detection layer, (ii) the deformable convolutional networks v2 (DCNv2) module, and (iii) the spatial pyramid pooling fast-large separable kernel attention (SPPF-LSKA) module into it. The new model is called YOLOv8s-ZS and the dataset used is VisDrone2019. The experimental results show that YOLOv8s-ZS achieves an improvement of up to 3.8% in terms of precision, recall, and mean average precision, as compared with some models, e.g., YOLOv8s, BDH-YOLO, and CoT-YOLOv8. These results demonstrate that YOLOv8s-ZS provides a better solution for the problem of UAV small target detection.
Semantic segmentation is essential in medical image analysis. In this paper, we propose a lightweight vision transformer-based semantic segmentation method for medical images. The proposed method is composed of three parts: encoder, decoder, and segmentation head. The encoder is a light-weight ViT called MPiFormer developed based on iFormer and the split-and-merge strategy. A decoder is designed to fuse the multi-scale feature maps and a segmentation head is developed to predict the semantic segmentation map. Experiments on Barrett’s esophagus and Melanoma datasets demonstrate that the proposed method achieves an F1-score of 96.01% and an mIoU of 92.24% with limited annotations. Additionally, the split-and-merge mechanism reduces parameters and computation by 72.63% and 71.75%, respectively, while maintaining stable performance.
Detecting forged images and Deepfake videos remains a pressing challenge. Most existing detection methods rely on datasets that contain specific types of tampering. However, as forgery techniques continue to evolve, new forms of manipulation emerge and may undermine these approaches. Furthermore, media content often undergoes compression during dissemination, which can obscure manipulation traces and reduce detection accuracy. This study proposes a forgery detection scheme that leverages block-level consistency. A source-related feature extractor is employed, followed by a Siamese network to classify the compression levels. For image tampering detection, the heatmap is examined to determine manipulated regions. In Deepfake video detection, the proposed scheme focuses on facial regions and evaluates frame-to-frame similarity to assess authenticity. The effectiveness of the proposed approach is verified using publicly available datasets.
As DNN-based AI models have rapidly developed, the risk of cyber-attacks against them has also increased. This paper focuses on Model Inversion Attacks (MIA), which is the attack to reveal a victim model’s training data by estimating input samples that causes the victim model to produce adversaries’ designated output. Modern MIA methods often tackle the task of Label-Only MIA, where a black-box image classification model is assumed as a victim model whose network structure and parameters are not disclosed to adversaries. A key technique for attacking such a model is to exploit an image generation model (called "attack model") trained by the adversaries’ own image set and its corresponding class label set provided by the victim model. However, to get the class label set, existing methods have to send a massive amount of images to the victim model as queries, which is highly suspicious behavior. To solve this unpracticality, we propose a Label-Only MIA method that requires no queries to train the attack model. The proposed method employs a Conditional Diffusion Model (CDM) as the attack model to obtain plausible attack results. Specifically, we first train an image feature extractor as an auxiliary module and use it to embed all images in the adversaries’ own dataset to a feature space. Then, we train a CDM-based attack model using a set of pairs of an image and its embedded feature. These training procedures are performed independently from the victim model. We experimentally confirmed that the proposed method achieves an attack performance comparable to white-box-oriented MIA methods, demonstrating that MIA risks are significant even for practical image classification models that are capable of blocking malicious users who send too many queries.
