Video moment retrieval uses a text query to locate a moment from a given untrimmed video reference. Locating corresponding video moments with text queries helps people interact with videos efficiently. Current solutions for this task have not considered conflict within location results from different models, so various models cannot integrate correctly to produce better results. This study introduces a reinforcement learning-based video moment retrieval model that can scan the whole video once to find the moment's boundary while producing its locational evidence. Moreover, we proposed a multi-agent system framework that can use evidential learning to resolve conflicts between agents' localization output. As a side product of observing and dealing with conflicts between agents, we can decide whether a query has no corresponding moment in a video (out-of-scope) without additional training, which is suitable for real-world applications. Extensive experiments on benchmark datasets show the effectiveness of our proposed methods compared with state-of-the-art approaches. Furthermore, the results of our study reveal that modeling competition and conflict of the multi-agent system is an effective way to improve RL performance in moment retrieval and show the new role of evidential learning in the multi-agent framework.

Label assignment is a critical component in object detectors, particularly within DETR-style frameworks where the one-to-one matching strategy, despite its end-to-end elegance, suffers from slow convergence due to sparse supervision. While recent works have explored one-to-many assignments to enrich supervisory signals, they often introduce complex, architecture-specific modifications and typically focus on a single auxiliary strategy, lacking a unified and scalable design. In this paper, we first systematically investigate the effects of “one-to-many” supervision and reveal a surprising insight that performance gains are driven not by the sheer quantity of supervision, but by the diversity of the assignment strategies employed. This finding suggests that a more elegant, parameter-efficient approach is attainable. Building on this insight, we propose LoRA-DETR, a flexible and lightweight framework that seamlessly integrates diverse assignment strategies into any DETR-style detector. Our method augments the primary network with multiple Low-Rank Adaptation (LoRA) branches during training, each instantiating a different one-to-many assignment rule. These branches act as auxiliary modules that inject rich, varied supervisory gradients into the main model and are discarded during inference, thus incurring no additional computational cost. This design promotes robust joint optimization while maintaining the architectural simplicity of the original detector. Extensive experiments on different baselines validate the effectiveness of our approach. Our work presents a new paradigm for enhancing detectors, demonstrating that diverse “one-to-many” supervision can be integrated to achieve state-of-the-art results without compromising model elegance.

Standard softmax self-attention excels in vision tasks but incurs quadratic complexity O(N^2), limiting high-resolution deployment. Linear attention reduces the cost to O(N), yet its compressed state representations can impair modeling capacity and accuracy. We present an analytical study that contrasts linear and softmax attention for visual representation learning from a layer-stacking perspective. We further conduct systematic experiments on layer-wise hybridization patterns of linear and softmax attention. Our results show that, compared with rigid intra-block hybrid designs, fine-grained layer-wise hybridization can match or surpass performance while requiring fewer softmax layers. Building on these findings, we propose SoLA-Vision (Softmax-Linear Attention Vision), a flexible layer-wise hybrid attention backbone that enables fine-grained control over how linear and softmax attention are integrated. By strategically inserting a small number of global softmax layers, SoLA-Vision achieves a strong trade-off between accuracy and computational cost. On ImageNet-1K, SoLA-Vision outperforms purely linear and other hybrid attention models. On dense prediction tasks, it consistently surpasses strong baselines by a considerable margin. Code will be released.

Bird's-eye-view (BEV) map layout estimation requires an accurate and full understanding of the semantics for the environmental elements around the ego car to make the results coherent and realistic. Due to the challenges posed by occlusion, unfavourable imaging conditions and low resolution, \emph{generating} the BEV semantic maps corresponding to corrupted or invalid areas in the perspective view (PV) is appealing very recently. \emph{The question is how to align the PV features with the generative models to facilitate the map estimation}. In this paper, we propose to utilize a generative model similar to the Vector Quantized-Variational AutoEncoder (VQ-VAE) to acquire prior knowledge for the high-level BEV semantics in the tokenized discrete space. Thanks to the obtained BEV tokens accompanied with a codebook embedding encapsulating the semantics for different BEV elements in the groundtruth maps, we are able to directly align the sparse backbone image features with the obtained BEV tokens from the discrete representation learning based on a specialized token decoder module, and finally generate high-quality BEV maps with the BEV codebook embedding serving as a bridge between PV and BEV. We evaluate the BEV map layout estimation performance of our model, termed VQ-Map, on both the nuScenes and Argoverse benchmarks, achieving 62.2/47.6 mean IoU for surround-view/monocular evaluation on nuScenes, as well as 73.4 IoU for monocular evaluation on Argoverse, which all set a new record for this map layout estimation task. The code and models are available on \url{https://github.com/Z1zyw/VQ-Map}.

Multilingual multimodal pre-training has garnered significant attention, but it faces challenges due to the substantial need for diverse multilingual text-image data, especially for minor languages. This article introduces UniGen, a unified strategy for efficient multilingual multimodal pre-training inspired by internet data distribution observations. Leveraging the richer availability and higher quality of multilingual text-English text and English text-image data, UniGen aligns the latent space of multilingual text with visual information to a unified semantic space. This alignment, with English as a reference, proves effective in enhancing cross-modal understanding. UniGen reduces reliance on multilingual text-image data, surpassing comparable models in multilingual multimodal benchmark IGLUE by a notable 7%. Notably, UniGen is the first multilingual multimodal model to unify all pre-training tasks within a generative pre-training framework.

Named entity recognition (NER) is an important task for the natural language processing of biomedical text. Currently, most NER studies standardized biomedical text, but NER for unstandardized biomedical text draws less attention from researchers. Named entities in online biomedical text exist with errors and polymorphisms, which negatively impact NER models’ performance and impede support from knowledge representation methods. In this paper, we propose a neural network method that can effectively recognize entities in unstandardized online medical/health text. We introduce a new pre-training scheme that uses large-scale online question-answering pairs to enhance transformers’ model capacity on online biomedical text. Moreover, we supply models with knowledge representations from a knowledge base called multi-channel knowledge labels, and this method overcomes the restriction from languages, like Chinese, that require word segmentation tools to represent knowledge. Our model outperforms other baseline methods significantly in experiments on a dataset for Chinese online medical entity recognition and achieves state-of-the-art results.

With the aging population, health and senior care are becoming to be crucial issues for the whole world. Because the number of healthcare professionals is far from fulfilling increasing patients’ and seniors’ needs, seeking services from non-professional healthcare staff, such as home caregivers, is indispensable. Methods that support locating video moments with natural language queries can improve the normalization of operations for the non-professional healthcare staff and reduce their time expenditure on specific action moment retrieval. Addressing this problem, we propose a cross-modal neural network model for effective health and senior care video localization. Our model learns procedures in the video reference and uses procedure knowledge to improve the model’s localization performance. We conduct experiments on a dataset for health and senior care video localization and an open-accessible dataset about medical instruction. Experiment results show procedure knowledge can remarkably improve the model’s capacity for video moment localization. We hope our dataset and method could promote the development of cross-modal research and application for health and senior care.

This paper introduces an online self-supervised method that leverages inter- and intra-level variance for video representation learning. Most existing methods tend to focus on instance-level or inter-variance encoding but ignore the intra-variance existing in clips. The key observation to solving this problem is the underlying correlation between visual and audio, in which the distribution of flow patterns in feature space is diverse, but expresses complementary similar semantics. And in the semantic feature space, the horizontal dimension of the feature matrix could be regarded as cluster labels. These cluster labels should be consistent for different modalities of the same video clip. Based on this idea, we propose an end-to-end inter-intra cross-modality contrastive clustering scheme to simultaneously optimize the inter- and intra-level contrastive loss. Experiments show that our proposed approach is able to considerably outperform previous methods for self-supervised learning on HMDB51 and UCF101 when applied to video retrieval and action recognition tasks.

Open- vocabulary object detection aims to detect novel object categories beyond the training set. The advanced open- vocabulary two-stage detectors employ instance-level visual-to- visual knowledge distillation to align the visual space of the detector with the semantic space of the Pre-trained Visual-Language Model (PVLM). However, in the more efficient one-stage detector, the absence of class-agnostic object proposals hinders the knowledge distil-lation on unseen objects, leading to severe performance degradation. In this paper, we propose a hierarchical visual-language knowledge distillation method, i.e., Hi-erKD, for open-vocabulary one-stage detection. Specifi-cally, a global-level knowledge distillation is explored to transfer the knowledge of unseen categories from the PVLM to the detector. Moreover, we combine the proposed global-level knowledge distillation and the common instance-level knowledge distillation to learn the knowledge of seen and unseen categories simultaneously. Extensive experiments on MS-COCO show that our method significantly surpasses the previous best one-stage detector with 11.9% and 6.7% AP50 gains under the zero-shot detection and generalized zero-shot detection settings, and reduces the AP 50 performance gap from 14% to 7.3% compared to the best two-stage detector. Code will be released at this url 1 1 https://qithub.com/menqqiDyanqqe/HierKD.

Background: The infectious disease Coronavirus Disease 2019 (COVID-19) outbroke in 2019 spread to multiple countries. The quick spread of the virus and isolation strategies may trigger psychological problems. Our aim was to explore the dynamic network structure of the psychological state before and during the epidemic.Methods: A web-based survey was conducted in two stages: the T1 stage (1 January 2019 to 31 December 2019) and the T2 stage (1 February 2020 to 8 March 2020). In both stages, the Patient Health Questionnaire-9, General Anxiety Disorder-7, and Pittsburgh Sleep Quality Index were used to assess depression, anxiety, and sleep, respectively.Results: We matched the data based on IP addresses. We included 1,978, 1,547, and 2,061 individuals who completed the depression, anxiety, and sleep assessments, respectively, at both stages. During epidemics, psychomotor agitation/retardation, inability to relax, restless behavior, and the frequency of using medicine had high centrality. Meanwhile, the network structure of psychological symptoms becomes stronger than before the epidemic.Conclusion: Symptoms of psychomotor agitation/retardation, inability to relax, and restless behavior should be treated preferentially. It is necessary to provide mental health services, including timely and effective early psychological intervention. In addition, we should also pay attention to the way patients use medicines to promote sleep quality.

In this paper, we propose an attention-based multi-task neural network model for text classification and sequence tagging and then apply it to the named entity recognition and the intent analysis of Chinese online medical questions. We found that the use of both attention and multi-task learning improved the performance of these tasks. Our method achieved superior performance in named entity recognition and intent analysis compared with other baseline methods; the method is a light-weight solution that is suitable for deployment on small servers. Furthermore, we took advantage of the model's capabilities for these two tasks and built a simple question-answering system for cardiovascular issues. Users and service providers can monitor the logic of the answers generated by this system.

Question answering (QA) is an important natural language processing (NLP) task and has received much attention in academic research and industry communities. Existing QA studies assume that questions are raised by humans and answers are generated by machines. Nevertheless, in many real applications, machines are also required to determine human needs or perceive human states. In such scenarios, machines may proactively raise questions and humans supply answers. Subsequently, machines should attempt to understand the true meaning of these answers. This new QA approach is called reverse-QA (rQA) throughout this paper. In this work, the human answer understanding problem is investigated and solved by classifying the answers into predefined answer-label categories (e.g., True, False, Uncertain). To explore the relationships between questions and answers, we use the interactive attention network (IAN) model and propose an improved structure called semi-interactive attention network (Semi-IAN). Two Chinese data sets for rQA are compiled. We evaluate several conventional text classification models for comparison, and experimental results indicate the promising performance of our proposed models.

In this paper, we address the problem of representing and recognizing human actions from videos on matrix manifolds. For this purpose, we propose a new vector representation method, named tangent Fisher vector, to describe video sequences in the Fisher kernel framework. We first extract dense curved spatio-temporal cuboids from each video sequence. Compared with the traditional ’straight cuboids’, the dense curved spatio-temporal cuboids contain much more local motion information. Each cuboid is then described using a linear dynamical system (LDS) to simultaneously capture the local appearance and dynamics. Furthermore, a simple yet efficient algorithm is proposed to learn the LDS parameters and approximate the observability matrix at the same time. Each video sequence is thus represented by a set of LDSs. Considering that each LDS can be viewed as a point in a Grassmann manifold, we propose to learn an intrinsic GMM on the manifold to cluster the LDS points. Finally a tangent Fisher vector is computed by first accumulating all the tangent vectors in each Gaussian component, and then concatenating the normalized results across all the Gaussian components. A kernel is defined to measure the similarity between tangent Fisher vectors for classification and recognition of a video sequence. This approach is evaluated on the state-of-the-art human action benchmark datasets. The recognition performance is competitive when compared with current state-of-the-art results.

Background: Behavior regulation and clinical intervention have a significant effect on depression treatments. This study aims to make a comparison between behavior regulation and clinical intervention for depression based on a large-scale dataset. Methods: We collect user-reported data from an online survey tool including depression symptoms, treatments and effectiveness of treatments (n = 91873). A deep learning approach is used to build an effective model to evaluate the effects on treatment methods for depression. The Skip-gram model is chosen to generate meaningful vector representations of symptoms and methods. Precision, recall and F1 score are calculated to evaluate the model performance. Results: Unidirectional model achieves higher F1 score than non-unidirectional model (0.71 vs. 0.63). The behavior regulation is better than the clinical intervention for mild depression symptoms. However, the clinical intervention for moderate or severe depression symptoms has obvious advantages. Conclusions: These experiments prove that the symptoms have unidirectional influence on the choice of regulatory methods. The behavior regulation and clinical treatment have different advantages for depression. These findings could help clinicians to choose better depression treatments.

With user-generated content explosively growing, how to find valuable posts from discussion threads in web communities becomes a hot topic. Although many learning algorithms have been proposed for mining the thread contents, there are still two problems that are not effectively considered. First, the learning algorithms are usually complicated so as to deal with various kinds of threads in web communities, which damages the generalization performance of the algorithms and takes the risk of overfitting to the learning models. Second, the small sample size problem exists when the training data for learning is divided into many isolated groups and each group is trained separately in order to avoid overfitting. In this paper, we propose a metadata-based clustered multi-task learning method, which takes full use of the metadata of threads and fuses it in the multi-task learning based on a divide-and-learn strategy. Our method provides an effective solution to the above problems by finding the geometric structure or context of semantics of threads in web communities and constructing the relations among training thread groups and their corresponding learning tasks. In addition, a soft-assigned clustered multi-task learning model is employed. Our experimental results show the effectiveness of our method.

In this paper, we consider the link sign prediction in social networks with friend and foe relationships. We view the sign prediction as a user-to-user recommendation problem with trust or distrust information. Not only do we take the topological relationships such as the social structural balance and status theories into consideration, but also the social factors that whether a user is trustworthy and whether the user easily trust others are involved. We propose a probabilistic matrix factorization method with social trust and distrust ensembles and the structural theories from social psychology in order to predict link signs in social networks. The experimental results show that our proposed method outperforms those of the previous studies on this problem.

The association rules mining process enables the end users to analyze, understand, and use the extracted knowledge in an intelligent system or to support the decision-making processes. To find valuable association rules from a large number of redundant rules, this paper proposes a deeper mining process, multi-mode and high value association rules mining (MH-ARM). This method takes into account the category information, the size of the item set, natural semantics, various metrics, and effective visualization of results. The process can effectively reduce the number of rules and improve the value and accuracy of the rules screened out for auxiliary diagnosis. In the end, the experimental data of rhinitis were analyzed and the effectiveness of the process was verified.

In this paper, we address the problem of human action recognition through combining global temporal dynamics and local visual spatio-temporal appearance features. For this purpose, in the global temporal dimension, we propose to model the motion dynamics with robust linear dynamical systems (LDSs) and use the model parameters as motion descriptors. Since LDSs live in a non-Euclidean space and the descriptors are in non-vector form, we propose a shift invariant subspace angles based distance to measure the similarity between LDSs. In the local visual dimension, we construct curved spatio-temporal cuboids along the trajectories of densely sampled feature points and describe them using histograms of oriented gradients (HOG). The distance between motion sequences is computed with the Chi-Squared histogram distance in the bag-of-words framework. Finally we perform classification using the maximum margin distance learning method by combining the global dynamic distances and the local visual distances. We evaluate our approach for action recognition on five short clips data sets, namely Weizmann, KTH, UCF sports, Hollywood2 and UCF50, as well as three long continuous data sets, namely VIRAT, ADL and CRIM13. We show competitive results as compared with current state-of-the-art methods.

In this paper, we address the problem of recognizing human actions with motion dynamics alone. For this purpose, we propose to use silhouette sequences to represent the human actions by discarding the appearance information, and then model the sequences with linear dynamical systems (LDSs). Recognition is achieved by directly comparing the distance between LDSs, rather than resorting to complex Bayesian learning and inference. In particular, we introduce an efficient optimization method to learn robust LDSs, and develop a shift invariant distance metric to measure the similarity on the LDSs space. We evaluate our approach on the human action data set and achieve comparable results.