In this paper, we introduce a self-supervised deep SLAM method that robustly operates in dynamic scenes while accurately identifying dynamic components. Our method leverages a dual-flow representation for static flow and dynamic flow, facilitating effective scene decomposition in dynamic environments. We propose a dynamic update module based on this representation and develop a dense SLAM system that excels in dynamic scenarios. In addition, we design a self-supervised training scheme using DINO as a prior, enabling label-free training. Our method achieves superior accuracy compared to other self-supervised methods. It also matches or even surpasses the performance of existing supervised methods in some cases. All code and data will be made publicly available upon acceptance.

Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.

Visual-inertial odometry (VIO) has made significant progress in various applications. However, one of the key challenges in VIO is the efficient and robust fusion of visual and inertial measurements, particularly while mitigating the impact of sensor failures. To address this challenge, we propose a new learning-based VIO system, i.e., DW-VIO, which is able to integrate multiple sensors and provide robust state estimations. To this end, we design a novel deep learning-based data-fusion approach that dynamically associates information from multiple sensors to predict sensor weights for optimization. Moreover, in order to improve the efficiency, we present several real-time optimization techniques including a fast patch graph constructor and an efficient GPU-accelerated multi-factor bundle adjustment layer. Experimental results show that DW-VIO outperforms most state-of-the-art (SOTA) methods on the EuRoC MAV, ETH3D-SLAM, and KITTI-360 benchmarks across various challenging sequences. Additionally, it maintains a minimum of 20 frames per second (FPS) on a single RTX 3060 GPU with high-resolution input, highlighting its efficiency.

One of the key challenges of visual Simultaneous Localization and Mapping (SLAM) in large-scale environments is how to effectively use global localization to correct the cumulative errors from long-term tracking. This challenge presents itself in two main aspects: first, the difficulty for robots in revisiting previous locations to perform loop closure, and second, the considerable memory resources required to maintain point-cloud-based global maps. Recent solutions have resorted into neural networks, using satellite images as the references for ground-level localization. However, most of these methods merely provide cross-view patch-matching results, which leads to unfeasible in integration with the SLAM system. To address these issues, we present a semantic-based cross-view localization method. This approach combines semantic information with a reward and penalty mechanism, enabling us to obtain a global probability map and achieve precise 3-degree-of-freedom (3-DoF) localization. Based on that, we develop a SLAM system that capitalizes on satellite imagery for global localization. This strategy effectively bridges the gap between SLAM and real-world coordinates while also substantially reducing accumulated errors. Our experimental results demonstrate that our global localization method significantly outperforms existing satellite-based systems. Moreover, in scenarios where the robot struggles to find loop closures, employing our localization method improves the SLAM accuracy.

It is well known that it is difficult to have a reliable and robust framework to link multi-agent deep reinforcement learning algorithms with practical multi-robot applications. To fill this gap, we propose and build an open-source framework for multi-robot systems called MultiRoboLearn(1). This framework builds a unified setup of simulation and real-world applications. It aims to provide standard, easy-to-use simulated scenarios that can also be easily deployed to real-world multir-obot environments. Also, the framework provides researchers with a benchmark system for comparing the performance of different reinforcement learning algorithms. We demonstrate the generality, scalability, and capability of the framework with two real-world scenarios using different types of multi-agent deep reinforcement learning algorithms in discrete and continuous action spaces.

Depth completion plays a crucial role in enabling real-world applications such as obstacle avoidance and SLAM for robot navigation. This letter focuses on addressing the depth completion challenge for single-line LiDAR, commonly used in conjunction with visual cameras. The sparsity of valid depth points makes supervised methods inadequate, while existing self-supervised approaches are only applicable to 64-line LiDARs. In this letter, we propose a novel self-supervised approach for single-line LiDAR depth completion. Our approach makes two key contributions. Firstly, we introduce the Relative-to-Metric (R2M) depth distillation framework, which estimates a pixel-wise metric depth map using an RGB image and its corresponding single-line depth map. This is achieved by distilling relative depth predictions from a monocular depth estimator trained on RGB images. Secondly, we propose the Line Depth Prior (LDP), a model-agnostic geometry regularization technique that promotes depth completion. Through extensive experiments, we demonstrate that our proposed method can: i) accurately reconstruct complete depth maps from single-line depth inputs without requiring additional depth supervision, except for the observed entries, and ii) facilitate downstream SLAM tasks when using single-line LiDAR.

Performing accurate localization while maintaining the low-level communication bandwidth is an essential challenge of multi-robot simultaneous localization and mapping (MR-SLAM). In this paper, we tackle this problem by generating a compact yet discriminative feature descriptor with minimum inference time. We propose descriptor distillation that formulates the descriptor generation into a learning problem under the teacher-student framework. To achieve real-time descriptor generation, we design a compact student network and learn it by transferring the knowledge from a pre-trained large teacher model. To reduce the descriptor dimensions from the teacher to the student, we propose a novel loss function that enables the knowledge transfer between two different dimensional descriptors. The experimental results demonstrate that our model is 30% lighter than the state-of-the-art model and produces better descriptors in patch matching. Moreover, we build a MR-SLAM system based on the proposed method and show that our descriptor distillation can achieve higher localization performance for MR-SLAM with lower bandwidth.

Monocular depth estimation (MDE) methods are often either too computationally expensive or not accurate enough due to the trade-off between model complexity and inference performance. In this paper, we propose a lightweight network that can accurately estimate depth maps using minimal computing resources. We achieve this by designing a compact model that maximally reduces model complexity. To improve the performance of our lightweight network, we adopt knowledge distillation (KD) techniques. We consider a large network as an expert teacher that accurately estimates depth maps on the target domain. The student, which is the lightweight network, is then trained to mimic the teacher’s predictions. However, this KD process can be challenging and insufficient due to the large model capacity gap between the teacher and the student. To address this, we propose to use auxiliary unlabeled data to guide KD, enabling the student to better learn from the teacher’s predictions. This approach helps fill the gap between the teacher and the student, resulting in improved data-driven learning. The experiments show that our method achieves comparable performance to state-of-the-art methods while using only 1% of their parameters. Furthermore, our method outperforms previous lightweight methods regarding inference accuracy, computational efficiency, and generalizability.

The core problem of visual multi-robot simultaneous localization and mapping (MR-SLAM) is how to efficiently and accurately perform multi-robot global localization (MR-GL). The difficulties are two-fold. The first is the difficulty of global localization for significant viewpoint difference. Appearance-based localization methods tend to fail under large viewpoint changes. Recently, semantic graphs have been utilized to overcome the viewpoint variation problem. However, the methods are highly time-consuming, especially in large-scale environments. This leads to the second difficulty, which is how to perform real-time global localization. In this letter, we propose a semantic histogram based graph matching method that is robust to viewpoint variation and can achieve real-time global localization. Based on that, we develop a system that can accurately and efficiently perform MR-GL for both homogeneous and heterogeneous robots. The experimental results show that our approach is about 30 times faster than Random Walk based semantic descriptors. Moreover, it achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.

The RGB-Thermal (RGB-T) information for semantic segmentation has been extensively explored in recent years. However, most existing RGB-T semantic segmentation usually compromises spatial resolution to achieve real-time inference speed, which leads to poor performance. To better extract detail spatial information, we propose a two-stage Feature-Enhanced Attention Network (FEANet) for the RGB-T semantic segmentation task. Specifically, we introduce a Feature-Enhanced Attention Module (FEAM) to excavate and enhance multi-level features from both the channel and spatial views. Benefited from the proposed FEAM module, our FEANet can preserve the spatial information and shift more attention to high-resolution features from the fused RGB-T images. Extensive experiments on the urban scene dataset demonstrate that our FEANet outperforms other state-of-the-art (SOTA) RGB-T methods in terms of objective metrics and subjective visual comparison (+2.6% in global mAcc and +0.8% in global mIoU). For the 480 × 640 RGB-T test images, our FEANet can run with a real-time speed on an NVIDIA GeForce RTX 2080 Ti card.

As vision based perception methods are usually built on the normal light assumption, there will be a serious safety issue when deploying them into low light environments. Recently, deep learning based methods have been proposed to enhance low light images by penalizing the pixel-wise loss of low light and normal light images. However, most of them suffer from the following problems: 1) the need of pairs of low light and normal light images for training, 2) the poor performance for dark images, 3) the amplification of noise. To alleviate these problems, in this letter, we propose a two-stage unsupervised method that decomposes the low light image enhancement into a pre-enhancement and a post-refinement problem. In the first stage, we pre-enhance a low light image with a conventional Retinex based method. In the second stage, we use a refinement network learned with adversarial training for further improvement of the image quality. The experimental results show that our method outperforms previous methods on four benchmark datasets. In addition, we show that our method can significantly improve feature points matching and simultaneous localization and mapping in low light conditions.

传统的学生综合评价方法主要是将得分进行加权累加并求平均值,人为因素对结果影响较大.该方法介绍了带有3种不同否定关系的广义模糊集GFScom的基本特性,描述了带有3种否定关系的语言变量表示方法,归纳了应用GFScom原理解决模糊决策问题的过程,并将该方法应用于毕业生综合评价中.实验表明,该方法在保证结果正确的同时,能够降低评价过程对人为因素的依赖,具有较好的客观性.

Each method, machine learning-based and rule-based, for extracting PPI (Protein-Protein Interactions) from biomedical literatures has advantages and disadvantages. In order to utilise the superiorities of these methods reasonably, this paper designs a new structure for the relational word dictionary, uses weakly supervised method to find dictionary items and fill them into the PPI relational word dictionary, and presents a method to learn PPI relational rules automatically based on slot-filling principle. Moreover, this method takes the PPI relation instances without apparent relational words into consideration aiming to improve the final performance. We conduct the experiments with five authoritative biomedical PPI corpuses, and discover some distribution features about PPI relational words. Finally, we also compare our method with several recent research achievements, and the results show that the performance of our method is better than the average level among these methods.

实体关系抽取的核心问题是实体关系特征的选择.以往的研究通常都以词法特征、实体原始特征等来刻画实体关系,其抽取效果已难再提高.在传统方法的基础上,该文提出一种基于句法特征、语义特征的实体关系抽取方法,融入了依存句法关系、核心谓词、语义角色标注等特征,选择SVM作为机器学习的实现途径,以真实新闻文本作为语料进行实验.实验结果表明该方法的F1值有明显提升.

In the research of named-entity relation extraction based on supervision, selecting relation features for traditional methods are usually finished by people, and it’s hard to implement these methods for large-scale corpus. On the other hand, fixing relation types is the premise, so the practicabilities of these methods are not so ideal. This paper presents a weakly supervised method for Chinese named-entity relation extraction without man-made annotations, and the relation types in this method are not chosen artificially. The method collects entity relation types from the structured knowledge in encyclopedia pages, and then automatically annotates the relation instances existing in the texts based on these relation types. Simultaneously, the syntactic and semantic features of entity relations will be considered in this method, then the machine learning data will be completed, finally we use Support Vector Machine (SVM) model to train relation classifiers from training data, and these classifiers could try to extract entity relations from testing data. We carry out the experiment with the data from Chinese Baidu Encyclopedia pages, and the results show the effectiveness of this method, the overall F1 value reaches to 83.12

Predicting disease genes in PPI network has attracted a lot of attention over the years. Based on the assumption that the phenotypes of the genes in the same complex where candidate gene located in are more similar to disease, the candidate gene is more possible to be disease gene, we propose a new disease gene identification method based on protein complex phenotype similarity. First, our method mines protein complexes in PPI network by resolution-limit-free clustering algorithm and maps the candidate genes to complexes. Second, we define phenotype similarity according to phenotype ontology, and calculate phenotype similarity value between gene and disease. Third, we add up the phenotype similarity value of whole genes in the complex as weight of candidate gene and rank the candidate gene according to the sum of phenotype similarity value. Finally, we test our method by leave-one-out cross validation. The results show that our method is effective and outperforms other methods such as NetRank, NetScore, NetZcore, Flow, RWR and NDRC. Importantly, we predict the disease gene of Prader-Willi syndrome (MIM: 176270) and Renal tubular dysgenesis (MIM: 267430) successfully, which do not exist in our disease-gene datasets but exists in online databases and scientific publications.

信息抽取的任务是从大量数据中准确、快速地获取目标信息,提高信息的利用率.目前,信息抽取已经成为NLP领域的一个重要分支.随着互联网应用的发展,其价值也正日益显现,学术界和工业界对此都寄予厚望.首先回顾了信息抽取的发展历程;接着从命名实体识别、指代消解、关系抽取和事件抽取4个方面总结了信息抽取关键技术的研究进展;然后分析了信息抽取目前面临的若干主要问题;最后对信息抽取的研究趋势作了预测.

Sentiment classification aims to automatically predict sentiment polarity (e.g., positive or negative) of user generated sentiment data (e.g., reviews, blogs). In real applications, these users generated sentiment data can span so many different domains that it is difficult to manually label training data for all of them. In this article, we develop a general solution to cross-domain sentiment classification when we do not have any labeled data in a target domain but have some labeled data in a source domain. To bridge the gap between domains, we propose a novel algorithm, called topical correspondence transfer (TCT). This is achieved by learning the domain-specific information from different domains into unified topics, with the help of shared topics across all domains. In this way, the topical correspondences behind the shared topics can be used as a bridge to reduce the gap between domains. We conduct experiments on a benchmark composed of reviews of 4 types of Amazon products. Experimental results show that our proposed TCT significantly outperforms the baseline method, and achieves an accuracy which is competitive with the state-of-the-art methods for cross-domain sentiment classification.

Using rules to extract protein-protein interactions (PPI) from biomedical literatures has shown recognized positive effect, but the process of making rules is time-costing and expensive. Relation dictionary-based rule is an effective way to solve the problem, while it also encounters a new problem: how to design an excellent dictionary fast and correctly. This paper proposes a weakly supervised method to construct the PPI relation dictionary, and presents a slot-filling method to learn PPI relation rules automatically according to the position of proteins and relation words. Moreover, this method does not depend on much more manual intervention. We conduct the experiment using 5 types of authoritative biomedical PPI corpus, and the results show that our method can improve the PPI extraction effect obviously.