Multi-label image recognition is a fundamental task in computer vision. Recently, Vision-Language Models (VLMs) have made notable advancements in this area. However, previous methods fail to effectively leverage the rich knowledge in language models and often incorporate label semantics into visual features unidirectionally. To overcome these problems, we propose a Split-and-Synthesize Prompting with Gated Alignments (SSPA) framework to amplify the potential of VLMs. Specifically, we develop an in-context learning approach to associate the inherent knowledge from LLMs. Then we propose a novel Split-and-Synthesize Prompting (SSP) strategy to first model the generic knowledge and downstream label semantics individually and then aggregate them carefully through the quaternion network. Moreover, we present Gated Dual-Modal Alignments (GDMA) to bidirectionally interact visual and linguistic modalities while eliminating redundant cross-modal information, enabling more efficient region-level alignments. Rather than making the final prediction by a sharp manner in previous works, we propose a soft aggregator to jointly consider results from all image regions. With the help of flexible prompting and gated alignments, SSPA is generalizable to specific domains. Extensive experiments on nine datasets from three domains (i.e., natural, pedestrian attributes and remote sensing) demonstrate the state-of-the-art performance of SSPA. Further analyses verify the effectiveness of SSP and the interpretability of GDMA. The code will be made public.

Due to the complex features of train chassis images, conventional registration methods perform poorly. To improve registration accuracy and efficiency, this paper proposes a Transformer-based approach incorporating frequency-domain processing. Specifically, cosine similarity between template and deformed chassis images is computed first, serving as an indicator of deformation magnitude. Features are then extracted using an enhanced backbone network, with the cosine similarity integrated into these features to boost registration precision. For efficiency improvement, a neural network predicts the low-dimensional Fourier-domain representation of image pairs. A parameter-free decoder subsequently decodes this representation into the final full-resolution deformation field, accelerating inference. Experiments confirm the proposed method surpasses existing approaches in both accuracy and efficiency for train chassis image registration.

Identifying multiple novel classes in an image, known as open-vocabulary multi-label recognition, is a challenging task in computer vision. Recent studies explore the transfer of powerful vision-language models such as CLIP. However, these approaches face two critical challenges: (1) The local semantics of CLIP are disrupted due to its global pre-training objectives, resulting in unreliable regional predictions. (2) The matching property between image regions and candidate labels has been neglected, relying instead on naive feature aggregation such as average pooling, which leads to spurious predictions from irrelevant regions. In this paper, we present RAM (Recover And Match), a novel framework that effectively addresses the above issues. To tackle the first problem, we propose Ladder Local Adapter (LLA) to enforce refocusing on local regions, recovering local semantics in a memory-friendly way. For the second issue, we propose Knowledge-Constrained Optimal Transport (KCOT) to suppress meaningless matching to non-GT labels by formulating the task as an optimal transport problem. As a result, RAM achieves state-of-the-art performance on various datasets from three distinct domains, and shows great potential to boost the existing methods. Code: https://github.com/EricTan7/RAM.

Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable generalization capabilities to downstream tasks. However, existing prompt tuning based frameworks need to parallelize learnable textual inputs for all categories, suffering from massive GPU memory consumption when there is a large number of categories in the target dataset. Moreover, previous works require to include category names within prompts, exhibiting subpar performance when dealing with ambiguous category names. To address these shortcomings, we propose Compound Text-Guided Prompt Tuning (TGP-T) that significantly reduces resource demand while achieving superior performance. We introduce text supervision to the optimization of prompts, which enables two benefits: 1) releasing the model reliance on the pre-defined category names during inference, thereby enabling more flexible prompt generation; 2) reducing the number of inputs to the text encoder, which decreases GPU memory consumption significantly. Specifically, we found that compound text supervisions, i.e., category-wise and content-wise, is highly effective, since they provide inter-class separability and capture intra-class variations, respectively. Moreover, we condition the prompt generation on visual features through a module called Bonder, which facilitates the alignment between prompts and visual features. Extensive experiments on few-shot recognition and domain generalization demonstrate that TGP-T achieves superior performance with consistently lower training costs. It reduces GPU memory usage by 93% and attains a 2.5% performance gain on 16-shot ImageNet. The code is available at https://github.com/EricTan7/TGP-T.

The field of computer vision is currently transitioning from closed-set to open-set tasks. Vision foundation models have already demonstrated success in open-set scenarios. Building on these models, the utilization of a feature supervision framework can further enhance results. Our paper introduces a new method called SAMS (Segment Anything Model using Similar Images), which is a type of feature supervision framework. It is designed to segment specific masks from visual supervision features. Our framework comprises a pre-trained segmentation model and an efficient, novel prompt generation model capable of generating new prompts based on pre-extracted image features. This innovation eliminates the need for manually crafted prompts in the mask generation phase by integrating the principles of one-shot or few-shot learning with visual instructions from similar images. The effectiveness of the SAMS method is evident in its performance across various tasks, particularly in open-set tasks where traditional models tend to struggle. The pretrained model not only achieves impressive mean Intersection over Union (mIOU) scores without incurring additional time loss, but also demonstrates potential for further improvement through targeted module training.

Multi-label image recognition is a fundamental task in computer vision. Recently, vision-language models have made notable advancements in this area. However, previous methods often failed to effectively leverage the rich knowledge within language models and instead incorporated label semantics into visual features in a unidirectional manner. In this paper, we propose a Prompt-driven Visual-Linguistic Representation Learning (PVLR) framework to better leverage the capabilities of the linguistic modality. In PVLR, we first introduce a dual-prompting strategy comprising Knowledge-Aware Prompting (KAP) and Context-Aware Prompting (CAP). KAP utilizes fixed prompts to capture the intrinsic semantic knowledge and relationships across all labels, while CAP employs learnable prompts to capture context-aware label semantics and relationships. Later, we propose an Interaction and Fusion Module (IFM) to interact and fuse the representations obtained from KAP and CAP. In contrast to the unidirectional fusion in previous works, we introduce a Dual-Modal Attention (DMA) that enables bidirectional interaction between textual and visual features, yielding context-aware label representations and semantic-related visual representations, which are subsequently used to calculate similarities and generate final predictions for all labels. Extensive experiments on three popular datasets including MS-COCO, Pascal VOC 2007, and NUS-WIDE demonstrate the superiority of PVLR.

Previous Sign Language Translation (SLT) methods achieve superior performance by relying on gloss annotations. However, labeling high-quality glosses is a labor-intensive task, which limits the further development of SLT. Although some approaches work towards gloss-free SLT through jointly training the visual encoder and translation network, these efforts still suffer from poor performance and inefficient use of the powerful Large Language Model (LLM). Most seriously, we find that directly introducing LLM into SLT will lead to insufficient learning of visual representations as LLM dominates the learning curve. To address these problems, we propose Factorized Learning assisted with Large Language Model (FLa-LLM) for gloss-free SLT. Concretely, we factorize the training process into two stages. In the visual initialing stage, we employ a lightweight translation model after the visual encoder to pre-train the visual encoder. In the LLM fine-tuning stage, we freeze the acquired knowledge in the visual encoder and integrate it with a pre-trained LLM to inspire the LLM's translation potential. This factorized training strategy proves to be highly effective as evidenced by significant improvements achieved across three SLT datasets which are all conducted under the gloss-free setting.

We propose a new neural network design paradigm Reversible Column Network (RevCol). The main body of RevCol is composed of multiple copies of subnetworks, named columns respectively, between which multi-level reversible connections are employed. Such architectural scheme attributes RevCol very different behavior from conventional networks: during forward propagation, features in RevCol are learned to be gradually disentangled when passing through each column, whose total information is maintained rather than compressed or discarded as other network does. Our experiments suggest that CNN-style RevCol models can achieve very competitive performances on multiple computer vision tasks such as image classification, object detection and semantic segmentation, especially with large parameter budget and large dataset. For example, after ImageNet-22K pre-training, RevCol-XL obtains 88.2% ImageNet-1K accuracy. Given more pre-training data, our largest model RevCol-H reaches 90.0% on ImageNet-1K, 63.8% APbox on COCO detection minival set, 61.0% mIoU on ADE20k segmentation. To our knowledge, it is the best COCO detection and ADE20k segmentation result among pure (static) CNN models. Moreover, as a general macro architecture fashion, RevCol can also be introduced into transformers or other neural networks, which is demonstrated to improve the performances in both computer vision and NLP tasks. We release code and models at https://github.com/megvii-research/RevCol

Facial age estimation has received a lot of attention for its diverse application scenarios. Most existing studies treat each sample equally and aim to reduce the average estimation error for the entire dataset, which can be summarized as General Age Estimation. However, due to the long-tailed distribution prevalent in the dataset, treating all samples equally will inevitably bias the model toward the head classes (usually the adult with a majority of samples). Driven by this, some works suggest that each class should be treated equally to improve performance in tail classes (with a minority of samples), which can be summarized as Long-tailed Age Estimation. However, Long-tailed Age Estimation usually faces a performance trade-off, i.e., achieving improvement in tail classes by sacrificing the head classes. In this paper, our goal is to design a unified framework to perform well on both tasks, killing two birds with one stone. To this end, we propose a simple, effective, and flexible training paradigm named GLAE, which is two-fold. First, we propose Feature Rearrangement (FR) and Pixel-level Auxiliary learning (PA) for better feature utilization to improve the overall age estimation performance. Second, we propose Adaptive Routing (AR) for selecting the appropriate classifier to improve performance in the tail classes while maintaining the head classes. Moreover, we introduce a new metric, named Class-wise Mean Absolute Error (CMAE), to equally evaluate the performance of all classes. Our GLAE provides a surprising improvement on Morph II, reaching the lowest MAE and CMAE of 1.14 and 1.27 years, respectively. Compared to the previous best method, MAE dropped by up to 34%, which is an unprecedented improvement, and for the first time, MAE is close to 1 year old. Extensive experiments on other age benchmark datasets, including CACD, MIVIA, and Chalearn LAP 2015, also indicate that GLAE outperforms the state-of-the-art approaches significantly.

Traveling by high-speed rail and railway transportation have become an important part of people's life and social production. Track is the basic equipment of railway transportation, and its performance directly affects the service lifetime of railway lines and vehicles. The anomaly detection of rail fasteners is in a priority, while the traditional manual method is extremely inefficient and dangerous to workers. Therefore, this paper introduces efficient computer vision into the railway detection system not only to locate the normal fasteners, but also to recognize the fasteners states. To be more specific, this paper mainly studies the rail fastener detection based on improved You can Only Look Once version 5 (YOLOv5) network, and completes the real-time classification of fastener states. The improved YOLOv5 network proposed contains five sections, which are Input, Backbone, Neck, Head Detector and a read-only Few-shot Example Learning module. Themain purpose of this project is to improve the detection precision and shorten the detection time. Ultimately, the rail fastener detection system proposed in this paper is confirmed to be superior to other advanced algorithms. This model achieves on-line fastener detection by completing the "sampling-detection-recognition-warning" cycle of a single sample before the next image is sampled. Specifically, the mean average precision of model reaches 94.6%. And the model proposed reaches the speed of 12 ms per image in the deployment environment of NVIDIA GTX1080Ti GPU.

Long-tailed visual recognition has received increasing attention in recent years. Due to the extremely imbalanced data distribution in long-tailed learning, the learning process shows great uncertainties. For example, the predictions of different experts on the same image vary remarkably despite the same training settings. To alleviate the uncertainty, we propose a Nested Collaborative Learning (NCL++) which tackles the long-tailed learning problem by a collaborative learning. To be specific, the collaborative learning consists of two folds, namely inter-expert collaborative learning (InterCL) and intra-expert collaborative learning (IntraCL). In-terCL learns multiple experts collaboratively and concurrently, aiming to transfer the knowledge among different experts. IntraCL is similar to InterCL, but it aims to conduct the collaborative learning on multiple augmented copies of the same image within the single expert. To achieve the collaborative learning in long-tailed learning, the balanced online distillation is proposed to force the consistent predictions among different experts and augmented copies, which reduces the learning uncertainties. Moreover, in order to improve the meticulous distinguishing ability on the confusing categories, we further propose a Hard Category Mining (HCM), which selects the negative categories with high predicted scores as the hard categories. Then, the collaborative learning is formulated in a nested way, in which the learning is conducted on not just all categories from a full perspective but some hard categories from a partial perspective. Extensive experiments manifest the superiority of our method with outperforming the state-of-the-art whether with using a single model or an ensemble. The code will be publicly released.

The phenomenon of long-tailed data distribution widely exists in every corner of life. And in fact, the fundamental solution to long-tailed distribution problem is actually the combination of imbalanced classification and few-shot learning. At present, the mainstream methods under the framework of deep learning are re-sampling, re-weighting and other strategies like transfer learning. This paper proves that when dealing with long-tailed data, the conventional deep learning methods turn to show a low accuracy of few-shot samples and many-shot samples. Therefore, this paper proposes a long-tailed recognition model based on Dual Branch Learning Network (DBLN). While the imbalanced learning branch is using class-balanced loss function to modify model preference for head data, DBLN leads in a data augmentation learning branch based on improved rebalanced Mixup algorithm. These two branches update the weights by sharing dynamic parameters. DBLN model shows an excellent performance in improving the tendency of the deep learning algorithm to prefer the head classes and the recognition ability of the tail data. The experiments on two benchmark datasets (imbalanced CIFAR-10/100 dataset and rail fastener dataset which contains 5,000 rail pictures about 4 types of fasteners) show that the proposed improved dual branch long-tailed recognition model is superior to other advanced methods.

Textbook Question Answering (TQA) is a complex multimodal task to infer answers given large context descriptions and abundant diagrams. Compared with Visual Question Answering (VQA), TQA contains a large number of uncommon terminologies and various diagram inputs. It brings new challenges to the representation capability of language model for domain-specific spans. And it also pushes the multimodal fusion to a more complex level. To tackle the above issues, we propose a novel model named MoCA, which incorporates multi-stage domain pretraining and multimodal cross attention for the TQA task. Firstly, we introduce a multi-stage domain pretraining module to conduct unsupervised post-pretraining with the span mask strategy and supervised pre-finetune. Especially for domain post-pretraining, we propose a heuristic generation algorithm to employ the terminology corpus. Secondly, to fully consider the rich inputs of context and diagrams, we propose cross-guided multimodal attention to update the features of text, question diagram and instructional diagram based on a progressive strategy. Further, a dual gating mechanism is adopted to improve the model ensemble. The experimental results show the superiority of our model, which outperforms the state-of-the-art methods by 2.21% and 2.43% for validation and test split respectively.

The Ordovician acritarch genus Barakella Cramer & Diez 1977 and two species, B. felix and B. fortunata, bearing characteristic anastomosing filamentous elements at one pole were first described from the subsurface of the Kasba Tadla Basin, Morocco. Subsequently, another species was described from the South Chinese Ordovician and was later attributed to the genus as Barakella rara. New material from the type locality in Morocco and additional sections in South China are used for a review of the genus Barakella. Based on this review, including statistical analysis, Barakella rara is considered a junior synonym of Barakella felix. Barakella is a useful taxon for biostratigraphical applications in the Ordovician. Its first appearance datum (FAD) lies in the early Floian (Early Ordovician, time slices 2a-2b) in both South China and Wales. Except for a doubtful record from Sweden, all records of Barakella are from the peri-Gondwana margin, making the genus an indicator of the peri-Gondwana palaeoprovince. The appearance of Barakella in South China indicates that Barakella possibly originated in shallow-water nearshore environments, and later spread to offshore environments.

With the development of social and the process of informatization, various types of computer systems have been widely used. However, the environmental impact of computer systems is also becoming increasingly prominent. Proposal of the concept of Green Computing is in order to save energy, improve efficiency, and achieve environmental protection and energy saving. This paper proposed the concept of Green Computing and Green Computing system based on reviewing the origin of the Green Computing. Then, the principles of Green Computing evaluation were analyzed and the index system of Green Computing systems evaluation was established. In the subsequent sections, Green Computing system evaluation process was proposed and the specific evaluation methods using Fuzzy Synthetic Estimation with the practical examples validated were given.

A moderately diverse acritarch and prasinophyte assemblage is described from the Dacao and Yingpan formations of the Houping section, Chengkou, Chongqing, southwestem China (Upper Yangtze platform). The palynomorph assemblage consists of 41 species attributed to 25 genera (39 species and 23 genera of acritarchs and two species and two genera of prasinophytes). It is dominated by Polygonium, Leiosphaeridia, Rhopaliophora, and Peteinosphaeridium.The first appearances of several taxa show a potential for stratigraphical correlation in the Floian (Early Ordovician). First appearance data (FADs) of Aureotesta, Petaloferidium, Striatotheca, the Veryhachium lairdii group and the V. trispinosum group indicate levels near the base of the Floian, whereas Ampullula, Arbusculidium filamentosum, Coryphidium bohemicum, Sacculidium and Tongzia first appear slightly higher. The acritarch assemblages from ten localities in South China are analysed using principal component and cluster analysis for ecological discussion. Four acritarch associations are distinguished from ten localities and related to different marine environments respectively. (C) 2012 Elsevier B.V. All rights reserved.

Multimedia stream monitoring refers to removing unwanted and malicious records from multimedia streams. In this application, a large number of filtering queries are registered on time-critical multimedia streams. Each filtering query contains multiple meta filters and a meta filter is shared among multiple filtering queries. The filtering queries and meta filters form a bipartite graph, and the objective is to minimize the overall evaluation time of the queries in the bipartite graph. In order to achieve this goal, some heuristic algorithms were proposed to order the shared meta filters in the graph to reduce the overall evaluation cost. While these methods can achieve near-optimal solutions in ideal stream environments that have stationary probability distributions, in this paper we propose an Adaptive Shared-filter Ordering Model (ASOM) for efficient filtering in dynamic data stream environments. To capture new trends and patterns along dynamic data streams, ASOM uses a time-based exponential smoothing forecasting method to adaptively order the shared meta filters for fast estimation. Experiments demonstrate that ASOM outperforms existing heuristic ordering methods in dynamic stream environments.

China's Reality and Global Vision, pp. 227-246 (2011) No AccessEmpirical Study of the Viscous Knowledge Transfer Effectiveness in Software EnterprisesLingling Zhang, Anqiang Huang, Yang Liu, Jun Li, Li Wang, and Yong ShiLingling ZhangManagement School of Chinese Academy of Sciences, Beijing 1001902, ChinaResearch Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China, Anqiang HuangManagement School of Chinese Academy of Sciences, Beijing 1001902, ChinaResearch Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China, Yang LiuManagement School of Chinese Academy of Sciences, Beijing 1001902, China, Jun LiResearch Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, ChinaYingDa TaiHe Property Insurance Co., Ltd. Beijing 100005, China, Li WangBeijing Institute of Development Strategy, Beijing 100081, China, and Yong ShiResearch Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, ChinaCollege of Information Science & Technology, University of Nebraska at Omaha, Omaha, NE, 68182, USAhttps://doi.org/10.1142/9789814287913_0012Cited by:0 (Source: Crossref) PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail Abstract: As knowledge has become the most important resource of an organization, the related knowledge becomes a vital approach to acquire core competitive power. In order to improve knowledge-transfer effectiveness, people have to identify significant influential factors of knowledge transfer and pay special attention to them. Based on existing research results at home and abroad, integrated with W-S-R systematic theory, this paper advances a model of influential factors of knowledge transfer, carries out empirical study of some China software enterprises by handing out questionnaires and gets some influential factors in the real world and some original results. Keywords: Knowledge transferEmpirical studyInfluence factorSoftware industry FiguresReferencesRelatedDetails Recommended China's Reality and Global VisionMetrics History KeywordsKnowledge transferEmpirical studyInfluence factorSoftware industryPDF download