The current 3D human pose estimators face challenges in adapting to new datasets due to the scarcity of 2D-3D pose pairs in target domain training sets. We present the Multi-Hypothesis Pose Synthesis Domain Adaptation (PoSynDA) framework to overcome this issue without extensive target domain annotation. Utilizing a diffusion-centric structure, PoSynDA simulates the 3D pose distribution in the target domain, filling the data diversity gap. By incorporating a multi-hypothesis network, it creates diverse pose hypotheses and aligns them with the target domain. Target-specific source augmentation obtains the target domain distribution data from the source domain by decoupling the scale and position parameters. The teacher-student paradigm and low-rank adaptation further refine the process. PoSynDA demonstrates competitive performance on benchmarks, such as Human3.6M, MPI-INF-3DHP, and 3DPW, even comparable with the target-trained MixSTE model. This work paves the way for the practical application of 3D human pose estimation1. The source code is available at https://github.com/hbing-l/PoSynDA.

Existing Visual Object Tracking (VOT) only takes the target area in the first frame as a template. This causes tracking to inevitably fail in fast-changing and crowded scenes, as it cannot account for changes in object appearance between frames. To this end, we revamped the tracking framework with Progressive Context Encoding Transformer Tracker (ProContEXT), which coherently exploits spatial and temporal contexts to predict object motion trajectories. Specifically, ProContEXT leverages a context-aware self-attention module to encode the spatial and temporal context, refining and updating the multi-scale static and dynamic templates to progressively perform accurately tracking. It explores the complementary between spatial and temporal context, raising a new pathway to multi-context modeling for transformer-based trackers. In addition, ProContEXT revised the token pruning technique to reduce computational complexity. Extensive experiments on popular benchmark datasets such as GOT-10k and TrackingNet demonstrate that the proposed ProContEXT achieves state-of-the-art performance 1 .

Recent works on semantic segmentation witness significant performance improvement by utilizing global contextual information. In this paper, an efficient multi-granularity based semantic segmentation network (MGSeg) is proposed for real-time semantic segmentation, by modeling the latent relevance between multi-scale geometric details and high-level semantics for fine granularity segmentation. In particular, a light-weight backbone ResNet-18 is first adopted to produce the hierarchical features. Hybrid Attention Feature Aggregation (HAFA) is designed to filter the noisy spatial details of features, acquire the scale-invariance representation, and alleviate the gradient vanishing problem of the early-stage feature learning. After aggregating the learned features, Fine Granularity Refinement (FGR) module is employed to explicitly model the relationship between the multi-level features and categories, generating proper weights for fusion. More importantly, to meet the real-time processing, a series of light-weight strategies and simplified structures are applied to accelerate the efficiency, including light-weight backbone, channel compression, narrow neck structure, and so on. Extensive experiments conducted on benchmark datasets Cityscapes and CamVid demonstrate that the proposed method achieves the state-of-the-art performance, 77.8%@50fps and 72.7%@127fps on Cityscapes and CamVid datasets, respectively, having the capability for real-time applications.

Image-to-image translation is a fundamental task for a wide range of applications, such as image style transfer, video effect generation, cross-domain retrieval, etc. Due to the limited number of labeled data, complex scenes, abstract semantics and various involved domains, image translation remains a challenging task. Compared to the supervised approaches for image translation that need a large collection of paired images for training, the unsupervised methods can significantly reduce the training cost. In this paper, an unsupervised end-to-end generative adversarial network is proposed, named BranchGAN, for mutual image-to-image transfer between two domains. A structure with one single encoder and dual decoders is novelly proposed to capture the cross-domain distributions and generate the images in both domains. Three factors, that is, pixel-level overall style, region semantics, and domain distinguishability are comprehensively considered to constrain the training process of the proposed model, corresponding to reconstruction loss, encoding loss, and adversarial loss, respectively. Experiments conducted on three benchmark datasets demonstrate the effectiveness of the proposed method that outperforms the unsupervised state-of-the-art approaches and has the competitive performance as the supervised method.

As one of the most common human helminths, hookworm is a leading cause of maternal and child morbidity, which seriously threatens human health. Recently, wireless capsule endoscopy (WCE) has been applied to automatic hookworm detection. Unfortunately, it remains a challenging task. In recent years, deep convolutional neural network (CNN) has demonstrated impressive performance in various image and video analysis tasks. In this paper, a novel deep hookworm detection framework is proposed for WCE images, which simultaneously models visual appearances and tubular patterns of hookworms. This is the first deep learning framework specifically designed for hookworm detection in WCE images. Two CNN networks, namely edge extraction network and hookworm classification network, are seamlessly integrated in the proposed framework, which avoid the edge feature caching and speed up the classification. Two edge pooling layers are introduced to integrate the tubular regions induced from edge extraction network and the feature maps from hookworm classification network, leading to enhanced feature maps emphasizing the tubular regions. Experiments have been conducted on one of the largest WCE datasets with WCE images, which demonstrate the effectiveness of the proposed hookworm detection framework. It significantly outperforms the state-of-the-art approaches. The high sensitivity and accuracy of the proposed method in detecting hookworms shows its potential for clinical application.

In this paper, a deep end-to-end network for sketch recognition, named Deep Visual-Sequential Fusion model (DVSF) is proposed to model the visual and sequential patterns of the strokes. To capture the intermediate states of sketches, a three-way representation learner is first utilized to extract the visual features. These deep features are simultaneously fed into the visual and sequential networks to capture spatial and temporal properties, respectively. More specifically, visual networks are novelly proposed to learn the stroke patterns by stacking the Residual Fully-Connected (R-FC) layers, which integrate ReLU and Tanh activation functions to achieve the sparsity and generalization ability. To learn the patterns of stroke order, sequential networks are constructed by Residual Long Short-Term Memory (R-LSTM) units, which optimize the network architecture by skip connection. Finally, the visual and sequential representations of the sketches are seamlessly integrated with a fusion layer to obtain the final results. Experiments conducted on the benchmark sketch dataset TU-Berlin demonstrate the effectiveness of the proposed method, which outperforms the state-of-the-art approaches.

The high-speed rail system provides a fast, reliable and comfortable means to transport large number of travelers over long distances. The existence of bird nests in overhead catenary system (OCS) can hazard to the safety of the high-speed rails, which will potentially result in long time delays and expensive damages. A vision-based intelligent inspection system capable of automatic detection of bird nests built on overhead catenary would avoid the damages and increase the reliability and punctuality, and therefore is attractive for a high-speed railway system. However, OCS images exhibit great variations with lighting changes, illumination conditions and complex backgrounds, which pose great difficulty for automatic recognition. This paper addresses the problem of automatic recognition of bird nests for OCS images. Based on the unique properties of bird nests, we propose a novel framework, which is composed of five steps: adaptive binarization, trunk/branch detection, hovering point detection, streak extraction and pattern learning, for bird nest detection. Two histograms, Histogram of Orientation of Streaks (HOS) and Histogram of Length of Streaks (HLS), are novelly proposed to capture the distributions of orientations and lengths of detected twig streaks, respectively. They are modeled with Support Vector Machine to learn the patterns of bird nests. Experiments on different high-speed train lines demonstrate the effectiveness and efficiency of the proposed work. The first work to systematically analyze the properties of bird nests in OCS images.The first work to automatically detect the bird nest for high-speed rail system.A five-phase framework is novelly proposed to detect the bird nest.HOS and HLS histograms are proposed to represent the distributions of streaks.Experiments on real high-speed rail lines demonstrate the effectiveness.