Monocular dense reconstruction plays more and more important role in AR application. In this paper, we present a new reconstruction system, which combines the Direct Sparse Odometry (DSO) and dense reconstruction into a uniform framework. The DSO can successfully track and build a semi-dense map even in low texture environment. The dense reconstruction is built on the fast superpixel segmentation and location consistency. However, a big gap between the semi-dense map and the dense reconstruction still needs to be bridged. To this end, we develop several elaborate methods including map points selection strategy, container for data sharing, and coordinate system transforming. We compare our system with a state-of-the-art monocular dense reconstruction system DPPTAM. The comparison experiments run on the public monocular visual odometry dataset. The experimental results show that our system has better performance and can run robustly, effectively in indoor and outdoor scenarios.
Face recognition is a challenging issue due to various appearances under different conditions of the face of a person. Meanwhile, conventional face representation methods always lead to high computational complexity. To overcome these shortcomings, in this paper, we propose a novel discriminative projection and representation method for face recognition. This method tries to seek a discriminative representation of the face image on a low-dimension space. Our method consists of two stages, namely face projection and face representation. In the face projection stage, a mapping matrix is produced by jointly maximizing the covariance of dissimilar samples and minimizing the covariance of similar samples. In the face representation stage, the representation result for each face image is obtained by minimizing the sum of representation results of each class. The proposed method achieves two-fold discriminative properties and provides a computational efficient algorithm. The experiments evaluated on diverse face datasets demonstrate that the proposed method has great superiority for face recognition task.
This paper addresses the problem of generating a super-resolution (SR) image from a low-resolution (LR) image assisted by nearby high-resolution (HR) image. The scaling factor of the super-resolved image is up to 8 times and even more, which is much larger than the ordinary super-resolution scaling factor. Combined patch match and learning based method for image super-resolution using a cross-resolution input. The method is used to super-resolve the images captured by a hybrid light field system consisting of a standard LF camera and a HR DSLR camera. We take the central high-resolution image as a reference, to deal with the around low-resolution images. Unlike other relative algorithm, the proposed method is exploited for the existence of large parallax between the captured images. The main process of our method is a combination of patch based (i.e., example based) algorithm and learning based (e.g., convolutional neural network) method, and does not require any calibration information. Experimental results show that our proposed method performs better than existing method on challenging scenes containing complex texture, specularity and large parallax. Both accuracy and visual improvements in our results are noticeable.
Compared to ground-based observation, space-based observation is an effective approach to catalog and monitor increasing space targets. In this paper space target detection in video satellite image with star image background is studied. An adaptive space target detector based on prior information is proposed. Firstly, bilateral filter is used to decrease noise. Then a single frame image is segmented using adaptive thresholding. Considering the continuity of target motion and brightness change, adaptive thresholding is based on local image properties and prior information of previous frames detection and Kalman filter. Then the algorithm uses the correlation of target motion in multi-frame to detect the target from stars. Experimental results with video image from Tiantuo-2 satellite show that this algorithm provides a good way for space target detection.
In this paper, a saliency fusion based content-based image retrieval method is proposed. Different saliency detection methods were conducted firstly and the output saliency maps were fused by double low rank matrix recovery method. Then the images were segmented into foreground and background according to the fusion result. As the foreground and background had the different impacts on the semantic understanding of the image, different features represented in the form of histogram were extracted. Finally, a fusion of z-score normalized Chi-Square distance is adopted as the similarity measurement. This proposal has been implemented on three widely used benchmark databases and the results evaluated in terms of mean Average Precision (mAP), precision, recall, and F1-measure show that our proposal outperforms the referred state-of-the-art approaches.
In this paper, we propose a new method for template matching method with rotation invariance. Our template matching can not only find the location of the object, but also annotate its rotation angle. The key idea is to firstly rectify the local rotation patches according to their intensity centroids, and then to find the corresponding patch-features between template and target images under an iterative matching framework. We adopt the coarse-to-fine search ways, so the patch size should be updated accordingly, which is time-consuming. To tackle this problem, we use the integral image to update the intensity centroid to accelerate the computing speed. The corresponding feature matching is based on the Best-Buddies Pairs (BBPs), which is robust to the non-rigid transform of local range and outliers. Experimental results demonstrate the effectiveness and robustness of the proposed algorithm.
Considering the problems that camera imaging model is complex and operation is complicated, a binocular camera calibration method of RBF neural network based on k-means and gradient method is proposed in this paper. The data center selection method based on the law of clustering error function can obtain hidden nodes and data centers of RBF network accurately. Dynamic learning of data centers, spread constants and weight values based on gradient method can contribute to improving the precision. Experimental results show that the proposed method has high precision and can be well applied in machine vision.
Visual question answering combines the fields of computer vision and natural language processing. It has received much attention in recent years. Image question answering (Image QA) targets to automatically answer questions about visual content of an image. Different from Image QA, video question answering (Video QA) needs to explore a sequence of images to answer the question. It is difficult to focus on the local region features which are related to the question from a sequence of images. In this paper, we propose a forget memory network (FMN) for Video QA to solve this problem. When the forget memory network embeds the video frame features, it can select the local region features that are related to the question and forget the irrelevant features to the question. Then we use the embedded video and question features to predict the answer from multiple-choice answers. Our proposed approaches achieve good performance on the MovieQA [21] and TACoS [28] dataset.
Discriminatively learned correlation filters (DCF) have been widely used in online visual tracking filed due to its simplicity and efficiency. These methods utilize a periodic assumption of the training samples to construct a circulant data matrix, which is also introduces unwanted boundary effects. Spatially Regularized Correlation Filters (SRDCF) solved this issue by introducing penalization on correlation filter coefficients. However, which breaks the circulant structure used in DCF. We propose Faster SRDCF (FSRDCF) via reintroduction of circulant structure. The circulant structure of training samples in the spatial domain is fully used, more importantly, we exploit the circulant structure of regularization function in the Fourier domain, which allows the problem to be solved more directly and efficiently. Our approach demonstrates superior performance over other non-spatial-regularization trackers on the OTB2013 and OTB2015.
With the development of consumer light field cameras, the light field imaging has become an extensively used method for capturing the three-dimensional appearance of a scene. The depth estimation often requires a dense sampled light field in the angular domain or a high resolution in the spatial domain. However, there is an inherent trade-off between the angular and spatial resolutions of the light field. Recently, some studies for super-resolving the trade-off light field have been introduced. Rather than the conventional approaches that optimize the depth maps, these approaches focus on maximizing the quality of the super-resolved light field. In this article, we investigate how the depth estimation can benefit from these super-resolution methods. Specifically, we compare the qualities of the estimated depth using (a) the original sparse sampled light fields and the reconstructed dense sampled light fields, and (b) the original low-resolution light fields and the high-resolution light fields. Experiment results evaluate the enhanced depth maps using different super-resolution approaches.
With mass data collected by seafloor observation networks, an autonomous system which helps to annotate these pictures are in great demand. In this paper, we study the relationship between the network architecture and the classification accuracy for the Plankton Dataset collected by Oregon State University’s Hatfield Marine Science Center. We use multiple classic deep convolutional neural networks (CNN) models to compare the benefit and cost of deeper models which have performed quite well in ImageNet Large Scale Visual Recognition Challenge (ILSVRC) ( http://www.image-net.org/challenges/LSVRC ) competitions and we discover a hidden degeneration phenomenon. Then we conclude some skills to make CNN smaller and finally propose a more efficient network architecture whose model is much smaller (only 1.5 MB), faster (32.2 fps) and achieve a top-5 accuracy of 96
Motion model and model updater are two necessary components for online visual tracking. On the one hand, an effective motion model needs to strike the right balance between target processing, to account for the target appearance and scene analysis, and to describe stable background information. Most conventional trackers focus on one aspect out of the two and hence are not able to achieve the correct balance. On the other hand, the admirable model update needs to consider both the tracking speed and the model drift. Most tracking models are updated on every frame or fixed frames, so it cannot achieve the best performance. In this article, we solve the motion model problem by collaboratively using salient region detection and image segmentation. Particularly, the two methods are for different purposes. In the absence of prior knowledge, the former considers image attributes like color, gradient, edges, and boundaries then forms a robust object; the latter aggregates individual pixels into meaningful atomic regions by using the prior knowledge of target and background in the video sequence. Taking advantage of their complementary roles, we construct a more reasonable confidence map. For model update problems, we dynamically update the model by analyzing scene with image similarity, which not only reduces the update frequency of the model but also suppresses the model drift. Finally, we use these improved building blocks not only to do comparative tests but also to give a basic tracker, and extensive experimental results on OTB50 show that the proposed methods perform favorably against the state-of-the-art methods.
High-quality insulator region proposals play important roles in the process of transmission line inspection images. A generation method of insulator region proposals based on edge boxes is proposed in this paper, and edge boxes are applied to the localization of insulators in inspection images creatively. We take a series of operations to generate insulator region proposals: K-means cluster is used on curvature scale space (CSS) points extracted from edge images, the most appropriate cluster number is chosen, and the circle is drawn on the insulator subclass. We consider the characteristics of insulators’ edge images, and combine these characteristics with edge boxes. As a result, more insulator region proposals are displayed. The experimental results show that our method can effectively reduce the interference area, meanwhile, has high quality of region proposals with fast calculation speed.
Existing projection-based person re-identification methods usually suffer from long time training, high dimension of projection matrix, and low matching rate. In addition, the intra-class instances may be much less than the inter-class instances when a training dataset is built. To solve these problems, a novel relative distance metric leaning based on clustering centralization and projection vectors learning (RDML-CCPVL) is proposed. When constructing training dataset, the images of a same target person are clustering centralized with FCM. The training datasets are built by these clusters in order to alleviate the imbalanced data problem of the training datasets. In addition, during learning projection matrix, the resulted projection vectors can be approximately orthogonal by using an iteration strategy and a conjugate gradient projection vector learning method to update training datasets. The advantage of this method is its quadratic convergence, which can promote the convergence. Experimental results show that the proposed algorithm has higher efficiency. The matching rate can be significantly improved, and the time of training is much shorter than most of existing algorithms of person re-identification.
Traffic sign recognition (TSR) is an important component of automated driving systems. It is a rather challenging task to design a high-performance classifier for the TSR system. In this paper, we propose a new method for TSR system based on deep convolutional neural network. In order to enhance the expression of the network, a novel structure (dubbed block-layer below) which combines network-in-network and residual connection is designed. Our network has 10 layers with parameters (block-layer seen as a single layer): the first seven are alternate convolutional layers and block-layers, and the remaining three are fully-connected layers. We train our TSR network on the German traffic sign recognition benchmark (GTSRB) dataset. To reduce overfitting, we perform data augmentation on the training images and employ a regularization method named “dropout”. The activation function we employ in our network adopts scaled exponential linear units (SELUs), which can induce self-normalizing properties. To speed up the training, we use an efficient GPU to accelerate the convolutional operation. On the test dataset of GTSRB, we achieve the accuracy rate of 99.67%, exceeding the state-of-the-art results.
Conventional compressive sensing (CS) reconstruction is very slow for its characteristic of solving an optimization problem. Convolutional neural network can realize fast processing while achieving comparable results. While CS image recovery with high quality not only depends on good reconstruction algorithms, but also good measurements. In this paper, we propose an adaptive measurement network in which measurement is obtained by learning. The new network consists of a fully-connected layer and ReconNet. The fully-connected layer which has low-dimension output acts as measurement. We train the fully-connected layer and ReconNet simultaneously and obtain adaptive measurement. Because the adaptive measurement fits dataset better, in contrast with random Gaussian measurement matrix, under the same measurement rate, it can extract the information of scene more efficiently and get better reconstruction results. Experiments show that the new network outperforms the original one.
One of the major challenges in object detection is to propose detectors with highly accurate localization of objects. The online sampling of high-loss region proposals (hard examples) uses the multitask loss with equal weight settings across all loss types (e.g., classification and localization, rigid and non-rigid categories) and ignores the influence of different loss distributions throughout the training process, which we find essential to the training efficacy. In this paper, we present the Stratified Online Hard Example Mining (S-OHEM) algorithm for training higher efficiency and accuracy detectors. S-OHEM exploits OHEM with stratified sampling, a widely-adopted sampling technique, to choose the training examples according to this influence during hard example mining, and thus enhance the performance of object detectors. We show through systematic experiments that S-OHEM yields an average precision (AP) improvement of 0.5% on rigid categories of PASCAL VOC 2007 for both the IoU threshold of 0.6 and 0.7. For KITTI 2012, both results of the same metric are 1.6%. Regarding the mean average precision (mAP), a relative increase of 0.3% and 0.5% (1% and 0.5%) is observed for VOC07 (KITTI12) using the same set of IoU threshold. Also, S-OHEM is easy to integrate with existing region-based detectors and is capable of acting with post-recognition level regressors.
A new kind of geometric invariants is proposed in this paper, which is called affine weighted moment invariant (AWMI). By combination of affine differential invariants and the framework of global integral, they can more effectively extract features of images, increase the number of low-order invariants. The experimental results show that AWMIs have good stability and distinguishability. Also, we find that the accuracy of image retrieval and classification has been improved by combining AWMIs with other traditional moment invariants.
We propose a fine-grained visual classification algorithm based on image foreground and sub-category similarity. In the processing of feature extracting, our model calculates the gradient of image pixels in a classification network to obtain the foreground of the image. Then input the foreground image and the original image into the bilinear convolution network to obtain the feature of the image. At the classification stage, we propose an improved SD-SVM algorithm, which takes the advantages of the similarities among sub-categories and the differences among the similarities of sub-category. Experimental results manifest that our algorithm can achieve 85.12% accuracy on the CUB-2011 dataset and 85.21% accuracy on the FGVC-aircrafts dataset even with only the category labels, which outperforms state-of-the-art fine-grained categorization methods.
With the rapid development of light rail transit, tramway detection based on video analysis is becoming the prerequisite and necessary task in driver assistance system. The system should be capable of automatically detecting the trackway using on-board camera in order to determine the train driving limit. However, due to the diversification of ground types, the diversity of weather conditions and the differences in illumination situations, this goal is very challenging. This paper presents a real-time tramway detection method that can effectively deal with various challenging scenarios in the real world of urban rail transit environment. It first uses an adaptive multi-level threshold to segment the ROI of the trolley track, where the local cumulative histogram model is used to estimate the threshold parameters. And then use the regional growth method to reduce the impact of environmental noise and predict the trend of tramway. We have experimentally proved that the method can correctly detect the tramway even in many undesirable situations and use less computational time to meet real-time requirements.
