ROI Pooled Correlation Filters for Visual Tracking

computer vision and pattern recognition, pp.5783-5791, (2019)

Cited by: 37|Views58
EI
Weibo:
We propose the ROI pooled correlation filters for visual tracking

Abstract:

The ROI (region-of-interest) based pooling method performs pooling operations on the cropped ROI regions for various samples and has shown great success in the object detection methods. It compresses the model size while preserving the localization accuracy, thus it is useful in the visual tracking field. Though being effective, the ROI...More

Code:

Data:

0
ZH
Full Text
Bibtex
Weibo
Introduction
  • Visual tracking aims to localize the manually specified target object in the successive frames, and it has been densely studied in the past decades for its broad applications in the automatic drive, human-machine interaction, behavior recognition, etc.
  • The correlation filter (CF) has become one of the most widely used formulas in visual tracking for its computation efficiency.
  • The primal correlation filter algorithms have limited tracking performance due to the boundary effects and the over-fitting problem.
  • Though the boundary effects have been well addressed in several recent papers (e.g., SRDCF [9], DRT [29], BACF [12] and ASRCF [5]), the over-fitting problem is still not paid much attention to and remains to be a challenging research hotspot
Highlights
  • Visual tracking aims to localize the manually specified target object in the successive frames, and it has been densely studied in the past decades for its broad applications in the automatic drive, human-machine interaction, behavior recognition, etc
  • ∗Corresponding Author: Dr Wang lation filter mainly comes from two aspects: first, by exploiting the property of circulant matrix, the CF-based algorithms do not need to construct the training and testing samples explicitly, and can be efficiently optimized in the Fourier domain, enabling it to handle more features; second, optimizing a correlation filter can be equivalently converted to solving a system of linear functions, the filter weights can either be obtained with the analytic solution (e.g., [9, 8]) or be solved via the optimization algorithms with quadratic convergence [9, 7]
  • The phenomenon of boundary effects is caused by the periodic assumptions of the training samples, while the over-fitting problem is caused by the unbalance between the numbers of model parameters and the training samples
  • We study the influence of the pooling operation in visual tracking, and propose a novel ROI pooled correlation filters algorithm
  • Different from the ECO tracker, our method introduces the ROIbased pooling operation into a correlation filter formula, which does address the over-fitting problem and makes the learned filter weights more robust to deformations
  • Our method improves the second best tracker ECO by 1.9% in terms of distance precision (DP) rates, and has comparable performance according to the success plots
  • We propose the ROI pooled correlation filters for visual tracking
Methods
  • The authors evaluate the proposed RPCF tracker on the OTB-2013 [31], OTB-2015 [32] and VOT2017 [20] datasets.
  • The authors follow the one-pass evaluation (OPE) rule on the OTB-2013 and OTB-2015 datasets, and report the precision plots as well as the success plots for the performance measure.
  • On the VOT-2017 dataset, the authors evaluate the tracker in terms of the Expected Average Overlap (EAO), accuracy raw value (A) and robustness raw value (R) measure the overlap, accuracy and robustness respectively
Results
  • The authors' tracker improves the Baseline method by 4.4% and 2.0% in precision and success plots respectively.
  • The authors' method improves the second best tracker ECO by 1.9% in terms of DP rates, and has comparable performance according to the success plots.
  • The authors' RPCF tracker has a 92.9% DP rate and a 69.0% AUC score
  • It improves the second best tracker ECO by 1.9% in terms of the precision plots.
  • Among all the compared trackers, the RPCF method has a 31.6% EAO score which improves the ECO method by 3.5%
Conclusion
  • Since the correlation filter algorithm does not extract real-world training samples, it is infeasible to perform the pooling operation for each candidate ROI region like the previous methods.
  • Based on the mathematical derivations, the authors provide an alternative solution for the ROI-based pooling with the circularly constructed virtual samples.
  • The authors evaluate the proposed RPCF tracker on OTB-2013, OTB-2015 and VOT2017 benchmark datasets.
  • Extensive experiments demonstrate that the method performs favourably against the stateof-the-art algorithms on all the three datasets
Summary
  • Introduction:

    Visual tracking aims to localize the manually specified target object in the successive frames, and it has been densely studied in the past decades for its broad applications in the automatic drive, human-machine interaction, behavior recognition, etc.
  • The correlation filter (CF) has become one of the most widely used formulas in visual tracking for its computation efficiency.
  • The primal correlation filter algorithms have limited tracking performance due to the boundary effects and the over-fitting problem.
  • Though the boundary effects have been well addressed in several recent papers (e.g., SRDCF [9], DRT [29], BACF [12] and ASRCF [5]), the over-fitting problem is still not paid much attention to and remains to be a challenging research hotspot
  • Methods:

    The authors evaluate the proposed RPCF tracker on the OTB-2013 [31], OTB-2015 [32] and VOT2017 [20] datasets.
  • The authors follow the one-pass evaluation (OPE) rule on the OTB-2013 and OTB-2015 datasets, and report the precision plots as well as the success plots for the performance measure.
  • On the VOT-2017 dataset, the authors evaluate the tracker in terms of the Expected Average Overlap (EAO), accuracy raw value (A) and robustness raw value (R) measure the overlap, accuracy and robustness respectively
  • Results:

    The authors' tracker improves the Baseline method by 4.4% and 2.0% in precision and success plots respectively.
  • The authors' method improves the second best tracker ECO by 1.9% in terms of DP rates, and has comparable performance according to the success plots.
  • The authors' RPCF tracker has a 92.9% DP rate and a 69.0% AUC score
  • It improves the second best tracker ECO by 1.9% in terms of the precision plots.
  • Among all the compared trackers, the RPCF method has a 31.6% EAO score which improves the ECO method by 3.5%
  • Conclusion:

    Since the correlation filter algorithm does not extract real-world training samples, it is infeasible to perform the pooling operation for each candidate ROI region like the previous methods.
  • Based on the mathematical derivations, the authors provide an alternative solution for the ROI-based pooling with the circularly constructed virtual samples.
  • The authors evaluate the proposed RPCF tracker on OTB-2013, OTB-2015 and VOT2017 benchmark datasets.
  • Extensive experiments demonstrate that the method performs favourably against the stateof-the-art algorithms on all the three datasets
Tables
  • Table1: Performance evaluation for 10 state-of-the-art algorithms on the VOT-2017 public dataset. The best three results are marked in red, blue and green fonts, respectively
Download tables as Excel
Related work
  • The recent papers on visual tracking are mainly based on the correlation filters and deep networks [21], many of which have impressive performance. In this section, we primarily focus on the algorithms based on the correlation filters and briefly introduce related issues of the pooling operations.

    Discriminative Correlation Filters. Trackers based on correlation filters have been the focus of researchers in recent years, which have achieved the top performance in various datasets. The correlation filter algorithm in visual tracking can be dated back to the MOSSE tracker [2], which takes the single-channel gray-scale image as input. Even though the tracking speed is impressive, the accuracy is not satisfactory. Based on the MOSSE tracker, Henriques et al advance the state-of-the-art by introducing the kernel functions [18] and higher dimensional features [19]. Ma et al [24] exploit the rich representation information of deep features in the correlation filter formula, and fuse the responses of various convolutional features via a coarse-tofine searching strategy. Qi et al [25] extend the work of [24] by exploiting the Hedge method to learn the importance for each kind of feature adaptively. Apart from the MOSSE tracker, the aforementioned algorithms learn the filter weights in the dual space, which have been attested to be less effective than the primal space-based algorithms [8, 9, 19]. However, correlation filters learned in the primal space are severely influenced by the boundary effects and the over-fitting problem. Because of this, Danelljan et al [9] introduce a weighted regularization constraint on the learned filter weights, encouraging the algorithm to learn more weights on the central region of the target object. The SRDCF tracker [9] has become a baseline algorithm for many latter trackers, e.g., CCOT [11] and SRD-
Funding
  • This paper is supported in part by National Natural Science Foundation of China #61725202, #61829102, #61872056 and #61751212, and in part by the Fundamental Research Funds for the Central Universities under Grant #DUT18JC30
  • This work is also sponsored by CCF-Tencent Open Research Fund
Study subjects and analysis
available public datasets: 3
With the computed Lagrangian multipliers, the paper aims to use the conjugate gradient method for filter learning, and develops efficient optimization strategy for each step. • This paper conducts large amounts of experiments on three available public datasets. The experimental results validate the effectiveness of the proposed method

datasets: 3
Finally, we evaluate the proposed RPCF tracker on OTB-2013, OTB-2015 and VOT2017 benchmark datasets. Extensive experiments demonstrate that our method performs favourably against the stateof-the-art algorithms on all the three datasets. Acknowledgement

Reference
  • Luca Bertinetto, Jack Valmadre, Stuart Golodetz, Ondrej Miksik, and Philip HS Torr. Staple: Complementary learners for real-time tracking. In CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • David S. Bolme, J. Ross Beveridge, Bruce A. Draper, and Yui Man Lui. Visual object tracking using adaptive correlation filters. In CVPR, 2010.
    Google ScholarFindings
  • Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning, 3(1):1– 122, 2011.
    Google ScholarLocate open access versionFindings
  • Angelika Bunse-Gerstner and Ronald Stover. On a conjugate gradient-type method for solving complex symmetric linear systems. Linear Algebra and its Applications, 287(1-3):105– 123, 1999.
    Google ScholarLocate open access versionFindings
  • Kenan Dai, Dong Wang, Huchuan Lu, Chong Sun, and Jianhua Li. Visual tracking via adaptive spatially-regularized correlation filters. In CVPR, 2019.
    Google ScholarFindings
  • Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.
    Google ScholarLocate open access versionFindings
  • Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg, et al. Eco: Efficient convolution operators for tracking. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
  • Martin Danelljan, Gustav Hager, Fahad Khan, and Michael Felsberg. Accurate scale estimation for robust visual tracking. In BMVC, 2014.
    Google ScholarLocate open access versionFindings
  • Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg. Learning spatially regularized correlation filters for visual tracking. In ICCV, 2015.
    Google ScholarLocate open access versionFindings
  • Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg. Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking. In CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • Martin Danelljan, Andreas Robinson, Fahad Shahbaz Khan, and Michael Felsberg. Beyond correlation filters: Learning continuous convolution operators for visual tracking. In ECCV, 2016.
    Google ScholarLocate open access versionFindings
  • Hamed Kiani Galoogahi, Ashton Fagg, and Simon Lucey. Learning background-aware correlation filters for visual tracking. In ICCV, 2017.
    Google ScholarLocate open access versionFindings
  • Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • Ross Girshick. Fast r-cnn. In ICCV, 2015.
    Google ScholarLocate open access versionFindings
  • Erhan Gundogdu and A Aydın Alatan. Good features to correlate for visual tracking. IEEE Transactions on Image Processing, 27(5):2526–2540, 2018.
    Google ScholarLocate open access versionFindings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • Zhiqun He, Yingruo Fan, Junfei Zhuang, Yuan Dong, and HongLiang Bai. Correlation filters with weighted convolution responses. In ICCV Workshops, 2017.
    Google ScholarLocate open access versionFindings
  • Joao F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. Exploiting the circulant structure of tracking-bydetection with kernels. In ECCV, 2012.
    Google ScholarLocate open access versionFindings
  • Joao F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3):583–596, 2015.
    Google ScholarLocate open access versionFindings
  • Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman P. Pflugfelder, Luka Cehovin Zajc, Tomas Vojır, and Gustav Hager. The visual object tracking vot2017 challenge results. In ICCV Workshops, 2017.
    Google ScholarLocate open access versionFindings
  • Peixia Li, Dong Wang, Lijun Wang, and Huchuan Lu. Deep visual tracking: Review and experimental comparison. Pattern Recognition, 76:323–338, 2018.
    Google ScholarLocate open access versionFindings
  • David G Lowe. Distinctive image features from scaleinvariant keypoints. International journal of computer vision, 60(2):91–110, 2004.
    Google ScholarLocate open access versionFindings
  • Alan Lukezic, Tomas Vojir, Luka Cehovin Zajc, Jiri Matas, and Matej Kristan. Discriminative correlation filter with channel and spatial reliability. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
  • Chao Ma, Jia-Bin Huang, Xiaokang Yang, and Ming-Hsuan Yang. Hierarchical convolutional features for visual tracking. In ICCV, 2015.
    Google ScholarLocate open access versionFindings
  • Yuankai Qi, Shengping Zhang, Lei Qin, Hongxun Yao, Qingming Huang, Jongwoo Lim, and Ming-Hsuan Yang. Hedged deep tracking. In CVPR, 2016.
    Google ScholarLocate open access versionFindings
  • Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015.
    Google ScholarLocate open access versionFindings
  • Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
    Findings
  • Chong Sun, Huchuan Lu, and Ming-Hsuan Yang. Learning spatial-aware regressions for visual tracking. In CVPR, 2018.
    Google ScholarLocate open access versionFindings
  • Chong Sun, Dong Wang, Huchuan Lu, and Ming-Hsuan Yang. Correlation tracking via joint discrimination and reliability learning. In CVPR, pages 489–497, 2018.
    Google ScholarLocate open access versionFindings
  • Ning Wang, Wengang Zhou, Qi Tian, Richang Hong, Meng Wang, and Houqiang Li. Multi-cue correlation filters for robust visual tracking. In CVPR, 2018.
    Google ScholarLocate open access versionFindings
  • Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Online object tracking: A benchmark. In CVPR, 2013.
    Google ScholarFindings
  • Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1834–1848, 2015.
    Google ScholarLocate open access versionFindings
  • Jianming Zhang, Shugao Ma, and Stan Sclaroff. Meem: robust tracking via multiple experts using entropy minimization. In ECCV, 2014.
    Google ScholarLocate open access versionFindings
  • Tianzhu Zhang, Changsheng Xu, and Ming-Hsuan Yang. Multi-task correlation particle filter for robust object tracking. In CVPR, 2017.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments