Robust Visual Tracking by Embedding Combination and Weighted-Gradient Optimization

•We propose a negative sample embedding combination network specific for handling the imbalance between positive and negative samples in the tracking-by-detection framework.•We propose a weighted-gradient loss to handle the imbalance between easy and hard samples by balancing the total gradient contributions of them.•We conduct extensive experiments on tracking benchmarks. The results demonstrate that the proposed algorithm improves the performance of the baseline and performs favorably against state-of-the-art trackers.