Mask RFCT: Recursive Fully Convolutional Tracker for Video Instance Segmentation

Video instance segmentation has recently attracted a lot of attention due to the challenge of simultaneous segmentation and tracking in video clips. In this work, we propose a new video instance segmentation method, called Mask RFCT, which performs tracking via a precise feature vector matching by generating feature vectors from a recursive fully convolutional tracking branch. Our main contribution is the novel recursive fully convolutional tracker (RFCT) that generates corresponding feature vector for each instance within a bounding-box, which will be used for the following instance category assignment as tracking process. Model evaluation is conducted on YOUTUBE-VIS dataset. It shows that our Mask RFCT outperforms the baseline model, MaskTrack RCNN, obtaining an absolute gain of 2.8% for mean AP and 2.65% for mean AR. And we also conduct comparison experiments on different trackers we proposed and the baseline. We hope our work will provide a new method for architecture modification and instance tracking.