Lightweight convolutional neural networks for player detection and classification.

•We propose an end-to-end framework to jointly detect players and classify their team membership.•In the proposal network, we design a multi-branch network to effectively generate candidate image patches with different aspect ratios.•In the classification network, we design a cascaded CNN and a joint learning objective so that the learned model can quickly reject negative examples and output accurate team membership.•Our trained model is very compact (less than 100KB). It saves 1000 time in memory compared with previous methods without sacrificing performance. It is also very efficient in testing (about 10 fps for images of 1280 × 720 with un-optimized Matlab code).