Improving dynamic gesture recognition in untrimmed videos by an online lightweight framework and a new gesture dataset ZJUGesture

•We propose an online lightweight two-stage framework for dynamic gesture recognition in raw video streams, which can handle untrimmed videos and achieve high precision with a low computational cost.•A detection network that combines the texture and motion features in the untrimmed video stream through RGB images and differential images is introduced to locate gestures in time series. And a classification network is employed to deduce temporal relationships at multiple time scales. Both networks are highly efficient and effective.•We present a new gesture dataset, termed ZJUGesture, which focuses on solving one-handed operation scenarios in practice. This dataset aims to improve the diversity of gestures, like different speeds, illumination changes, and lots of scenarios. Besides, we employ more fine-grained annotation to reduce frequent error responses in real systems.