Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentation
2020 IEEE Winter Conference on Applications of Computer Vision (WACV)(2018)
摘要
We propose novel Stacked Spatio-Temporal Graph Convolutional Networks (Stacked-STGCN) for action segmentation, i.e., predicting and localizing a sequence of actions over long videos. We extend the Spatio-Temporal Graph Convolutional Network (STGCN) originally proposed for skeleton-based action recognition to enable nodes with different characteristics (e.g., scene, actor, object, action, etc.), feature descriptors with varied lengths, and arbitrary temporal edge connections to account for large graph deformation commonly associated with complex activities. We further introduce the stacked hourglass architecture to STGCN to leverage the advantages of an encoder-decoder design for improved generalization performance and localization accuracy. We explore various descriptors such as frame-level VGG, segment-level I3D, RCNN-based object, etc. as node descriptors to enable action segmentation based on joint inference over comprehensive contextual information. We show results on CAD120 (which provides pre-computed node features and edge weights for fair performance comparison across algorithms) as well as a more complex real-world activity dataset, Charades. Our Stacked-STGCN in general achieves 4.0 performance improvement over the best reported results in F1 score on CAD120 and 1.3
更多查看译文
关键词
skeleton-based action recognition,graph deformation,stacked hourglass architecture,action segmentation,stacked-STGCN,stacked spatio-temporal graph convolutional networks,feature descriptors,temporal edge connections,encoder-decoder design,frame-level VGG,segment-level I3D,RCNN-based object
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络