Motion-Based Sign Language Video Summarization using Curvature and Torsion
CoRR(2023)
摘要
An interesting problem in many video-based applications is the generation of
short synopses by selecting the most informative frames, a procedure which is
known as video summarization. For sign language videos the benefits of using
the t-parameterized counterpart of the curvature of the 2-D signer's wrist
trajectory to identify keyframes, have been recently reported in the
literature. In this paper we extend these ideas by modeling the 3-D hand motion
that is extracted from each frame of the video. To this end we propose a new
informative function based on the t-parameterized curvature and torsion of
the 3-D trajectory. The method to characterize video frames as keyframes
depends on whether the motion occurs in 2-D or 3-D space. Specifically, in the
case of 3-D motion we look for the maxima of the harmonic mean of the curvature
and torsion of the target's trajectory; in the planar motion case we seek for
the maxima of the trajectory's curvature. The proposed 3-D feature is
experimentally evaluated in applications of sign language videos on (1)
objective measures using ground-truth keyframe annotations, (2) human-based
evaluation of understanding, and (3) gloss classification and the results
obtained are promising.
更多查看译文
关键词
sign language video summarization,curvature,motion-based
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要