Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera
CoRR(2024)
摘要
We present a lightweight and affordable motion capture method based on two
smartwatches and a head-mounted camera. In contrast to the existing approaches
that use six or more expert-level IMU devices, our approach is much more
cost-effective and convenient. Our method can make wearable motion capture
accessible to everyone everywhere, enabling 3D full-body motion capture in
diverse environments. As a key idea to overcome the extreme sparsity and
ambiguities of sensor inputs, we integrate 6D head poses obtained from the
head-mounted cameras for motion estimation. To enable capture in expansive
indoor and outdoor scenes, we propose an algorithm to track and update floor
level changes to define head poses, coupled with a multi-stage
Transformer-based regression module. We also introduce novel strategies
leveraging visual cues of egocentric images to further enhance the motion
capture quality while reducing ambiguities. We demonstrate the performance of
our method on various challenging scenarios, including complex outdoor
environments and everyday motions including object interactions and social
interactions among multiple individuals.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要