Evaluating the Impact of Noisy Point Clouds on Wireless Gesture Recognition Systems

Point cloud data gathered through wireless sensors has garnered increasing attention for its critical applications, including automotive radars, security systems, and notably, gesture recognition. It provides a non-intrusive and robust approach towards human-computer interactions. However, its reliance on real-time data makes resilience of paramount concern and attacks on or imperfections with these sensors can have catastrophic effects. From real-time spoofing to data poisoning attacks or even just faulty data, systems based on 2D and 3D point cloud machine learning models can be extremely vulnerable. Despite this, there exist few studies prioritizing evaluations on the robustness of these systems over noisy time-sensitive point clouds. This study presents an in-depth examination on the effects of noisy data being used in training various millimeter wave based gesture recognition systems. Noisy point clouds can be introduced during the training stage where imperfect data is fed to a model, causing the model to misclassify test-time samples and lowering its overall accuracy. We stage and evaluate the impact of four different, simple data noising scenarios to observe potential vulnerabilities within these systems. Our findings reveal the respective susceptibilities and resiliencies of transformer, long-short term memory, and convolutional models, highlighting the importance to not only dedicate time and research towards innovations in wireless gesture recognition, but also towards optimizing these systems in order to proactively prevent undesirable effects.