Real-time finger motion recognition in free space via skin-compatible electronics

Abstract Interpreting and tracking finger motion in free space are important for human communication in the reality–virtuality continuum. End-user form-factor-free sensors integrated with neuromorphic computing capabilities can expedite the implementation of human–machine interfaces with accurate finger motion recognition. We introduce and design an ultra-flexible TiO2 artificial synapse array and organic proximity sensor conformably attachable to the finger skin surface for real-time finger motion recognition in free space. Ultra-flexible synaptic device and sensor exhibit well-defined synaptic and light-responsive electrical properties, along with mechanical robustness, and the synapse–sensor integration enables optical–electrical signal conversion. Accordingly, various finger motions for time-resolved digit patterns following a Eulerian trail in free space can be learned and recognized with up to ~95 % accuracy for different individuals at varying strains and repeated cycles. Our proposal may contribute to the development of human–machine interaction wearables for extended reality.