A study on hand gesture recognition algorithm realized with the aid of efficient feature extraction method and convolution neural networks: design and its application to VR environment

Humans maintain and develop interrelationships through various forms of communication, including verbal and nonverbal communications. Gestures, which constitute one of the most significant forms of nonverbal communication, convey meaning through diverse forms and movements across cultures. In recent decades, research efforts aimed at providing more natural, human-centered means of interacting with computers have garnered increasing interest. Technological advancements in real-time, vision-based hand motion recognition have become progressively suitable for human–computer interaction, aided by computer vision and pattern recognition techniques. Consequently, we propose an effective system for recognizing hand gestures using time-of-flight (ToF) cameras. The hand gesture recognition system outlined in the proposed method incorporates hand shape analysis, as well as robust fingertip and palm center detection. Furthermore, depth sensors, such as ToF cameras, enhance finger detection and hand gesture recognition performance, even in dark or complex backgrounds. Hand shape recognition is performed by comparing newly recognized hand gestures with pre-trained models using a YOLO algorithm-based convolutional neural network. The proposed hand gesture recognition system is implemented in real-world virtual reality applications, and its performance is evaluated based on detection performance and recognition rate outputs. Two distinct gesture recognition datasets, each emphasizing different aspects, were employed. The analysis of results and associated parameters was conducted to evaluate the performance and effectiveness. Experimental results demonstrate that the proposed system achieves competitive classification performance compared to conventional machine learning models evaluated on standard evaluation benchmarks.