Design and Validation of a Unity-Based Simulation to Investigate Gesture Based Control of Semi-autonomous Vehicles.

The objective of this investigation is to explore the use of hand gestures to control semi-autonomous vehicles. This was achieved through the use of simulations built in Unity and real-life demonstrations. Screen-spaced simulations modeled the control of a recreational quadcopter while Virtual Reality simulations followed by actual demonstrations used a small ground vehicle. The purpose of the actual demonstrations was to validate observations and lessons learned about vehicle control and human performance from the simulations. The investigative process involved identifying natural gestures to control basic functions of a vehicle, matching them to the selected gesture capture technology, developing algorithms to interpret those gestures for vehicle control, and arranging the appropriate visual environment using the Unity game engine to investigate preferred use of those gestures. Observations and participant feedback aided in refining the gesture algorithms for vehicle control and visual information, indicated the simulations provided suitable learning experiences and environments from which to assess human performance. Results indicate that the gesture-based approach holds promise given the availability of new technology.