OpenMarshall: Open-Set Recognition of Aircraft Marshalling Signals for Safe Docking

Aircraft marshallers provide visual docking guidance to the pilots to safely maneuver large aircraft while avoiding unseen obstacles. Although the marshallers are well-trained, there exists a huge gesture variance for the aircraft marshallers worldwide in performing a specific marshalling signal. It creates great difficulty for a pilot from a different geographical location to interpret the desired hand signal. Besides, poor visibility in adverse weather coupled with the high pilot sitting position in big aircraft increases the signal deciphering complexity. To tackle this, we propose a novel high-fidelity gesture-tracking model, OpenMarshall, which extracts salient body joints for marshalling and performs real-time classification of hand signals into a set of known marshalling signals and anomalous gestures with the help of a sequential learning unit. On one hand, our approach can help the Aircraft Marshallers to rectify gestures in their training phase, and on the other hand, vision-based automatic deciphered signals in the cockpit can reduce the workload for a pilot in the docking phase. Additionally, we augment substantial occlusion-simulating scenarios during OpenMarshall training to deal with heavy occlusions in airports. OpenMarshall is a lightweight and generic detector-tracker pipeline that makes it suitable to deploy in platform-agnostic configurations, such as autonomous UAVs, passenger jets, commercial aircraft, etc.