Refining autonomous vehicle situational awareness due to varying sensor error

Pilots use their senses and training to generate situational awareness (SA). They then use this SA to make sound aeronautical decisions. Autonomous vehicles, by contrast, cannot rely on pilot expertise in off-nominal situations. They must rely on their onboard sensors to build SA of the environment. As these sensors degrade, it is hypothesized that a point exists where the SA generated by these sensors is inadequate to allow the autonomous vehicle to make sound aeronautical decisions. In previous work, a point was defined based on broad assumptions within a modeling and simulation environment (i.e., the error within each sensor was known and not random). This research used a larger data set that contained random errors within the sensors. The data was then used to build predictive equations through a Monte Carlo simulation in the same simulation environment as previous work. While the data showed there was a statistically significant relationship between the error values in each sensor and the fused distance value, the resulting predictive equations were not able to provide adequate SA to make sound aeronautical decisions. This research highlights multiple issues the test and evaluation community will face when trying to develop new techniques for the verification and validation of autonomous systems.