This paper describes a modeling method for predicting a human's task-level intent through the use of Markov Decision Processes. Intent prediction can be used by a robot to improve decision-making when human and robot operate in a shared physical space. This work presumes human and robot goals are independent such that the robot seeks to avoid interfering with the human rather than directly assisting the human. The proposed human intent prediction system transforms goal sequences the human is expected to complete, a limited past action history, and a correlation of observed behaviors with actions into a prediction of the in-progress or next action the humans is most likely to take. An intra-vehicle activity space robotics application example is presented.
A prototype demonstration was created to illustrate a delegation control approach for future unmanned aerial systems (UAS) applications. With the goal of being able to demonstrate a flexible, natural, multi-level architecture for UAS control, the simulation illustrates four different control modes, ranging from manual (pilot controls the vehicle's flight with stick and throttle control) to high level "plays" (pilot's verbal command initiates planning for a series of automated tasks). The present paper will provide a detailed description of the concept demonstration, including key hardware and software components. This demonstration was used to support an 'operator-centered' effort that involves acquiring feedback from operators to guide interface designs for future UAS applications. Feedback obtained to date is summarized in a companion paper titled "Future Unmanned Aerial Systems Control: Feedback on Highly Flexible Operator-Automation Delegation Interface Concept."
We present an idling, dynamic priority scheduling policy for non-preemptive task sets with precedence, wait constraints, and deadline constraints. The policy operates on a well-formed task model where tasks are related through a hierarchical temporal constraint structure found in many real-world applications. In general, the problem of sequencing according to both upperbound and lowerbound temporal constraints requires an idling scheduling policy and is known to be NP-complete. However, we show through empirical evaluation that, for a given task set, our polynomial-time scheduling policy is able to sequence the tasks such that the overall duration required to execute the task set, the makespan, is within a few percent of the theoretical, lowerbound makespan.
A terrain avoidance extension based primarily on the requirements of Terrain Awareness and Warning Systems (TAWS) using US Geological Survey data was recently integrated into an autonomous airborne sense and avoid system (ABSAA) at the University of North Dakota’s Unmanned Aircraft Systems Engineering team (UASE). Given the native capabilities of a system using an Automatic Dependent Surveillance Broadcast (ADS-B) transceiver, a robust terrain avoidance component is an apt extension for ABSAA. This extension was successfully flight tested in August 2011, demonstrating the ability of the autonomous avoidance system to incorporate and balance additional environmental inputs and desired navigational objectives.
Achieving an optimal balance of system autonomy with human interaction is critical for envisioned applications of multi-unmanned vehicle supervisory control. While research to date has helped inform control station interface design in terms of what to automate, when to automate, and how much to automate, more research is needed examining automation levels across tasks and to better understand what factors (personality, arousal, alerting, motivation, etc.) mediate effective automation reliance and automation transference (automation level of one task impacting performance of other tasks). The present experiment employed a multi-unmanned aerial vehicle simulation to support the collection of objective performance data on several mission-related tasks, as well as measures of individual differences (e.g., personality). Additionally, the autonomy level for two primary supervisory control tasks was manipulated. The results of this initial evaluation of personality drivers of supervisory control performance showed that individual difference data can vary as a function of automation configuration and that there is a complex interplay between personality factors and task type that may inform interface design.
This paper describes a particle-filter based visual processing algorithm that detects approaching aircraft in video feed with complex background images. The algorithm isolates moving aerial objects from homographic image differences of successive frames taken from an onboard camera, and characterizes them as approaching aerial vehicles based on the output of a particle filter with multi-threshold binarization. This algorithm is suitable for low-altitude flight tests as it effectively suppresses noise in complex, dynamically shifting background images, and it is less sensitive than other algorithms to color variations and outdoor sunlight conditions. The performance of the algorithm is validated by applying it to the onboard video feed recorded during a circle-turn flight test using two UAVs.
Unmanned air systems with video capturing systems for surveillance and visual tracking of ground targets have worked relatively well when employing gimbaled cameras controlled by two or more operators: one to fly the vehicle, and one to orient the camera and visually track ground targets. However, autonomous operation to reduce operator workload and crew levels is more challenging when the camera is strapdown, or fixed to the airframe without a pan-and-tilt capability, rather than gimbaled, so that the vehicle must be steered to orient the camera field of view. Visual tracking becomes even more difficult when the target follows an unpredictable path. This paper investigates a machine learning algorithm for visual tracking of stationary and moving ground targets by unmanned air systems with nongimbaling, fixed pan-and-tilt cameras. The algorithm is based on Q learning, and the learning agent initially determines an offline control policy for vehicle orientation and flight path such that a target can be tracked in the image frame of the camera without the need for operator input. Performance of the control policy is demonstrated with simulation test case scenarios for stationary, linear, and random moving targets with changes in target speeds and trajectories. Monte Carlo results presented in the paper demonstrate that the learned policies are capable of tracking stationary and moving targets with path perturbations, provided the perturbations are small. The learned policies are robust to small changes in target trajectory; therefore, learning separate policies for every type of trajectory is not required. The approach is judged to have merit for autonomous visual tracking of both fixed and randomly moving ground targets.
This paper employ a simulation technique to judge accurately the capability of a SAM system to intercept a fighter aircraft in its course of flight. This analysis is automated in a closed loop requiring extensive calculations of the points of interception for the fighter aircraft using the features of commercial software MATLAB. A region is formulated by taking different points and checking the proximity hit of the SAM system with the Aircraft. Every point is tested as the initial flight locked on by the SAM system and its launch. The point is chosen with keeping in mind the maximum aircraft capability with its maximum rate of climb and vehicle horizontal velocity. The initial points of the simulation are each started with the Aircraft’s best abilities, keeping in mind the rate of climb and the maximum velocity. Two anti-parallel tangents for the region of influence are drawn at each point keeping in view of the aircrafts capabilities. The aircraft is simulated to follow those tangential lines with the velocities it has. If the SAM system intercepts both the tangential, then a confirm interception of that system on that point is acknowledged and then that point is locked and the simulation moves on to the next point. The simulation starts at a guessed initial point from where the SAM locks onto the aircraft. If the interception occurs, a point further away from the SAM system is chosen, the target would be failed to intercept. When the initial point of the simulation is captured successfully, another point near the first point is taken with the same hypothesis. After optimizing on to the next point, the simulation continues to capture the next furthest points of interceptions. This process goes on to the time that every point has been captured forming a closed figure of points. These points are then linked together forming a two-dimensional figure. The key results of the extensive simulation are presented numerically with the help of graphs and discussions are made on the results to provide contribution to the current literature.
Over the past decade, the tasks of autonomous localization and tracking of mobile ground targets using cooperative, multiple small unmanned aerial vehicles (UAVs) have been gaining an increasing amount of interest among researchers in the UAV community. Robust solutions have been elusive due to a number of challenges including sudden, unpredictable route changes of targets; inaccurate computation of small platform attitudes; and limited sensor field of views. In our previous works1,2 we demonstrated the merits of the Out-Of-Order Sigma-Point Kalman Filter (O3SPKF) and the concept of Sensor Fusion Quality (SFQ) as multiple UAVs effectively tracked and located mobile ground targets moving in a linear fashion. We showed that the O3SPKF enables UAVs to optimally incorporate randomly time-delayed sensor information, also called out-of-order data, while the SFQ technique removes invalid sensor data, enhancing the target localization accuracy for linearly moving targets. In this paper we present an improved O3SPKF/SFQ method that uses both linear and circular target tracking models to geo-locate mobile targets with non-linear motions.
The ability to reason within an Artificially Intelligent System (AIS) denotes the ability to infer about information, knowledge, observations, and experiences, and affect changes within the AIS that enable it to perform new tasks previously unknown or to perform tasks already learned more efficiently. The act of reasoning, or inferring, allows the AIS to construct or modify representations of that the AIS is experiencing and learning. Reasoning allows the AIS to fill in skeletal or incomplete information or specifications about one or more of its domains (selfassessment). Presented here is a discussion of reasoning within the AIS that takes the form induction and abduction:
Robonaut, a humanoid robot, was launched to the International Space Station (ISS) in 2011. The purpose of this mission is to demonstrate the ability of a humanoid robot to assist astronauts in both IVA and EVA tasks. Various levels of supervised autonomous controls are normally employed to manage the activities of the robot. However, in some cases, controlling the robot by commanding motions that mimic the motions of a human operator, called teleoperation, is desirable. The teleoperation control method puts the human directly in the control loop to perceive and understand the robot's environment and take desired actions within the workspace of the robot. Robonaut is continuously expanding its operational envelope through continued checkout and experiments with the goal of routinely performing tasks that range from mundane (e.g. cleaning and other housekeeping tasks) to risky (e.g. EVA duty). The Robonaut Teleoperation System (RTS) has been shown to be a useful tool for handling unexpected or unique circumstances in the robot's workspace and allowing the crew to perform 'hands on' operations remotely. The paper discusses the development of the RTS including design considerations for use in space, software approach, and crew training for ISS operations. As with any system on the ISS, special considerations were made to ensure the safety of the crew, the robot, and the Space Station. Issues related to the flight certification of this system are also addressed.
This paper describes the design and development of a system to provide a Space Plugand-play Avionics (SPA) interface in a CubeLab form-factor to extend the capabilities of the International Space Station (ISS) to interface directly with SPA-1 compliant devices. The CubeLab is a new payload standard for access to the ISS for small, rapid turn-around microgravity experiments. CubeLabs are small (less than 16”x8”x4” and under 10kg) modular payloads that interface with the NanoRacks Platform aboard the ISS, receive power and transfer data using plug-and-Play Universal Serial Bus (USB) standard. The SPA architecture is a modular technology for spacecraft that provides an infrastructure for modular satellite components to reduce the time to orbit and development costs for satellites. The new system described in this paper allows developers to easily operate their SPA-1 based experiments and payloads aboard the ISS. In addition, developers of new SPA-1 devices can rapidly access the microgravity environment of space.
In the current experiment, we simulated a military multitasking environment and evaluated the effects of RoboLeader on the performance of human operators (i.e., vehicle commanders) who had the responsibility of supervising the plans/routes for a convoy of three vehicles while maintaining proper 360° local security around their own vehicle. We evaluated whether – and to what extent – operator individual differences (spatial ability, attentional control, and video gaming experience) impacted the operator’s performance. In two out of three mission scenarios, the participants had access to the assistance of an intelligent agent, RoboLeader. Results showed that RoboLeader’s level of autonomy had a significant impact on participants’ concurrent target detection task performance and perceived workload. Those participants who played action video games frequently had significant better situation awareness of the mission environment. Those participants with lower spatial ability had increasingly better situation awareness as RoboLeader’s level of autonomy increased; however, those with higher spatial ability did not exhibit the same trend.
In contrast to traditional parabolic dish antennas which must be mechanically steered to point at satellites, phased array antennas operate by electronically activating subarrays in configurations that can be maneuvered across the surface of the antenna without any physical movement. This leads to many benefits including an increase in capacity from the fact that multiple active areas can be enabled simultaneously on the same phased array. Under this concept, a single antenna can support multiple simultaneous contacts, although there are still constraints specific to any implementation. The phased array is made up of subarrays with Transmit and Receive modules, which present their own specific limitations. In one implementation, an individual Transmit / Receive module can simultaneously support two Receive beams from two distinct satellites, but only one Transmit beam. As the active areas for separate beams move across the surface of the antenna, the conditions where they overlap may overload specific modules in the overlapping area. Thus when constructing automated logic for allocating supports to antennas, a predictive compatibility assessment mechanism is required to determine if a trial allocation will lead to conditions that violate the constraints of the antenna hardware. When two or more supports will be active on the same phased array, a predictive assessment must consider their active areas and their paths across the surface over time to identify conflicts and then determine if such conflicts can be remedied. A phased array antenna allows active areas to be shifted away from their optimal positions, as long as they are also increased in size to compensate. This paper describes a central piece of the assessment mechanism implemented in conjunction with an automated scheduling algorithm, which predicts whether beam conflicts will occur, and whether they can be deconflicted with changes in position and size. A series of small experiments was conducted to determine boundary conditions for the deconfliction algorithm, such as a threshold number of iterations for incrementally separating beams, and impacts from the positions of active areas such as proximity to an edge of the antenna. These experiments also explored how the deconfliction thresholds change with different combinations of larger and smaller active areas. This paper summarizes the results of these experiments, and also related elements of the algorithm such as the information preserved from the deconfliction process to inform the logic for trying alternative allocations when necessary. Finally, this paper also presents results from a larger experiment conducted to compare performance in two scenarios – a baseline scenario with only parabolic antennas, and an alternate scenario with phased array antennas substituted at several ground stations. Using a sample set of satellite support requests over a 24 hour period, we compare overall performance in the phased array scenario with the baseline, in terms of the ability to satisfy requests in each case via the automated scheduling algorithm.
Adaptive systems are critical for future space and other unmanned and intelligent systems. Verification of these systems is also critical for their use in systems with potential harm to human life or with large financial investments. Due to their nondeterministic nature and extremely large state space, current methods for verification of software systems are not adequate to provide a high level of assurance for them. The combination of stabilization science, high performance computing simulations, compositional verification and traditional verification techniques, plus operational monitors, provides a complete approach to verification and deployment of adaptive systems that has not been used before. This paper gives an overview of this approach.
This paper presents a description of the difficulties for automatic optimized scheduling for the Space Surveillance Network (SSN) and then describes a design and algorithm to overcome those difficulties. These difficulties includes a large number of objects to track, which requires an even larger number of observations to schedule, different types of observation tasks, different regimes of space (near earth and deep space) and different types of requirements for the different types of objects (e.g., some have frequent visit requirements). The solution must therefore be similarly multifaceted with different algorithms appropriate for the different types of regimes while still being combined into a single whole.
Because metallic aircraft components are subject to a variety of in-service loading conditions, predicting their fatigue life has become a critical challenge. To address the failure mode mitigation of aircraft components and at the same time reduce the life-cycle costs of aerospace systems, a reliable prognostics framework is essential. In this paper, a hybrid prognosis model that accurately predicts the crack growth regime and the residual-useful-life estimate of aluminum components is developed. The methodology integrates physics-based modeling with a data-driven approach. Different types of loading conditions such as constant amplitude, random, and overload are investigated. The developed methodology is validated on an Al 2024-T351 lug joint under fatigue loading conditions. The results indicate that fusing the measured data and physics-based models improves the accuracy of prediction compared to a purely data-driven or physics-based approach.
This paper describes a vision-based positioning system for Unmanned Aerial Vehicles. The proposed method is matching landmarks found in an aerial image to a reference database and uses the match data to estimate the current position of the air vehicle. Research in the area has focused on matching raw aerial image data to a reference data set. While these methods can be designed to provide acceptable results in specific scenarios they struggle with variations in lighting, seasonal changes and changing environments. We present a new multi-stage method that aims to overcome these challenges by analysing the image for key features that can be matched to known ground objects. The new approach can be divided into a set of sub-problems: Detection, Fingerprinting, Matching and State Estimation. Results from the system show that it can accurately detect and match landmarks to a prior database, enabling the system to determine the position of the vehicle. The proposed system is more flexible than current methods as the core methods are independent of the sensor used for detection and can be used with any type of landmarks. Therefore the system can be applied to a wide range of problems, from pure vision-based navigation for small UAVs to planetary exploration.
Airborne Obstacle Tracking has a significant role onboard Unmanned Aerial Systems. The primary function of this system is to detect and track location of obstacles in the flight path in an accurate and timely manner. Advanced filtering methodologies, such as Particle Filters, can provide very accurate estimates of the target state when the aircraft trajectories are described by non-linear dynamic models and linear filters could cause a loss of accuracy. The paper focuses on algorithm and test results from an Obstacle Tracking system based on Particle Filter. A customized version of the Particle Filter has been developed by exploiting data acquired during flight tests by means of a Very Light Aircraft in the framework of a research project carried out by the Italian Aerospace Research Centre and the University of Naples “Federico II”. First of all, a brief description of the hardware/software architecture installed onboard the aircraft is presented. Then, the Particle Filter performance is analyzed in order to point out the impact of non-linear filters on the estimate of the Distance at Closest Point of Approach.
