For the decades that followed the publishing of the Cooper-Harper report that formalized a standard and universally recognized handling qualities pilot rating scale, researchers have sought to correlate pilot compensation—as well as physical and mental workload—with the assigned rating. A quantitative correlation remains elusive. In recent years, new physiological measurement devices have been developed that together with software processing tools can provide accurate measures of psychophysiological measures including cognitive workload, distraction, and high/low engagement based on electroencephalogram (EEG) and electrocardiogram (ECG) measures (i.e., brain waves and heart rate variability). The pilot compensation referred to in the Cooper-Harper scale is also a function of task performance measures that reflect aircraft characteristics and inceptor activity that reflects upon physical workload. Using a new piloted simulation test database generated in Manned Flight Simulator's containerized rotary-wing simulator with ten experienced test pilots, a machine learning-based software algorithm that integrates a disparate mix of pilot-vehicle system, physiological, and task performance measures was used to further develop an approach to predict handling qualities levels and ratings.

In application, ADS-33E-PRF provides the means to effectively predict rotorcraft handling qualities via validated criteria and demonstrate actual handling qualities in flight test using mission task elements (MTE). With decades of successful outcomes achieved by integrated industry and government test teams, international users, and researchers, this approach provides an effective means to evaluate handling qualities of advanced rotorcraft designs. The requirement for at least three test pilot evaluators of each MTE expands the flight hours required for test and hence increases costs. To reduce flight hours required, while maintaining process effectiveness, the Naval Air Warfare Center Aircraft Division (NAWCAD) is interested in better understanding the relationship between pilot workload and assigned handling qualities ratings such that predictive tools, if proven effective, can reduce this burden. To meet this challenge, Systems Technology, Inc. (STI), Charles River Analytics, Inc. (CRA), Mitchell Aerospace Research, and Advanced Brain Monitoring, Inc. joined with NAWCAD to explore through piloted simulation the relationship between physiological measures of pilot workload and assigned pilot ratings as experienced test pilots conducted handling qualities evaluations using three exemplar MTEs. This paper describes the piloted simulation study and summarizes initial results.

View Video Presentation: https://doi.org/10.2514/6.2023-1367.vid Since the first powered flight over a century ago, pilot-induced oscillations (PIO) continue to provide a source of handling qualities cliffs in all forms of piloted aircraft from fixed wing to powered lift. With the introduction of Electronic Flight Control Systems (EFCS), fly-by-wire, a number of high-profile PIO events have been documented, primarily in military aircraft. Despite a lower profile, such events are also found in civil aviation. This paper explores the use of a wavelet scalogram-based metric as Inceptor Peak Power-Phase (IPPP) as applied to a number of civil airplane accidents/incidents in which PIO is considered to be a contributing factor. IPPP is a wavelet scalogram-based metric that uses time-varying frequency responses to identify a PIO signature, that is, pilot-vehicle system behavior wherein a key output response is 180 degrees out of phase with the pilot input at a magnitude and frequency that typically characterizes such events. The metric is applied to data that are obtained by simply digitalizing the airplane accident/incident investigation report in an attempt to identify the actual occurrence of PIO and also to estimate its onset. Examined results include not only suspected PIO cases, but also non-PIO cases, because any successful metric must be able to properly identify both cases.

Led by the Federal Aviation Administration (FAA), research is being conducted to develop certification means of compliance methods for small aircraft including the emerging class of advanced air mobility vehicles that will transition from thrust-borne flight to wing-borne flight and vice versa. These vehicles typically feature highly augmented fly-by-wire flight control systems that offer advanced flight control modes designed for simplified vehicle operations and in many cases completely autonomous flight operations. The emerging means of compliance methods will feature special classes of flight test maneuvers that address the assessments of the vehicle system, flying qualities, handling qualities, and increasing automation. Inspiration for this approach comes from the Army's Aeronautical Design Standard ADS-33-E-PRF that introduced a mission-oriented approach to address handling qualities via predictive requirements and specified flight test maneuvers - Mission Task Elements. Whereas ADS-33E-PRF prescribes methodologies through a procurement process, the FAA seeks a means to determine that a minimum safety standard has been met by an aircraft presented for certification. Thus, the flight test maneuvers as conceived here form a holistic approach to determine acceptable handling qualities via a process that encompasses the period between Type Certification (TC) application to the point where the TC is granted. To facilitate the process further, a team led by Systems Technology, Inc. (STI) is developing a flight test guide that will support the safe and repeatable execution of these flight test maneuvers. This paper provides an overview of not only the holistic approach to handling qualities assessments, but also the key elements of the flight test guide.

Urban Air Mobility represents a revolution in air travel. Novel aircraft designs, combined with increasingly complex levels of automation, will provide opportunities for quick and convenient commuter and air taxi service at an affordable price. As aircraft are introduced that look little like those with which we are familiar, blending elements of vertical and short takeoff-and-landing capability with conventional wingborne flight, civil certification will become more and more challenging. Adding to the challenge will be the increasing use of autonomous control, to the extent that future aircraft will be pilotless. Certification testing of such designs requires not only a thorough understanding of the flight characteristics of the basic vehicle, but an appreciation for the range of possible paths that can be adopted by highly augmented and autonomous control systems. This paper reviews the challenges in certification and discusses three levels of sophistication in the process, matching design steps that are employed in modern aircraft design: analysis, simulation, and flight test. We will suggest that designers of modern aircraft be allowed to apply each of those design steps throughout the certification cycle.

Modern CDCL SAT solvers learn clauses rapidly, and an important heuristic is the clause deletion scheme. Most current solvers have two (or more) stores of clauses. One has ``valuable'' clauses which are never deleted. Most learned clauses are added to the other, with an aggressive deletion strategy to restrict its size. Recent solvers in the MapleSAT family, have comparatively complex deletion scheme, and perform well. Many solvers store only binary clauses permanently, but MapleLCMDistChronoBT stores clauses with small LBD permanently. We report an experimental study of the permanent clause store in MapleLCMDistChronoBT. We observe that this store can get quite large, but several methods for limiting its size reduced performance. We also show that alternate size and LBD based criteria improve performance, while still having large permanent stores. In particular, saving clauses up to size 8, and adding small numbers of high-centrality clauses, both improved performance, with the best improvement using both methods.

In application, the Aeronautical Design Standard for the handling qualities of military rotorcraft, ADS-33E-PRF, provides the means to effectively predict rotorcraft handling qualities via validated criteria and demonstrate actual handling qualities in flight test using mission task elements. Besides a definition, a note that rotorcraft shall have no tendencies, and a note regarding Attitude Command Response-Types and gain bandwidth frequency, the topic of pilotinduced oscillations (PIO) is not addressed via specific criteria or flight test techniques. As the use of full authority fly-by-wire flight control continues to expand in Vertical Takeoff and Landing (VTOL) aircraft, the likelihood of encountering PIO will also expand. In the fixed wing world where PIO has been commonplace, at least in developmental test if not operations, predictive analytical methods that can also be used for detection of PIO in realtime have been developed, which can also be applied to rotorcraft. Furthermore, recent time-frequency domain methods that have been developed to differentiate VTOL piloting techniques are directly applicable to PIO scenarios. Using a flight test database generated with the UH-60L in-flight simulators at the U.S. Naval Test Pilot School (USNTPS), this paper explores the utility of these methods for identifying rotorcraft PIO tendencies.

We review our formalized problem solving context, and describe a scheme for problem solving in this context based on “grounding” or instantiation. Grounding-based schemes work as follows. A pair consisting of a problem specification, which contains quantified variables, and an instance, are mapped on to a quantifier-free representation, which is then solved by a “ground solver”. Grounding-based schemes are the basis of most current, competitive systems for solving search, decision and optimization problems from declarative specifications.

CDCL SAT solvers generate many “learned” clauses, so effective clause database reduction strategies are important to performance. Over time reduction strategies have become complex, increasing the difficulty of evaluating particular factors or introducing new refinements. At the same time, it has been unclear if the complexity is necessary. We introduce a simple online clause reduction scheme, which involves no sorting. We instantiate this scheme with simple mechanisms for taking into account clause activity and LBD within the winning solver from the 2018 SAT Solver Competition, obtaining performance comparable to the original. We also present empirical data on the effects of simple measures of clause age, activity and LBD on performance.

A common topic of discussion among members of the flight-test community is pilot gain. Every seasoned flight-test professional is capable of identifying test pilots that are considered to be either high-gain or low-gain pilots. It is considered a given in the testing world that high-gain pilots are better at exposing shortcomings in aircraft handling qualities, including the potential for explosive pilot-induced oscillation. By inference, low-gain pilots are often judged to be just the opposite. Despite this interpretation, the concept of pilot gain is itself not well understood. How do we define, quantify, and (perhaps most importantly) modify pilot gain? Using recent research data, pilot gain can be identified as an element in the pilot-vehicle closed-loop system. It is demonstrated that pilot gain, in closed-loop control, is a misnomer and that more proper terms might be pilot-vehicle bandwidth or crossover frequency.

Combinatorial problem solving is often carried out by reducing problems to SAT or some other finite domain constraint language. Explicitly defining reductions can be avoided by using so-called “model and solve” systems. In this case the user writes a declarative problem specification in a constraint modelling language, such as MiniZinc. The specification implicitly defines a reduction, which is implemented by the constraint solving system. Unfortunately, reductions can destroy useful instance structure, such has having small treewidth. We show that reductions defined by certain guarded first order formulas preserve bounded treewidth. We also show such reductions can be executed automatically from problem specifications written in a guarded existential second order logic ( ∃ SO) by simple grounding or “flattening” algorithms. Many constraint modelling languages are essentially extensions of ∃ SO, and this result applies to natural, useful, fragments of these languages.

This chapter aims to familiarize flight control system designers with the latest concepts and criteria that have been implemented into, or are planned for, the handling-qualities specifications. It presents these concepts and criteria in the context of functional requirements for the flight control system. The use of the handling-qualities specification as a source of functional requirements for the flight control system is rarely, if ever, carried out in practice. Active control technology (ACT) has become the basis for the flight control system design on many new commercial and military aircraft. The quantitative specification criteria should be used as functional requirements for the design of the flight control system. An understanding of the strengths, weaknesses and idiosyncrasies of each Response-Type is essential for a successful design of the flight control system. Moderate-amplitude maneuvering has become more important for aircraft that employ ACT flight control systems, because actuator rate limiting tends to occur during such maneuvers.

We present a method we call structure-based preferential bumping,as a low-cost way to exploit formula structure in VSIDS-based SAT solvers.We show that the Glucose SAT solver, when modified with preferential bumpingof certain easily identified structurally important variables,out-performs unmodified Glucose on the industrial formulasfrom recent SAT solver competitions.

It is often convenient to let the universe (or part of the universe) of a structure be some prefix of the natural numbers (or the natural numbers plus 0). We will do this frequently in the following notes. When this is the case, we will often assume that our vocabulary includes the standard order < on the natural numbers. When the universe is [n] = {1, 2, . . . , n}, we often assume our vocabulary has the constant symbols 1 and max, which will always be interpreted as 1 and n, and sometimes addition or subtraction of 1.

There are many reasons to think that SAT solvers should be able to exploit formula structure, but no standard techniques in modern CDCL solvers make explicit use of structure. We describe modifications to modern decision and clause-deletion heuristics that exploit formula structure by using variable centrality. We show that these improve the performance of Maple LCM Dist, the winning solver from Main Track of the 2017 SAT Solver competition. In particular, using centrality in clause deletion results in solving 9 more formulas from the 2017 Main Track. We also look at a number of measures of solver performance and learned clause quality, to see how the changes affect solver execution.

Motivated by the question of how to efficiently do model finding or theorem proving for multi-valued logics, we study the relative reasoning power of resolution proofs and a natural family of model-finding algorithms for Signed CNF Formulas. The conflict-driven clause learning (CDCL) algorithm for SAT is the basis of model finding software systems (SAT solvers) that have impressive performance on many families of propositional formulas. CDCL with restarts (CDCL-R) has been shown to have essentially the same reasoning power as unrestricted propositional resolution. More precisely, they p-simulate each other. We show that this property generalizes to two families of Signed CNF formulas, those with unrestricted signs, and those where the truth value set is a lattice and all signs are regular. We show that a natural generalization of CDCL-R to these formulas has essentially the same reasoning power as natural generalizations of resolution found in the literature. Moreover, the algorithm efficiently simulates bounded width resolution in these systems. These families of signed formulas are possible reduction targets for a number of multi-valued logics, and thus this algorithm has potential as a basis for efficient implemented reasoning systems for many multi-valued logics.

To appear in the proceedings of LPAR 21. Solving complex problems can involve non-trivial combinations of distinct knowledge bases and problem solvers. The Algebra of Modular Systems is a knowledge representation framework that provides a method for formally specifying such systems in purely semantic terms. Formally, an expression of the algebra defines a class of structures. Many expressive formalism used in practice solve the model expansion task, where a structure is given on the input and an expansion of this structure in the defined class of structures is searched (this practice overcomes the common undecidability problem for expressive logics). In this paper, we construct a solver for the model expansion task for a complex modular systems from an expression in the algebra and black-box propagators or solvers for the primitive modules. To this end, we define a general notion of propagators equipped with an explanation mechanism, an extension of the alge- bra to propagators, and a lazy conflict-driven learning algorithm. The result is a framework for seamlessly combining solving technology from different domains to produce a solver for a combined system.

We describe a method and prototype tool for purely declarative creation of “solvers” for a wide range of problems where instances are presented as strings. Examples include model finders for logics of moderate expressive power. The method exploits existing specification-based declarative problem-solving systems (“model-and-solve” systems), adding a front-end tool to declaratively map problem instances in arbitrary form into system-specific instance formats. We illustrate application of our prototype tool with examples including graph problems and non-classical propositional logics.

We present an algorithm, CDCL-AMS, for solving Modular Systems consisting of a set of modules where, for each module, we have a simple “black-box” solver. The algorithm is based on the Conflict-Directed Clause Learning algorithm for SAT, and communicates asynchronously with the black-box solvers to accommodate high variability in response latencies.

Conflict-directed clause learning (CDCL) is the basis of SAT solvers with impressive performance on many problems. CDCL with restarts (CDCL-R) has been shown to have essentially the same reasoning power as unrestricted resolution (formally, they p-Simulate each other). We show that this property generalizes to multi-valued CNF formulas. In particular, for Signed (or Multi-Valued) CNF formulas, and Regular Formulas, we show that a natural generalization of CDCL-R to these logics has essentially the same reasoning power as natural generalizations of resolution from the literature. These formulas are possible reduction targets for a number of multi-valued logics, and thus a possible basis for efficient reasoning systems for these logics.