On July 14, 2015, the New Horizons mission accomplished the first flyby of Pluto-Charon, achieving full mission success during its primary mission. Less than 4 years later, during its first extended mission, New Horizons flew by Arrokoth, a 36-km contact binary trans-Neptunian object in the Kuiper Belt, on January 1, 2019. Along the way, New Horizons imaged numerous distant Kuiper Belt objects, performed important heliophysics science including complex Lyman-alpha radiation scans, and measured the dust and zodiacal light from regions never before explored. This article provides an overview of the New Horizons spacecraft and its engineering performance, as well as potential strategies for extending the mission far beyond its original design lifetime. Details on the mass and power budgets, as well as descriptions of key innovations to meet the challenges posed by the mission, offer insight into the engineering accomplishments that led to mission success. Trended data on the power, thermal, and propulsion systems substantiate projections of the mission's potential to continue its exploration beyond the heliopause until similar to 2050.

The spacecraft trajectory and the associated course-correction maneuvers provided a primary means for accomplishment of the scientific objectives of the MErcury Surface, Space ENvironment, GEochemistry, and Ranging (MESSENGER) mission. Whereas other articles in this issue offer a quantitative performance assessment of MESSENGER's course-correction maneuvers, this account identifies unique aspects and lessons learned from the examination of the processes and team interactions for maneuver design through maneuver reconstruction at the core of this successful NASA mission. Keys to mission success included forward thinking in the creative use of maneuvers as a means of preparing for future important maneuvers, as well as exercising flexibility to allow change from the nominal plan when this change would either increase scientific return or enable new scientific observations to answer questions that arose during the mission. Cautious use of propellant reserves and a willingness to accept higher risk near the end of flight operations enabled a 3.1-year extension of the yearlong orbital phase of the primary mission.

Two orbit-correction maneuvers (OCMs) were required during MESSENGER's extended mission to reduce the orbital period from 11.6 to 8 hours. The OCMs were designed as a pair to maximize use of the bi-propellant propulsion system. The first maneuver was designed to be flexible to a range of values for the amount of oxidizer remaining in the system. A special autonomy scheme was necessary to respond to oxidizer depletion and continue the maneuver without interruption using only monopropellant thrusters. The second maneuver executed four days later and was designed on the basis of the performance of the first maneuver.

During the Radiation Belt Storm Probes (RBSP) mission, a test framework was developed by the Embedded Applications Group in the Space Department at the Johns Hopkins Applied Physics Laboratory (APL). The test framework is implemented for verification of the Command and Data Handling (C&DH) Flight Software. The RBSP C&DH Flight Software consists of applications developed for use with Goddard Space Flight Center's core Flight Executive (cFE) architecture. The test framework's initial concept originated with tests developed for verification of the Autonomy rules that execute with the Autonomy Engine application of the RBSP C&DH Flight Software. The test framework was adopted and expanded for system and requirements verification of the RBSP C&DH Flight Software. During the evolution of the RBSP C&DH Flight Software test framework design, a set of script conventions and a script library were developed. The script conventions and library eased integration of system and requirements verification tests into a comprehensive automated test suite. The comprehensive test suite is currently being used to verify releases of the RBSP C&DH Flight Software. In addition to providing the details and benefits of the test framework, the discussion will include several lessons learned throughout the verification process of RBSP C&DH Flight Software. Our next mission, Solar Probe Plus (SPP), will use the cFE architecture for the C&DH Flight Software. SPP also plans to use the same ground system as RBSP. Many of the RBSP C&DH Flight Software applications are reusable on the SPP mission, therefore there is potential for test design and test framework reuse for system and requirements verification.

The National Aeronautics and Space Administration’s (NASA’s) Radiation Belt Storm Probe (RBSP) is an Earth-orbiting mission that launched August 30, 2012, and is the latest science mission in NASA’s Living with a Star Program. The RBSP mission will investigate, characterize and understand the physical dynamics of the radiation belts, as well as the influence of the Sun on the Earth’s environment, by measuring particles, electric and magnetic fields and waves that comprise geospace. The mission is composed of two identically instrumented spinning observatories in an elliptical orbit around earth with 600 km perigee, 30,000 km apogee and 10 ∘ inclination to provide full sampling of the Van Allen radiation belts. The twin RBSP observatories (recently renamed the Van Allen Probes) will follow slightly different orbits and will lap each other four times per year, offering simultaneous measurements over a range of observatory separation distances. A description of the observatory environment is provided along with protection for sensitive electronics to support operations in the harsh radiation belt environment. Spacecraft and subsystem key characteristics and instrument accommodations are included that allow the RBSP science objectives to be met.

NASA's Radiation Belt Storm Probe (RBSP) is an Earth-orbiting mission scheduled to launch in September 2012 and is the next science mission in NASA's Living with a Star Program. The RBSP mission will investigate, characterize and understand the physical dynamics of the radiation belts, and the influence of the sun on the earth's environment, by measuring particles, electric and magnetic fields and waves that comprise the geospace. The mission is composed of two identically instrumented spinning spacecraft in an elliptical orbit around earth from 600 km perigee to 30,000 km apogee at 10 degree inclination to provide full sampling of the Van Allen radiation belts. The twin spacecraft will follow slightly different orbits and will lap each other 4 times per year; this offers simultaneous measurements over a range of spacecraft separation distances. A description of the spacecraft environment is provided along with spacecraft and subsystem key characteristics and accommodations that protect sensitive spacecraft electronics and support operations in the harsh radiation belt environment.

This paper describes techniques to design stress tests, classifies the types of problems found during these types of tests, and analyzes why these problems are not discovered with traditional unit testing or acceptance testing. The findings are supported by citing examples from three recent embedded software development programs performed by the Johns Hopkins University Applied Physics Laboratory where formal stress testing was employed. It also defines what is meant by the robustness and elasticity of a software system. These findings will encourage software professionals to incorporate stress testing into their formal software development process. 1. INTRODUCTON Traditional Software Acceptance Testing is a standard phase in nearly every software development methodology. Test engineers develop and execute tests that are defined to validate software requirements. The tests tend to be rigid with specific initial conditions and well-defined expected results. The tests typically execute software within the limits prescribed by the software design. While these tests are often complemented with System Tests or Use Case Tests, these higher level scenarios still conform within the design bounds of the software. OUTLINE 1. INTRODUCTON 1.1 What can be learned by “breaking” the software 2. DESIGNING A STRESS TEST 2.1 What is a reasonable target CPU load? 2.2 Other ways to stress the system 2.3 Characteristics of a Stress Test 3. REAL WORLD RESULTS 3.1 Case #1: Software Missed Receiving Some Commands When CPU Was Heavily Loaded 3.2 Case #2: Processor Reset when Available Memory Buffers Were Exhausted 3.3 Case #3: Unexpected Command Rejection When CPU Was Heavily Loaded 3.4 Case #4: Processor Reset When RAM Disk Was Nearly Full 3.5 Synopsis of All Problems Found During Stress Testing 4. SUMMARY ESC-302 Stress Testing Embedded Software Applications Page 2 of 2 Software Stress Testing, however, runs counter to these traditional approaches as demonstrated in Table 1. Stress testing involves intentionally subjecting software to unrealistic loads while denying it critical system resources. The software is intentionally exercised “outside the box” and known weaknesses and vulnerabilities in the software design may be specifically exploited. Degraded performance of a system under stress may be deemed perfectly acceptable, thus, the interpretation of test results and definition of pass / fail criteria is more subjective. Furthermore, a test that stresses one aspect of the software may lead to undesirable side effects in another area of the software thus the entire system behavior as a whole must be evaluated to properly analyze the results. Software Acceptance Testing Software Stress Testing Black Box Testing. No need to understand software internals Tests are designed to verify that software meets requirements Tests exercise software within acceptable bounds Pass / Fail criteria are clearly defined White Box Testing. Tests target weak spots in the software design Tests attempt to “break” the software Tests intentionally violate constraints to stress software Pass / Fail criteria are subjective Table 1 Comparison of Acceptance Testing versus Stress Testing 1.1 What can be learned by “breaking” the software For critical embedded software applications, the user has an expectation that the software is robust. The dictionary defines robustness as the property of being powerfully built or sturdy; one step below bulletproof. For software, this means that the implementation should demonstrate resiliency. It is permissible for the software to operate in a degraded fashion (e.g., dropping commands or missing data) while under stress but the degradation should be graceful and recoverable. A lack of robustness would lead to unacceptable behavior such as writing corrupted data or resetting the processor. Secondarily, the embedded software should demonstrate elasticity. The dictionary defines elasticity as the property of returning to an initial form or state following deformation. In a software context this means that when the stress is removed the system should return back into its normal operating state and have full functionality restored. A lack of elasticity would leave the software operating in a degraded mode after the stress has been relieved. ESC-302 Stress Testing Embedded Software Applications Page 3 of 3 While stress testing can validate the robustness and elasticity of the software system, that validation may not be enough to justify the time and effort required to develop and execute such tests. However, there is an additional benefit to stress testing that may not be as readily apparent. Stress testing exposes design flaws and implementation bugs that are difficult or impossible to discover under traditional testing approaches. Furthermore, these flaws and bugs are often present even when the system is not under stress. In such cases, the stress test magnifies these inherent problems allowing them to be more easily detected. Stress testing allows these defects to be corrected before the software is deployed into the user community. 2. DESIGNING A STRESS TEST Although there is no single approach to designing a stress test, a recommended approach is shown in Figure 1. With this method, the CPU loading is increased at the start of the overall test. Next, a series of subtests are run against the heavily loaded processor. Each subtest exercises some type of targeted stress. The type of stress is highly dependent upon the system under test and could include items such as overflowing input buffers, simulating rapid keystrokes, etc. It is perfectly valid to inspect the software design and inject stress that targets specific areas such as input queues, interrupt service routines and critical resources. As each subtest executes, a qualitative measure of performance should be validated (i.e., robustness). This requires establishing an acceptable level of degraded behavior for each subtest. Finally, after the series of subtests completes, the CPU is returned to nominal levels and it is verified that the software system returns to its nominal operating state to confirm the elasticity of the system. To simplify development, the subtests can often be developed, debugged and executed “standalone” (while the system is not under stress) to validate the test scripts and establish a baseline behavior before they are incorporated into the larger Software Stress Test. This provides a level of modularity to the overall stress test. Figure 1 Structure of a Software Stress Test 2.1 What is a reasonable target CPU load? One obvious way to place the software under stress is to perform steps to increase the CPU load. Increasing the CPU load is especially important when ESC-302 Stress Testing Embedded Software Applications Page 4 of 4 stress testing pre-emptive multitasking systems. Software designers spend considerable effort assigning task priorities in a pre-emptive system to ensure that all real-time deadlines are met. It has been shown that even if task priorities are poorly assigned, task deadlines will usually be met when the overall CPU load is less than about 85%. Said another way, having 15% overall free CPU load can provide enough margin to mask flaws regarding task priority assignments [LEHOCZKY01]. Since one purpose of the stress test is to expose these flaws, the test should have a target CPU load of 90% or more, if possible. 2.2 Other ways to stress the system Increasing CPU load is just one of the ways to stress a software system. Other stress methods, which are typically employed in the subtests, include: • Maximizing I/O data rates • Maximizing data bus usage • Maximizing interrupt rate • Exhausting available memory • Overflowing queues 2.3 Characteristics of a Stress Test Ideally, stress tests should be scripted and repeatable. While stress testing may seem less precise than traditional testing because of the subjectivity in evaluating system performance and defining acceptable behavior, it does not mean that the test process is any less formal. One possibility with stress testing is that a problem may be observed one time but cannot be repeated. While not a foolproof solution, using scripts that can repeat a test will greatly increase the likelihood that intermittent problems can be recreated improving the probability that engineers can isolate the root cause. Additionally, post-test analysis should always be performed to identify any unexpected anomalies. Checklists can ensure that all necessary data points are verified. If possible, scripting can to verify checklist items during the stress test. Frequently there is a side effect when stressing the system in one area that may have an unexpected effect on another area. Without a rigorous approach to reviewing all data, these side effects may not be observed which undermines a key benefit of Software Stress Testing. 3. REAL WORLD RESULTS The Johns Hopkins University Applied Physics Laboratory (JHU/APL), located in Laurel, Maryland, has developed spacecraft flight software for three recently launched NASA missions. The embedded software is highly critical (a serious fault can result in loss of mission) and is expected to operate continuously for long periods of time (i.e., years) as shown in Table 2. ESC-302 Stress Testing Embedded Software Applications Page 5 of 5 Mission Launch Duration MErcury Surface, Space ENvironment, GEochemistry, and Ranging (MESSENGER) August 2004 8 years New Horizons (Pluto-Kuiper Belt Mission) January 2006 9+ years Solar TErrestrial RElations Observatory (STEREO) October 2006 2+ years Table 2 Recent JHU/APL Software Development Programs All three software development efforts followed a formal software development life cycle including requirements analysis, design, implementation, unit and integration testing, requirements-based testing and system-level acceptance testing. Formal Software Stress Testing was performed on the software near the end of the softwar

The New Horizons spacecraft was launched on 19 January 2006. The spacecraft was designed to provide a platform for seven instruments designated by the science team to collect and return data from Pluto in 2015. The design meets the requirements established by the National Aeronautics and Space Administration (NASA) Announcement of Opportunity AO-OSS-01. The design drew on heritage from previous missions developed at The Johns Hopkins University Applied Physics Laboratory (APL) and other missions such as Ulysses. The trajectory design imposed constraints on mass and structural strength to meet the high launch acceleration consistent with meeting the AO requirement of returning data prior to the year 2020. The spacecraft subsystems were designed to meet tight resource allocations (mass and power) yet provide the necessary control and data handling finesse to support data collection and return when the one-way light time during the Pluto fly-by is 4.5 hours. Missions to the outer regions of the solar system (where the solar irradiance is 1/1000 of the level near the Earth) require a radioisotope thermoelectric generator (RTG) to supply electrical power. One RTG was available for use by New Horizons. To accommodate this constraint, the spacecraft electronics were designed to operate on approximately 200 W. The travel time to Pluto put additional demands on system reliability. Only after a flight time of approximately 10 years would the desired data be collected and returned to Earth. This represents the longest flight duration prior to the return of primary science data for any mission by NASA. The spacecraft system architecture provides sufficient redundancy to meet this requirement with a probability of mission success of greater than 0.85. The spacecraft is now on its way to Pluto, with an arrival date of 14 July 2015. Initial in-flight tests have verified that the spacecraft will meet the design requirements.

NASA's MESSENGER mission, part of its Discovery program, is the first mission to return to the planet Mercury since the Mariner 10 flybys in 1974 and 1975. The spacecraft incorporates many innovative features, including a sunshade made of ceramic cloth for protection from the Sun, a pair of electronically steerable phased-array antennas, and specially hardened solar panels. A suite of seven miniaturized science instruments, along with the antennas, will globally characterize the planet's composition, structure, atmosphere, and charged particle environment. MESSENGER was launched on August 3, 2004, and performed its single Earth flyby on August 2, 2005. The spacecraft will make two flybys of Venus and three of Mercury prior to orbiting the planet for one Earth-year beginning in March 2011. Highlights of a busy first year of flight operations include initial testing of all spacecraft systems and instruments, execution of six trajectory control maneuvers, and instrument observations of the Earth and Moon surrounding the August flyby

The MESSENGER spacecraft, the first mission to the planet Mercury since 1975, will achieve Mercury orbit in 2011. The spacecraft uses two opposite-facing mission-enabling X-band (8.4 GHz) phased-array antennas to achieve high-rate downlink communications. The spacecraft orientation is constrained such that a preferred direction faces the Sun; rotation about the Sun-line is allowable. The main beam of each antenna is steerable in one dimension. These two degrees of freedom allow the main beam of the phased array to be pointed in any direction about the spacecraft. A novel system-level design requires many different subsystems of the spacecraft to interact together to achieve accurate beam-pointing, and thus, high-rate downlink data from Mercury to Earth

This paper describes a process of spacecraft flight software development that reuses requirements, designs, and implementations. Beginning in 1997, The Johns Hopkins University Applied Physics Laboratory (JHU/APL) developed the flight software for five different space missions: (1) Thermosphere, Ionosphere, Mesosphere Energetics and Dynamics (TIMED), (2) COmet Nucleus TOUR (CONTOUR), (3) MErcury Surface, Space ENvironment, GEochemistry, and Ranging (MESSENGER), (4) Solar TErrestrial RElations Observatory (STEREO), and (5) New Horizons, a mission to Pluto and beyond. JHU/APL met strict constraints of budget and schedule by reusing products from each mission for the following one in a successively more comprehensive fashion. Keys to the success of this reuse are consistent external interface protocols, rigorous requirements management, retention of the original development documentation, use of a consistent development process, and a group organization that fosters reuse. The reused packages include bootstrap, task scheduling, uplink, command handling, autonomy, and 1553 bus support.