..................................................................................................................... iii Table of

Depiction Level Concrete Depiction Level Figure 4: The most difficult part of interactive graphics programming is maintaining the association between application objects and their depictions. When a structured intermediate representation (an abstract depiction) is interposed between application objects and their depictions, this task can b~ considerably simplified. The system can maintain the relationship between the abstract and concrete depictions, and associate input events with particut~ components of the abstract depiction. The artist that created that particular component can then be notified; it need only maintain the relationship between high-level graphical objects and the application objects they depict.

Early software environments have supported a narrow range of activities ( programming environments) or else been restricted to a single “hard-wired” software development process. The Arcadia research project is investigating the construction of software environments that are tightly integrated, yet flexible and extensible enough to support experimentation with alternative software processes and tools. This has led us to view an environment as being composed of two distinct, cooperating parts. One is the variant part, consisting of process programs and the tools and objects used and defined by those programs. The other is the fixed part, or infrastructure , supporting creation, execution, and change to the constituents of the variant part. The major components of the infrastructure are a process programming language and interpreter, object management system, and user interface management system. Process programming facilitates precise definition and automated support of software development and maintenance activities. The object management system provides typing, relationships, persistence, distribution and concurrency control capabilities. The user interface management system mediates communication between human users and executing processes, providing pleasant and uniform access to all facilities of the environment. Research in each of these areas and the interaction among them is described.

The synchronize and stabilize lifecycle model defines an overall approach for developing and managing large-scale software systems. Organizations use this lifecycle model to continually synchronize what people are doing as individuals and as members of parallel teams and periodically stabilize a product in increments as a project proceeds, rather than once at the end of a project. This lifecycle approach originated from experiences and insights on large-scale projects at Microsoft, and this research examines the applicability of this approach and related ideas to other problem domains.

Mission-critical embedded software performs the core processing logic for pervasive systems that affect people and enterprises everyday, ranging from aerospace systems to financial markets to automotive systems. In order to function properly, these embedded software systems rely on and are highly interdependent with other hardware and software systems. This research identifies design principles for large-scale mission-critical embedded software and investigates their application in development strategies, architectures, and techniques. We have examined actual embedded software systems from two different problem domains, advanced robotic spacecraft and financial market systems, and these analyses established the foundations for these design principles. Both system types embody solutions that respond to detailed specifications defined and modeled with heavy user involvement. Both system types possess mission-critical logic represented using state machines and other structured techniques. They both use a layered architecture approach with a foundation that provides infrastructure services, a layer with a simple set of foreground and background tasks, a layer with deterministic synchronous processing steps, and a layer with event-driven monitoring, commanding, and sequencing capabilities. The architectural approach supports a domain-specific command sequencing macro language that defines table-driven executable specifications and enables developers to work at higher abstraction levels throughout the lifecycle. The architectural approach also facilitates extensibility, reuse, and portability across multi-processor execution environments. The systems rely on extensive built-in self-tests, invariants, and redundant calculations that assess states and detect faults. From a development standpoint, both systems use risk-driven incremental lifecycles, system modeling, end-to-end prototyping, and statistical analysis of development processes. Based on insights gained from embedded software design principles and their application on these as well as other systems, improvement opportunities and research directions will be identified.

Measurement-driven system development focuses on using quantitative data to evaluate capabilities, benefits, progress, and tradeoffs as well as identify improvement opportunities. This paper describes a controlled study that addresses software testing effectiveness and focuses on the combination of individual testing techniques into team-based testing strategies. This analysis is intended to enable measurement-driven process improvement by characterizing how testing effectiveness relates to several factors, including testing strategy, software type, and developer expertise. In this study, a representative group of software development professionals applied common testing techniques to different types of software. This study compares the six possible team combinations of three testing techniques: (1) code reading by stepwise abstraction, (2) functional testing using equivalence partitioning and boundary value analysis, and (3) structural testing using 100% statement coverage criteria. Thirty-two professional developers applied the techniques to three unit-sized programs in a fractional factorial experimental design. The major results of this study are the following: ♦ The six combined testing strategies detected 17% more of the programs’ faults on the average than did the three single techniques, which was a 35% improvement in fault detection. ♦ The highest percentages of the programs’ faults were detected when there was a combination of either two code readers or a code reader and a functional tester. However, a pairing of two code readers detected more faults per hour than did a pairing of a code reader and a functional tester. ♦ The pairing of two individuals of advanced expertise resulted in the highest percentage of faults being detected. ♦ The most cost-effective (number of faults detected per hour) testing strategy overall was when code reading was applied by an individual. The most cost-effective combined testing strategy was when a code reader was paired with either another code reader or a structural tester. ♦ Both the percentage of faults detected and the fault detection cost-effectiveness depended on the type of software being tested. In conclusion, we outline future research directions that build on these strategies and ideas.

Economics‐driven systems engineering integrates system design principles with measurement‐based analyses of value, options, and tradeoffs. This study helps enable economics‐driven systems engineering by describing an example of measurement‐based analysis of software reuse in large‐scale systems. Software reuse enables developers to leverage past accomplishments and facilitates significant improvements in software productivity and quality. Software reuse catalyzes gains in productivity by avoiding redevelopment and gains in quality by incorporating components whose reliability has already been established. The purpose of this study is to characterize software reuse empirically by investigating one development environment that actively reuses software. Twenty‐five software systems ranging from 3000 to 112,000 source lines have been selected for analysis from a NASA systems development environment. The amount of software either reused or modified from previous systems averages 32% per project in this environment. Non‐parametric statistical models are applied to examine numerous development variables across the software modules in the systems. This research focuses on initial results and graphical characterizations of the data. Four classes of software modules are analyzed: (a) modules reused without revision, (b) modules reused with slight revision (< 25% changes), (c) modules reused with major revision (≥ 25% changes), and (d) newly developed modules. The modules reused without revision had the fewest faults, lowest fault correction effort, and lowest fault densities. In conclusion, we outline future research directions that build on these strategies and ideas.

We are convening a symposium to honor Barry W. Boehm's lifetime contributions to the Software Engineering community and co-locating this event with the 29th International Conference on Software Engineering.

System developers and managers continually strive to identify, undertake, and realize improvements in system development methods. Measurement‐driven systems engineering guides the identification of improvement opportunities and enables the quantitative evaluation of progress and benefits. Six Sigma techniques provide a structured approach for using measurement‐driven methods to decrease the variances and shift the means of user‐defined metrics such as defect densities, development cycletimes, and resource expenditures. This research investigates the effectiveness of software defection detection using peer reviews across 12 system development phases on 14 large‐scale systems. This study analyzes 3418 defects from 731 peer reviews and benchmarks the defect injection and detection performance across the 12 system development phases. Six Sigma techniques including the define‐measure‐analyze‐improve‐control (DMAIC) method, root cause analysis, and control charts helped achieve inphase detection of 95 percent of defects and realize over 50 percent improvements in defect densities and closure cycletimes for certain peer review types.

We have investigated the problem of immigrating software artifacts from one software development environment (SDE) to another for the purpose of upgrading to new SDEs as technology improves, while continuing development or maintenance of existing software systems. We rst taxonomize the larger problem of data migration, to establish the scope of immigration. We then classify SDEs in terms of the ease of immigrating software artifacts out of the data repository of the source SDE without knowledge of its internal representation. A framework is presented for constructing automatic immigration tools as utilities provided by destination SDEs. We describe a speci c immigration tool, called Marvelizer, that we have implemented as part of the Marvel SDE and discuss our experience using the tool.

ABSTRACTMeasurement‐driven system development focuses on using quantitative data to evaluate capabilities, benefits, progress, and tradeoffs as well as identify improvement opportunities. This paper describes a controlled study that addresses software testing effectiveness and focuses on the combination of individual testing techniques into team‐based testing strategies. This analysis is intended to enable measurement‐driven process improvement by characterizing how testing effectiveness relates to several factors, including testing strategy, software type, and developer expertise. In this study, a representative group of software development professionals applied common testing techniques to different types of software. This study compares the six possible team combinations of three testing techniques: (1) code reading by stepwise abstraction, (2) functional testing using equivalence partitioning and boundary value analysis, and (3) structural testing using 100% statement coverage criteria. Thirty‐two professional developers applied the techniques to three unit‐sized programs in a fractional factorial experimental design.The major results of this study are the following. The six combined testing strategies detected 17% more of the programs' faults on the average than did the three single techniques, which was a 35% improvement in fault detection. The highest percentages of the programs' faults were detected when there was a combination of either two code readers or a code reader and a functional tester. However, a pairing of two code readers detected more faults per hour than did a pairing of a code reader and a functional tester. The pairing of two individuals of advanced expertise resulted in the highest percentage of faults being detected. The most cost‐effective (number of faults detected per hour) testing strategy overall was when code reading was applied by an individual. The most cost‐effective combined testing strategy was when a code reader was paired with either another code reader or a structural tester. Both the percentage of faults detected and the fault detection cost‐effectiveness depended on the type of software being tested. In conclusion, we outline future research directions that build on these strategies and ideas.

Software engineering training can incorporate approaches for using measurement-driven methods such as Six Sigma by using a 16-week project planning and tollgate review process. We discuss example software measurement-driven improvement methods, projects, and results.

ABSTRACTSystems engineering must tackle the challenges of computational systems that are increasingly large‐scale and software‐intensive in terms of system size, component breadth and maturity, and development heterogeneity. This research describes and empirically evaluates techniques for generating predictive models for enabling large‐scale system development and management. We describe two types of metric‐driven decision models, decision trees and neural networks, which classify software components in large systems according to their likelihood of having user‐specified properties such as high fault‐proneness or high development effort. The metric‐driven decision models enable coarse‐grain analysis of large‐scale multi‐component heterogeneous systems, and they identify high‐payoff areas for directing the application of fine‐grain analysis techniques for fault detection or redesign. The decision models serve as metric integration mechanisms that enable the synergistic use of numerous metrics simultaneously and integrate measurements collected by development tools or infrastructure. Model generation techniques automatically generate the decision models to calibrate them to new projects and organizations.We evaluate the predictive effectiveness of the decision models in terms of correctness, consistency, and completeness using fault and effort data from large NASA systems. Correctness is defined as the percent of components correctly identified, consistency is defined as 100% minus the percent of false positives, and completeness is defined as 100% minus the percent of false negatives. On average, the decision models had 83.44% correctness, 71.96% consistency, and 65.25% completeness in predictions of high fault and high effort software components. The network models had 89.63% correctness, 79.49% consistency, and 69.09% completeness, while the tree models had 77.25% correctness, 64.42% consistency, and 61.40% completeness. Non‐parametric ANOVA comparisons showed that the network models were statistically more accurate than the tree models (α < 0.0001).

Measurement-driven dashboards provide a unifying mechanism for understanding, evaluating, and predicting the development, management, and economics of large-scale systems and processes. Dashboards enable interactive graphical displays of complex information and support flexible analytic capabilities for user customizability and extensibility. Dashboards commonly include software requirements and design metrics because they provide leading indicators for project size, growth, and stability. This paper focuses on dashboards that have been used on actual large-scale projects as well as example empirical relationships revealed by the dashboards. The empirical results focus on leading indicators for requirements and design of large-scale systems. In the first set of 14 projects focusing on requirements metrics, the ratio of software requirements tosource-lines-of-code averaged 1:46. Projects that far exceeded the 1:46 requirements-to-code ratio tended to be more effort-intensive and fault-prone during verification. In the second set of 16 projects focusing on design metrics, the components in the top quartile of the number of component internal states had 6.2 times more faults on average than did the components in the bottom quartile, after normalization by size. The components in the top quartile of the number of component interactions had 4.3 times more faults on average than did the components in the bottom quartile, after normalization by size. When the number of component internal states was in the bottom quartile, the component fault-proneness was low even when the number of component interactions was in the upper quartiles, regardless of size normalization. Measurement-driven dashboards reveal insights that increase visibility into large-scale systems and provide feedback to organizations and projects.