This study explores the translation of agile management practices (agile) into a traditional industry context. Using a case study of agile adoption in a development project in a large traditional company of the aerospace and defense industry, this research investigates how the translation process unfolds by studying how both idea and context are adapted to create shared meaning. This research makes two main contributions. First, this research introduces translation theory to the innovation management literature and details its applicability in explaining variations in outcomes of applying management practices in new organizational contexts. Our research suggests that this constitutes a complementary theoretical lens to diffusion theory, enabling the explanation of the process to create shared meaning when idea and recipient context have a low level of compatibility. Second, this research shows the process of translating agile into a traditional context through a non-linear joint creation of meaning. The process was shaped largely by the experience and resolution of project-external (i.e., with the surrounding organization) and project-internal conflicts, which in turn motivated the idea or context to adapt. The study identifies two central concepts, namely isolation, and shielding, which determine how the translation process unfolds and how meaning is created. Managerial implications based on these contributions are presented and discussed.

AbstractScaled agile development of large systems has primarily followed the approach used in traditional systems engineering – system decomposition and static teams assigned to subsystems. However, this arrangement may result in an inefficient allocation of resources and uneven progress. This paper presents an alternative approach in which problem-based decomposition replaces system-architecture-based decomposition, and resources are flexibly allocated to problems for each sprint using ad hoc teams. Using a field study approach, we examine a mechatronic system development project utilizing these agile adjustments and discuss situations in which these methods may be successfully utilized in other projects and organizations.

AbstractAgile has its roots in software, improving the pace and quality of development projects. The application of Agile to hardware is less mature, and anecdotal evidence suggests there are specific challenges in this space. Based upon qualitative interviews with organizations working simultaneously across hardware and software applications, this paper presents three key challenges we identified. We also document several useful adjustments to Agile practice which provide a toolset that teams can use to more successfully apply Agile to hardware projects.

Scaled agile projects require coordination of work dependencies across groups of teams called Agile Release Trains (ARTs). ARTs work on developing and delivering bundles of product features called Capabilities. Coordination can be especially complex when dependencies cut across organizational levels and product domains. We develop a field study involving Multi-Domain Matrix (MDM) mapping of dependencies across multiple ARTs and product capabilities. Our analysis finds that organizational dependencies (within a single team or an ART) associated with features that lead up to a single capability typically get tracked through periodic Program Increment (PI) planning processes. Product capabilities that depend directly on other capabilities are visible through customer delivery processes. However, some dependencies that link organizational and product capability domains may be episodic and may not be addressed through PI planning. MDM analysis can be used to both formalize PI coordination and to create dedicated problem-solving groups to improve such episodic coordination.

Process knowledge is of considerable significance to the digitalization and intelligentization of the manufacturing industry. Current research on the process knowledge representation of decision-making in engineering design has predominantly focused on either mathematical models of individual decisions at the micro-level or organizational models of group decision consensus at the macro-level. However, the management of complexity and uncertainty in the model-based realization of engineered systems is critical to achieving rational, comprehensive, and robust decisions, especially in terms of knowledge-intensive design. The efficiency and effectiveness of decisions in system design are intrinsically linked to the process, knowledge, and system concepts involved, necessitating a more flexible and systematic decision process representation scheme that supports both the management of complexity and uncertainty. Hence, in this paper, we propose a decision-centric design process representation scheme named the Phase-Event-Information X (PEI-X) diagram and its corresponding systematic design guidance method for designing decision workflows. Using the proposed method, designers have the ability to (1) model hierarchical decision processes that cover vertical and horizontal interaction patterns, and (2) exploit the synthesis of the “Formulating-Identifying-Reusing-Exploring” iterative process to extend the understanding and prediction of decision process behaviors in design. We achieve the aforesaid abilities through the implementation of a knowledge-based design guidance system for collaborative decision support and we demonstrate the efficacy by adopting a specific multi-stage manufacturing process design problem, hot rod rolling system design, and carry out an integrated design of materials, products, and related manufacturing processes.

Program Increments (PIs) are sequences of consecutive sprints during SAFe implementation of agile developments. SAFe work is planned at two levels of granularity: (i) stories, which create tasks (within a sprint team) and account for task interactions at a fine level of granularity; and (ii) features (often decomposed into stories), which account for interactions at a coarser level of granularity. A common practice in PI planning, involving 10-15 teams, is to suppress interactions at the story level and focus on interactions at the feature level instead. We create two DSMs for a PI planning process – one based on story interactions, and another based on feature interactions. We find a nearly 9X increase in interactions at the finer level, i.e. more granular interaction, compared to a DSM based on coarser level interaction data. We discuss theory and practice implications for using more granular DSMs during PI Planning and oversight processes.

The technology readiness level (TRL) scale was developed at the National Aeronautics and Space Administration (NASA) in the 1970s as a standardized technology maturity assessment tool for use in complex system development. Today, TRL assessments are used to make multimillion-dollar decisions at NASA and beyond, yet anecdotal evidence suggests that there are challenges associated with TRL use in practice. In this paper, we systematically uncover the practitioners' view, first via 19 interviews with employees from seven organizations. We identify 15 challenges of TRL implementations in three categories: system complexity, planning and review, and validity of assessment. Next, we prioritize these challenges via a survey of TRL practitioners, using a best-worst choice experiment. Finally, we identify best practices and proposed extensions to address the challenges. We find that system complexity challenges are most critical to TRL users, despite being addressed in the literature. We posit that addressing these opportunities could result in substantial improvements to decision processes and outcomes in complex engineering projects.

AbstractWe study a DevOps software development process for a Product-Service System (PSS) using a design structure matrix (DSM) representation. We find unique features such as nested, planned iterations at differing rates which are not evident in conventional engineering development projects. We describe the impact of integrating ongoing operations into a development process and identify some of the enablers that lead to adoption of a DevOps process. We conclude by discussing the implications of our findings and raise questions for further research.

Agile and the Scaled Agile Framework (SAFe) methodologies are increasingly being deployed, both for software development and also by teams in broader work settings. This is a proof-of-concept study to develop a DSM for a SAFe development environment. Key takeaways from this study are: (i) Agile and SAFe DSMs are characterized by a unique set of nested information dependencies that are absent in waterfall development; (ii) these dependencies are not readily evident in the conventional and layered rendering of SAFe process documentation. We discuss the organizational and analytical process improvement opportunities available based on such DSM analysis and conclude by identifying opportunities for follow-on research.

A rigorous, in-depth analysis is a common approach in complex system design. Elsewhere, however, more iterative and agile processes and open innovation have become commonplace. We experiment with an agile hackathon-type design sprint for solving industry-provided, complex system engineering problems. In a typical complex system project, significant domain expertise is expected and only one in-depth analysis is typically conducted to make recommendations for a given problem. The question we explore is whether a quick sprint with non-domain experts can result in useful insights for further analysis. We tasked seven teams in parallel to conduct analysis and suggest recommendations for a given company case in only a few hours. The industry challenge was to propose system changes that would mitigate risks due to the long lifecycle of the system and long time from order to delivery. The teams were given two a priori decomposed design structure matrices, representing the product architecture at two levels of granularity, as well as access to several analysis tools. The design sprint resulted in seven sets of recommendations, each with unique insights. The results and their variety highlighted the type of recommendations any given analysis direction would give if pursued further. It provided insights about the many different ways to potentially address the given challenge. As expected, it also highlighted the difficulty of analysis due to lack of detailed system knowledge. Nevertheless, the sprint was considered successful and meaningful as well as an effective means to augment traditional complex system analysis.

In a staged development process, the planned work is not always complete at the gate review, yet a gate decision must be made. We present a more complete explanation of the reality of gate decision options, with the addition of Waiver (with and without re-review), Back-up plan, and Delay, along with Go and Kill. We also show how it is feasible to extend the simple decision tree modelling approach currently used for the Go/Kill choice to analyze the expected value of the broader set of options available. We demonstrate this new approach with studies from industrial application of the method. These case studies show that it is possible to estimate the parameters needed to conduct the decision tree analysis. Our case work also identifies heuristics that are prevalent in gate decisions. Coupling such heuristics with our decision analysis can formalize the underlying trade-offs and inform decision makers on the risks (or benefits) of waiver, delay, and back-up plan options.

Risk estimation is a key activity for product development and technology integration programs. There are a number of decision support tools that help project managers identify and mitigate risks in a project, however few explicitly consider the effects of architecture on risk. We propose a novel risk estimation framework that includes considerations of the system architecture. By starting with traditional project management literature, we define risk as a combination of likelihood and impact. We use Technology Readiness Levels as our measure for likelihood, and given that change propagates through interfaces, we used measures that relate to connectivity to estimate impact. This framework is applied to an industry example and we visualize the data in different formats to aid in analysis. The insights gained from this analysis are discussed, and we conclude that the risk estimation framework provides estimates that are in line with the experience of engineers at the company.

AbstractThe Technology Readiness Level (TRL) method, used to assess the maturity of a technology, has been applied in different industries for over 40 years. TRLs assist engineers in managing risks, communicating development progress and specifying deliverables. As TRLs have been adopted in more industries, the original scope of TRL application has expanded and challenges in use have emerged. Little is known about the TRL user community and its opinions of the method, or of the method's practical challenges. A global industry survey, conducted with professional TRL users, investigated the relative importance of 15 previously identified challenges of TRL use. We find that the top four most critical TRL challenges as identified by our respondents are system‐related; they include the topics of integration, interfaces, system modifications, and overall system maturity. The results reveal a need for an extension of the TRL method from a component‐readiness‐context to a system‐readiness‐context.

ABSTRACT The engineering design literature has provided guidance on how to identify and analyze design activities and their information dependencies. However, a systematic characterization of process interfaces between engineering design activities is missing, and the impact of structural and compositional aspects of interfaces on process performance is unclear. To fill these gaps, we propose a new approach that characterizes process interfaces as organization networks consisting of people and their interactions when performing interfacing activities. Furthermore, we provide guidance on how to test and interpret the effect of those characteristics on interface problems. As a result, we show how structural and compositional aspects of the organization networks between information‐dependent activities provide valuable insights to better manage complex engineering design processes. The proposed approach is applied to the development of a power plant, analyzing 79 process interfaces. The study reveals a relationship between the structure and composition of the process interfaces and reported interface problems. Implications of this approach include the integration of information about process and organization architectures, the systematic identification of key performance metrics associated with interface problems, and improved support for engineering managers by means of a better overview of information flows between activities.

Citation Olechowski, Alison; Eppinger, Steven D. and Joglekar, Nitin. “Technology Readiness Levels at 40: A Study of State-of-the-Art Use, Challenges, and Opportunities.” 2015 Portland International Conference on Management of Engineering and Technology (PICMET), August 2-6 2015, Institute of Electrical and Electronics Engineers (IEEE), September 2015 © Institute of Electrical and Electronics Engineers (IEEE) As Published http://dx.doi.org/10.1109/PICMET.2015.7273196 Publisher Institute of Electrical and Electronics Engineers (IEEE)

No AccessModeling and managing complex systemsNov 2015Structuring a Product Development Organization Based on the Product Architecture and CommunicationRodolfo Reyes Luna, Steven D. EppingerRodolfo Reyes Luna, Steven D. Eppingerhttps://doi.org/10.3139/9783446447264.004SectionsAboutPDF ToolsAdd to FavoritesDownload CitationTrack CitationsCopy LTI LinkPDF key 'share (en)' returned an object instead of string.FacebookTwitterEmailLinkedIn previous chapternext chapter FiguresReferencesRelatedDetails 2015Pages: 29-39Print ISBN: 978-3-446-44573-4eISBN: 978-3-446-44726-4 Copyright & Permissions© 2015 Carl Hanser Verlag GmbH & Co. KGPDF DownloadLoading ...

The pattern of information flow through the network of interdependent design activities is thought to be an important determinant of engineering design process results. A previously unexplored aspect of such patterns relates to the temporal dynamics of information transfer between activities as those activities are implemented through the network of people executing the project. To address this gap, we develop a dynamic modeling method that integrates both the network of people and the network of activities in the project. We then employ a large dataset collected from an industrial setting, consisting of project-related e-mails and activity records from the design and development of a renewable energy plant over the course of more than three years. Using network metrics for centrality and clustering, we make three important contributions: 1) We demonstrate a novel method for analyzing information flows between activities in complex engineering design projects; 2) we show how the network of information flows in a large-scale engineering project evolved over time and how network analysis yields several managerial insights; and 3) we provide a useful new representation of the engineering design process and thus support theory-building toward the evolution of information flows through systems engineering stages. Implications include guidance on how to analyze and predict information flows as well as better planning of information flows in engineering design projects according to their individual stage and activity characteristics.

The technology readiness level (TRL) scale was introduced by NASA in the 1970s as a tool for assessing the maturity of technologies during complex system development. TRL data have been used to make multi-million dollar technology management decisions in programs such as NASA's Mars Curiosity Rover. This scale is now a de facto standard used for technology assessment and oversight in many industries, from power systems to consumer electronics. Low TRLs have been associated with significantly reduced timeliness and increased costs across a portfolio of US Department of Defense programs. However, anecdotal evidence raises concerns about many of the practices related to TRLs. We study TRL implementations based on semi-structured interviews with employees from seven different organizations and examine documentation collected from industry standards and organizational guidelines related to technology development and demonstration. Our findings consist of 15 challenges observed in TRL implementations that fall into three different categories: system complexity, planning and review, and validity of assessment. We explore research opportunities for these challenges and posit that addressing these opportunities, either singly or in groups, could improve decision processes and performance outcomes in complex engineering projects.

This paper investigates how to identify potential misalignments between actual and expected information flow patterns among activities at each design stage. We develop a dynamic network model that quantifies how much information control each activity has over time and how the overall control of information is distributed among activities. The model is applied to the design of a biomass power plant. As a result we demonstrate an empirical relationship between information control patterns and design process stages that can be used to guide information flow improvements.