Context: Grounded Theory (GT) is a research method that facilitates theory development. Its application in Software Engineering (SE) often raises concerns among researchers. Objectives: This paper aims to highlight challenges in applying GT in SE and provide practical insights to overcome them. Method: I outline the top five challenges I faced while applying GT to develop a theory of value for value-based feature selection. The theory was developed taking a positivist stand. I also reflect on the lessons that I learned along the way. Results: The top five challenges are: 1) defining research questions, 2) keeping away from the related literature, 3) assuring the trustworthiness of "ground" data, 4) specifying the theory with a proper granularity level, and 5) reporting a GT study in a limited-length research paper. These challenges led to 17 lessons learned. Conclusions: While my experience with GT in SE has been positive, certain aspects of the method need thoughtful consideration, particularly when the research is conducted from a positivist stand. I hope that the experience I share in this paper is valuable for others in the SE community attempting to use GT for their studies.

Agile software development (ASD) aims to deliver high-quality software. However, software quality often receives less attention in ASD due to focus on functionality. Elicit requirements for, design, validate, implement, and evaluate an evidence-based quality-aware ASD process that organizations can use for better integration of quality management. Using design science research, we collaborated for over 1.5 years with four organizations applying ASD. Involving a total of 35 practitioners, we designed new evidence-based quality-aware ASD processes, which were validated through simulated scenarios, implemented, and evaluated by the practitioners. A general ASD process model, derived from the company-specific models, was created; it shows activities, artifacts, roles, and development phases where evidence can be used to improve software quality. The new process model contributed to higher quality awareness, improved bottleneck identification, release readiness, and quality issues prioritization in the studied companies. We also learned that companies must be mindful of the rules that determine the use of evidence. Integrating the use of quality related evidence into the ASD process can make it quality-aware, support software quality management, and enable timely reaction through fast feedback loops. The evidence-based quality-aware ASD process is applicable in any Scrum-like ASD process. Applicability in other forms of ASD, however, requires further research.

Technical Debt (TD) is a metaphor reflecting technical compromises that can yield short-term benefits but might hurt the long-term health of a software system. With the increasing amount of data generated when performing software development activities, an emergent research field has gained attention: applying Intelligent Techniques to solve Software Engineering problems. Intelligent Techniques were used to explore data for knowledge discovery, reasoning, learning, planning, perception, or supporting decision-making. Although these techniques can be promising, there is no structured understanding related to their application to support Technical Debt Management (TDM) activities. Within this context, this study aims to investigate to what extent the literature has proposed and evaluated solutions based on Intelligent Techniques to support TDM activities. To this end, we performed a Systematic Mapping Study (SMS) to investigate to what extent the literature has proposed and evaluated solutions based on Intelligent Techniques to support TDM activities. In total, 150 primary studies were identified and analyzed, dated from 2012 to 2021. The results indicated a growing interest in applying Intelligent Techniques to support TDM activities, the most used: Machine Learning and Reasoning under uncertainty. Intelligent Techniques aimed to assist mainly TDM activities related to identification, measurement, and monitoring. Design TD, Code TD, and Architectural TD are the TD types in the spotlight. Most studies were categorized at automation levels 1 and 2, meaning that existing approaches still require substantial human intervention. Symbolists and Analogizers are levels of explanation presented by most Intelligent Techniques, implying that these solutions conclude a general truth after considering a sufficient number of particular cases. Moreover, we also cataloged the empirical research types, contributions, and validation strategies described in primary studies. Based on our findings, we argue that there is still room to improve the use of Intelligent Techniques to support TDM activities. The open issues that emerged from this study can represent future opportunities for practitioners and researchers.

In agile software development (ASD), where minimal documentation and rapid delivery are the focus, Quality requirements (QRs) are often underspecified and not documented. Guidelines supporting QR documentation task are scarce. The study developed the Agile QR-Doc QR documentation guidelines, which aim to support QR documentation in ASD. We applied a design science research methodology (DSRM) to build the Agile QR-Doc. We used a survey questionnaire and open discussion with ten software practitioners, from two ASD companies to validate the Agile QR-Doc. The practitioners evaluated the guidelines in terms of usefulness, relevance, understandability, and coverage of important aspects for supporting QR documentation and their impact on the agility of the software development process. Agile QR-Doc list 12 recommendations that are grouped into two categories. The first category introduces three recommendations that focus on raising awareness about the significance of QRs, their documentation and related challenges. The second category lists nine recommendations that introduce artifacts, practices and important aspects for documenting QRs. The validation reveals the relevance, understandability and usefulness of the guidelines to support QR documentation in ASD. It also indicates that the guidelines consider important aspects for documenting QRs and that they do not negatively affect the agility of the software development process. Practitioners can utilize the practices, artifacts and knowledge from the guidelines to support QR documentation in ASD. Researchers can benefit from the knowledge on QR documentation in ASD, and application of DSRM in building artifacts.

Communication between developers and testers can be a rich source of insights into software development processes and practices, which may not be easily discoverable from other means like retrospectives or project roadmaps. With the objective of deriving and capitalizing on potential development-related insights, we analyzed developer-tester communication in an industrial setting. We conducted a case study at a software-intensive Agile company, within the context of the development of one of their flagship products from 2016 to 2018. We applied Latent Dirichlet Allocation (LDA) to analyze communication between developers and testers, and then invited two case-company practitioners to study the results for insights into their developments processes: The findings reveal the case company's efforts to improve their product stability, the growing emphasis on addressing end-user concerns and other quality-related issues. The practitioners interpreted these findings as indicators of evolution in their development process. Based on these findings and the state of the art, we propose an insight classification to highlight insights discoverable from developer-tester communication: Recognizing LDA's potential for deriving insights, the practitioners are keen on incorporating it into their software development practices. The findings from this study serve as evidence for use and benefits of text-mining techniques like LDA in industrial setting, which other practitioners could adapt to elicit their own context-influenced insights. Furthermore, the insight classification can serve as a foundation for further investigation into the extent and type of insights discoverable from developer-tester communication.

Context: In despite of agile and rapid software development (ARSD) being researched and applied extensively, managing quality requirements (QRs) are still challenging. As ARSD processes produce a large amount of data, measurement has become a strategy to facilitate QR management. Objective: This study aims to survey the literature related to QR management through metrics in ARSD, focusing on: bibliometrics, QR metrics, and quality-related indicators used in quality management. Methods: The study design includes the definition of research questions, selection criteria, and snowballing as search strategy. Results: We selected 61 primary studies (2001-2019). Despite a large body of knowledge and standards, there is no consensus regarding QR measurement. Terminology is varying as are the measuring models. However, seemingly different measurement models do contain similarities. Conclusion: The industrial relevance of the primary studies shows that practitioners have a need to improve quality measurement. Our collection of measures and data sources can serve as a starting point for practitioners to include quality measurement into their decision-making processes. Researchers could benefit from the identified similarities to start building a common framework for quality measurement. In addition, this could help researchers identify what quality aspects need more focus, e.g., security and usability that have surprisingly few metrics reported. (c) 2021 The Author(s). Published by Elsevier Inc.

Agile methods can deliver software that fulfills customer needs rapidly and continuously. Quality requirements (QRs) are important in this regard; however, detailed studies on how companies applying agile methods to manage QRs are limited, as are studies on the rationale for choosing specific QR management practices and related challenges. The aim of this study was to address why practitioners manage QRs as they do and what challenges they face. We also analyzed how existing practices mitigate some of the found challenges. Lastly, we connect the contextual elements of the companies with their practices and challenges. We conducted 36 interviews with practitioners from four companies of varying sizes. Since each company operates in different domains, comparing QR management strategies and related challenges in different contexts was possible. We found that the companies apply proactive, reactive, and interactive strategies to manage QRs. Additionally, our study revealed 40 challenges in six categories that companies applying agile methods may face in QR management. We also identified nine contextual elements that affect QR management practice choices and which, importantly, can explain many related challenges. Based on these findings, we constructed a theoretical model about the connection between context, QR management practices, and challenges. Practitioners in similar contexts can learn from the practices identified in this study. Our preliminary theoretical model can help other practitioners identify what challenges they can expect to face in QR management in different developmental contexts as well as which practices to apply to mitigate these challenges.

Background: Practitioners would like to take action based on software metrics, as long as they find them reliable. Existing literature explores how metrics can be made reliable, but remains unclear if there are other conditions necessary for a metric to be actionable. Context & Method: In the context of a European H2020 Project, we conducted a multiple case study to study metrics' use in four companies, and identified instances where these metrics influenced actions. We used an online questionnaire to enquire about the project participants' views on actionable metrics. Next, we invited one participant from each company to elaborate on the identified metrics' use for taking actions and the questionnaire responses (N=17). Result: We learned that a metric that is practical, contextual, and exhibits high data quality characteristics is actionable. Even a non-actionable metric can be useful, but an actionable metric mostly requires interpretation. However, the more these metrics are simple and reflect the software development context accurately, the less interpretation required to infer actionable information from the metric. Company size and project characteristics can also influence the type of metric that can be actionable. Conclusion: This exploration of industry's views on actionable metrics help characterize actionable metrics in practical terms. This awareness of what characteristics constitute an actionable metric can facilitate their definition and development right from the start of a software metrics program.

Numerous software companies are adopting value-based decision making. However, what does value mean for key stakeholders making decisions? How do different stakeholder groups understand value? Without an explicit understanding of what value means, decisions are subject to ambiguity and vagueness, which are likely to bias them. This case study provides an in-depth analysis of key stakeholders' value propositions when selecting features for a large telecommunications company's software-intensive product. Stakeholders' value propositions were elicited via interviews, which were analyzed using Grounded Theory coding techniques (open and selective coding). Thirty-six value propositions were identified and classified into six dimensions: customer value, market competitiveness, economic value/profitability, cost efficiency, technology & architecture, and company strategy. Our results show that although propositions in the customer value dimension were those mentioned the most, the concept of value for feature selection encompasses a wide range of value propositions. Moreover, stakeholder groups focused on different and complementary value dimensions, calling to the importance of involving all key stakeholders in the decision making process. Although our results are particularly relevant to companies similar to the one described herein, they aim to generate a learning process on value-based feature selection for practitioners and researchers in general.

Practitioners adopt software metrics programs to support their software development from the perspective of either overall quality, performance, or both. Current literature details and justifies the role of a metrics program in a software organization's software development, but empirical evidence to demonstrate its actual use and concomitant benefits remains scarce. In the context of an EU H2020 Project, we conducted a multiple case study to investigate how two software-intensive Agile companies utilized a metrics program in their software development. We invited practitioners from the two case companies to report on the actual use of the metrics program, the underlying rationale, and any benefits they may have witnessed. We also collected and analyzed metrics data from multiple use cases to explain the reported use of the metrics. The analysis revealed improvements like better code review practices and formalization of quality requirements management, either as a direct consequence or as a byproduct of the use of the metrics. The contrasting contexts like company size, project characteristics, and general perspective towards metrics programs could explain why one company viewed the metrics as a trigger for their reported improvements, while the other company saw metrics as the main driver for their improvements. Empirical evidence from our study should help practitioners adopt a more favorable view towards metrics programs, who were otherwise reluctant due to lack of evidence of their utility and benefits in industrial context.

Context: Agile software development has become commonplace in software development companies due to the numerous benefits it provides. However, conducting Agile projects is demanding in Small and Medium Enterprises (SMEs), because projects start and end quickly, but still have to fulfil customers' quality requirements. Objective: This paper aims at reporting a practical experience on the use of metrics related to the software development process as a means supporting SMEs in the development of software following an Agile methodology. Method: We followed Action-Research principles in a Polish small-size software development company. We developed and executed a study protocol suited to the needs of the company, using a pilot case. Results: A catalogue of Agile development process metrics practically validated in the context of a small-size software development company, adopted by the company in their Agile projects. Conclusions: Practitioners may adopt these metrics in their Agile projects, especially if working in an SME, and customise them to their own needs and tools. Academics may use the findings as a baseline for new research work, including new empirical studies.

Background: The existing literature in software engineering reports adverse effects of confirmation bias on software testing. Confirmation bias among software testers leads to confirmatory behavior, which is designing or executing relatively more specification consistent test cases (confirmatory behavior) than specification inconsistent test cases (disconfirmatory behavior). Objective: We aim to explore the antecedents to confirmatory and disconfirmatory behavior of software testers. Furthermore, we aim to understand why and how those antecedents lead to (dis)confirmatory behavior. Method: We follow grounded theory method for the analyses of the data collected through semi-structured interviews with twelve software testers. Results: We identified twenty antecedents to (dis)confirmatory behavior, and classified them in nine categories. Experience and Time are the two major categories. Experience is a disconfirmatory category, which also determines which behavior (confirmatory or disconfirmatory) occurs first among software testers, as an effect of other antecedents. Time Pressure is a confirmatory antecedent of the Time category. It also contributes to the confirmatory effects of antecedents of other categories. Conclusion: The disconfirmatory antecedents, especially that belong to the testing process, e.g., test suite reviews by project team members, may help circumvent the deleterious effects of confirmation bias in software testing. If a team’s resources permit, the designing and execution of a test suite could be divided among the test team members, as different perspectives of testers may help to detect more errors. The results of our study are based on a single context where dedicated testing teams focus on higher levels of testing. The study’s scope does not account for the testing performed by developers. Future work includes exploring other contexts to extend our results.

Agile software development (ASD) promotes minimal documentation and often prioritizes functional requirements over quality requirements (QRs). This may be beneficial in reducing the time to market of software. When considering QRs in ASD, the minimal documentation practice may be seen as a concern since QRs determine the success of software projects and are as well not easy to specify and document. Nevertheless, what do practitioners think of the necessity of documenting QRs in ASD? How do they perceive factors that may affect documentation of QRs in ASD? We conducted a multiple case study of three cases applying ASD, involving 12 participants. ASD practitioners identify that it is important to document QRs, and perceive that it contributes to ensuring quality, clarifying QRs, and helping in decision making. Time constraint, QR awareness and communication gaps on QRs influence the documentation of QRs in ASD. ASD teams may align their documentation practices to fit the sprint duration. The influence of QR awareness on documentation was dependent on project context and roles. Communication gaps can create confusion on QRs. Missing and outdated QR documentation may result in accruing technical debt, and lack of common understanding on QRs. The study synthesizes empirical evidence on the significance of documenting QRs in ASD and provides an insight into factors affecting documentation of QRs in ASD.

Value-Based Software Engineering stresses the role of value in software related decisions. In the context of feature selection, software features judged to provide higher value take priority in the development process. This paper focuses on what value means when selecting software features. Using grounded theory, we conducted and analyzed semi-structured interviews with 21 key stakeholders (decision-makers) from three software/software-intensive companies, within a context where value-based decision-making was already established. Our analysis led to the building of a theory of value for value-based feature selection that identifies the nature of value propositions considered by key stakeholders when selecting software features (i.e., decision-making criteria for deciding upon software features, as suggested by Boehm (2003)). We found that some value propositions were common to all three company cases (core value propositions), whereas others were dependent upon the context in which a company operates, and the characteristics of the product under development (specific value propositions). Moreover, value propositions vary according to the stakeholder group and the type of feature being assessed. Our study provides significant insight into value in the context of feature selection, and generates new concepts around value-based feature selection such as new value propositions.

One source of software project challenges and failures is the systematic errors introduced by human cognitive biases. Although extensively explored in cognitive psychology, investigations concerning cognitive biases have only recently gained popularity in software engineering research. This paper therefore systematically maps, aggregates and synthesizes the literature on cognitive biases in software engineering to generate a comprehensive body of knowledge, understand state-of-the-art research and provide guidelines for future research and practise. Focusing on bias antecedents, effects and mitigation techniques, we identified 65 articles (published between 1990 and 2016), which investigate 37 cognitive biases. Despite strong and increasing interest, the results reveal a scarcity of research on mitigation techniques and poor theoretical foundations in understanding and interpreting cognitive biases. Although bias-related research has generated many new insights in the software engineering community, specific bias mitigation techniques are still needed for software professionals to overcome the deleterious effects of cognitive biases on their work.

Context: Quality requirements (QRs) have a significant role in the success of software projects. In agile software development (ASD), where working software is valued over comprehensive documentation, QRs are often under-specified or not documented. Consequently, they may be handled improperly and result in degraded software quality and increased maintenance costs. Investigating the documentation of QRs in ASD, would provide evidence on existing practices, tools and aspects considered in ASD that other practitioners might utilize to improve documentation and management of QRs in ASD. Although there are some studies examining documentation in ASD, those that specifically investigate the documentation of QRs in depth are lacking. Method: we conducted a multiple case study by interviewing 15 practitioners of four ASD cases, to provide empirical evidence on documentation of QRs in ASD. We also run workshops with two of the cases, to identify important aspects that ASD practitioners consider when documenting QRs in requirements management repositories. Result and conclusions: ASD companies approach documentation of QRs to fit the needs of their context. They used tools, backlogs, iterative prototypes, and artifacts such as epic, and stories to document QRs, or utilized face-face communication without documenting QRs. We observed that documentation of QRs in ASD is affected by factors such as context (e.g. product domain, and size) and the experience of practitioners. Some tools used to document QRs also enhanced customer collaboration, enabling customers report and document QRs. Aspects such as levels of abstraction, the traceability of QRs, optimal details of information of QRs and verification and validation are deemed important when documenting QRs in ASD requirements management repositories.

Agile software development (ASD) promotes minimal documentation and often prioritizes functional requirements over quality requirements (QRs). This may be beneficial in reducing the time to market of software. When considering QRs in ASD, the minimal documentation practice may be seen as a concern since QRs determine the success of software projects and are as well not easy to specify and document. Nevertheless, what do practitioners think of the necessity of documenting QRs in ASD? How do they perceive factors that may affect documentation of QRs in ASD? We conducted a multiple case study of three cases applying ASD, involving 12 participants. ASD practitioners identify that it is important to document QRs, and perceive that it contributes to ensuring quality, clarifying QRs, and helping in decision making. Time constraint, QR awareness and communication gaps on QRs influence the documentation of QRs in ASD. ASD teams may align their documentation practices to fit the sprint duration. The influence of QR awareness on documentation was dependent on project context and roles. Communication gaps can create confusion on QRs. Missing and outdated QR documentation may result in accruing technical debt, and lack of common understanding on QRs. The study synthesizes empirical evidence on the significance of documenting QRs in ASD and provides an insight into factors affecting documentation of QRs in ASD.

Software quality poses continuously new challenges in software development, including aspects related to both software development and system usage, which significantly impact the success of software systems. The Q-Rapids H2020 project defines an evidence-based, data-driven quality-aware rapid software development methodology. Quality requirements (QRs) are incrementally elicited, refined and improved based on data gathered from software repositories, project management tools, system usage and quality of service. This data is analysed and aggregated into quality-related key strategic indicators (e.g., development effort required to include a given QR in the next development cycle) which are presented to decision makers using a highly informative dashboard. The Q-Rapids platform is being evaluated in-premises by the four companies participating in the consortium, reporting useful lessons learned and directions for new development.

Existing literature proposes success factors for establishing metrics programs. However, very few studies focus on factors that could ensure long-term use of metrics, and even fewer studies investigate such factors in the context of Agile Software Development (ASD). Motivated by this knowledge gap, we aim to identify success factors for operationalizing metrics in ASD, particularly, factors that could help in the long-term use of metrics. We conducted a multiple case study, where we operationalized process metrics at two software-intensive companies using ASD. We learned that data availability and development process are the two fundamental success factors for process metrics operationalization, albeit less prominent in literature. Companies prefer iterative and incremental operationalization of stable and functional process metrics, which is analogous to the agile way of working. Metrics trustworthiness plays a key role in successful operationalization of process metrics, and is potentially vital to ensuring their long-term use. By comparing the identified success factors with the existing literature, we conclude that success factors concerning data availability, development process, and metrics trustworthiness warrant greater attention, especially to maximize the chances of long-term use of process metrics.

Background: Despite the growth in the use of software analytics platforms in industry, little empirical evidence is available about the challenges that practitioners face and the value that these platforms provide. Aim: The goal of this research is to explore the benefits of using a software analytics platform for practitioners managing quality. Method: In a technology transfer project, a software analytics platform was incrementally developed between academic and industrial partners to address their software quality problems. This paper focuses on exploring the value provided by this software analytics platform in two pilot projects. Results: Practitioners emphasized major benefits including the improvement of product quality and process performance and an increased awareness of product readiness. They especially perceived the semi-automated functionality of generating quality requirements by the software analytics platform as the benefit with the highest impact and most novel value for them. Conclusions: Practitioners can benefit from modern software analytics platforms, especially if they have time to adopt such a platform carefully and integrate it into their quality assurance activities.