Software development is a dynamic activity that requires engineers to work effectively with tools, processes, and collaborative teams. As a result, the presence of friction can significantly hinder productivity, increase frustration, and contribute to low morale among developers. By contrast, higher satisfaction levels are positively correlated with higher levels of perceived productivity. Hence, understanding the factors that cause bad experiences for developers is critical for fostering a positive and productive engineering environment. In this research, we employed a mixed-method approach, including interviews, surveys, diary studies, and analysis of developer telemetry data to uncover and triangulate common factors that cause "bad days" for developers. The interviews involved 22 developers across different levels and roles. The survey captured the perception of 214 developers about factors that cause them to have "bad days," their frequency, and their impact on job satisfaction. The daily diary study engaged 79 developers for 30 days to document factors that caused "bad days" in the moment. We examined the telemetry signals of 131 consenting participants to validate the impact of bad developer experience using system data. Findings from our research revealed factors that cause "bad days" for developers and significantly impact their work and well-being. We discuss the implications of these findings and suggest future work.

Video meetings are notorious for difficulties with conversational turn-taking, which has impacts on inclusion and outcomes. We present a scalable automatic process to categorize turn-taking patterns in remote meetings based on eyes-off analysis of meeting transcripts. Drawing on a series of remote meetings (10 series, 34 total meetings) recorded in July-August 2021 by employees of a global technology company, we identified and parametrized patterns of cooperative and competitive overlaps of turns. The results show initial success characterizing people's behaviours as either likely to continue or cede turns based on either the amount of overlap that they produce during other's turns or the amount of overlap they experience in their own turns. With further development and validation, this method could be used to measure inclusion in remote and hybrid meetings.

Pair programming is routinely used in industry and taught in face-to-face programming classes. Research indicates that it improves self-confidence and students’ programming, communication and team working skills. We investigate the technology, barriers, and soft-skill benefits for distance-learning students who pair program with a remote partner online. In one study, students watched two tutors pair programming and then performed a pair programming task remotely with a student partner. Students felt significantly more positive with the latter compared to the former. As universities strive to provide a good student experience during a pandemic, these findings highlight the value of active remote pair programming using standard online communication tools.

Pair programming is routinely used in industry and taught in face-to-face programming classes. Research indicates that it improves self-confidence and students' programming, communication and team working skills. We investigate the technology, barriers, and soft-skill benefits for distance-learning students who pair program with a remote partner online. In one study, students watched two tutors pair programming and then performed a pair programming task remotely with a student partner. Students felt significantly more positive with the latter compared to the former. As universities strive to provide a good student experience during a pandemic, these findings highlight the value of active remote pair programming using standard online communication tools.

Predictable, rapid, and data-driven feature rollout; lightning-fast; and automated fix deployment are some of the benefits most large software organizations worldwide are striving for. In the process, they are transitioning toward the use of continuous deployment practices. Continuous deployment enables companies to make hundreds or thousands of software changes to live computing infrastructure every day while maintaining service to millions of customers. Such ultra-fast changes create a new reality in software development. Over the past four years, the Continuous Deployment Summit, hosted at Facebook, Netflix, Google, and Twitter has been held. Representatives from companies like Cisco, Facebook, Google, IBM, Microsoft, Netflix, and Twitter have shared the triumphs and struggles of their transition to continuous deployment practices—each year the companies press on, getting ever faster. In this chapter, the authors share the common strategies and practices used by continuous deployment pioneers and adopted by newcomers as they transition and use continuous deployment practices at scale.

Static analysis tools (SATs) often fall short of developer satisfaction despite their many benefits. An understanding of how developers in the real-world act on the alerts detected by SATs can help improve the utility of these tools and determine future research directions. The goal of this paper is to aid researchers and tool makers in improving the utility of static analysis tools through an empirical study of developer action on the alerts detected by Coverity, a state-of-the-art static analysis tool. In this paper, we analyze five open source projects as case studies (Linux, Firefox, Samba, Kodi, and Ovirt-engine) that have been actively using Coverity over a period of at least five years. We investigate the alert occurrences and developer triage of the alerts from the Coverity database; identify the alerts that were fixed through code changes (i.e. actionable) by mining the commit history of the projects; analyze the time an alert remain in the code base (i.e. lifespan) and the complexity of code changes (i.e. fix complexity) in fixing the alert. We find that 27.4% to 49.5% (median: 36.7%) of the alerts are actionable across projects, a rate higher than previously reported. We also find that the fixes of Coverity alerts are generally low in complexity (2 to 7 lines of code changes in the affected file, median: 4). However, developers still take from 36 to 245 days (median: 96) to fix these alerts. Finally, our data suggest that severity and fix complexity may correlate with an alert's lifespan in some of the projects.

Deciding the equivalence of SQL queries is a fundamental problem in data management. As prior work has mainly focused on studying the theoretical limitations of the problem, very few implementations for checking such equivalences exist. In this paper, we present a new formalism and implementation for reasoning about the equivalences of SQL queries. Our formalism, U-semiring, extends SQL's semiring semantics with unbounded summation and duplicate elimination. U-semiring is defined using only very few axioms and can thus be easily implemented using proof assistants such as Lean for automated query reasoning. Yet, they are sufficient enough to enable us reason about sophisticated SQL queries that are evaluated over bags and sets, along with various integrity constraints. To evaluate the effectiveness of U-semiring, we have used it to formally verify 68 equivalent queries and rewrite rules from both classical data management research papers and real-world SQL engines, where many of them have never been proven correct before.

The practice of continuous deployment enables product teams to release content to end users within hours or days, rather than months or years. These faster deployment cycles, along with rich product instrumentation, allows product teams to capture and analyze feature usage measurements. Product teams define a hypothesis and a set of metrics to assess how a code or feature change will impact the user. Supported by a framework, a team can deploy that change to subsets of users, enabling randomized controlled experiments. Based on the impact of the change, the product team may decide to modify the change, to deploy the change to all users, or to abandon the change. This experimentation process enables product teams to only deploy the changes that positively impact the user experience. The goal of this research is to aid product teams to improve their deployment process through providing an empirical characterization of an experimentation process when applied to a large-scale and mature service. Through an analysis of 21,220 experiments applied in Bing since 2014, we observed the complexity of the experimental process and characterized the full deployment cycle (from code change to deployment to all users). The analysis identified that the experimentation process takes an average of 42 days, including multiple iterations of one or two week experiment runs. Such iterations typically indicate that problems were found that could have hurt the users or business if the feature was just launched, hence the experiment provided real value to the organization. Further, we discovered that code changes for experiments are four times larger than other code changes. We identify that the code associated with 33.4% of the experiments is eventually shipped to all users. These fully-deployed code changes are significantly larger than the code changes for the other experiments, in terms of files (35.7%), changesets (80.4%) and contributors (20.0%).

Continuous deployment involves automatically testing incremental software changes and frequently deploying them to production environments. With it, developers' changes can reach customers in days or even hours. Such ultrafast changes create a new reality in software development. To understand the emerging practices surrounding continuous deployment, researchers facilitated a one-day Continuous Deployment Summit at the Facebook campus in July 2015, at which participants from 10 companies described how they used continuous deployment. From the resulting conversation, the researchers derived 10 adages about continuous-deployment practices. These adages represent a working set of approaches and beliefs that guide current practice and establish a tangible target for empirical validation by the research community.

Proactive security reviews and test efforts are a necessary component of the software development lifecycle. Resource limitations often preclude reviewing the entire code base. Making informed decisions on what code to review can improve a team's ability to find and remove vulnerabilities. Risk-based attack surface approximation (RASA) is a technique that uses crash dump stack traces to predict what code may contain exploitable vulnerabilities. The goal of this research is to help software development teams prioritize security efforts by the efficient development of a risk-based attack surface approximation. We explore the use of RASA using Mozilla Firefox and Microsoft Windows stack traces from crash dumps. We create RASA at the file level for Firefox, in which the 15.8% of the files that were part of the approximation contained 73.6% of the vulnerabilities seen for the product. We also explore the effect of random sampling of crashes on the approximation, as it may be impractical for organizations to store and process every crash received. We find that 10-fold random sampling of crashes at a rate of 10% resulted in 3% less vulnerabilities identified than using the entire set of stack traces for Mozilla Firefox. Sampling crashes in Windows 8.1 at a rate of 40% resulted in insignificant differences in vulnerability and file coverage as compared to a rate of 100%.

Since its introduction 10 years ago, GIT has taken the world of version control systems (VCS) by storm. Its success is partly due to creating opportunities for new usage patterns that empower developers to work more efficiently. However, the resulting change in both user behavior and the way GIT stores changes impacts data mining and data analytics procedures [6], [13]. While some of these unique characteristics can be managed by adjusting mining and analytical techniques, others can lead to severe data loss and the inability to audit code changes, e.g. knowing the full history of changes of code related to security and privacy functionality. Thus, switching to GIT comes with challenges to established development process analytics. This paper is based on our experience in attempting to provide continuous process analysis for Microsoft product teams who switching to GIT as their primary VCS. We illustrate how GIT's concepts and usage patterns create a need for changing well-established data analytic processes. The goal of this paper is to raise awareness how certain GIT operations may damage or even destroy information about historical code changes necessary for continuous data development process analytics. To that end, we provide a list of common GIT usage patterns with a description of how these operations impact data mining applications. Finally, we provide examples of how one may counteract the effects of such destructive operations in the future. We further provide a new algorithm to detect integration paths that is specific to distributed version control systems like GIT, which allows us to reconstruct the information that is crucial to most development process analytics.

It would appear that developing software programs or services is one of the easiest things in the world to do. The theorist would have us believe that all practitioners need to do is to write the code in specific software language (e.g. functional language) and follow specific processes (e.g. agile) and ideally deploy as a continuously evolving service and you will reach perfection. If you run into problems then that is because your engineers are not good enough and/or you are not following the process correctly. The reality is very different, especially when developing software at scale. This talk will describe why there are no universal development processes that can be applied across all software product and service. While practitioners are well aware of this reality, there is little assistance, based on empirical evidence, to help them either choose a suitable development process for their product or service, or in optimizing the solution they have already chosen. Over the last few years I and other researchers have being looking at how to assist developers in optimizing their process, based on the attributes of the product or services they are developing. In this talk I will describe the current state of our research in this space.

The software industry has existed for over 60 years and is now a massive multinational, multi-billion dollar industry. During this time, the number of people working in the software industry has expanded significantly, as has the number of software engineering researchers. In theory, these software engineering researchers should be driving the future of software development, but even the most optimistic software engineering researcher would not make this claim. In reality the number of practitioners attending Software Engineering conferences is decreasing. This chapter examines research limitations that may explain its apparent disconnect with the software engineering industry.

Vulnerability prediction models (VPM) are believed to hold promise for providing software engineers guidance on where to prioritize precious verification resources to search for vulnerabilities. However, while Microsoft product teams have adopted defect prediction models, they have not adopted vulnerability prediction models (VPMs). The goal of this research is to measure whether vulnerability prediction models built using standard recommendations perform well enough to provide actionable results for engineering resource allocation. We define 'actionable' in terms of the inspection effort required to evaluate model results. We replicated a VPM for two releases of the Windows Operating System, varying model granularity and statistical learners. We reproduced binary-level prediction precision (~0.75) and recall (~0.2). However, binaries often exceed 1 million lines of code, too large to practically inspect, and engineers expressed preference for source file level predictions. Our source file level models yield precision below 0.5 and recall below 0.2. We suggest that VPMs must be refined to achieve actionable performance, possibly through security-specific metrics.

Security testing and reviewing efforts are a necessity for software projects, but are time-consuming and expensive to apply. Identifying vulnerable code supports decision-making during all phases of software development. An approach for identifying vulnerable code is to identify its attack surface, the sum of all paths for untrusted data into and out of a system. Identifying the code that lies on the attack surface requires expertise and significant manual effort. This paper proposes an automated technique to empirically approximate attack surfaces through the analysis of stack traces. We hypothesize that stack traces from user-initiated crashes have several desirable attributes for measuring attack surfaces. The goal of this research is to aid software engineers in prioritizing security efforts by approximating the attack surface of a system via stack trace analysis. In a trial on Windows 8, the attack surface approximation selected 48.4% of the binaries and contained 94.6% of known vulnerabilities. Compared with vulnerability prediction models (VPMs) run on the entire codebase, VPMs run on the attack surface approximation improved recall from .07 to .1 for binaries and from .02 to .05 for source files. Precision remained at .5 for binaries, while improving from .5 to .69 for source files.

Testing is a key element of software development processes for the management and assessment of product quality. In most development environments, the software engineers are responsible for ensuring the functional correctness of code. However, for large complex software products, there is an additional need to check that changes do not negatively impact other parts of the software and they comply with system constraints such as backward compatibility, performance, security etc. Ensuring these system constraints may require complex verification infrastructure and test procedures. Although such tests are time consuming and expensive and rarely find defects they act as an insurance process to ensure the software is compliant. However, long lasting tests increasingly conflict with strategic aims to shorten release cycles. To decrease production costs and to improve development agility, we created a generic test selection strategy called THEO that accelerates test processes without sacrificing product quality. THEO is based on a cost model, which dynamically skips tests when the expected cost of running the test exceeds the expected cost of removing it. We replayed past development periods of three major Microsoft products resulting in a reduction of 50% of test executions, saving millions of dollars per year, while maintaining product quality.

The challenges facing software engineering teams are complex and are evolving over time. Many teams aim to produce software more frequently without sacrificing quality. These teams may be considering changing their development processes to meet these new challenges. Teams creating products in such a changing and challenging environment can benefit from data to make decisions based on evidence rather than based on intuition. In many cases, software engineers have a tremendous amount of data, and sometimes, they are overloaded with it. However, the challenge is to determine what data is relevant, how it should be analyzed, and how the analysis should be interpreted to guide decision-making. Due to the great diversity of products made by means of software engineering, no single type of dataset, which can be interpreted through metrics, is applicable to all software engineering efforts. The guidance accentuates the need to establish clear goals, identifies the important components of a decision support program, identifies common pitfalls when choosing metrics, and discusses interpretation challenges.

The development of software products are often managed through the use of Version Control Systems (VCS). VSCs allow for versioning of source code and manage the process of merging newly developed code into the product primarily through the use of branches. The branching architecture and the development processes used by the product groups are inter-related and have often evolved over many releases to reflect unique aspects of the product, the development preferences of the team, and the structure of the development organization. Within Microsoft, there is a lot of commonality in the development tools used by the product teams, but there is great diversity in the development processes, and the structure and use of branching. This diversity in the development processes across Microsoft provides a unique learning platform to characterize the effectiveness of specific practices when applied to specific product developments. Understanding the characteristics of different software development processes is becoming increasingly important as a number of product groups are migrating from developing a box set product to simultaneously developing a product deployed as a service, released as a stand-alone product, and shared as a component with other product groups within Microsoft. To assist the product group in making their decisions regarding the most appropriate branching architecture for their product, this document examines the benefits and drawbacks for different branching architectures. It also delineates the issues product groups often have in managing the simultaneous development and release of multiple products that share the same code base. The document should aid engineering teams in choosing the best branch structure for the software product they are developing and in understanding the requirements for sharing code between products.

The organizational and social aspects of software engineering (SE) are now increasingly well investigated. This paper proposes that there are a number of approaches taken in research that can be distinguished not by their method or topic but by the different views they construct of the human agent acting in SE. These views have implications for the pragmatic outcome of the research, such as whether systems design suggestions are made, proposals for the development of practical reasoning tools or the effect of Social Network Systems on engineer’s sociability. This paper suggests that these studies tend to underemphasize the felt-life of engineers, a felt-life that is profoundly emotional though played in reference to ideas of moral propriety and ethics. This paper will present a study of this felt-life, suggesting it consists of a form of digital dwelling. The perspective this view affords are contrasted with process and ‘scientific’ approaches to the human agent in SE, and with the more humanistic studies of SE reasoning common in CSCW.