Background: Developers increasingly review multi-file code changes generated by LLM-based agents, yet no validated end-to-end workflow or IDE tooling design exists for this scenario. Aims: We investigate (RQ1) the challenges developers face when reviewing LLM-generated multi-file changes and (RQ2) how developers envision effective workflows for this task. Method: In collaboration with JetBrains, we conducted a participatory design study structured using the double-diamond design process with Discover, Define, Develop, and Deliver phases. Industry practitioners participated in the Discover phase (N=17); seven of these returned for the Develop phase. The Define phase was an author-led synthesis. The Deliver phase produced a conceptual design and a high-fidelity semi-interactive prototype evaluated through a follow-up survey with N=43 practitioners. Results: Participants identified trust-calibration as the central challenge. The study yielded a three-level review workflow (overview, file-analysis, code snippet review) supported by seven design constructs (chunk, risk-per-line, risk-per-file, judge, walk-through, zooming in/out, and security cage). In the validation survey, all three workflow levels scored above the neutral midpoint (means 3.50–3.91 on a five-point scale). Of the respondents, 63 Conclusions: Reviewing LLM-generated multi-file changes is a trust-calibration problem rather than a diffing problem. The three-level workflow and the seven constructs we report give tool designers a conceptual framework for building AI-ready code review tools that surface risk and confidence signals at the granularity at which developers allocate attention.

Program comprehension is an essential activity in software engineering. Not only does it often challenge professionals, but it can also hinder novices from advancing their programming skills. Gaze, an emerging modality in developer tools, has so far primarily been utilized to improve our understanding of programmers' visual attention and as a means to reason about programmers' cognitive processes. There has been limited exploration of integrating gaze-based assistance into development environments to support programmers, despite the tight links between attention and gaze. We also know that joint attention is important in collaboration, further suggesting that there is value in exploring collective gaze. In this paper, we investigate the effect of visualizing gaze patterns gathered from experts to novice programmers to assist them with program comprehension in a new codebase. To this end, we present GazePrinter, designed to provide gaze-orienting visual cues informed by experts to aid novices with program comprehension. We present the results of a mixed-methods study conducted with 40 novices to study the effects of using GazePrinter for program comprehension tasks. The study included a survey, a controlled experiment, and interviews. We found that visualization of expert gaze can have a significant effect on novice programmers' behavior in terms of which path they take through the code base; with GazePrinter, novices took a path closer to the path taken by experts. We also found indications of reduced time and cognitive load among novices using GazePrinter.

Code review is a well-established and valued practice in the software engineering community contributing to both code quality and interpersonal benefits. However, there are challenges in both tools and processes that give rise to misalignments and frustrations. Recent research seeks to address this by automating code review entirely, but we believe that this risks losing the majority of the interpersonal benefits such as knowledge transfer and shared ownership. We believe that by better understanding the cognitive processes involved in code review, it would be possible to improve tool support, with or without AI, and make code review both more efficient, more enjoyable, while increasing or maintaining all of its benefits. In this paper, we conduct an ethnographic think-aloud study involving 10 participants and 34 code reviews. We build a cognitive model of code review bottom up through thematic, statistical, temporal, and sequential analysis of the transcribed material. Through the data, the similarities between the cognitive process in code review and decision-making processes, especially recognition-primed decision-making, become apparent. The result is the Code Review as Decision-Making (CRDM) model that shows how the developers move through two phases during the code review; first an orientation phase to establish context and rationale and then an analytical phase to understand, assess, and plan the rest of the review. Throughout the process several decisions must be taken, on writing comments, finding more information, voting, running the code locally, verifying continuous integration results, etc. Analysis software and process-coded data publicly available at: https://doi.org/10.5281/zenodo.15758266

AI assistants, like GitHub Copilot and Cursor, are transforming software engineering. While several studies highlight productivity improvements, their impact on maintainability requires further investigation. This study investigates whether co-development with AI assistants affects software maintainability, specifically how easily other developers can evolve the resulting source code. We conducted a two-phase, preregistered controlled experiment involving 151 participants, 95

We are entering a hybrid era in which human developers and AI coding agents work in the same codebases. While industry practice has long optimized code for human comprehension, it is increasingly important to ensure that LLMs with different capabilities can edit code reliably. In this study, we investigate the concept of “AI-friendly code” via LLM-based refactoring on a dataset of 5,000 Python files from competitive programming. We find a meaningful association between CodeHealth, a quality metric calibrated for human comprehension, and semantic preservation after AI refactoring. Our findings confirm that human-friendly code is also more compatible with AI tooling. These results suggest that organizations can use CodeHealth to guide where AI interventions are lower risk and where additional human oversight is warranted. Investing in maintainability not only helps humans; it also prepares for large-scale AI adoption.

Context: Developing compilers and static analysis tools ("language tools") is a difficult and time-consuming task. We have previously presented *property probes*, a technique to help the language tool developer build understanding of their tool. A probe presents a live view into the internals of the compiler, enabling the developer to see all the intermediate steps of a compilation or analysis rather than just the final output. This technique has been realized in a tool called CodeProber. Inquiry: CodeProber has been in active use in both research and education for over two years, but its practical use has not been well studied. CodeProber combines liveness, AST exploration and presenting program analysis results on top of source code. While there are other tools that specifically target language tool developers, we are not aware of any that has the same design as CodeProber, much less any such tool with an extensive user study. We therefore claim there is a lack of knowledge how property probes (and by extension CodeProber) are used in practice. Approach: We present the results from a mixed-method study of use of CodeProber in an educational setting, with the goal to discover if and how property probes help, and how they compare to more traditional techniques such as test cases and print debugging. In the study, we analyzed data from 11 in-person interviews with students using CodeProber as part of a course on program analysis. We also analyzed CodeProber event logs from 24 students in the same course, and 51 anonymized survey responses across two courses where CodeProber was used. Knowledge: Our findings show that the students find CodeProber to be useful, and they make continuous use of it during the course labs. We further find that the students in our study seem to partially or fully use CodeProber instead of other development tools and techniques, e.g. breakpoint/step-debugging, test cases and print debugging. Grounding: Our claims are supported by three different data sources: 11 in-person interviews, log analysis from 24 students, and surveys with 51 responses. Importance: We hope our findings inspire others to consider live exploration to help language tool developers build understanding of their tool.

The use of large language models (LLMs) is being explored for a multitude of tasks in software engineering (SE), ranging from code generation to bug report assignment. Although LLMs provide impressive results, they require more time and energy than some other machine learning models. For some tasks, simpler models may be more sustainable than LLMs. In this paper, we construct natural language classifiers of different complexity for a use case in the SE domain: commit message classification. We compare the performance of each model with the state-of-the-art with regard to energy consumption for training and inference. We find that simpler models based on Naïve Bayes and LSTM perform similarly to LLMs, while using a fraction of the energy, suggesting that choosing a small model can lead to significant reduction in power usage without compromising performance. Replication package: https://doi.org/10.5281/zenodo.15641782 .

The remarkable advances in AI and Large Language Models (LLMs) have enabled machines to write code, accelerating the growth of software systems. However, the bottleneck in software development is not writing code but understanding it; program understanding is the dominant activity, consuming approximately 70% of developers' time. This implies that improving existing code to make it easier to understand has a high payoff and - in the age of AI-assisted coding - is an essential activity to ensure that a limited pool of developers can keep up with ever-growing codebases. This paper introduces Augmented Code Engineering (ACE), a tool that automates code improvements using validated LLM output. Developed through a data-driven approach, ACE provides reliable refactoring suggestions by considering both objective code quality improvements and program correctness. Early feedback from users suggests that AI-enabled refactoring helps mitigate code-level technical debt that otherwise rarely gets acted upon.

DevOps has significantly improved the software development life-cycle by enabling fast, automated, and continuous integration and deployment (CI/CD). However, its application to Cyber-Physical Systems (CPS) presents unique challenges that require rethinking of traditional practices. DevOps in CPS becomes even more challenging when we consider sustainability, which is becoming increasingly important in all parts of society. In this position paper, we suggest that sustainability, though often overlooked, needs more attention in DevOps for CPS. Through insights gathered from a multi-industry workshop, we identify relevant sustainability goals, such as optimizing resource usage, and key barriers to sustainability in DevOps for CPS, such as difficulties in measuring sustainability improvements. To overcome these challenges, we propose strategies, such as using tools for reproducible builds, but also incorporation of sustainability goals into the organization and possibly in alignment with financial goals. We further see a need for more guidelines and frameworks specifically targeting sustainability.

The use of static program analysis tools can be highly beneficial in software development, but usage is hindered by usability issues. One method of better understanding these issues is to gather user feedback, but it is difficult for developers to invest effort in giving feedback. In this paper, we investigate whether gamification can increase user engagement in feedback collection for static analysis tools. We conducted a mixed-method study at a large software company, with initial user research, design of a feedback system with context-sensitive gamification, deployment of the system to 900 developers over 14 days in an experiment with randomized assignment, and a final collection of qualitative user input. We found that user engagement increased for the gamification group in terms of the amount of feedback (2.47 times more feedback), but the difference is not statistically significant, and participants also expressed a mix of enthusiasm and skepticism in our follow-up interviews.

The C programming language, while offering high performance and low-level control, is memory-unsafe. This makes it prone to programming errors that can result in serious software vulnerabilities and system instability. In this paper, we investigate the feasibility of transitioning a large code base from C to a memory-safe alternative, specifically Rust, in the context of a large company in telecommunications and networking. We explore technical challenges with translation, building on automatic translation with C2Rust, analyze performance impact, and survey developers’ attitudes toward migration. We find that while C2Rust can generate correct code, significant manual effort is required in the translation process, especially to get the benefit of Rust’s safety guarantees. In the internal environment, Rust integration is currently limited, in part by missing compiler support for code closely tied to hardware. Performance results from open-source projects relevant to the company indicate that Rust does not perform significantly worse than C, although further research is needed. Developer attitudes towards a switch were generally positive. Our conclusion is that full migration is currently unfeasible, but gradual adoption is possible. Replication package: https://zenodo.org/records/15754034 .

Code review is a well-established and valuable software development practice associated with code quality, interpersonal, and team benefits. However, it is also time-consuming, with developers spending 10-20% of their working time doing code reviews. With recent advances in AI capabilities, there are more and more initiatives aimed at fully automating code reviews to save time and streamline software developer workflows. However, while automated tools might succeed in maintaining the code quality, we risk losing interpersonal and team benefits such as knowledge transfer, shared code ownership, and team awareness. Instead of automating code review and losing these important benefits, we envision a code review platform where AI is used to support code review to increase benefits for both code quality and the development team. We propose an AI agent-based architecture that collects and combines information to support the user throughout the code review and adapt the workflow to their needs. We analyze this design in relation to the benefits of code review and outline a research agenda aimed at realizing the proposed design.

Context: AI technologies are increasingly embedded in products and software engineering processes of industrial IoT, autonomous systems, and cyber-physical systems. It is therefore essential to ensure alignment with safety, reliability, and ethical standards. However, practical software engineering methods for managing misalignment risks remain underdeveloped. Objective: This study aims to explore industry awareness of misalignment risks and current practices for monitoring them within real-world software engineering contexts. Method: We conducted seven interviews with industry professionals to examine perceptions of misalignment risks, gather insights into existing practices, and understand approaches to alignment across various industrial settings. Three recently proposed taxonomies guided our discussions: one on ethical guidelines for trustworthy AI published by the EU, another summarizing identified AI risks, and a third addressing “double-edged components” (aspects of AI systems that can simultaneously yield positive and negative effects.) Results: Our analysis identified common misalignment risks across these settings and revealed limited use of dedicated testing or monitoring for AI alignment. Most organizations rely on general oversight rather than specialized tools. Conclusion: These findings highlight the need to develop tailored governance practices for alignment in industrial software engineering settings.

[Background/Context] AI assistants like GitHub Copilot are transforming software engineering; several studies have highlighted productivity improvements. However, their impact on code quality, particularly in terms of maintainability, requires further investigation. [Objective/Aim] This study aims to examine the influence of AI assistants on software maintainability, specifically assessing how these tools affect the ability of developers to evolve code. [Method] We will conduct a two-phased controlled experiment involving professional developers. In Phase 1, developers will add a new feature to a Java project, with or without the aid of an AI assistant. Phase 2, a randomized controlled trial, will involve a different set of developers evolving random Phase 1 projects - working without AI assistants. We will employ Bayesian analysis to evaluate differences in completion time, perceived productivity, code quality, and test coverage.

Software development is a complex activity that needs a lot of tool assistance. Over the years there has been a lot of effort put into development of automated assistance to help with activities such as detection of issues via program analysis, or refactoring of code. Recently, the landscape of developer tool assistance is being disrupted with the entry of AI tools, such as Copilot and ChatGPT, powered via Large Language Models. Other kinds of tool assistance, for instance, gaze-driven assistance, is around the corner. What are programmers’ perceptions on tool assistance today? What do they see as good directions for the future? In this paper, we present the results of a survey where we asked developers about their programming practices, experience with program analysis, and attitudes and views on enabling technologies, like AI and eye-tracking. We received 68 replies from a diverse group of developers from 12 countries. We found that 50% of the participants use program analysis and that many participants (N=28) already use AI-enabled tools for programming. We found that our participants were positive toward AI-powered tools, neutral toward eye-tracking, and negative toward gamification. We discuss these and other findings and point out directions for future work.

The widespread availability of large language models (LLMs) has presented the opportunity for novice programmers to make use of them for the purpose of understanding and synthesising code. In this paper, we discuss a small pilot study intended to explore the user experience of doing so in a limited way, and the attitudes of a group of novice programmers towards this style of programming. We also draw parallels to the seminal work of Lisanne Bainbridge, and discuss the way in which her "ironies of automation" are also present when attempting to automate the activity of programming.

Background. Open science aims to improve research accessibility, replicability, and consequently its quality. Empirical software engineering entails both data and artifacts, which may be shared more or less openly, to support transparency. However, the trade-offs involved in balancing the openness against integrity and secrecy concerns need methodological guidance. Aim. We aim to derive such advice, based on our own experiences from a research project, in the field of gaze-assisted code reviews - the Gander case. Method. We draw on literature about open data and artifacts in socio-technical research. Next, we describe our case project and derive a conceptual framework of steps in research data analysis and artifact development, using our data and artifacts as illustrating examples. Results. The conceptual framework contains 1) a categorization of humans involved as participants and their concerns, 2) four steps for data analysis, each resulting in corresponding data and meta-data, and 3) three steps of artifact distribution, matching different levels of openness. We derive a preliminary set of recommendations for open science practices for data and artifacts. Conclusion. The conceptual framework has proven useful in summarizing and discussing data and artifacts in the studied case project. We envision that the framework and recommendations will provide a foundation for further advancement of open science research practices in empirical, socio-technical software engineering.

Code review occupies a significant amount of developers’ work time and is an established practice in modern software development. Despite misaligments between users’ goals and the code review tools and processes pointed out by recent research, the code review tooling has largely stayed the same since the early 90s. Improving these tools, even slightly, has the potential for a large impact spread out over time and the large developer community. In this paper, we use the Double Diamond design process to work together with a team of industry practitioners to find, refine, prototype, and evaluate ways to make it easier to compare refactored code blocks and find previously hard-to-see changes in them. The results show that a flexible comparison modal integrated into Gerrit could reduce the mental load of code review on refactored code. Potentially, it could also have effects on how code is written by no longer discouraging refactoring due to it complicating the review. The user interface created in this collaborative manner was also intuitive enough for all of the participants to be able to use it without any hints or instructions.

Developers need to comprehend new code on a regular basis; a demanding task that gets complicated as projects grow in complexity and size. To help developers, documentation is typically added in the form of code comments, design documents, system documents, to name a few examples. Despite all these different kinds of documentation code base comprehension is still challenging and there is room for further exploration of assistance in this space. In this paper, we explore storytelling as a strategy to assist developers in comprehending a code base, by exploring attachment of stories to developer tasks. We carry out an exploratory study using a design science based research structure with three iterations with feedback from four practitioners working at a consultancy company. We find that our participants typically miss information about rationale and alternatives explored in the documentation, and resort to getting this information from other developers. When asked to write their own stories for tasks our participants sought to add this missing information, but overall found story-writing challenging. We further saw that stories on task-level presents challenges connected to exposure of personal information and integration. We make recommendations for future designs in this space based on our results and list a couple of interesting directions for future work.

We report on an investigation into the nature of the experience that developers have with the spaces in which they work and live based on the reflections of the authors. We describe a few aspects of the structure of developers’ experience around the malleability, ownership and control of the spaces and the emotions they induce, and consider what this may mean for future research.