While much prior work examines Large Language Models (LLMs) for solo development tasks (e.g., coding), far less is known about how LLMs shape collaborative group work in software engineering. This study focuses on one such collaborative task, namely software design. It presents the results of an exploratory laboratory study of 18 pairs of software professionals who could use an LLM however they saw fit, to design a University campus bicycle parking application. Our findings reveal that introducing an LLM leads to distinct patterns of joint use: shared-instance use facilitated shared understanding, whereas parallel use across separate instances sometimes led to ”context drift”. We also observe wide variation in reliance, from non-use to treating the LLM as an information source or producer. Across these modes, professionals scrutinized and reflected on LLM responses, often yielding design insights; however, early anchoring sometimes curtailed exploration. We provide implications for tools to aid designers while retaining the human-centricity important to design.
A rapidly growing body of research is examining how LLMs influence developers when they code. To date, this research has tended to focus on productivity and code quality outcomes, rather than the underlying cognitive processes involved in programming. To address this gap, we report on the results of an exploratory laboratory study of ten advanced student developers (five with support from AI and five without) who had to make a non-trivial extension to a sizable software system. Leveraging Polya's four problem-solving phases and 25 inductively-generated codes detailing distinct problem-solving behaviors as the primary lenses, we examined: (1) how AI impacted the problem-solving approach the developers used to solve the programming task, and (2) how AI impacted their progress when they became stuck. For the analysis, we triangulated data across multiple sources (e.g., think-aloud, code changes, web searches, and LLM prompts). Unexpectedly, while developers in the AI group repeatedly turned to the AI tool to offload certain aspects of the process, all detailed problem-solving behaviors appeared in both groups. We also found that nine out of ten participants found themselves stuck in their work, but with key differences in how they became stuck and unstuck. We highlight seven distinct causes for being stuck and highlight how AI in some cases helped and in other cases hindered becoming unstuck.
While the use of Large Language Models (LLMs) in programming has been extensively studied, there is limited understanding of how LLMs support collaborative work where creativity plays a central role. Software design, as a collaborative and creative activity, provides a valuable context for exploring the influence of LLMs on creativity. This study investigates how and where creativity naturally emerges when software designers collaborate with an LLM during a design task. In a laboratory setting simulating a workplace environment, 18 pairs of software professionals with design experience were asked to complete a design task. Each pair had 90 minutes to produce a software design based on a set of requirements, with optional access to a custom LLM interface. Pairs were not primed to be creative. We find that creativity was present in all pairs in design processes, with 13 producing design documents containing creativity. We primarily attribute creativity to the human designers, driven by traits such as prior experience, empathy, and the use of analogies. The LLM contributed by producing novel ideas and elaborating human ideas. However, in some cases, the LLM appeared to hinder creativity by suggesting complex solutions or adding to unproductive digressions. LLMs can support creativity in collaborative software design, but human insights remain central. To effectively augment human creativity, designers must be intentional in their engagement with LLMs.
With the rapid adoption of Generative AI (GenAI) tools, software engineering educators have grappled with how best to incorporate them into the classroom. While some research discusses the use of GenAI in the context of learning to code, there is little research that explores the use of GenAI in the classroom for other areas of software development. This paper provides an experience report on introducing GenAI into an undergraduate software design class. Students were required to use GenAI (in the form of ChatGPT) to help complete a team-based assignment. The data collected consisted of the ChatGPT conversation logs and students' reflections on using ChatGPT for the assignment. Subsequently, qualitative analysis was undertaken on the data. Students identified numerous ways ChatGPT helped them in their design process while recognizing the need to critique the response before incorporating it into their design. At the same time, we identified several key lessons for educators in how to deploy GenAI in a software design class effectively. Based on our experience, we believe students can benefit from using GenAI in software design education as it helps them design and learn about the strengths and weaknesses of GenAI.
Background: Software engineering requires both technical skills and creative problem-solving. Blind and low-vision software professionals (BLVSPs) encounter numerous workplace challenges, including inaccessible tools and collaboration hurdles with sighted colleagues. Objective: This study explores the innovative strategies employed by BLVSPs to overcome these accessibility barriers, focusing on their custom solutions and the importance of supportive communities. Methodology: We conducted semi-structured interviews with 30 BLVSPs and used reflexive thematic analysis to identify key themes. Results: Findings reveal that BLVSPs are motivated to develop creative and adaptive solutions, highlighting the vital role of collaborative communities in fostering shared problem-solving. Conclusion: For BLVSPs, creative problem-solving is essential for navigating inaccessible work environments, in contrast to sighted peers, who pursue optimization. This study enhances understanding of how BLVSPs navigate accessibility challenges through innovation.
Teamwork is beneficial for students' learning experiences. However, inherent challenges persist, including free-riding, conflicts, and communication issues. To tackle these issues, the use of peer assessment has been widely advocated, and practiced. For various reasons, however, peer assessment tends to be performed infrequently, limiting its utility for instructors to detect issues in, and intervene with, student teams during a course. One reason is that existing tools do not present peer assessment results over time, not for individual team members, nor for teams as a whole. In this experience report, we introduce a new dashboard that is specifically designed to address time, and report and reflect on the experiences of three instructors who used the dashboard to monitor teams in their courses and intervene when they thought it was necessary.
Creativity is an integral part of software development. This paper introduces the special issue by first providing a brief overview of creativity in the context of software development before introducing the nine articles comprising the special issue.
Climate econometrics is a rapidly advancing field, driven by accelerating climate change, the growing availability of climate data, and new computational methods for climate projection and statistical analysis. Many climate econometrics articles use a similar workflow, yet little effort has been made to provide software toolkit support. The Climate Econometric Toolkit integrates aspects from each step of this workflow into a single software package, supporting: (1) aggregating gridded climate data, (2) constructing and evaluating econometric models, and (3) computing climate impacts. The toolkit is written in Python and provides both a programming and graphical user interface (GUI). We present the functionality of the toolkit, the results of a qualitative evaluation, and two applications based on replications of published papers in the field. The goals of this toolkit are to improve the quality of climate econometrics science by standardizing researchers’ treatment of critical statistical factors, reduce code duplication between articles, and ease the statistical burden on climate econometrics researchers.
Creativity has always been considered a major differentiator to separate the good from the great, and we believe the importance of creativity for software development will only increase as GenAI becomes embedded in developer tool-chains and working practices. This paper uses the McLuhan tetrad alongside scenarios of how GenAI may disrupt software development more broadly, to identify potential impacts GenAI may have on creativity within software development. The impacts are discussed along with a future research agenda comprising six connected themes that consider how individual capabilities, team capabilities, the product, unintended consequences, society, and human aspects can be affected.
The software development workplace poses numerous technical and collaborative accessibility challenges for blind and low vision software professionals (BLVSPs). Though Generative AI (GenAI) is increasingly adopted within the software development industry and has been a rapidly growing topic of interest in research, to date, the unique perspectives of BLVSPs have yet to be consulted. We report on a qualitative study involving 39 semi-structured interviews with BLVSPs about what the introduction of GenAI has meant for their work. We found that BLVSPs used GenAI for many software development tasks, resulting in benefits such as increased productivity and accessibility. However, significant costs were also accompanied by GenAI use as they were more vulnerable to hallucinations than their sighted colleagues. Sometimes, organizational policies prevented use. Based on our findings, we discuss the higher-risks and higher-returns that BLVSPs had to carefully weigh when deciding whether and when to use GenAI tools for work.
Software design has been a topic of interest in the research community since the emergence of software engineering. Indeed, in the past, a dedicated, thriving workshop existed (IWSSD [Library(2000)], last held in 2000), but gradually many topics from this workshop were absorbed into the traditional venues such as ICSE, TSE, and other venues. Today, however, software design is at a crossroads, facing several challenges that we believe necessitate bringing together a dedicated community of software design researchers and practitioners.
Team projects are an essential part of any software engineering program, and are known to help students in learning the primary course materials while imparting valuable teamwork skills. At the same time, teamwork can also present challenges, to both students and their instructors. Well-known problems include teams not following recommended practices (e.g., producing deliverables out of order or at the last minute) and problematic team dynamics interfering with progress (e.g., social loafing or last-minute heroes). This paper presents TeamCAST, a tool we designed to address both kinsd of problems. The key idea underneath TeamCAST is that, by tapping into the intermediate work that students produce as part of their projects, it becomes possible to gain insight into both team progress and contributions. TeamCAST is designed to provide an easy-to-use platform for students to submit such intermediate work and for instructors to monitor the ongoing projects in an intuitive dashboard. This experience report introduces the primary design decisions underlying TeamCAST and presents the results of an exploratory pilot use in a 179 student software design class.
Software design is a set of activities and decisions involved in constructing a software system, ranging from high-level architecture and conceptual design to code design. Decisions made during the design process have a long-lasting impact on various qualities of a system, such as modularity, maintainability, scalability, robustness, security, usability, and performance. Despite its important role in software development, design is still considered something of an art-particularly in terms of design as an activity rather than design as a product or outcome. With the increasing societal impact of software and the potential use of AI in development, design has a new and larger role to play in the engineering of modern software systems. The second Designing workshop aims to bring together researchers, practitioners, and educators who are interested in any aspect of software design, to discuss novel approaches to design, and to identify open challenges and future directions for the field.
Existing commercial and in-house software development tools are often inaccessible to Blind and Low Vision Software Professionals (BLVSPs), hindering their participation and career growth at work. Building on existing research on Do-It-Yourself (DIY) Assistive Technologies and customized tools made by programmers, we shed light on the currently unexplored intersection of how DIY tools built and used by BLVSPs support accessible software development. Through semi-structured interviews with 30 BLVSPs, we found that such tools serve many different purposes and are driven by motivations such as desiring to maintain a professional image and a sense of dignity at work. These tools had significant impacts on workplace accessibility and revealed a need for a more centralized community for sharing tools, tips, and tricks. Based on our findings, we introduce the "Double Hacker Dilemma" and highlight a need for developing more effective peer and organizational platforms that support DIY tool sharing.
The recent surge of research on software developer mental health challenges highlights the importance and urgency of studying solutions to support developer wellbeing. Self-Determination Theory (SDT) offers a valuable framework for exploring wellbeing at work, emphasizing the need to satisfy three psychological needs: autonomy, competence, and relatedness. This paper presents an interview study with 31 software developers in the United States that uses SDT as a guide, exploring how these three needs are perceived and influenced in the work of software developers. We identify specific factors and processes at work and work tools and designs that impact developers’ psychological needs and satisfaction. Results from our study can help design targeted solutions to satisfy developers’ psychological needs, which indirectly support developer wellbeing. This paper highlights the necessity of healthy work cultures in software development and presents design considerations for creating tools for developers.
Recent progress in Generative AI (GenAI) impacts different software engineering (ES) tasks in software development cycle, e.g., from code generation to program repair, and presents a promising avenue for enhancing the productivity of development teams. GenAI based tools have the potential to change the way we develop software and have received attention from industry and academia. However, although some studies have been addressing the adoption of these tools in the software industry, little is known about what are developers' real experiences in a professional software development context, aside the hype. In this paper, we explore the use of GenAI tools by a large Brazilian media company that has teams developing software inhouse. We observed practitioners for six weeks and used online surveys at different time points to understand their expectations, perceptions, and concerns about these tools in their daily work. In addition, we automatically collected quantitative data from the company's development systems, aiming at getting insights about how GenAI impacts the development process during the period. Our results provide insights into how practitioners perceive and utilize GenAI in their daily work in software development.
Richard Newton Taylor合作论文数School of Information and Computer Sciences, University of California5
David G. Kay合作论文数Informatics Department
Computer Science Department
Donald Bren School of Information and Computer Sciences
4
Scott A. Hendrickson合作论文数Donald Bren School of Information and Computer Sciences
University of California3
Cláudia Maria Lima Werner合作论文数Federal University of Rio de Janeiro3