In this paper, we present Wandercode, a novel interaction design for recommender systems that recommend code locations to aid programmers in software development tasks. In particular, our design aims to improve upon prior designs by reducing information overload, by better supporting the exploration of recommendations, and by making more efficient use of screen space. During our design process, we developed a set of design dimensions to aid others in the design of code recommenders. To validate our design, we implemented a prototype of our design as an Atom code editor extension with support for the Java programming language, and conducted an empirical user evaluation comparing our graph-based Wandercode design to a control design representative of prior list-based interaction designs for code recommenders. The results showed that, compared with the control design, Wandercode helped participants complete tasks more quickly, reduced their cognitive load, and was viewed more favorably by participants.

We present here a novel instructional resource, called DeepCode, to support deep code comprehension and learning in intro-to-programming courses (CS1 and CS2). DeepCode is a set of instructional code examples which we call a codeset and which was annotated by our team with comments (e.g., explaining the logical steps of the underlying problem being solved) and related instructional questions that can play the role of hints meant to help learners think about and articulate explanations of the code. While DeepCode was designed primarily to serve our larger efforts of developing an intelligent tutoring system (ITS) that fosters the monitoring, assessment, and development of code comprehension skills for students learning to program, the codeset can be used for other purposes such as assessment, problem-solving, and in various other learning activities such as studying worked-out code examples with explanations and code visualizations. We present here the underlying principles, theories, and frameworks behind our design process, the annotation guidelines, and summarize the resulting codeset of 98 annotated Java code examples which include 7,157 lines of code (including comments), 260 logical steps, 260 logical step details, 408 statement level comments, and 590 scaffolding questions.

This experience report describes the delivery of round-the-clock help to students using Discord (a popular messaging and voice/video calling platform) in a remote software engineering course. Students in the course learn full-stack web development using Ruby on Rails and PostgreSQL, and work in teams to develop web applications. Our central goal in offering round-the-clock help using Discord was to increase the amount of help that students receive from teachers (i.e., teaching assistants and the instructor). Indeed, we found that our 24/7-Discord approach led to a considerable increase in the amount of student-teacher interaction versus the approach used previously, which emphasized in-person office hours and a question-and-answer forum in Piazza. Moreover, students from underrepresented groups in computer science interacted with teachers at a rate comparable to other students, and we received consistently positive feedback from students regarding the approach. We also made several key observations about when students tended to seek help, including that they sought help the most between 7:00 p.m. and midnight, that help seeking spiked right before deadlines, that students posted the fewest help messages on weekends, and that students posted significantly more messages during the first half of the course, which emphasized skills assignments, versus the second half, which focused on team project work.

JupyterLab is a widely used platform for programming and data science using computational notebooks, but it has not been widely used in the educational data mining community as a source of student data. We have developed three JupyterLab extensions to enable educational data mining research in CSEd and data science. Our Blockly extension supports blocks-based programming in JupyterLab and logs both event-level blocks actions as well as kernel actions and errors. Our self-explanation extension appends self-explanation prompts to codes cells and logs the input text for further analysis. Finally, our HTML injection extension allows injection of arbitrary HTML and Javascript into JupyterLab notebooks to enable pedagogies and data collection currently unsupported by JupyterLab. All extensions are open-source and distributed through NPM.

Blocks languages are widely used to teach children programming, and research over the past decade has generally supported their benefits in terms of motivation and actual learning. However, little work has been done on using blocks languages to teach adults, and even less work has looked at blocks languages for data science. We have integrated Blockly, a blocks-based programming environment, into JupyterLab, one of the leading computational notebook development environments for data science. Our integration, which is publicly released as a JupyterLab extension, allows users to assemble blocks-based programs in a GUI workspace and then render the blocks as textual code (e.g., Python) in a computational notebook cell. Additional features of the extension are notebook sync, which clears and restores the blocks workspace as the user navigates to different notebook cells, and intelliblocks, dynamically generated blocks that are created when users load software packages. In this demonstration, we show how learners can use blocks to solve data science problems in a JupyterLab computational notebook. We have released 20 worked example companion notebooks at https://github.com/memphis-iis/datawhys-content-notebooks and an extended tutorial on the extension at https://youtu.be/-luPzplPDI0. This material is based upon work supported by the National Science Foundation under Grant No. 1918751.

This experience report describes the use of frequent, timed coding tests in a project-intensive software engineering course in which students first learn full-stack web development using Ruby on Rails and then apply their skills in a team project. The goal of the skills tests was twofold: (1) to help motivate students to engage in distributed practice and, thus, gain adequate coding skills to be an effective team member during the team project and (2) to accurately assess whether students had acquired the requisite skills and, thereby, catch deficiencies early, while there was still time to address them. Regarding the first goal, although several students indicated that the tests motivated them to engage in substantial practice coding, it was ultimately inconclusive as to the extent of the tests' impact on students' distributed practice behavior and on their preparation for the project. Regarding the second goal, the skills testing approach was indeed considerably more effective than graded homework assignments for assessing coding skill and detecting struggling students early. Lessons learned from our experiences included that students had significant concerns about the strict time limit on the tests, that the tests caused a spike in mid-semester withdrawals from the course that disproportionately impacted students from underrepresented groups, and that detecting struggling students was one thing, but effectively helping them catch up was a whole other challenge.

The difficulty of learning data science is believed to arise from its deep prerequisites in statistics, programming, and machine learning. This poster explores how blocks languages may be used to reduce the cognitive load of learning data science. Unlike blocks languages for introductory programming, blocks languages for data science naturally align with a high level of abstraction and almost exclusively sequential execution. However, these gains in simplicity are offset by the high level of parameterization at the block level. Three designs for blocks languages are presented and compared, and implications for abstraction, sequential execution, and parameterization on cognitive load are discussed.

In this paper, we present CodeDeviant, a novel tool for visual dataflow programming environments that assists programmers by helping them ensure that their code-restructuring changes did not accidentally alter the behavior of the application. CodeDeviant aims to integrate seamlessly into a programmer's workflow, requiring little or no additional effort or planning. Key features of CodeDeviant include transparently recording program execution data, enabling programmers to efficiently compare program outputs, and allowing only apt comparisons between executions. We report a formative qualitative-shadowing study of LabViewprogrammers, which motivated CodeDeviant's design, revealing that the programmers had considerable difficulty determining whether code changes they made resulted in unintended program behavior. To evaluate Code-Deviant, we implemented a prototype CodeDeviant extension for LabViewand used it to conduct a laboratory user study. Key results included that programmers using CodeDeviant discovered behavior-altering changes more accurately and in less time than programmers using standard LabView.

Collaborative code review has become an integral part of the collaborative design process in the domain of software development. However, there are well-documented challenges and limitations to collaborative code review---for instance, high-quality code reviews may require significant time and effort for the programmers, whereas faster, lower-quality reviews may miss code defects. To address these challenges, we introduce CFar, a novel tool design for extending collaborative code review systems with an automated code reviewer whose feedback is based on program-analysis technologies. To validate this design, we implemented CFar as a production-quality tool and conducted a mixed-method empirical evaluation of the tool usage at Microsoft. Through the field deployment of our tool and a laboratory study of professional programmers using the tool, we produced several key findings showing that CFar enhances communication, productivity, and review quality in human--human collaborative code review.

Design principles are a key tool for creators of interactive systems; however, a cohesive set of principles has yet to emerge for the design of code editors. In this paper, we conducted a between-subjects empirical study comparing the navigation behaviors of 32 professional LabVIEW programmers using two different code-editor interfaces: the ubiquitous tabbed editor and the experimental Patchworks editor. Our analysis focused on how the programmers arranged and navigated among open information patches (i.e., code modules and program output). Key findings of our study included that Patchworks users made significantly fewer click actions per navigation, juxtaposed patches side by side significantly more, and exhibited significantly fewer navigation mistakes than tabbed-editor users. Based on these findings and more, we propose five general principles for the design of effective navigation affordances in code editors.

In this paper, we report an evaluation study of predictive models of programmer navigation. In particular, we compared two operationalizations of navigation from the literature (click-based versus view-based) to see which more accurately records a developer's navigation behaviors. Moreover, we also compared the predictive accuracy of seven models of programmer navigation from the literature, including ones based on navigation history and code-structural relationships. To address our research goals, we performed a controlled laboratory study of the navigation behavior of 10 participants engaged in software evolution tasks. The study was a partial replication of a previous comprehensive evaluation of predictive models by Piorkowski et al., and also served to test the generalizability of their results. Key findings of the study included that the click-based navigations agreed closely with those reported by human observers, whereas view-based navigations diverged significantly. Furthermore, our data showed that the predictive model based on recency was significantly more accurate than the other models, suggesting the strong potential for tools that leverage recency-type models. Finally, our model-accuracy results had a strong correlation with the Piorkowski results, however, our results differed in several noteworthy ways, potentially caused by differences in task type and code familiarity.

The design of programming tools is slow and costly. To ease this process, we developed a design pattern catalog aimed at providing guidance for tool designers. This catalog is grounded in Information Foraging Theory (IFT), which empirical studies have shown to be useful for understanding how developers look for information during development tasks. New design patterns, authored by members of the research community for the catalog, concretely explain how to apply IFT in tool design. In our evaluation, qualitative analyses revealed the community-written design patterns compared well in quality to patterns that we had ourselves published in a smaller, peer-reviewed catalog.

In this paper, we present the Yestercode tool for supporting code changes in visual dataflow programming environments. In a formative investigation of LabVIEW programmers, we found that making code changes posed a significant challenge. To address this issue, we designed Yestercode to enable the efficient recording, retrieval, and juxtaposition of visual dataflow code while making code changes. To evaluate Yestercode, we implemented our design as a prototype extension to the LabVIEW programming environment, and ran a user study involving 14 professional LabVIEW programmers that compared Yestercode-extended LabVIEW to the standard LabVIEW IDE. Our results showed that Yestercode users introduced fewer bugs during tasks, completed tasks in about the same time, and experienced lower cognitive loads on tasks. Moreover, participants generally reported that Yestercode was easy to use and that it helped in making change tasks easier.

Empirical studies have revealed that software developers spend 35%–50% of their time navigating through source code during development activities, yet fundamental questions remain: Are these percentages too high, or simply inherent in the nature of software development? Are there factors that somehow determine a lower bound on how effectively developers can navigate a given information space? Answering questions like these requires a theory that captures the core of developers' navigation decisions. Therefore, we use the central proposition of Information Foraging Theory to investigate developers' ability to predict the value and cost of their navigation decisions. Our results showed that over 50% of developers' navigation choices produced less value than they had predicted and nearly 40% cost more than they had predicted. We used those results to guide a literature analysis, to investigate the extent to which these challenges are met by current research efforts, revealing a new area of inquiry with a rich and crosscutting set of research challenges and open problems.

Developers performing maintenance activities must balance their efforts to learn the code vs. their efforts to actually change it. This balancing act is consistent with the "production bias" that, according to Carroll's minimalist learning theory, generally affects software users during everyday tasks. This suggests that developers' focus on efficiency should have marked effects on how they forage for the information they think they need to fix bugs. To investigate how developers balance fixing versus learning during debugging, we conducted the first empirical investigation of the interplay between production bias and information foraging. Our theory-based study involved 11 participants: half tasked with fixing a bug, and half tasked with learning enough to help someone else fix it. Despite the subtlety of difference between their tasks, participants foraged remarkably differently-making foraging decisions from different types of "patches," with different types of information, and succeeding with different foraging tactics.

Although there have been many advances in end-user programming environments, recent empirical studies report that programming still remains difficult for end-users. We hypothesize that one reason may be lack of effective support for helping end-user programmers problem-solve their own way around barriers they encounter. Therefore, in this paper, we describe the Idea Garden, a concept designed to help end-user programmers generate new ideas and problem-solve when they run into barriers. The Idea Garden has its roots in Minimalist Learning Theory and problem-solving theories. Our proof-of-concept prototype of the Idea Garden concept in the CoScripter end-user programming environment currently targets three barriers reported in end-user programming literature. It does so using an integrated, just-in-time combination of scaffolding for problem-solving strategies, for design patterns and for programming concepts. Our empirical results showed that this approach helped end-user programmers overcome all three types of barriers that our prototype targeted.