Oral exams provide a compelling alternative to traditional evaluation methods, expanding or replacing traditional written work. Interest in oral exams is growing rapidly in computer science (CS) education due to shifts to remote learning and concerns around AI-supported programming. This Birds of a Feather (BoF) session is broadly applicable to many in the CS education community, whether they have previously tried oral exams, have concerns about the use of oral exams in CS education, or are curious to hear more about how oral exams might work. The BoF session will provide a forum to discover and discuss previous approaches to oral exams, dive into common themes of interest in small groups, and collectively identify promising future directions for oral exams in CS courses.

Oral exams are an inviting alternative to traditional paper-and-pencil exams. However, they are largely under-utilized in computer science education. In this report, we describe our design for comprehensive final oral exams in five software engineering class sections, across two different small institutions. We present our exam format and our subjective assessment of the exam format in assessing student knowledge as instructors. We also gather quantitative and qualitative data from student surveys. We surveyed students before and after the oral exam to assess their perceptions of it, including their predicted grade and their subjective opinions and experiences. Our work shows evidence that oral exams are effective and practical mechanisms for software engineering classes of a smaller size (approximately 20 students). Student survey responses indicated favorable feedback for our oral exam format; students viewed oral exams as a good assessment of their knowledge and useful beyond that individual class.

We developed an interactive software tool tailored for introductory computer science education, specifically for courses taught in Python. The software is designed to engage students in ethical decision-making through the simulation of emergency scenarios involving self-driving vehicles. In these scenarios, where the vehicle's brakes fail, students must refine the car's "ethics engine" to determine the best course of action: continuing straight, swerving into another lane, or crashing into a barrier. Students develop algorithms that consider various factors such as age, gender, race, and physical ability, emphasizing that algorithms are not neutral but reflect embedded societal values. This hands-on, iterative process encourages students to critically evaluate their decisions and understand the broader ethical implications of programming choices. By providing real-time feedback on the consequences of their algorithms, the software fosters critical thinking and ethical awareness, helping students to see how technology and societal values intersect in real-world applications. This project highlights the essential role of ethics in CS education, preparing students to navigate complex moral landscapes in technology.

Program coverage customization selectively adds instrumentation to a compiled computer program so that a limited amount of directly observed data can be used to infer other program coverage information after a run. A good instrumentation plan can reduce run-time overheads while still giving software developers the information they need. Unfortunately, optimal coverage planning is NP-hard, limiting either the quality of heuristic plans or the sizes of programs that can be instrumented optimally. We exploit the monotonicity property of feasible instrumentations to formulate this problem as an intraprocedural set covering problem. Our formulation has an exponential number of constraints, and we design a polynomial-time separation algorithm to incrementally add the necessary subset of these inequalities. Our approach reduces expected run-time probing costs compared with existing methods, offers a guarantee of the optimality of the instrumentation, and has compilation-time overhead suitable for wide practical use. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work was supported by the National Science Foundation [Grants CCF-1318489, CCF-1420866, and CCF-1423237] and the Air Force Research Laboratory [Grant FA8750-14-2-0270]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0349 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0349 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

ABSTRACT:Preventing medication errors remains a priority in nursing education. The implementation of Barcode Medication Administration (BCMA) systems is one strategy that has been used to reduce medication errors. Practice using BCMA in simulated settings may enhance the transfer of these skills to the clinical practice setting. However, the purchase of BCMA educational products available for nursing students can be cost prohibitive for many nursing programs. To overcome the barrier of cost, an interdisciplinary and innovative collaborative approach was used to create a fully functional low-cost BCMA system.

An oral exam is an assessment approach involving verbal explanations of key concepts or thought process to achieve a solution to a problem, sometimes accompanied by written or typed work. As a complement or refreshing alternative to standard written assessments, oral exams are being implemented in CS courses in a variety of formats. This Birds of a Feather (BoF) session will bring together the growing community of CS educators who have previously used oral exams and those interested in alternative assessment approaches to share ideas and discuss best practices. The discussion leaders represent a wide range of institutions and have varied previous expertise in the design, implementation, and study of oral exams in CS courses. The BoF session will involve (1) introduction to oral exams and previous approaches led by the discussion leaders, (2) small group discussions based on shared interests or concerns with oral exams, and (3) discussion of best practices for oral exams in CS courses.

This software is an instrumenting compiler for lightweight, customizable control-flow tracing. The compiler has embedded within it calls to the mixed-integer programming package Gurobi to implement the cutting plane method as described in the paper.

This poster presents SynpleTest, a teaching tool created for introductory computer science courses. SynpleTest uses program synthesis to generate code based on test cases given by the user. Students must continue to add test cases until SynpleTest generates the program they desire. By doing this, our program teaches the student the importance of producing thoughtful and diverse test cases to improve fundamental understanding of testing programs. Preliminary experimentation in classrooms is underway and initial results of usability and effectiveness are being analyzed.

Assessment of student knowledge is a crucial and challenging part of course design. Especially in computer science courses in the United States, written examinations are very common. While written exams offer a number of advantages in convenience and familiarity, they are also inflexible and prone to question misinterpretation. In contrast to written tests, oral exams offer the prospect of an interactive conversation where students can express their knowledge in a variety of ways while asking clarifying questions. In this paper, we present and assess our implementation of oral exams in an introductory computer science course. We describe the motivation for and resulting features of our design, including a simplified rubric style for equitable, on-the-fly grading. We also perform an assessment relative to more traditional written exams. We find the time commitment for instructors to be manageable and comparable to traditional exams. Through post-semester surveys, students self-report spending slightly more time studying for oral exams, but rate the difficulty as similar to written exams. Both qualitative and quantitative student feedback indicates that oral exams can be effective and well-received.

Debugging post-deployment failures is difficult, in part because failure reports from these applications usually provide only partial information about what occurred during the failing execution. We introduce approaches that answer control-flow queries about a failing program’s execution based on failure constraints given as formal languages. A key component of our approach is the introduction of a new class of subregular languages, the unreliable trace languages (UTL), which allow us to answer many common queries in polynomial time. This report supplements the description of these new approaches with formal proofs. Specifically: we prove completeness for our context-insensitive query problem, tightly bind polynomial-time decidability of query recovery to the UTL class, and prove partial correctness for our approach to answering user queries with UTL constraints.

Debugging is difficult. When software fails in production, debugging is even harder, as failure reports usually provide only an incomplete picture of the failing execution. We present a system that answers control-flow queries posed by developers as formal languages, indicating whether the query expresses control flow that is possible or impossible for a given failure report. We consider three separate approaches that trade off precision, expressiveness for failure constraints, and scalability. We also introduce a new subclass of regular languages, the unreliable trace languages, which are particularly suited to answering control-flow queries in polynomial time. Our system answers queries remarkably efficiently when we encode failure constraints and user queries entirely as unreliable trace languages.

The paper describes Vertx, a tool for validating software binary transformations. Vertx enables software developers and system administrators to automatically check the correctness of software binary transformations such as security hardening and optimization. Transformation validation increases user trust in binary-transformation technology and allows binary transformations to be safely used in critical applications that have narrow margins of error.

Debugging is difficult and costly. As a human programmer looks for a bug, it would be helpful to see a complete trace of events leading to the point of failure. Unfortunately, full tracing is simply too slow to use in deployment, and may even be impractical during testing. We aid post-deployment debugging by giving programmers additional information about program activity shortly before failure. We use latent information in post-failure memory dumps, augmented by low-overhead, tunable run-time tracing. Our results with a realistically-tuned tracing scheme show low enough overhead (0-5%) to be used in production runs. We demonstrate several potential uses of this enhanced information, including a novel postmortem static slice restriction technique and a reduced view of potentially-executed code. Experimental evaluation shows our approach to be very effective, such as shrinking stack-sensitive interprocedural static slices by 49-78% in larger applications.

Program coverage is used across many stages of software development. While common during testing, program coverage has also found use outside the test lab, in production software. However, production software has stricter requirements on run-time overheads, and may limit possible program instrumentation. Thus, optimizing the placement of probes to gather program coverage is important. We introduce and study the problem of customized program coverage optimization. We generalize previous work that optimizes for complete coverage instrumentation with a system that adapts optimization to customizable program coverage requirements. Specifically, our system allows a user to specify desired coverage locations and to limit legal instrumentation locations. We prove that the problem of determining optimal coverage probes is NP-hard, and we present a solution based on mixed integer linear programming. Due to the computational complexity of the problem, we also provide two practical approximation approaches. We evaluate the effectiveness of our approximations across a diverse set of benchmarks, and show that our techniques can substantially reduce instrumentation while allowing the user immense freedom in defining coverage requirements. When naive instrumentation is dense or expensive, our optimizations succeed in lowering execution time overheads.

Debugging is difficult and costly, especially for failures that occur after deployment. In prior work, we developed a suite of instrumentation and analysis tools, collectively titled the Crash Scene Investigation toolkit (CSI). These tools aid developers by providing additional information about failing program executions using latent data in post-failure memory dumps. While we showed that our technique is effective in reducing execution ambiguity, it lacked a proper user interface for developers. In this paper, we present CSIclipse, a work-in-progress plugin for the Eclipse integrated development environment (IDE) that brings our analyses directly to the user. The goal of our plugin is to ease the burden of debugging production failures by conveniently presenting CSI trace and analysis data with intuitive source code overlays and powerful data exploration mechanisms. While designed for our CSI data, our plugin is likely general enough to support trace data from a variety of program analyses.

Debugging is difficult and costly. Developers greatly value full traces and complete, reproducible crash recordings, but these are impractical for deployed software. Fortunately, failing applications can leave behind a snapshot of their crashing state in the form of a core dump. Unfortunately, crash data alone often leaves substantial ambiguity in the program's execution. My thesis work aims both to improve the quality of information extracted from core dumps and enhance this readily-available information. My prior work showed that automated postmortem analysis results can be significantly improved by targeted, lightweight, and tunable instrumentation (0-5% run-time overhead). My thesis aims to expand this work in three directions: improved tracing and dump data recovery, expanded postmortem analyses, and improved tracing efficiency based on previously-observed failures.