We present the PROTEAN Spectre defense-the first to be altogether comprehensive, covering all side-channels and speculation; programmer-transparent, requiring no source modifications; and programmable, tailoring its hardware protections to software's security needs. Several Spectre defenses offer the first two features, but protect a hardware-defined subset of architectural state from transiently leaking. Meanwhile, many Spectrevulnerable programs process secrets in ways that such rigid protections cannot both performantly and fully secure. PROTEAN overcomes this limitation through: (1) ProtISA, an ISA extension that allows software to tell hardware which architectural registers and memory bytes require protection from transiently leaking at each program point; (2) ProtCC, a compiler that automatically infers and programs ProtISA protections for vulnerable code with minimal user input; and (3) ProtDelay and ProtTrack, two alternative hardware mechanisms that performantly enforce software-defined ProtISA protections. By flexibly tailoring a hardware Spectre defense to a program ' s data protection needs, PROTEAN significantly reduces the overhead of fully securing vulnerable programs. With ProtDelay/ProtTrack, it averages 0.27x/0.18x and 0.42x/0.34x of the runtime overhead of the best secure baseline for programs with and without mixed security needs, respectively, at lower/comparable hardware complexity.

A central task in online learning is to design and test learning experiences that blend good pedagogy with elegant user interaction. In this work, we show that LLM-based web agents provide an efficient and helpful way to get immediate feedback on a learning design. We create a web agent that engages with an online learning experience much like a student would – autonomously navigating the interface and analyzing the website content – and generates a comprehensive description of the student experience. We validate the agent’s ability to understand student behavior by showing that it is useful in predicting student outcomes in a massive, open-access, online CS1 course. Specifically, agent-generated descriptions significantly improve our ability to predict student dropout in a lesson, outperforming all other measured approaches. We also demonstrate how designers could use the agent to compare potential lesson designs through case studies. Our qualitative analysis confirms that the agent can be used to identify confusing lesson content and provide actionable feedback for the designer. Such AI web agents have the potential to accelerate meaningful advancements in online learning by giving fast, low-cost, and helpful insights for new learning experiences.

We present Coding Pathfinder, a platform to help non-programmers learn to code for a specific purpose. This paper explores how we can scaffold generative AI to provide structure and ensure mastery in informal learning settings, introducing a new approach to coding education. In the current iteration of Pathfinder, a user describes the coding task that they are working on. After collecting some details and scoping the project, Pathfinder identifies the skills that the user will master upon successful completion of the project. It then assesses which of the skills our users already has, and designs a personalised learning journey. The guided journey consists of instructions, explanations, tasks and videos. We also incorporate a chat feature so users can ask questions and engage as if they are working with a tutor.

We introduce the Agent GPA (Goal-Plan-Action) framework, driven by the fundamental insight that critical agent failures emerge at the intersections of setting goals, devising plans, and executing actions. We operationalize the framework with a factorized suite of LLM judges designed to measure distinct elements of Goal-Plan-Act alignment. To make this methodology scalable and generalizable across diverse agent architectures and datasets, we use state-of-the-art automated prompt optimization techniques to systematically generate domain-specific evaluation criteria. We validate this approach across three benchmarks: a multi-agent research setting (TRAIL/GAIA), a single coding agent setting (TRAIL/SWE-bench), and a private, enterprise data-agent setting (Snowflake Intelligence). Extensive evaluation on TRAIL/GAIA demonstrates the core validity of the framework, which identifies a broad range of agent failures (95

We present uSpectre, a new class of transient execution attacks that exploit microcode branch mispredictions to transiently leak sensitive data. We find that many long-known and recently-discovered transient execution attacks, which were previously categorized as Spectre or Meltdown variants, are actually instances of uSpectre on some Intel microarchitectures. Based on our observations, we discover multiple new uSpectre attacks and present a defense against uSpectre vulnerabilities, called uSLH.

We conducted a large-scale randomized controlled trial (RCT) in a massive, open-access, online CS1 course to examine how a chatbot's (1) placement within the course interface, (2) degree of personification, and (3) technical functionality impact student outcomes. When the chatbot was placed in the course's Integrated Development Environment instead of the lessons interface, over five times the proportion of students sent a message. Additionally, these students sent four times as many messages and asked about the homework four times as often. Subtle design choices -- such as greeting students with ''Hello'' -- nearly doubled the percentage of students who engaged with the tool. Despite these significant differences in chatbot usage, we found no meaningful impact on student performance in the course. The study included 6,515 students from 149 countries, divided into 10 experiment groups - each receiving a distinct chatbot implementation - and one control group. Our results show that chatbot design dramatically influences how students engage with the tool, providing valuable insights for further development of AI-based pedagogical tools.

We explore the potential for productive team-based collaboration between humans and Artificial Intelligence (AI) by presenting and conducting initial tests with a general framework that enables multiple human and AI agents to work together as peers. ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role, autonomously engage in tasks and communication within Slack, and remain agnostic to whether their collaborators are human or AI. Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities, coordinate with other agents, and await requested inputs or deliverables before proceeding. In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task. We also propose an automated method for analyzing collaboration dynamics that effectively identifies behavioral characteristics of agents with distinct roles, allowing us to quantitatively compare collaboration dynamics in a range of experimental conditions. For example, in comparing ChatCollab AI agents, we find that an AI CEO agent generally provides suggestions 2-4 times more often than an AI product manager or AI developer, suggesting agents within ChatCollab can meaningfully adopt differentiated collaborative roles. Our code and data can be found at: https://github.com/ChatCollab.

People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life. However, practice environments for social skills are typically out of reach for most people. How can we make social skill training more available, accessible, and inviting? Drawing upon interdisciplinary research from communication and psychology, this perspective paper identifies social skill barriers to enter specialized fields. Then we present a solution that leverages large language models for social skill training via a generic framework. Our AI Partner, AI Mentor framework merges experiential learning with realistic practice and tailored feedback. This work ultimately calls for cross-disciplinary innovation to address the broader implications for workforce development and social equality.

Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy.

The rise of Generative AI (GenAI) brings about transformative potential across sectors, but its dual-use nature also amplifies risks. Governments globally are grappling with the challenge of regulating GenAI, balancing innovation against safety. China, the United States (US), and the European Union (EU) are at the forefront with initiatives like the Management of Algorithmic Recommendations, the Executive Order, and the AI Act, respectively. However, the rapid evolution of GenAI capabilities often outpaces the development of comprehensive safety measures, creating a gap between regulatory needs and technical advancements. A workshop co-organized by Google, University of Wisconsin, Madison (UW-Madison), and Stanford University aimed to bridge this gap between GenAI policy and technology. The diverse stakeholders of the GenAI space -- from the public and governments to academia and industry -- make any safety measures under consideration more complex, as both technical feasibility and regulatory guidance must be realized. This paper summarizes the discussions during the workshop which addressed questions, such as: How regulation can be designed without hindering technological progress? How technology can evolve to meet regulatory standards? The interplay between legislation and technology is a very vast topic, and we don't claim that this paper is a comprehensive treatment on this topic. This paper is meant to capture findings based on the workshop, and hopefully, can guide discussion on this topic.

Drawing on platforms and methods that had been developed but not widely adopted, the COVID-19 pandemic suddenly forced remote learning and remote work worldwide. While the first few months were too rushed to provide time for iteration or reflection, continuing years gave educators, students, employers, and workers an unprecedented opportunity to explore and find value in remote operation. Recognizing the potential long-term impact of this period, this special issue collects contributions submitted in response to a broad call for articles on pandemic-period remote learning and work. The articles selected by peer review discuss the challenges faced by college students across the United States; describe educational innovations drawing on the motivational advantages of escape rooms; and explain the evolving characteristics of the remote work landscape, including a specific study of gender equality and challenges faced by women during this time. While all of these articles reflect the difficulties and hardships of the period, all of the articles also report resourceful creativity and positive directions for the future.

We present Serberus, the first comprehensive mitigation for hardening constant-time (CT) code against Spectre attacks (involving the PHT, BTB, RSB, STL and/or PSF speculation primitives) on existing hardware. Serberus is based on three insights. First, some hardware control-flow integrity (CFI) protections restrict transient control-flow to the extent that it may be comprehensively considered by software analyses. Second, conformance to the accepted CT code discipline permits two code patterns that are unsafe in the post-Spectre era. Third, once these code patterns are addressed, all Spectre leakage of secrets in CT programs can be attributed to one of four classes of taint primitives--instructions that can transiently assign a secret value to a publicly-typed register. We evaluate Serberus on cryptographic primitives in the OpenSSL, Libsodium, and HACL* libraries. Serberus introduces 21.3% runtime overhead on average, compared to 24.9% for the next closest state-of-the-art software mitigation, which is less secure.

To inspire student engagement in middle school math, we explore the possibility of using generative AI to enhance the creativity of math learning. We present the Math IDE, a math education environment in which students learn about math concepts by building artifacts. We aimed to create a platform in which students can engage with mathematical concepts, create an artifact that embodies the math that they are learning about, and practice their high-level specification skills. In the current iteration of the Math IDE, students can create custom web pages by describing and demonstrating understanding of the math that is involved in the web page. In this short overview, we describe our process and discuss several open questions regarding the design and application of this novel method of math education.

In this paper, we evaluate the most effective error message types through a large-scale randomized controlled trial conducted in an open-access, online introductory computer science course with 8,762 students from 146 countries. We assess existing error message enhancement strategies, as well as two novel approaches of our own: (1) generating error messages using OpenAI's GPT in real time and (2) constructing error messages that incorporate the course discussion forum. By examining students' direct responses to error messages, and their behavior throughout the course, we quantitatively evaluate the immediate and longer term efficacy of different error message types. We find that students using GPT generated error messages repeat an error 23.1% less often in the subsequent attempt, and resolve an error in 34.8% fewer additional attempts, compared to students using standard error messages. We also perform an analysis across various demographics to understand any disparities in the impact of different error message types. Our results find no significant difference in the effectiveness of GPT generated error messages for students from varying socioeconomic and demographic backgrounds. Our findings underscore GPT generated error messages as the most helpful error message type, especially as a universally effective intervention across demographics.

Decomposition is considered one of the four cornerstones of computational thinking, which is essential to software development [36]. It requires the ability to assess a problem at a high level, develop a strategy to combat it, and then design a solution. Our study focuses on the metacognitive aspect of decomposition. We try to understand the learner's thought process and, specifically, what makes the novice programmer decide to break down a function. Researchers have studied decomposition in introductory programming courses through guided experiments, case studies, and surveys [23, 37]. In this work, we follow a different, more scalable approach. We develop an automated system to analyze 45,000 code snapshots from 168 students for a challenging CS1 programming assignment, detect the pivotal moments when they decide to decompose their programs, and identify what drives their decisions from the code. We then classify the students and study the relationship between the different categories, the code complexity, and the time to derive the final solution. We evaluate the impact of decomposition on the student's performance in the assignment and the course exams. Finally, we discuss the implications of our results for computing education.

Every major technical invention resurfaces the dual-use dilemma -- the new technology has the potential to be used for good as well as for harm. Generative AI (GenAI) techniques, such as large language models (LLMs) and diffusion models, have shown remarkable capabilities (e.g., in-context learning, code-completion, and text-to-image generation and editing). However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks. This paper reports the findings of a workshop held at Google (co-organized by Stanford University and the University of Wisconsin-Madison) on the dual-use dilemma posed by GenAI. This paper is not meant to be comprehensive, but is rather an attempt to synthesize some of the interesting findings from the workshop. We discuss short-term and long-term goals for the community on this topic. We hope this paper provides both a launching point for a discussion on this important topic as well as interesting problems that the research community can work to address.

When students first learn to program, they often focus on functionality: does a program work? In an era where software volume and complexity increase exponentially, it is equally important that they learn to write programs with style so that they are readable and extendable. Writing quality code starts with the building blocks for any program, its functions. A carefully chosen name is vital for program maintainability and manageability. The identifier is the most portable and concise way to summarize what the function does. What makes for the right choice? And can we automatically assess the quality of function names? Using natural language processing, we were able to create a probabilistic model to evaluate their clarity. Using functionality encodings, we attempt to learn the relationship between functions in different programs to improve their names. We analyzed a total of 5,400 programs tackling five novice programming tasks submitted by over 1,000 students in CS1. We developed a software system to automate labor-intensive tasks, detect poor function names and recommend replacements. Our findings suggest that less than 2.5% of name substitutions have an adverse outcome, and in most cases, more than 50% result in an improvement.

Decomposition is a problem-solving technique that is essential to software development. Nonetheless, it is perceived as the most challenging programming skill for learners to master. Researchers have studied decomposition in introductory programming courses through guided experiments, case studies, and surveys. We believe that the rapid advancements in scientific fields such as machine learning and natural language processing (NLP) opened up opportunities for more scalable approaches. We study the relationship between problem-related entities and functional decomposition. We use an automated system to collect 78,500 code snapshots from two CS1 programming assignments of 250 students and then apply NLP techniques to quantify the learner's ability to break down a problem into a series of smaller, more straightforward tasks. We compare different behaviors and evaluate at scale the impact of decomposition on the time it takes to deliver the solution, its complexity, and the student's performance in the assignment and the course exams. Finally, we discuss the implications of our results for teaching and future research.

In introductory CS programming courses, student learning is often assessed on the basis of the submitted code. However, the final artifact fails to capture the problem-solving journey. Active learning occurs when a student stumbles upon conceptual unclarities, design dilemmas, algorithmic challenges. How can we shine light on hidden programming aspects to help teachers provide insightful feedback to learners? We developed a tool to analyze programs snapshots from the entire development process, visualize their evolution over time, filter syntax errors, and detect complex code. The tool is equipped with an editor to explore different ideas and execute the program on the fly, especially in one-on-one feedback sessions with the learner. We are eager to share our work with researchers and educators in other institutions and look forward to their feedback and ideas for improvement.