The introduction of AI into the developer workflow has fundamentally changed the conversation around productivity, but merely adopting tools is not enough. Drawing on insights from the 2025 DORA State of AI-assisted Software Development report, this keynote argues that AI is an amplifier that magnifies an organization's pre-existing technical and cultural capabilities. It will show why the greatest returns on AI investment come not from the models themselves, but from the maturity of the underlying software delivery system. We will explore a real-world transformation, using the journey of Unico IDTech https://unico.io as a prime example. This story demonstrates how creating a robust and consistent engineering structure Dev Prime coupled with a radical shift to user-centric monitoring SLOs and reliability engineering created the necessary high-quality Platform foundation. When this foundation is present-characterized by clear governance, healthy data ecosystems, and foundational practices-AI's benefits such as throughput and organizational performance gains are dramatically amplified. This session provides technology leaders with a clear, data-driven mandate: to maximize the value of AI, you must first invest in and rationalize your foundational structure, treating your internal platform as the critical enabler for sustainable AI-driven excellence.
How does fault tolerance affect latency in a system? This talk examines this issue in the context of crash faults for vote-based consensus in blockchains. Specifically, the timers that are often used for fault tolerance can make consensus time non-monotonic with respect to timer value and number of faults. Also, the protocol design for fault tolerance can affect its performance scalability, but this depends on the underlying network topology.
Neural Code Models (NCMs) are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models arrive at decisions or whether practitioners trust NCMs' outcomes. In this talk, I will introduce doCode, a post hoc interpretability framework specific to NCMs that can explain model predictions. doCode is based upon causal inference to enable programming languageoriented explanations. While the theoretical underpinnings of doCode are extensible to exploring different model properties, we provide a concrete instantiation that aims to mitigate the impact of spurious correlations by grounding explanations of model behavior in properties of programming languages. doCode can generate causal explanations based on Abstract Syntax Tree information and software engineeringbased interventions. To demonstrate the practical benefit of doCode, I will present empirical results of using doCode for detecting confounding bias in NCMs as well as detecting code smells in the code generated by state-of-the-art NCMs, such as CodeLlama and Mistral. Finally, I will showcase results of our recent work on how causal inference can be used in the context of secure software engineering research.
As the use of machine learning ML becomes more prevalent in many practical software systems, maintaining the long-term availability of ML systems has emerged as a new critical challenge. In ML system operations, ML models continuously confront the risk of performance degradation due to dataset shifts or security attacks. As the system performance and quality may deteriorate over time, implementing proper mitigation methods to ensure the high availability of ML systems is essential. This talk introduces recent approaches to enhance the reliability and availability of ML systems through N-version configuration, software rejuvenation, and ML model updates. While these methods are based on wellestablished techniques, several unique perspectives, specifically tailored for ML-based systems, are discussed. The talk will also highlight future challenges for high-availability ML systems, which may consist of multiple dependent ML and AI models.
The growing sophistication of cyber threats demands dynamic and adaptive security approaches, especially in cloud computing environments where shared resources create a large attack surface. In this context, Moving Target Defense (MTD) emerges as a promising paradigm that hinders the predictability of the infrastructure by dynamically altering its properties. This paper presents a practical analysis of the MTD technique, implemented through virtual machine (VM) migration in an environment managed by Apache CloudStack. The objective is to evaluate its effectiveness as a proactive security strategy against distinct attack vectors, including Distributed Denial of Service (DDoS) and microarchitectural stress attacks targeting the CPU. The methodology involved configuring a cloud environment with multiple physical hosts, simulating attacks, and monitoring the system. Preliminary results indicate that MTD contributes to stabilization and reduction of resource consumption (CPU and memory) under attack, reinforcing its potential as a complementary defense strategy to enhance availability.
Microservices are typically deployed as replica sets to serve high workloads of requests while guaranteeing high availability and reliability. Being designed for continuous longterm operation, replica-based systems suffer from software aging, leading to progressive performance degradation and possibly to failures. This problem is exacerbated by the heterogeneity of the endpoints of a microservice in terms of workload characteristics, software aging effects produced on the replicas, and QoS requirements. Thus, configuration of replica sets is challenging. In this work, we address quantitative modeling and evaluation of software aging and rejuvenation in replica sets. Specifically, we define Generalized Stochastic Petri Nets (GSPNs) that model the workload, service process, and aging effects characterizing each endpoint, as well as software rejuvenation and repair of the replicas. Steady-state analysis of these models enables deriving quantitative measures of interest, including the expected number of requests yielding a replica failure and the expected number of rejected requests. The analysis results show the effectiveness of our approach in determining the convenience of different replica set configurations, i.e., joined configurations, where two endpoints share a set of replicas, versus separate configurations, where each endpoint has exclusive access to a subset of replicas.
Bug reproduction is a fundamental task in software maintenance and debugging, essential for fault localization, patch generation, and regression testing. However, it remains challenging due to vague descriptions, missing steps, and inconsistent environments in bug reports. These difficulties are amplified in deep learning (DL) frameworks such as TensorFlow, PyTorch, and MXNet, which involve complex computation graphs, hardware dependencies, and rapidly evolving APIs. To address this challenge, we propose ARBRGPT, a prompt-based framework that leverages large language models (LLMs) to automatically reproduce DL bugs from natural language reports. ARBRGPT extracts key entities and builds structured prompts incorporating keyword (KW) abstraction, sequence to reproduce (S2R), chain-of-thought (CoT) reasoning, and few-shot examples. These prompts guide the LLM to generate minimal executable code that faithfully reproduces the reported issue. Evaluated on 155 real-world bug reports from PyTorch, TensorFlow, and MXNet, ARBRGPT achieves an average reproduction success rate exceeding 80 %, significantly surpassing template-based baselines. The results demonstrate that carefully designed prompt components play complementary roles in boosting reproduction performance, offering actionable insights into building more effective LLMbased debugging systems for DL frameworks.
This position paper argues that the assurance of LLMs is particularly prone to a ‘Context Fallacy'. We define the context fallacy as the assumption or presumption that practices that hold true for different contexts - either time or operational contexts - can, will or do apply to LLMs. We introduce examples of the context fallacy and briefly suggest future research directions to ameliorate this issue.
Moving Target Defense (MTD) is a cybersecurity strategy aimed at increasing the complexity and cost for attackers by continuously shifting or modifying a system's attack surface. A possible way to introduce an MTD strategy in private cloud systems is to move a virtual machine (VM) from one host to another. However, this operation may generate software aging that gradually decreases system performance and increases failure rates during prolonged operation. This paper investigates the effects of prolonged system operation in cloud computing environments, focusing on resource consumption and performance degradation over time. A dedicated experimental setup was used to monitor VM behavior via automated workloads. The results showed progressive increases in RAM and CPU usage, indicating clear signs of software aging. This study highlights the need for continuous monitoring and proactive maintenance to ensure the long-term reliability and efficiency of cloud-based systems.
Memory integrity in AI systems remains a foundational weakness in both training reproducibility and operational trustworthiness. Large-scale vector data, embeddings, and internal memory traces are discarded, rewritten, or silently drift over time without cryptographic anchors or verifiable context. This paper introduces AnchorChain, a mechanism for binding AI memory and volume-based data structures (e.g. embedding matrices, long-term memory vector stores) to blockchain-indexed anchors. Each memory state is summarised into a cryptographically verifiable Merkle root, indexed in a deterministic fashion, and immutably committed to a transaction on a Bitcoin-based ledger. This enables traceable recovery, rollback, and accountability in AI training and inference workflows-resolving the problem of memory state erosion across system restarts, deployments, or trust boundaries. The system supports deterministic reference resolution, anchor-based recall indexing, and verifiable memory finality, solving a critical missing reliability gap in AI-driven software engineering.
Large Language Models (LLMs) are increasingly wrapped in agents that orchestrate external tools, memories, and planning components to perform mission-critical functions. Despite growing interest in security testing of LLMs, the dependability of such agents remains largely unexplored. Existing robustness evaluations focus on prompt-level attacks and overlook non-adversarial system-level faults that routinely arise in real deployments (e.g., tool crashes or timeouts). We present ChaosLLM, a lightweight fault injection framework that sits between an LLM agent and its environment, allowing researchers to emulate realistic failures and quantify their impact on task success and recovery behavior. This paper focuses on tool-calling capabilities as a starting point for a broader investigation. We detail our experimental design, define the dependability metrics, and run the experiments on a LangChain ReAct agent. Our preliminary results motivate the need for further research efforts. Finally, we discuss how ChaosLLM can be extended in future work.
Early-exit Deep Neural Networks (EE-DNNs) embed intermediate classifiers along the model architecture, enabling local inference when the confidence of an early exit surpasses a threshold. While this design reduces dependence on cloud processing, fixed thresholds are unable to adapt to performance degradation caused by transient software faults or hardware-software interaction faults. Such faults may manifest as latency spikes, processing slowdowns, or increased drop rates, especially in edge-cloud collaborative inference scenarios. This work proposes a fault-resilient adaptive offloading framework that combines multi-armed bandit (MAB) threshold selection with a priority-based input buffer. The adaptive controller responds to varying conditions, including simulated network degradation, cloud-side slowdowns, and burst overloads, by jointly optimizing the accuracy-latency trade-off and ensuring timely processing of high-priority inputs. We evaluate three buffer policies-FIFO, Strict Priority, and a Hybrid PriorityAdmission FIFO-in a simulation-based environment using MobileNetV2 with a single early exit. Results show that the hybrid policy consistently achieves higher goodput and lower drop rates under fault-injection scenarios, demonstrating the benefits of combining adaptive thresholding with priority-aware buffering for robust EE-DNN operation in fault-prone edge-cloud systems.
Coverage-guided fuzzing has proven highly effective in uncovering software vulnerabilities. However, many industrial systems cannot be instrumented or inspected due to security, legal, or operational constraints. In such settings, blackbox test generation remains the only viable testing strategy. This paper presents a case study in collaboration with our industrial partner, Mettle Networks, where we were tasked with testing the protocol-handling subsystem of a virtualized packet processing engine without access to the source code or internal documentation. To overcome this constraint, we applied blackbox grammar inference techniques to reverse-engineer the input protocol accepted by the system, achieving an F1 score of 0.94 for the inferred protocol grammar. Our blackbox test campaign resulted in 33.6% code coverage, as measured by Mettle Networks after the test campaign concluded. We discuss our testing methodology and the practical challenges encountered. This case study highlights current limitations of automatic testing frameworks when deployed in blackbox industrial environments, and offers actionable insights for improving their effectiveness in such settings.
AI-driven intelligent testing has advanced rapidly, enabling automated test-case generation, defect prediction, and risk assessment. However, the absence of explicit integration of human factors into the testing process often leads to the neglect of testers' cognitive attributes and domain expertise, thereby amplifying cognitive biases and exacerbating safety risks. This paper proposes a three-layer Human-in-the-Loop Intelligent Testing (HITL-IT) framework that systematically incorporates human factors into AI-based testing for safety-critical software. The framework consists of a human-factor modeling layer, an AI testing core, and an interactive feedback loop, collectively forming a closed-cycle mechanism of “suggestion-challenge-refinement-relearning.” In the context of AI test-case generation, this framework is designed to substantially improve both the quality and efficiency of generated cases. Preliminary applications show promising potential for this approach. By embedding explicit human-factor models and closed-loop feedback into the testing workflow, HITL-IT provides a novel and practical paradigm for building more trustworthy, resilient, and safety-critical AI testing systems.
Accessibility defects in modern web applications often stem from human factors such as cognitive overload, insufficient training, and the difficulty of interpreting complex accessibility audit reports. In Brazil, despite the legal requirements of the Brazilian Inclusion Law (Lei Brasileira de InclusaoLBI, Law 13.146/2015), recent studies show that less than 1of national websites comply with the Web Content Accessibility Guidelines (WCAG) 2.1. These gaps are not only technical but also cognitive and organizational, reflecting a persistent mismatch between detection capabilities and remediation practices. In large-scale React applications, the component-based architecture amplifies the challenge by introducing framework-specific pitfalls that require specialized expertise. This paper introduces AccessibilityAgent, a human-aware, multi-agent architecture that autonomously repairs accessibility violations using locally deployed Large Language Models (LLMs). By automating repetitive and error-prone tasks, the system reduces developers' decision fatigue and cognitive burden, while its validation agents preserve functional correctness and maintain trustworthiness. Unlike monolithic approaches, our design coordinates specialized agents for auditing, planning, retrieving, fixing, and validating repairs, enabling an adaptive workflow that integrates human oversight where necessary. We discuss how this architecture addresses human-factor-related risks in accessibility maintenance, improves software dependability, and supports inclusive design at scale.
