Automated unit test generation promises to reduce the cost of software quality assurance, and hence, is attracting attention from both academia and industry. Yet, generating assertions that are executable, meaningful to developers, and able to catch faults remains an unsolved challenge. Existing approaches either randomly enumerate assertions that are plausible based on static program analysis without considering whether they naturally fit the test prefix or query an LLM to generate assertions based on local context only, such as the test prefix and the focal method. However, we observe that local context alone is insufficient for LLMs to generate high-quality assertions because many desirable assertions are built from components that are almost impossible to guess for an LLM, such as sequences of multiple method calls. This paper presents STARS, a novel test assertion generation technique that combines the benefits of static program analysis and LLM-based synthesis. The key idea is to first gather a set of assertion components based on static program analysis and to then combine, concretize, prioritize, and improve them with an LLM. The resulting assertions go beyond what an LLM alone could realistically guess based on the test prefix and focal method, and they naturally fit the given test case. Empirical results show that STARS consistently outperforms the state-of-the-art baseline in five evaluation metrics: STARS achieves an exact-match rate of at most 66.2%, and its mutation score nearly triples that of the baseline (10.97% vs. 3.97%), approaching that of developer-written assertions, while consuming 24.75% fewer tokens and 30.6% fewer LLM queries
Large language models (LLMs) have achieved impressive performance on text-based programming tasks, yet they remain unreliable for block-based languages like Scratch. Scratch programs feature deeply nested, non-linear structures, event-driven concurrency across multiple sprites, and tight coupling between code and multimedia assets, properties that are fundamentally different from those of textual code. As a result, LLMs frequently misinterpret Scratch semantics and propose large, invasive edits that are syntactically valid yet semantically misaligned when repairing buggy programs. We introduce ScratchNet, the first executable benchmark designed to systematically evaluate and advance LLM-based repair methods for Scratch programs, spanning program understanding, debugging, analysis, and repair. The benchmark comprises 100 carefully curated Scratch projects from the public repository, each chosen for high structural and semantic complexity. Every project is paired with executable test suites, bug descriptions and corresponding fixes, block-level edit constraints that define minimal semantically correct repairs, and associated multimedia assets. We construct the benchmark via a human-in-the-loop pipeline that combines automated project mining with expert validation of trigger mechanism–outcome semantics and representative bug patterns, with particular focus on event ordering, concurrency, and state management. To enable rigorous and reproducible evaluation, we propose a three-layer, executable protocol that measures functional correctness via VM-level execution, repair quality in terms of minimality and semantic fidelity using block-level edit distance and behavioral trajectory comparisons, and explanation quality through structured rubrics assessing alignment between the model’s reasoning and its patch. Using this benchmark, we investigate fundamental research questions such as the impact of domain-specific fine-tuning on repair success, the effectiveness of different training data sources, and the ability of models to generalize to unseen bug types. Our benchmark establishes a reproducible foundation and a closed-loop framework for evaluating and post-training LLMs on block-based programming tasks.
Background: The use of large language models (LLMs) for automated code review has brought significant change to a time-consuming part of software engineering. Prior work has shown that LLM-based code tools can improve code quality and enable more robust software development processes. As the tools get more powerful, the explanations behind their decisions remain hard to understand. Developers struggle to assess the validity of LLM-generated code reviews, making it difficult to gauge how much trust they should place in them. While the application of automated code review with LLMs has been extensively researched, the inclusion of Explainable AI (XAI) for transparency in code reviews and its impact on trust are yet to be explored. Objective: We aim to address this research gap by collecting empirical evidence on the influence of XAI on the trust of software developers in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants from diverse programming backgrounds, comparing three experimental LLM-based automated code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants were shown a series of real-world code change requests along with the AI-generated code reviews. During the study, we measured trust perceptions for each system using a questionnaire, agreement with the AI recommendation, the reasoning for accepting or rejecting the code change, and the time taken to review the code change. Results: Our quantitative results show that the level of explanation significantly influences both the level of trust of software developers and their agreement with AI recommendations, but in different ways. Full explanations (Condition A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement with AI recommendations, whereas moderate explanations (Condition B) achieve the highest AI agreement (89.22%). This could suggest that more explanations prompt developers to question AI recommendations more frequently. In contrast, providing no explanations (Condition C) results in the lowest levels of trust and agreement. We also find that the level of explanation did not significantly impact the time taken to accept or reject a code change. Across all conditions, the most commonly cited reasons for code change decisions were changes in code readability and the correctness of the implementation. Conclusion: Overall, these findings indicate that incorporating XAI into the code review process significantly changes the trust perceptions and agreement with AI recommendations for software developers. These results provide insights for the design and evaluation of trustworthy AI-based code review systems, and support researchers in the design of studies on the human factors of AI-assisted software development.
State-of-the-art log compressors typically rely on a decoupled “parse-then-compress” workflow, where parsing is optimized for semantic accuracy (i.e., event identification) rather than storage efficiency. Through a comprehensive empirical study, we reveal that this architectural decoupling prevents the exploitation of deep correlations between static templates and dynamic variables. To address it, we propose LogNexus based on the principle of unified redundancy encoding, a new log compression paradigm that co-designs structural extraction and variable encoding. LogNexus constructs a Unified Redundancy Tree (URT) using a hierarchical strategy that progressively mines frequent “structure+variable” patterns in logs. Such a design captures deep contextual redundancies ignored by traditional methods while minimizing computational overhead by pre-emptively encoding dominant patterns. Extensive evaluation on 16 benchmark datasets demonstrates that LogNexus establishes a new state-of-the-art. It achieves the highest compression ratio on 14 datasets (outperforming baselines by 9.48% to 89.13%) and the fastest speed (1.51× to 40.06× faster than competitors). Furthermore, when configured in non-chunked mode to maximize global pattern discovery, LogNexus boosts its compression ratio by 285.13%, which is 27.08% higher than the best baseline, while retaining a 2.43× speed advantage. The decompression audit further shows that LogNexus successfully restores every token on all 16 datasets.
WebAssembly (Wasm) is a prominent programming language enabling high-performance execution across diverse computing environments. However, bugs in Wasm runtimes, which execute Wasm binaries, can lead to severe security breaches and system failures. Manually debugging Wasm binaries that trigger such runtime bugs is exceedingly difficult due to their poor human readability, stemming from low-level stack-based instructions, complex control flow structures, and extraordinary length, often containing millions of instructions. Thus, there is a critical need for automated reduction techniques that minimize Wasm binaries while preserving the ability to trigger the original bug. However, existing reducers often suffer from significant limitations in effectiveness and efficiency: language-agnostic reducers frequently generate invalid variants without Wasm validation awareness, while Wasm-specific reducers still lack efficient delta-debugging-style intra-function instruction-sequence reduction and reduce non-function definitions inefficiently. To address these challenges, we propose E$^2$WR, an effective and efficient reduction framework for Wasm binaries. E$^2$WR introduces a two-stage approach for intra-function instruction-sequence reduction, consisting of operand-dependency guided instruction reduction and combinational redundancy elimination to enable efficient delta debugging while avoiding unnecessary padding instructions, and a non-function-oriented reduction approach that combines static analysis with delta-debugging-guided trials to remove property-irrelevant non-function definitions efficiently. Compared with Wasm-Shrink and Wasm-Reduce, E$^2$WR produces binaries that are 95.4% and 63.9% smaller, respectively, and achieves reduction speed improvement of 11.1$\times$ and 5.1$\times$, respectively.
