Given the discrete nature of many software engineering (SE) optimization problems, trapping at local optima remains a crucial issue in the optimizer designs. Meta Multi-Objectivization (MMO) is an optimization model for overcoming local optima by introducing an extra, auxiliary objective. Yet, MMO is originally designed only for configuration tuning, leaving its suitability for other SE problems in doubt, particularly regarding the choice of the auxiliary objective. In this paper, we study the possibility and resolution for specializing MMO as a general baseline across a range of SE optimization problems, making it a fundamental counterpart for future SE optimizers that seek to mitigate the issue of local optima. We do so by providing both theoretical remit and empirical evidence: we analytically show how the choice of the auxiliary objective is crucial while demonstrating the effectiveness of MMO under nine SE optimization problems comprising 212 cases when the right one can be chosen, i.e., MMO is able to outperform the other state-of-the-art and general-purpose optimizers in 79
Reinforcement learning (RL) pipelines coordinate practices such as configuration, reproducibility, logging, checkpointing, training, and evaluation. Information about these practices is often distributed across source-code files and framework-specific constructs, making RL pipelines difficult to understand, particularly for developers with limited RL experience. We investigated whether automatically generated architectural reports, provided alongside source code, support the comprehension of RL pipeline practices. In a controlled experiment with 174 participants, each participant inspected two RL pipelines: one using source code alone and the other using source code accompanied by a report generated by the RL Pipeline Insights Service. We assessed comprehension using task correctness as the primary outcome and task duration as a secondary process measure. Linear mixed-effects models accounted for repeated observations and examined the associations of report availability, inspected pipeline, and their interaction with both outcomes. Report availability was associated with an average increase of 0.0549 in task correctness, corresponding to approximately 5.5 percentage points (95% CI [0.0341, 0.0757], Benjamini–Hochberg-adjusted p < 0.001). Report-assisted tasks also took approximately 9.4% longer on average (log-scale estimate = 0.0898, 95% CI [0.0127, 0.1669], adjusted p = 0.034). The report-by-pipeline interaction was not statistically supported for either correctness or duration, and no statistically supported relationship was found between correctness and duration. These findings suggest that generated architectural reports can support more accurate identification of selected RL lifecycle and architectural practices, but this improvement is accompanied by additional reading and cross-referencing time. The reports should therefore be considered a complement to source-code inspection rather than a means of reducing task-completion time.
Developers insert logging statements in source code to record its execution, and the resulting logs are essential for detecting and troubleshooting system anomalies. A key part of these statements is the log level (e.g., DEBUG, INFO, WARN, ERROR), which determines whether a message is recorded. Choosing it is a trade-off: levels set too low mask important information in production, while levels set too high create noisy logs and extra I/O and storage overhead. Recent work uses Large Language Models (LLMs) with In-Context Learning (ICL) to predict log levels, but typically augments the prompt with randomly sampled examples. We argue that this overlooks the socio-technical nature of logging, where conventions vary by component and development team. Off-the-shelf LLMs using random examples are reported to achieve median AUCs of only 0.56 to 0.76, leaving room for improvement without fine-tuning. In this paper, we propose a retrieval approach for LLM-powered log level prediction that prioritizes context relevance over random selection and basic similarity. Our method retrieves logging examples relevant along two socio-technical dimensions: (i) functionality (code performing similar operations) and (ii) ownership (code maintained by the same developers). Evaluating four LLMs across ten popular Java projects, we find that this retrieval improves AUC over random baselines by 0.07 (median), and combining both signals yields median AUCs of 0.83 to 0.97. Our ablation studies show that each signal improves on random retrieval, and that combining them outperforms the stronger of the two on all ten projects. Our results also significantly exceed state-of-the-art supervised predictors (e.g., DeepLV) by a median AUC of 0.12 to 0.19. Our findings highlight the importance of socio-technical signals such as code functionality and ownership for LLM-powered log level prediction.
Flaky tests are test cases that fail non-deterministically without code modifications. They disrupt continuous integration, waste developer time, and obscure real faults. While prior work largely treats flakiness test-by-test, we show that failures often co-occur in ways that suggest shared causes, a phenomenon we call systemic flakiness. This raises the possibility that developers could reduce repair effort by identifying shared failure mechanisms and investigating co-failing tests together. We study this across 36 open-source Java and Python projects by combining two datasets with 10,000 (Java) and 12,000 (Python) test suite runs per project. We cluster flaky tests by failure co-occurrence to quantify the phenomenon and analyze how different rerun budgets influence its manifestation. We then evaluate machine learning models trained on static test case distance metrics to predict co-occurrence as a way to suggest which known flaky tests may warrant joint inspection, and qualitatively examine likely causes. Systemic flakiness is prevalent: 77% of flaky tests belong to a cluster, with a mean cluster size of 21.1. On average, 47% of flaky tests and 49% of the number of clusters are identified by 2,000 test suite runs. The proportion of detected flaky tests grouped into a cluster remains high even under reduced rerun budgets, although the resultant clusters only partially approximate the clusters computed from the full set of runs. The Extra Trees model achieved a mean R2 of 0.79 when predicting failure co-occurrence between flaky tests using distance metrics that include those based on embeddings of test case source code, reflecting partial predictive power. Qualitative analysis shows that systemic flakiness mainly arises from intermittent network problems and unstable external services or libraries, with filesystem pollution and resource mismanagement also being common.
Mutation analysis is a fundamental dynamic technique for software testing and analysis. Its typical applications, such as evaluating test effectiveness and fault localization, have proven highly effective. However, its inherent scalability limitations hinder its adoption in practice. Recently, a family of fork-based mutation analysis approaches has significantly accelerated mutation analysis by reducing redundant executions among mutants. To achieve this, fork-based mutation analysis carries all mutants in one process and forks new child processes when needed. For example, when the execution engine reaches a mutated statement, it interprets each active mutant and clusters them by their states. The engine forks a new child for each cluster to share the common execution among the mutants. However, the existing fork-based approaches suffer from two key limitations: (1) they compare mutant states at the granularity of individual statements, which limits opportunities for clustering and merging mutants; and (2) newly forked processes still incur substantial redundant execution. To address these limitations, we propose WinMut-Turbo, a novel fork-based mutation analysis approach that (1) expands the analysis scope from individual statements to basic blocks; (2) partitions oversized basic blocks to reduce dynamic analysis cost; (3) eliminates redundant interpreter computations; and (4) enables output-aware early termination of child processes whose mutants are guaranteed to be killed under the append-only condition. We evaluate WinMut-Turbo on ten real-world, large-scale C programs, with more than 20 million mutants and 1,149 tests. Experimental results show that WinMut-Turbo achieves a geometric mean speedup of 9.7× over the state-of-the-art fork-based approach AccMut, substantially improving the efficiency of mutation analysis. Moreover, our ablation study confirms that each optimization component contributes non-trivially to the overall performance improvement.
