Assessing the security and safety robustness of Automated Driving Systems (ADS) requires more than simulation mileage: rare semantic combinations of surrounding-agent behaviors and environmental context create blind spots that expose brittle planning decisions. We propose SPEC , an exposure metric characterizing how extensively an ADS's planning response is exercised across vehicle state, environment, and interaction criticality. SPEC is computed over a unified per-scene abstraction, HSR (Holistic Scene Representation) , a digital fingerprint for forensic analysis of behaviors and failure patterns. Across over one million scenario runs, HSR yields more consistent behavioral partitions and SPEC correlates strongly with the discovery of distinct failure manifestations. To operationalize exposure discovery, we introduce COAX ( C ollective O rchestration of A gents for e X ploration), an automated adversarial orchestrator that adaptively coordinates surrounding agents to steer the ADS into interaction regimes whose HSR fingerprints are absent from the accumulated corpus, thereby broadening SPEC-measured exposure. Under the same simulation budget, COAX uncovers up to 556% more distinct failure manifestations than baselines across multiple ADS platforms; fine-tuning downstream agents on the corpora it generates reduces collision rates by up to 75–78%. Together, SPEC and COAX provide a scalable framework for robustness assessment and vulnerability-oriented exposure discovery in ADS planning under multi-agent interaction.
Model-Driven Engineering (MDE) supports the development of large-scale, complex systems through abstraction, automation, and traceability. The rise of Large Language Models (LLMs) has renewed interest in reshaping MDE, with recent studies exploring their use in tasks such as model generation, model comprehension, as well as model to code and test generation. While early results suggest improved productivity and lower barriers to modeling, their overall impact remains unclear. To address this, we conduct a Systematic Literature Review (SLR) and analyze 235 primary studies. We examine MDE tasks addressed, how LLMs were leveraged, benchmarking and evaluation practices, and the availability of tools and datasets. Results indicate a rapidly growing research trend, covering 25 types of MDE tasks, with model generation as the dominant focus. Unified Modeling Language (UML), Domain-Specific Languages (DSLs), and Business Process Model and Notation (BPMN) are the most frequently targeted formalisms. GPT-family models and zero-shot prompting are most commonly used, with fine-tuning and retrieval-augmented generation (RAG) as typical enhancement techniques. API-based LLM access and single-LLM architectures are the prevailing design choices. Benchmarking and evaluation studies remain narrow in scope. Evaluations primarily emphasize correctness, with limited benchmarking in realistic settings and insufficiently comprehensive replication materials.
Missing security checks on critical variables are an important class of semantic bugs in the Linux kernel and can lead to severe vulnerabilities. Existing approaches largely rely on static analysis, but heuristic rule-based and statistical approaches may fail to capture the semantic context needed to determine whether a security check is required. Recent advances in large language models (LLMs) offer an opportunity to complement static analysis with semantic reasoning. However, directly applying LLMs to kernel code remains challenging, particularly in identifying critical variables and interpreting security-check semantics. To address these challenges, we present LLMSA-MC, a framework that combines static analysis with LLM-based semantic reasoning for missing-check bug detection. LLMSA-MC incorporates three strategies: (1) Statement Type Classification, which guides LLMs to identify critical variables; (2) Precise Code Context Extraction, which provides relevant code context for semantic reasoning; and (3) Function Label Generation, which annotates function return values with semantic labels. Evaluation on curated intra- and inter-procedural benchmarks shows that LLMSA-MC improves F1-score by up to 39.01 and 33.66 percentage points, respectively, over baseline chain-of-thought (CoT) prompting. When integrated with a representative missing-check bug detection tool, LLMSA-MC reduces candidate reports by 55.38% while confirming 168 missing-check bugs in Linux kernel v4.20-rc5.
The Model Context Protocol (MCP) introduces a standard specification that defines how Foundation Model (FM)-based agents should interact with external systems by invoking tools. However, to understand a tool’s purpose and features, FMs rely on natural-language tool descriptions, making these descriptions a critical component in guiding FMs to select the optimal tool for a given (sub)task and to pass the right arguments to the tool. While defects or smells in these descriptions can misguide FM-based agents, their prevalence and consequences in the MCP ecosystem remain unclear. Hence, we examine 856 tools spread across 103 MCP servers empirically, assess their description quality, and their impact on agent performance. We identify six components of tool descriptions from the literature, develop a scoring rubric utilizing these components, and then formalize tool description smells based on this rubric. By operationalizing this rubric through an FM-based scanner, we find that 97.1% of the analyzed tool descriptions contain at least one smell, with 56% failing to state their purpose clearly. While augmenting these descriptions for all components improves task success rates by a median of 5.85 percentage points and improves partial goal completion by 15.12%, it also increases the number of execution steps by 67.46% and regresses performance in 16.67% of cases. These results indicate that achieving performance gains is not straightforward; while execution cost can act as a trade-off, execution context can also impact. Furthermore, component ablations show that compact variants of different component combinations often preserve behavioral reliability while reducing unnecessary token overhead, enabling more efficient use of the FM context window and lower execution costs.
Code review is critical to software quality assurance, with defect detection as a primary objective. While LLMs enable automated review comment generation, most approaches treat code review as a general-purpose task, favoring frequent but low-risk feedback. We propose DefectReviewer , a defect-focused review comment generation approach combining task-specific fine-tuning with fine-grained localization signals. The key insight is that while task-specific fine-tuning establishes the foundation for defect-focused review, grounding the model in where the defect is provides complementary guidance to narrow the reasoning space for what the defect is . DefectReviewer uses a Just-In-Time localization tool to identify defect lines, grounding review comment generation through structural prompting and fine-tuning. DefectReviewer is designed as a specialized, selectively applicable component that complements general-purpose reviewers. We evaluate DefectReviewer on a large-scale real-world dataset against six representative baseline methods. DefectReviewer consistently outperforms state-of-the-art baselines, improving BLEU and ROUGE-L by 17.0% and 24.2%, respectively. A dual-track semantic evaluation combining LLM judges with human expert validation confirms that DefectReviewer produces comments with higher defect relevance and readability. Additional analyses show that it can be applied to different generation architectures and that its performance changes moderately as the number of input defect lines varies, although the trend differs across architectures.
