The rapid adoption of Generative AI (GenAI) in the software development life cycle (SDLC) increases computational demand, which can raise the carbon footprint of development activities. At the same time, organizations are increasingly embedding governance mechanisms into GenAI-assisted development to support trust, transparency, and accountability. However, these governance mechanisms introduce additional computational workloads, including repeated inference, regeneration cycles, and expanded validation pipelines, increasing energy use and the carbon footprint of GenAI-assisted development. This paper proposes Carbon-Aware Governance Gates (CAGG), an architectural extension that embeds carbon budgets, energy provenance, and sustainability-aware validation orchestration into human-AI governance layers. CAGG comprises three components: (i) an Energy and Carbon Provenance Ledger, (ii) a Carbon Budget Manager, and (iii) a Green Validation Orchestrator, operationalized through governance policies and reusable design patterns.

Context: While conceptual research on AI in project management is advancing, empirical evidence of actual usage among IT project managers practising is limited. Objective: We investigate how Finnish IT project and programme managers use generative AI in daily work, identifying practices, organisational constraints, and future visions through individual interviews. Method: Using qualitative descriptive design, we conducted semi-structured interviews with 12 experienced Finnish IT project management consultants who work in multiple client organisations. Data were analysed through hybrid inductive-deductive thematic analysis to identify usage practices and future usage. Results: GenAI adoption is fragmented and peripheral, mainly constrained by organisational policies rather than individual resistance. Participants use GenAI mainly for support tasks rather than core project management functions. Despite varied usage, participants converge on envisioning AI as an 'assistant not replacement,' reflecting professional boundary work that preserves human authority. Conclusions: The adoption of GenAI in IT project management is limited by organisational constraints such as security policies and governance structures, which means that organisations should prioritise integration and data protection over bottom-up experimentation. More advanced capabilities remain aspirational, making incremental adoption through low-risk use cases more realistic than broad automation of core project management functions.

Artificial Intelligence (AI) is shifting software engineering from tool-supported processes towards AI-first collaboration, where authority is dynamically distributed across human and artificial actors. However, existing method engineering approaches assume static, human-centric control and provide limited support explicitly capturing evolving autonomy. This paper presents a vision for autonomy-aware method engineering by proposing a metamodel that treats autonomy not as a fixed property of an actor, but as a derived, situation-dependent authority assignment determined by task, context, and collaboration pattern. The metamodel formalizes autonomy through four authority dimensions: task execution, task decomposition, task initiation, and collaboration reconfiguration. Through an analytical instantiation with a multi-agent requirements analysis tool, we illustrate how the metamodel supports dynamic authority assignment. This work provides a conceptual foundation for governance-aware, adaptable, and AI-first software engineering methods.

Smart environments increasingly integrate artificial intelligence (AI) to process sensor data, coordinate Internet of Things (IoT) devices, and actuate cyber-physical services.Within these environments, large language models (LLMs) are emerging as decision-making and interaction components. However, LLM integration expands the attack surface of these environments, and existing benchmarks focus largely on risks associated with text-based interactions and LLM-generated outputs, failing to capture cyber-physical risks that arise when models control physical actuators. This paper introduces Evil-AI Benchmark, an open-source evaluation framework for assessing LLM agents in smart environments through (i) five adversarial threat vectors—prompt injection, persuasion, attacker-in-the-middle (AITM) simulation, data leakage, and unsafe actions—and (ii) a capability validation that checks whether a model can correctly activate device-control tools. The benchmark instruments tool activations and forwards tool-trigger events to an external Arduino-driven servo actuator, providing physical verification of actuation events. To demonstrate the benchmark in practice, we evaluate eight representative LLMs using 350 executable scenarios: 250 adversarial tests spanning five smart-environment domains (smart home, healthcare IoT, industrial control, public infrastructure, and smart building), 50 capability checks, and 50 benign over-refusal tests. Responses are scored by two independent judge models and verified by a human evaluator. We report Evilness as the extent to which attacks succeed. The Evilness Score equals the number of successful attacks, and the Evilness Rate is the corresponding percentage. Across models, Evilness Rates range from 0.8% to 54.0% (2–135 successful attacks out of 250), while Defense Rates range from 46.0% to 99.2%. The most persistent weaknesses occur in persuasion, especially under sustained multi-turn pressure, and in prompt injection. Moreover, larger models are not necessarily more robust against adversarial attacks. We additionally quantify over-refusal on benign authorized requests, so that safety is not rewarded by indiscriminate refusal. Evil-AI Benchmark provides reproducible diagnostics and quantitative metrics that connect text-based safety evaluation with real-world cyber-physical operation.

Code-generating tools are increasingly used in software development, yet experience reports on conversational "vibe coding" under production constraints remain limited. This paper presents an experience report from a small full-stack team that applied contextual prompting and explicit architectural constraints to build (i) a multi-project agent learning platform designed for sustained, production-oriented use and (ii) an academic retrieval-augmented generation system. The agent platform supports multiple isolated projects, each with structured memory and background processing, thereby enforcing project-level isolation. The RAG system provides citation-grounded answers, role-based access control, and evaluation tracking. Across both systems, vibe coding accelerated scaffolding and integration. However, the generated code often under-specified isolation rules and infrastructure constraints when these were not explicitly defined. Consequently, aspects such as multi-tenancy, access control, memory policies, and asynchronous processing required deliberate architectural design and verification. We observe a shift in engineering effort from boilerplate implementation toward constraint specification and enforcement auditing. We also identify recurring architectural "non-delegation zones" where conversational code generation remains insufficient for production reliability.

Collaborative AI experimentation in industry and academia requires environments that support rapid trials while maintaining controlled access, organisational isolation, and traceable workflows. Although interest in AI sandboxes is increasing, practical guidance on designing and building governance-aware experimentation platforms remains limited. This work designs and operationalizes a governance-aware, multi tenant AI sandbox that supports structured experimentation and produces reusable evaluation evidence across stakeholders. The sandbox was developed in an industry and academia ecosystem using iteratively validated requirements gathered from industrial partners. The solution adopts a layered reference architecture that separates a multi tenant presentation layer from a backend control plane and isolates execution and data management concerns into dedicated layers. The sandbox supports governed onboarding, project based collaboration, controlled access to AI services, and traceable experimentation through approval workflows and audit logging. By structuring experiment context and governance decisions as persistent records, the sandbox enables evaluation evidence to be reused and compared across projects and stakeholders. The development experience yields lessons learned and practical considerations that inform deployment and future evolution of governance-aware sandbox platforms.

ABSTRACT The operationalization of AI ethical principles during the requirements engineering phase is paramount for creating ethically aligned AI systems. However, ethical requirements classification remains a significant challenge due to the abstract nature of ethical principles and the lack of practical tools to support developers. We introduce the EthicalRequirements4AI, a dataset comprising 1091 annotated requirements, including ethical and non‐ethical requirements. We also provide the Ethical Requirements Classification for AI (ERC4AI): a BERT‐based multi‐label classification model for classifying ethical requirements in AI, fine‐tuned on our labeled dataset. Passenger Flow and PROMISE datasets were used to construct the EthicalRequirements4AI dataset, encompassing 385 requirements aligned with 11 AI ethical principles and 706 non‐ethical requirements. Three transformer‐based models—XLM‐RoBERTa, BERT, and DistilBERT—were evaluated for multi‐label classification task. BERT demonstrated the most stable overall performance across seeds while achieving the highest macro‐average performance among the evaluated models (XLM‐RoBERTa, BERT, and DistilBERT), achieving a macro F1 score of 0.76 and a weighted F1 score of 0.83 on the test set. Although BERT showed the best aggregate results, performance varied across different ethical principles. ERC4AI offers a practical means for researchers and developers to address ethical concerns early in AI development. The dataset and model are publicly available to foster further research and advancements in practical AI ethics.

Organisations are examining how generative AI can support their operational work and decision-making processes. This study investigates how employees in a energy company understand AI adoption and identify areas where AI and LLMs-based agentic workflows could assist daily activities. Data was collected in four weeks through sixteen semi-structured interviews across nine departments, supported by internal documents and researcher observations. The analysis identified areas where employees positioned AI as useful, including reporting work, forecasting, data handling, maintenance-related tasks, and anomaly detection. Participants also described how GenAI and LLM-based tools could be introduced through incremental steps that align with existing workflows. The study provides an overview view of AI adoption in the energy sector and offers a structured basis for identifying entry points for practical implementation and comparative research across industries.

Requirements elicitation discussions are communication-intensive and require manual effort to produce structured outputs such as product vision. We introduce MARARE, a real-time multi-agent system that converts live team meeting discussions into draft product vision statements and feature descriptions using transcription and a team of agents. We implement a prototype and conduct an exploratory evaluation with practitioners. The results suggest that the system can support real-time documentation, while the generated outputs still require human validation.

Automated authoring of Gherkin Behavior-Driven Development (BDD) acceptance criteria remains a manual bottleneck in requirements engineering. This study investigates whether epic-organized LLM-generated Gherkin produces higher quality and coverage than requirement-aligned generation. We compare our Timeless (an epic-organized LLM pipeline) approach against a naive large language model (LLM) baseline on four requirements documents (107 requirements) from the PURE dataset. Evaluation covers structural metrics, automated requirement coverage via TF-IDF and dense embeddings, and blind expert assessment by four researchers. In our evaluation, the JSON-constrained pipeline produced structurally valid scenarios across all generated outputs, while the zero-shot baseline achieved 99

Context: Software organizations continue to adopt and use generative AI (GenAI). Many software organizations are currently exploring various use cases for GenAI in software engineering (SE). However, software companies face practical challenges in identifying and prioritizing good GenAI use cases.Objective: Our objective is to understand what factors software organizations could consider in practice when selecting GenAI use cases in SE. We study this both in terms of what are the practical cost and benefit considerations involved, as well as in terms of other potential factors to consider.Method: We conducted a single in-depth case study of a Nordic software company. We collected data through semi-structured interviews (n=15) and an internal questionnaire (n=20). Results: We analyzed 15 interview transcripts and 20 questionnaire responses using an inductive, open coding approach. The analysis focused on identifying and categorizing potential selection criteria related to GenAI use case selection and prioritization in SE. Based on 255 coded segments, we identified 5 main categories, split into a total of 21 subcategories of criteria.Conclusion: The five categories of GenAI use case selection criteria for SE contexts that we present in this paper provide a starting point for both researchers and practitioners to build on in relation to GenAI transformation and GenAI use case exploration in SE. The potential selection criteria are in and of themselves a practical contribution as well.

Large language models (LLMs) accelerate software development but often exhibit instability, non-determinism, and weak adherence to development discipline in unconstrained workflows. While test-driven development (TDD) provides a structured Red-Green-Refactor process, existing LLM-based approaches typically use tests as auxiliary inputs rather than enforceable process constraints. We present an AI-native TDD framework that operationalizes classical TDD principles as structured prompt-level and workflow-level governance mechanisms. Extracted principles are formalized in a machine-readable manifesto and distributed across planning, generation, repair, and validation stages within a layered architecture that separates model proposal from deterministic engine authority. The system enforces phase ordering, bounded repair loops, validation gates, and atomic mutation control to improve stability and reproducibility. We describe architecture and discuss encoding software engineering discipline directly into prompt orchestration, which we think offers a promising direction for reliable LLM-assisted development.

The ongoing evolution and societal impact of AI systems demand systematic methods to embed ethics into AI development. Existing approaches often struggle to translate high-level ethical principles into concrete, adaptable software requirements, resulting in “ethical debt” that risks reputational harm, regulatory issues, and diminished stakeholder trust. This paper introduces the Ethical Requirements Stack (ERS), a structured, multi-layered artifact designed to elicit, decompose, and manage ethical requirements (ERs) from abstract themes to actionable development tasks. The ERS is operationalized through a human–AI collaborative workflow that leverages GPT-based Large Language Models (LLMs) for scalable ideation, complemented by human oversight to ensure contextual and ethical alignment. Using a design science research methodology, we demonstrate how the ERS supports the translation of stakeholder-elicited ethical values—aligned with frameworks such as IEEE 7000™-2021—into traceable software specifications. Our findings show that the ERS enables structured ethical reasoning and highlights the complementary strengths of AI-generated breadth and human critical judgment. This work contributes a practical approach for integrating ethics into the AI development lifecycle, supporting responsible innovation and reducing ethical debt through a combination of human-centered design and LLM-assisted requirements engineering.

LLM-based agents for automated code repair have received significant attention in recent years from both research and software engineering practice perspectives. However, limited attention has been paid to patches that pass syntactic and functional verification but still retain or introduce security vulnerabilities. The aim of this research is to systematically identify and categorize such silent failures in LLM-based agentic code repair. We conducted an empirical study using 1,030 valid execution traces produced by seven agent frameworks with GPT-4o-mini across two security-focused datasets, SecurityEval and CVEfixes. Through three iterations of qualitative coding and manual verification, 170 confirmed silent failures were identified. The key results are: (i) Three main categories of silent failures were identified: Omission, Introduction, and Inadequacy. Omission accounts for 48.2% of the confirmed failures, Introduction for 30.6%, and Inadequacy for 21.2%. (ii) Ten fine-grained failure codes were classified under these three categories, showing how agents omit required security controls, apply incomplete defenses, or introduce new vulnerabilities during repair. (iii) Current test-passing evaluation and LLM-based reviewer roles were insufficient to expose or intercept these failures in the confirmed cases. (iv) Similar insecure solutions appeared across different frameworks, suggesting possible shared model-, prompt-, or task-level influences, while single-agent and multi-agent systems showed different failure profiles. The results of this study will assist researchers and practitioners in improving the evaluation of LLM-based agentic code repair and developing targeted verification methods that go beyond functional correctness and cover all generated artifacts.

[Context and motivation] User stories are central in agile software development, yet creating and managing high-quality ones remains challenging. Existing tools for user story quality enhancement often lack semantic analysis or project-specific context alignment. [Questions] This paper investigates how multi-agent systems (MAS) based on large language models (LLMs) can refine user story quality and strengthen traceability by linking them to project artifacts, while preserving human oversight. [Results] We present a context-aware system that integrates LLMs with retrieval-augmented generation (RAG) to refine, group, and link user stories to Jira tickets. Implemented using LangGraph, the system distributes reasoning across designed agents and allows feedback loops for oversight. The implementation was evaluated on three projects comprising ten user stories each, with project members participating as experts in the assessment. Results show notable improvements in user story quality in completeness (mean +2.32) and testability (mean +2.01), and overall quality (mean +0.84), while inter-rater agreement indicated high reliability for user story grouping ( α = 0.771) and modest agreement in linking stories with Jira tickets ( α = 0.491). [Contribution] The study contributes an LLM-based architecture for AI-assisted requirements refinement and management. It demonstrates how modular agent roles, RAG-based contextual information, and human-in-the-loop evaluation enhance the performance. It illustrates evidence of both the opportunities and current limitations of AI-driven requirements management.

Large Language Models (LLMs) have enabled multi-agent systems to perform autonomous code generation for complex tasks. Despite the recent growth in research and industrial applications in this area, there is little work on synthesizing evidence from both academic and industrial sources to capture the current state of research on LLM-based multi-agent systems for code generation. To this end, we conducted a Multi-Vocal Literature Review (MLR), combining insights from both academia and industry, including peer-reviewed studies and grey literature. The aim of this study is to systematically synthesize and analyze existing knowledge on LLM-based multi-agent systems for code generation. Specifically, the review examines the motivations for their use, employed benchmarks and models, key challenges, proposed solutions, and potential directions for future research. We selected and reviewed 114 studies, and the key findings are: 1) the identified reasons for adopting multi-agent systems for code generation were classified into nine categories; 2) the models and evaluation benchmarks utilized across the studies were systematically analyzed to provide a structured overview of commonly adopted LLM configurations and assessment practices; 3) the reported challenges and corresponding solutions were synthesized into six main categories and 26 subcategories; and 4) future research directions were identified and organized into six main categories and 18 subcategories. The results of this MLR will assist researchers and practitioners in pursuing further studies and supporting the real-world adoption of multi-agent systems in industrial settings.

Recent advances in agentic frameworks have enabled AI agents to perform complex reasoning and decision-making. However, evidence comparing their reasoning performance, efficiency, and practical suitability remains limited. To address this gap, we empirically evaluate 22 widely used agentic frameworks across three reasoning benchmarks: BBH, GSM8K, and ARC. The frameworks were selected from 1,200 GitHub repositories collected between January 2023 and July 2025 and organized into a taxonomy based on architectural design. We evaluated them under a unified setting, measuring reasoning accuracy, execution time, computational cost, and cross-benchmark consistency. Our results show that 19 of the 22 frameworks completed all three benchmarks. Among these, 12 showed stable performance, with mean accuracy of 74.6-75.9 We also found a sharp drop in mathematical reasoning. Mean accuracy on GSM8K was 44.35

Large Language Models are increasingly evaluated for code generation using test-based benchmarks. The validity of such evaluations depends on the reliability of their references and tests, while test-based correctness captures only part of the observable properties of generated code. We present CodeAssay, a taxonomy-first benchmark of 185 Python tasks across ten software-engineering categories. It combines audited ground truth, public tests for generation and repair, hidden tests for grading, mutation-based test-suite validation, and selected code-property measures. Regrading fixed model outputs after the audit changed 170 of 1,890 correctness labels (9.0