Large language models (LLMs) increasingly generate executable software, but functional correctness alone does not establish production readiness. This study evaluates GPT-5.6 Sol, Terra, and Luna across 15 algorithmic and production-oriented tasks using five generations per model-task pair, yielding 225 generated programs plus 15 fixed references. The layered testbed combines nominal and hidden-edge-case checks, moderate and severe production-like stress, metamorphic validation, reference-normalized runtime and memory, structural quality, static analysis, implementation-strategy inspection, and generation cost. All 800 functional and 368 applicable robustness checks passed, while production-stress success fell to 95.16% for Sol, 95.60% for Terra, and 93.38% for Luna. Cross-model stress differences were not significant, but generation cost differed strongly. A Quick Sort case showed more than a 500x runtime spread among behaviorally correct implementations. The results show that production-oriented evaluation must extend beyond pass/fail correctness to include stress behavior, resources, implementation strategy, and cost
Test automation remains essential but increasingly brittle as applications evolve rapidly. This journal article introduces a semantic intelligence approach to test automation, enabling contextdriven adaptation through Natural Language Understanding (NLU), contextual reasoning, and machine learning. Unlike prior work that emphasizes locator-level self-healing alone, this study treats semantic understanding and business context awareness as first-class capabilities for resilient, self-adapting test systems. The framework learns from application behavior patterns, user workflows, and business rules to guide test execution, validation, and adaptation decisions. Through evaluation across 47 enterprise applications spanning finance, retail, healthcare, and media, we show that semantic reasoning can reduce test maintenance overhead by up to 85%, improve defect detection by 62%, and achieve 94% accuracy in autonomous UI change adaptation. A key contribution is semantic context graphs that map business intent to technical implementations, allowing tests to interpret not only “what” changed, but “why” it matters. We further introduce quantitative metrics for test-suite semantic drift and practical mitigation strategies. Finally, we discuss challenges in domain adaptation, computational efficiency, and interpretability, and propose solutions based on transfer learning and explainable AI techniques.
Autonomous software engineering agents are progressing from code assistants to systems that can plan and execute multi-step repository changes. This shift requires evaluation methods that move beyond one-shot pass rate reporting. Many current studies still provide limited visibility into repeatability, failure recovery, and operational efficiency under realistic constraints. This journal article presents an expanded DevAgentBench and DevAgentEval methodology designed for deployment-oriented assessment. The benchmark covers bug fixing, test generation, refactoring, code review assistance, and long-horizon feature work. We organize analysis into three metric layers: task-level success and correctness, robustness under perturbation, and business-aligned operational efficiency. We also formalize a nine-category failure-mode taxonomy linked to trace-level evidence and remediation guidance. Baseline experiments across agent patterns and model families show that rankings are sensitive to context reduction, tool-output noise, transient execution failures, and tighter resource budgets. These findings indicate that average success rates alone are insufficient for production decisions. We therefore recommend condition-aware reporting, repeated-run variance estimation, and reproducible artifact release as minimum standards for autonomous software-agent benchmarking.
Cloud computing is a major current trend that involves virtually distributing processing and data across configurable execution environments. Developing and deploying software for the cloud presents a new scientific challenge in terms of expressing and accounting for variability. Indeed, cloud computing relies on the principles of heterogeneity and elasticity, thus offering numerous configuration and scalability possibilities. Migrating existing applications therefore remains the primary challenge. Several types of migration exist, depending on the three services offered by the cloud. This article presents a study on application migration to the cloud in general, and more specifically on the migration of the "Information Access Assistance Service" (IA2S) application to cloud computing. Migrating existing systems to the cloud offers several advantages but also raises new challenges, particularly due to restrictions imposed by the service provider. Furthermore, factors such as time, training, and significant re-engineering make the migration process lengthy and prone to errors. Although several techniques exist for the partial or complete migration of existing applications to the cloud, only a few can overcome these constraints. Our work primarily consists, firstly, of clarifying the concepts of cloud and "Information Access Assistance Service" (IA2S), and then of studying existing migration strategies in order to propose the most suitable one for migrating IA2S to the cloud.
The integration of Machine Learning (ML) components into modern software systems enhances datadriven decision-making but introduces new challenges for Software Product Line (SPL) engineering. Variability modeling, configuration, and reuse become increasingly complex when adaptive ML components are involved. Although previous studies have addressed variability in traditional SPLs and ML integration in standalone systems, limited work has systematically explored the intersection of these two domains. This paper presents a structured framework that extends SPL engineering to support ML-aware variability management. The framework enables the systematic modeling and configuration of ML components and has been implemented in the VariaMos web tool. A case study demonstrates the framework’s feasibility and applicability, illustrating how it supports the development of adaptive and intelligent product lines.
