Large Language Models (LLMs) are powerful tools used in several applications such as conversational AI, and code generation. However, significant robustness concerns arise with LLMs in production, such as hallucinations, prompt injection attacks, harmful content generation, and challenges in maintaining accurate domain-specific content moderation. Guardrails aim to mitigate these challenges by aligning LLM outputs with desired behaviors without modifying the underlying models. Nvidia NeMo Guardrails, for instance, rely on specifying acceptable/unacceptable behaviours. However, it is challenging to predict and address potential issues of LLMs in advance to create these guardrails. Also, manual updates from software engineers are often required to maintain and refine these guardrails. We introduce LLM-Guards, specialised machine learning (ML) models trained to function as protective guards. Additionally, we present an automation pipeline for training and continual fine-tuning of these guards using reinforcement learning from human feedback (RLHF). We evaluated several small LLMs, including Llama-3, Mistral, and Gemma, as LLM-Guards for challenges such as moderation and detecting off-topic queries, and compared their performance against NeMo Guardrails. The proposed Llama-3 LLM-Guard outperformed NeMo Guardrails in detecting offtopic queries, achieving an accuracy of 98.7% compared to 81%. Furthermore, the LLM-Guard detected 97.86% of harmful queries” surpassing NeMo Guardrails by 19.86%.

Enterprise software systems are increasingly integrating with diverse services to meet expanding business demands. Testing these highly interconnected systems presents a challenge due to the need for access to the connected services. Service virtualization has emerged as a widely used technique to derive service models from recorded interactions, for service response generation during system testing. Various methods have been proposed to emulate actual service behavior based on these interactions, but most fail to account for the service's state, which reduces the accuracy of service emulation and the realism of the testing environment, especially when dealing with stateful services. This paper proposes an approach to deriving service models from service interactions, which enhance the accuracy of response generation by considering service state. This is achieved by uncovering contextual dependencies among interaction messages and analyzing the relationships between message data values. The approach is evaluated using interaction traces collected from both stateful and stateless services, and the results reveal notable enhancements in accuracy and efficiency over existing approaches in service response generation.

Ensuring robustness in ML-enabled software systems requires addressing critical challenges, such as silent failures, out-of-distribution (OOD) data, and adversarial attacks. Traditional software engineering practices, which rely on predefined logic, are insufficient for ML components that depend on data and probabilistic decision-making. To address these challenges, we propose the ML-On-Rails protocol, a unified framework designed to enhance the robustness and trustworthiness of ML-enabled systems in production. This protocol integrates key safeguards such as OOD detection, adversarial attack detection, input validation, and explainability. It also includes a model-to-software communication framework using HTTP status codes to enhance transparency in reporting model outcomes and errors. To align our approach with real-world challenges, we conducted a practitioner survey, which revealed major robustness issues, gaps in current solutions, and highlighted how a standardised protocol such as ML-On-Rails can improve system robustness. Our findings highlight the need for more support and resources for engineers working with ML systems. Finally, we outline future directions for refining the proposed protocol, leveraging insights from the survey and real-world applications to continually enhance its effectiveness.

The Internet of Things (IoT) has a large and rapidly expanding number of deployed devices, which leads to a significant global energy consumption footprint. Diverse IoT use cases, including smart cities, smart grids, Industry 5.0, eHealth, and autonomous vehicles, are contributing to this increase in energy consumption. Optimising energy utilisation is crucial to sustaining the exponential growth of IoT applications, which demand stringent delays and latencies measured in milliseconds and microseconds. There are additional complexities with the emergence of edge, fog, and cloud computing and the need to manage the energy consumption at all the layers. In this paper, mechanisms that can be used to minimise energy consumption within an edge-fog-cloud IoT architecture for real-time IoT applications are being proposed. We investigate mechanisms for optimal node selection, primarily focusing on minimising energy usage while adhering to the Quality of Service (QoS) requirements of various IoT requests. The mechanisms include genetic, modified genetic, and delay-aware algorithms tailored explicitly for real-time IoT applications. We evaluated the proposed mechanisms using a simulation of diverse network scenarios. The results presented in the paper provide insight into balancing processing time and energy efficiency, which are critical considerations in sustainably developing IoT applications in an edge-fog-cloud IoT architecture.

Context In the last decade of data-driven decision-making, Machine Learning (ML) systems reign supreme. Because of the different characteristics between ML and traditional Software Engineering systems, we do not know to what extent the issue-reporting needs are different, and to what extent these differences impact the issue resolution process. Objective We aim to compare the differences between ML and non-ML issues in open-source applied AI projects in terms of resolution time and size of fix. This research aims to enhance the predictability of maintenance tasks by providing valuable insights for issue reporting and task scheduling activities. Method We collect issue reports from Github repositories of open-source ML projects using an automatic approach, filter them using ML keywords and libraries, manually categorize them using an adapted deep learning bug taxonomy, and compare resolution time and fix size for ML and non-ML issues in a controlled sample. Result 147 ML issues and 147 non-ML issues are collected for analysis. We found that ML issues take more time to resolve than non-ML issues, the median difference is 14 days. There is no significant difference in terms of size of fix between ML and non-ML issues. No significant differences are found between different ML issue categories in terms of resolution time and size of fix. Conclusion Our study provided evidence that the life cycle for ML issues is stretched, and thus further work is required to identify the reason. The results also highlighted the need for future work to design custom tooling to support faster resolution of ML issues.

Machine learning (ML) models have become essential components in software systems across several domains, such as autonomous driving, healthcare, and finance. The robustness of these ML models is crucial for maintaining the software systems performance and reliability. A significant challenge arises when these systems encounter out-of-distribution (OOD) data, examples that differ from the training data distribution. OOD data can cause a degradation of the software systems performance. Therefore, an effective OOD detection mechanism is essential for maintaining software system performance and robustness. Such a mechanism should identify and reject OOD inputs and alert software engineers. Current OOD detection methods rely on hyperparameters tuned with in-distribution and OOD data. However, defining the OOD data that the system will encounter in production is often infeasible. Further, the performance of these methods degrades with OOD data that has similar characteristics to the in-distribution data. In this paper, we propose a novel OOD detection method using the Gini coefficient. Our method does not require prior knowledge of OOD data or hyperparameter tuning. On common benchmark datasets, we show that our method outperforms the existing maximum softmax probability (MSP) baseline. For a model trained on the MNIST dataset, we improve the OOD detection rate by 4% on the CIFAR10 dataset and by more than 50% for the EMNIST dataset.

Energy-efficient computation is an increasingly important target in modern-day computing. Scientific computation is conducted using scientific workflows that are executed on highly scalable compute clusters. The execution of these workflows is generally geared towards optimizing run-time performance with the energy footprint of the execution being ignored. Evidently, minimizing both execution time as well as energy consumption does not have to be mutually exclusive. The aim of the research presented in this paper is to highlight the benefits of energy-aware scientific workflow execution. In this paper, a set of requirements for an energy-aware scheduler are outlined and a conceptual architecture for the scheduler is presented. The evaluation of the conceptual architecture was performed by developing a proof of concept scheduler which was able to achieve around 49.97% reduction in the energy consumption of the computation.

Mobile devices often use offloading nodes to reduce the amount of power used locally, or because of the low computational power of the device. Mobile devices can take advantage of offloading to reduce the power usage and increase the battery life of the device. Cloud offloading architectures are largely out of the users control and have higher latency than running the computation on the local device. Cloud computing architectures use specialised nodes, which need to be designed for the service provider. As more Internet of Things (IoT) devices are created and deployed, the number of idle devices also increases. These devices use power to stay in an idle state, which constitutes an inefficient usage of computational resources; both as network resources and physical materials used to make the device. To address an alternative to cloud offloading architectures, this work proposes an architecture for taking advantage of idle computing power of IoT devices for offloading computations from devices locally to improve privacy, latency, and sustainability. Testing of the proposed architecture demonstrates that computations can be successfully offloaded, with acceptable latency, and minimal increase to the individual power use of the connected IoT devices.

Machine learning (ML), especially with the emergence of large language models (LLMs), has significantly transformed various industries. However, the transition from ML model prototyping to production use within software systems presents several challenges. These challenges primarily revolve around ensuring safety, security, and transparency, subsequently influencing the overall robustness and trustworthiness of ML models. In this paper, we introduce ML-On-Rails, a protocol designed to safeguard ML models, establish a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers (software engineers). ML-On-Rails enhances the robustness of ML models via incorporating detection capabilities to identify unique challenges specific to production ML. We evaluated the ML-On-Rails protocol through a real-world case study of the MoveReminder application. Through this evaluation, we emphasize the importance of safeguarding ML models in production.

There is growing attention to energy efficiency in the software engineering field. This has been driven by modern technologies, for example, Internet of Things (IoT), Social Networking Services (SNS) and quantum computing. In addition to this, recent trends and concerns such as Environment, Social, and Governance (ESG) and human/societal/environmental well-being for responsible Artificial Intelligence (AI) have accelerated the use of energy efficient software. Despite this, energy concerns in this field have been less explored and studied. This limitation results in falling short to address and overcome greenability issues at the software level, and leaving critical challenges to be solved in this space. This study aims to address this limitation and fill the gap between previous studies. We survey green in software engineering framed by the ten knowledge areas of software engineering to not only cover the entire development life-cycle but also widen the scope of discussion to software process, method, and model management. Based on our comprehensive investigation, we discuss open challenges, trade-offs and implications of this study for both researchers and practitioners.

Internet of Things (IoT) architectures generally focus on providing consistent performance and reliable communications. The convergence of IoT, edge, fog, and cloud aims to improve the quality of service of applications, which does not typically emphasize energy efficiency. Considering energy in IoT architectures would reduce the energy impact from billions of IoT devices. The research presented in this paper proposes an optimization framework that considers energy consumption of nodes when selecting a node for processing an IoT request in edge-fog-cloud layered architecture. The IoT use cases considered in this paper include smart grid, autonomous vehicles, and eHealth. The proposed framework is evaluated using CPLEX simulations. The results provide insights into mechanisms that can be used to select nodes energy-efficiently whilst meeting the application requirements and other network constraints in multi-layered IoT architectures.

Machine Learning (ML) is used in critical highly regulated and high-stakes fields such as finance, medicine, and transportation. The correctness of these ML applications is important for human safety and economic benefit. Progress has been made on improving ML testing and monitoring of ML. However, these approaches do not provide i) pre/post conditions to handle uncertainty, ii) defining corrective actions based on probabilistic outcomes, or iii) continual verification during system operation. In this paper, we propose MLGuard, a new approach to specify contracts for ML applications. Our approach consists of a) an ML contract specification defining pre/post conditions, invariants, and altering behaviours, b) generated validation models to determine the probability of contract violation, and c) an ML wrapper generator to enforce the contract and respond to violations. Our work is intended to provide the overarching framework required for building ML applications and monitoring their safety.

Optimizing computation for energy consumption and performance requires consideration of the following three key factors: the energy consumption of the resources used; the performance of the workflow; and the time it takes to complete the computation. The increasing complexity of today's computing systems makes it essential to ensure that the resources allocated to a workload are used in the most effective and cost-efficient manner. In this paper, the importance of optimal scheduling solutions for scientific workflow computation is presented. A generic framework is proposed that takes into consideration the user constraints, performance, and factors that affect the energy consumption of different computations to develop an optimal schedule for workflow execution. The aim of the framework will be to improve the energy consumption and performance of the computation. The study also aims to motivate research in energy-efficient optimized scheduling.

Machine learning (ML) has revolutionized various industries by enabling the development of complex models that learn from data and make accurate predictions. However, moving from prototyping ML models to production software systems poses robustness challenges due to the lack of standardization around ML tools and processes. To tackle these challenges in deployed ML models, different elements of robustness must be addressed, including transparency, safety, and security along with adopting a unified language and terminologies. This paper aims to highlight the key robustness challenges that ML models encounter when deployed in production environments, and to emphasize the significance of proactively tackling these challenges. We also introduce various patterns for safeguarding ML models that need to be implemented on both the ML models and the software system sides.

Containers are an increasingly used mechanism for providing low-cost, lightweight, portable, standalone application deployments, particularly for service orchestration. Docker provides container technology that enables a single host to isolate several applications and deploy them rapidly in different environments. The increasing demand for container applications and the growing popularity of Docker has motivated extensive research into evaluating the performance, energy consumption, and running cost of Docker-based computation. This paper investigates the energy footprint of Docker containers and workloads. To motivate research in energy-efficient container development, this paper takes a practical approach to measure the energy consumption in common Docker containers under various workloads.

Contract cheating, a form of academic misconduct in which students outsource assessment activities to third parties, is a topic of concern among educators. As similarity-detection systems are ineffective at detecting contract cheating, some institutions have turned to intensely criticised proctoring systems, however student and educator bodies report high costs and privacy concerns. Oral assessment is an alternative assessment approach that provides valuable interpersonal and communication skills in graduates and can naturally help detect and deter cheating. However, oral assessment is typically time-consuming, and in larger courses, it is challenging to validate respondents' identity. Advancements in machine learning approaches can scale time-consuming tasks that previously required prohibitive educator effort. One such system, Deep Speaker, is a speaker identification and verification system that can verify if two audio samples resemble speech from the same person with high accuracy. This paper presents an innovative tool that integrates an online oral assessment tool, Real Talk, with Deep Speaker. This proposed system facilitates scalable student-tutor discussions while providing longitudinal student identity validation with minimal cost and impact for institutions and addressing student privacy concerns. We evaluated the system and showed that student audio responses collected via oral discussion tools are suitable for verification. We then discuss the impact our system may have when applied in higher education. We posit that institutions can use such approaches to detect cases of contract cheating, enhance learning outcomes, and pave the way for more student-friendly assessment and discussion models in online education.

Scientific workflows consist of multi-step compu-tational tasks executing in the form of data flow and task dependencies. These workflows are defined to be long running and fault tolerant. There is evidence of improving performance achieved through run-time adaptive changes made to the work-flow execution. The aim of the work presented in this paper is to highlight the benefits that adaptive scheduling of scientific workflows have on the energy consumption of the computation. In this paper, an architecture for the implementation of an energy-aware adaptive scheduler is presented. The monitoring, analysis, planning and execution (MAPE) model from autonomic computing is used to propose a set of run-time modifications that will be used by the scheduler to improve the performance and energy consumption of the workflow.

Scientific problems can be formulated as workflows to allow them to take advantage of cluster computing resources. Generally, the assumption is that the greater the resources dedicated to completing these tasks the better. This assumption does not take into account the energy cost of performing the computation and the specific characteristics of each workflow. In this paper, we present a unique approach to evaluating the energy consumption of scientific workflows on compute clusters. Two workflows from different domains, Astronomy and Bioinformatics, are presented and their execution is analyzed on a cluster of low powered small board computers. The paper presents a theoretical analysis of an energy-aware execution of workflows that can reduce the energy consumption of workflows by up to 68% compared to normal execution. We demonstrate that there are limitations to the benefits of increasing cluster sizes and there are trade-offs when considering energy vs. performance of the workflows and that the performance and energy consumption of any scientific workflow is heavily dependent on its underlying structure. The study concludes that the energy consumption of workflows can be optimized to improve both aspects of the workflow and motivates the development of an energy-aware scheduler.

It is generally accepted in Software Engineering that code clones – often the result of copy-and-paste of existing code – result in poorer maintainability of software systems. Consequently, a variety of techniques have been devised to detect cloned code in software systems and alert developers of duplicated code. Most techniques operate at the source-code level and require some combination of pretty-printing, tokenization and abstraction in order to improve the comparison of code fragments over purely string-based techniques. Avoiding some of the issues of source-code based approaches, we are investigating the effectiveness of using various similarity measures on Bytecode to identify code clones in Java-based systems in this work. The results of our evaluation on selected Java systems indicate that instruction sequences can be used to effectively detect identical code clones. Especially, we achieved the best performance when using the normalized edit distance among applied similarity measures.