Microservices constitute the state of the art for implementing distributed systems and have been seen as a potential solution towards open systems. The characteristics of open systems require structured microservice management, including grouping microservices that are functionally similar. Microservices use RESTful APIs, often documented via OpenAPI specifications, to demonstrate their functionalities. Existing similarity metrics for microservice APIs have primarily focused on individual RESTful endpoints. However, understanding the full functionality of a microservice within an open system requires that the entirety of its OpenAPI documentation be considered. Thus, an approach that can compute a measure of similarity between entire microservice definitions in open environments is needed. In this paper, we propose an approach that can extract key information from the OpenAPI descriptions of microservices using Natural Language Processing (NLP) techniques, vectorise the extracted information using GLoVe embeddings, and cluster similar microservices using embedded API file vectors. Evaluations were conducted on real-world OpenAPI documents to demonstrate the effectiveness of the proposed approach.

Advances in Internet of Things (IoT) technologies have resulted in a significant surge in the utilization of sensor devices across diverse domains for environmental sensing and monitoring. The applications of IoT sensor devices in environmental monitoring span a wide range, including the surveillance of biodiverse areas such as peatlands, forests, and oceans, as well as air quality monitoring, commercial agriculture, and the safeguarding of endangered species. This paper provides a long term evaluation of IoT sensors data quality in environmental monitoring networks, particularly focusing on peatland regions. IoT sensors have the capacity to provide high resolution spatiotemporal dataset in environmental monitoring networks. Sensor data quality plays significant role in increasing the adoption of IoT devices for environmental data gathering. However, due to the nature of deployment (i.e., in harsh and unfavourable weather conditions), coupled with the limitations of low-cost components, IoT sensors are prone to collection of erroneous data, also the nature of peatland ecosystems presents unique challenges in data quality assurance due to their complex and dynamic characteristics. This paper identifies specific challenges and issues related to IoT sensor data quality in different peatland ecotopes. These challenges include sensor placement and calibration, data validation and fusion, environmental interference, and the management of data gaps and uncertainties. To address these challenges, the paper presents and evaluates methods for improving data quality in peatland monitoring networks. These methods encompass advanced sensor calibration techniques, data validation algorithms, machine learning approaches, data processing and data fusion strategies.

As the complexity of network end devices and applications grows, network managers face increasing difficulty in meeting specific end user requirements, leading to reduced user experience and inefficient resource management. This paper introduces a Quality of Experience (QoE)-oriented routing strategy to enhance user experience by selecting routing paths based on application-specific QoE. Application key performance indicators (KPIs) and dynamic link metrics are utilized to represent real-time QoE and network state. This data builds QoE models for various applications such as video streaming, VoIP, and web map, using four learning methods. The trained models are implemented in a software-defined networking (SDN) controller for optimal QoE routing. Evaluations using the Mininet network simulator reveal that the proposed QoE routing strategy can select the best path 78.4% of the time which is almost 20% more than the top-performing state-of-the-art. This results in measurably higher application performance, proving the efficiency of the proposed approach in improving the application’s QoE.

As an emerging paradigm of heterogeneous networks (HetNets) towards 6G, the hybrid light fidelity (LiFi) and wireless fidelity (WiFi) networks (HLWNets) have potential to explore the complementary advantages of the optical and radio spectra. Like other cooperation-native HetNets, HLWNets face a crucial load balancing (LB) problem due to the heterogeneity of access points (APs). The existing literature mostly formulates this problem as joint AP selection and resource allocation (RA), presuming that each user equipment (UE) is served by one AP at a time, under the constraint of the traditional transmission control protocol (TCP). In contrast, multipath TCP (MPTCP), which allows for the simultaneous use of multiple APs, can significantly boost the UE's throughput as well as enhancing its network resilience. However, the existing TCP-based LB methods, particularly those aided by machine learning, are not suitable for the MPTCP scenario. In this paper, we discuss the challenges when developing learning-aided LB in MPTCP-enabled HLWNets, and propose a novel user-centric learning model to tackle this tricky problem. Unlike the conventional network-centric learning methods, the proposed method determines the LB solution for a single target UE, rendering low complexity and high flexibility in practical implementations. Results show that the proposed user-centric approach can greatly outperform the network-centric learning method. Against the TCP-based LB method such as game theory, the proposed method can increase the throughput of HLWNets by up to 40%.

Smart agriculture relies on accurate yield maps as a crucial tool for decision-making. Many yield maps, however, suffer from spatial errors that can compromise the quality of their data, while several approaches have been proposed to address some of these errors, detecting voids or holes in the maps remains challenging. Additionally, the quality of yield datasets is typically evaluated based on root mean squared errors after interpolation. This evaluation method relies on weighbridge data, which can occasionally be inaccurate, impacting the quality of decisions made using the datasets. This paper introduces a novel algorithm designed to identify voids in yield maps. Furthermore, it maps three types of spatial errors (GPS errors, yield surges, and voids) to two standard data quality dimensions (accuracy and completeness). Doing so provides a quality score that can be utilized to assess the quality of yield datasets, eliminating the need for weighbridge data. The paper carries out three types of evaluations: (1) evaluating the algorithm’s efficacy by applying it to a dataset containing fields with and without voids; (2) assessing the benefits of integrating void detection and other spatial error identification techniques into the yield data processing chain; and (3) examining the correlation between root mean squared error and the proposed quality score before and after filtering out spatial errors. The results of the evaluations demonstrate that the proposed algorithm achieves a 100% sensitivity, 91% specificity, and 82% accuracy in identifying yield maps with voids. Additionally, there is a decrease in the root mean squared error when various spatial errors, including voids after applying the proposed data pre-processing chain. The inverse correlation observed between the root mean squared error and the proposed quality score (−0.577 and −0.793, before and after filtering spatial errors, respectively) indicates that the quality score can effectively assess the quality of yield datasets. This assessment enables seamless integration into real-time big data quality assessment solutions based on various data quality dimensions.

The microservices architecture (MSA) is a form of distributed systems architecture that has been widely adopted in large-scale software systems in recent years. As with other distributed system architectures, one of the challenges that MSA faces is establishing trust between the microservices, particularly in the context of open systems. The boundaries of open systems are unlimited and unknown, which means that they can be applied to any use case. Microservices can leave or join an open system arbitrarily, without restriction as to ownership or origin, and MSA systems can scale extensively. The organisation of microservices (in terms of the roles they play and the communication links they utilise) can also change in response to changes in the environment in which the system is situated. The management of trust within MSAs is of great importance as the concept of trust is critical to microservices communication, and the operation of an open MSA system is highly reliant on communication between these fine-grained microservices. Thus, a trust model should also be able to manage trust in an open environment. Current trust management solutions, however, are often domain-specific and many are not specifically tailored towards the open system model. This motivates research on trust management in the context of open MSA systems. In this paper, we examine existing microservices trust models, identify the limitations of these models in the context of the principles of open microservices systems, propose a set of qualities for open microservices trust models that emerge from these limitations, and assess selected microservices trust models using the proposed qualities.

Timely data quality assessment has been shown to be crucial for the development of IoT-based applications. Different IoT applications’ varying data quality requirements pose a challenge, as each application requires a unique data quality process. This creates scalability issues as the number of applications increases, and it also has financial implications, as it would require a separate data pipeline for each application. To address this challenge, this paper proposes a novel approach integrating fusion methods into end-to-end data quality assessment to cater to different applications within a single data pipeline. By using real-time and historical analytics, the study investigates the effects of each fusion method on the resulting data quality score and how this can be used to support different applications. The study results, based on two real-world datasets, indicate that Kalman fusion had a higher overall mean quality score than Adaptive weighted fusion and Naïve fusion. However, Kalman fusion also had a higher computational burden on the system. The proposed solution offers a flexible and efficient approach to addressing IoT applications’ diverse data quality needs within a single data pipeline.

Continued development of communication technologies has led to widespread Internet-of-Things (IoT) integration into various domains, including health, manufacturing, automotive, and precision agriculture. This has further led to the increased sharing of data among such domains to foster innovation. Most of these IoT deployments, however, are based on heterogeneous, pervasive sensors, which can lead to quality issues in the recorded data. This can lead to sharing of inaccurate or inconsistent data. There is a significant need to assess the quality of the collected data, should it be shared with multiple application domains, as inconsistencies in the data could have financial or health ramifications. This article builds on the recent research on trust metrics and presents a framework to integrate such metrics into the IoT data cycle for real-time data quality assessment. Critically, this article adopts a mechanism to facilitate end-user parameterization of a trust metric tailoring its use in the framework. Trust is a well-established metric that has been used to determine the validity of a piece or source of data in crowd-sourced or other unreliable data collection techniques such as that in IoT. The article further discusses how the trust-based framework eliminates the requirement for a gold standard and provides visibility into data quality assessment throughout the big data model. To qualify the use of trust as a measure of quality, an experiment is conducted using data collected from an IoT deployment of sensors to measure air quality in which low-cost sensors were colocated with a gold standard reference sensor. The calculated trust metric is compared with two well-understood metrics for data quality, root mean square error (RMSE), and mean absolute error (MAE). A strong correlation between the trust metric and the comparison metrics shows that trust may be used as an indicative quality metric for data quality. The metric incorporates the additional benefit of its ability for use in low context scenarios, as opposed to RMSE and MAE, which require a reference for comparison.

Quality data is crucial to dependable, trustworthy data analytics and data-driven decision-making in IoT-based applications. Data pre-processing and data cleaning are the only considered strategies for data quality assurance. Whilst these have provided usable results, within IoT, a problem still exists. Data quality issues within IoT proliferate throughout all the stages of the data cycle, with the majority at the source. This work proposes a solution to achieve two main objectives; 1) improving data quality of IoT streams by using alternative sources. 2) demonstrate that improved data quality results in improved application performance. Firstly, results from two experimental settings based on weather data from a real-world deployment show that by seeking alternative sources, data quality improved by 15.6% and 0.2% for completeness and accuracy, respectively. Secondly, two distinct scenarios show that by improving data quality, the model's RMSE improved by an average of 2 and 2.5 from 4 and 17.5, respectively.

When equipped with a reliable calibration model, Low-Cost Sensor (LCS) can be relied upon as an effective option for gas concentration estimation, providing robust and high spatio-temporal resolution data to replace large-scale analytical instruments. In this paper, we present ProxySense, a rapid and efficient approach for gas concentration estimation. The ProxySense pipeline consists of gas sensing unit made up of array of metal oxide LCS, data pre-processing including an effective approach based on Variatioanl Autoencoders (VAE) for handling missing sensor data and Long Short Term Memory Reccurrent Neural Network (LSTM-RNN) prediction model. We investigate the capability of ProxySense in exploiting the deep characteristics that exist in multi-sensors’ responses to predict the concentration of a gas for which no specific sensor is included in a multi-sensor device. We evaluated ProxySense for benzene (C6H6) and carbon monoxide (CO) concentration predictions and compared the performances to multiple baselines by means of prediction error characterization. We further studied the relationship between model performance and training length and showed ProxySense to be highly accurate for gas concentration prediction even for small number of training period.

An increasing variety of network services and applications have led to a demand for service specific network management. QoE routing, routing service traffic on an individual basis, has been applied to target this demand. Learning tools have also been applied to automate and tailor management approach in real time within the network. A network manager can evaluate routing decisions to determine if expected performance was reached, and make adjustments to the routing model if not. The difficulty with this approach remains in collecting and evaluating the network state and service performance in real time to enable learning in the network. Such metrics must also be suitable for developing or adapting a routing model. This paper expands on a framework for real time feedback supported management. The aim of the paper is to identify and evaluate a suitable real time mechanism to collect network state data and a suitable application feedback metric. The metrics are evaluated for use in a routing model. The solution is unique as it provides a framework for a general service given a suitable feedback metric for that service. The paper examines application KPI metrics as suitable feedback metrics for two services, video streaming and VoIP, with APSNR and PESQ used as respective feedback metrics. The paper defines and evaluates link metrics as a mechanism for real time state determination. The framework is implemented and evaluated on an emulated SDN testbed.

When equipped with an appropriate data processing pipeline and a reliable prediction model, Low-Cost Sensor (LCS) can be relied upon as an effective option for gas concentration estimation, providing robust and high spatio-temporal resolution data to replace large-scale analytical instruments. In this paper, we present ProxySense, a rapid and efficient approach for gas concentration estimation. The ProxySense pipeline consists of gas sensing unit made up of array of metal oxide LCS, data processing including an effective approach based on Variational Autoencoders (VAE) for handling missing sensor data and Long Short Term Memory Recurrent Neural Network (LSTM-RNN) prediction model. We investigated the capability of ProxySense in exploiting the deep characteristics that exist in multi-sensors' responses to predict the concentration of a gas for which no specific sensor is included in a multi-sensor device. We evaluated the performance of ProxySense for predicting the atmospheric concentration of benzene (C6H6) and carbon monoxide (CO), comparing to multiple baselines by means of prediction error characterization. We also investigated the relationship between model performance and training length and showed ProxySense to be highly accurate for gas concentration prediction even for small number of training period.

IoT sensors are becoming increasingly important supplement to traditional monitoring systems, particularly for in-situ based monitoring. However, data collection based on IoT sensors are often plagued with missing values usually occurring as a result of sensor faults, network failures, drifts and other operational issues.

As higher volumes of new applications are dependant on the network for operational performance, network orchestration must manage each traffic type to maximise application performance. Application specific routing requires identification of traffic type, and subsequently a traffic model to best provision for this traffic. This paper presents an analysis of a link-based modeling approach to applica...

Content is a key influencing factor in Web Quality of Experience (QoE) estimation. A web user’s satisfaction can be influenced by how long it takes to render and visualize the visible parts of the web page in the browser. This is referred to as the Above-the-fold (ATF) time. SpeedIndex (SI) has been widely used to estimate perceived web page loading speed of ATF content and a proxy metric for Web QoE estimation. Web application developers have been actively introducing innovative interactive features, such as animated and multimedia content, aiming to capture the users’ attention and improve the functionality and utility of the web applications. However, the literature shows that, for the websites with animated content, the estimated ATF time using the state-of-the-art metrics may not accurately match completed ATF time as perceived by users. This study introduces a new metric, Plausibly Complete Time (PCT), that estimates ATF time for a user’s perception of websites with and without animations. PCT can be integrated with SI and web QoE models. The accuracy of the proposed metric is evaluated based on two publicly available datasets. The proposed metric holds a high positive Spearman’s correlation (rs=0.89) with the Perceived ATF reported by the users for websites with and without animated content. This study demonstrates that using PCT as a KPI in QoE estimation models can improve the robustness of QoE estimation in comparison to using the state-of-the-art ATF time metric. Furthermore, experimental result showed that the estimation of SI using PCT improves the robustness of SI for websites with animated content. The PCT estimation allows web application designers to identify where poor design has significantly increased ATF time and refactor their implementation before it impacts end-user experience.

The number of web applications for both personal and business use will continue to increase. The popularity of web applications has grown, increasing the need to estimate Quality of Experience for web applications (Web QoE). Web QoE helps providers to understand how their end-users perceive quality and point towards areas to improve. Waiting time has been proven to have a significant influence on user satisfaction. Most studies in the field of Web QoE have focused on modelling Web QoE for the user's first interaction with the application, e.g., the waiting time for the first page load to complete. This does not include a user's subsequent interactions with the application. Users keep interacting with the application beyond the first page load resulting in an experience that consists of a series of waiting times. In this study, we have chosen web maps as a use case to investigate how to measure waiting time for a user's interactions across a web browsing session, and to measure the correlation between waiting time and user-reported perceived quality. We provide a short survey of existing Web QoE estimation metrics and models. We then propose two new measures: interactive Load Time (iLT) and Total Completed interactive Load (TCiL) to establish the waiting time associated with a web application user's interactions. A subjective study confirms a logarithmic relationship for interactive web application sessions between iLT and perceived quality. We compare the correlation between QoE for iLT and the state of the art, non-interactive equivalent, Page Load Time (PLT)/Waiting Time. We demonstrate how the iLT/QoE fitting curve deviates from PLT/QoE. The number of clicks in completing tasks and TCiL are explored to explain the connections between user's interactions behaviour and the perceived quality.

Internet of Things devices and data sources areseeing increased use in various application areas. The pro-liferation of cheaper sensor hardware has allowed for widerscale data collection deployments. With increased numbers ofdeployed sensors and the use of heterogeneous sensor typesthere is increased scope for collecting erroneous, inaccurate orinconsistent data. This in turn may lead to inaccurate modelsbuilt from this data. It is important to evaluate this data asit is collected to determine its validity. This paper presents ananalysis of data quality as it is represented in Internet of Things(IoT) systems and some of the limitations of this representation. The paper discusses the use of trust as a heuristic to drive dataquality measurements. Trust is a well-established metric that hasbeen used to determine the validity of a piece or source of datain crowd sourced or other unreliable data collection techniques. The analysis extends to detail an appropriate framework forrepresenting data quality effectively within the big data modeland why a trust backed framework is important especially inheterogeneously sourced IoT data streams.

Recent developments in Internet of Things have heightened the need for data sharing across application domains to foster innovation. As most of these IoT deployments are based on heterogeneous sensor types, there is increased scope for sharing erroneous, inaccurate or inconsistent data. This in turn may lead to inaccurate models built from this data. It is important to evaluate this data as it is collected to establish its quality. This paper presents an analysis of data quality as it is represented in Internet of Things (IoT) systems and some of the limitations of this representation. The paper then introduces the use of trust as a heuristic to drive data quality measurements. Trust is a well-established metric that has been used to determine the validity of a piece or source of data in crowd sourced or other unreliable data collection techniques. The analysis extends to detail an appropriate framework for representing data quality within the big data model. To demonstrate the application of a trust backed framework, we used data collected from a IoT deployment of sensors to measure air quality in which a low cost sensor was co-located with a gold reference sensor. Using data streams modeled based on a dataset from an IoT deployment, our initial results show that the framework's trust score are consistent with the accuracy measure of the machine learning models.

Quality of Experience (QoE) for various types of websites has gained significant attention in recent years. In order to design and evaluate websites, a metric that can estimate a user's experienced quality robustly for diverse content is necessary. SpeedIndex (SI) has been widely adopted to estimate perceived web page loading progress. It measures the speed of rendering pixels for the webpage that is visible in the browser window. This is termed Above-The-Fold (ATF). The influence of animated content on the perception of ATF has been less comprehensively explored. In this paper, we present an experimental design and methodology to measure ATF perception for websites with and without animated elements for various page content categories. We found that pages with animated elements caused people to have more varied perceptions of ATF under different network conditions. Animated content also impacts the page load estimation accuracy of SI for websites. We discuss how the difference in the perception of ATF will impact the QoE management of web applications. We explain the necessity of revisiting the visual assessment of ATF to include the animated contents and improve the robustness of metrics like SI.