This work introduces a model-based framework that reveals the idle opportunity of modern servers running latency-critical applications. Specifically, three queuing models, M/M/1, cxM/M/1, and M/M/c, are used to estimate the theoretical idle time distribution at the CPU core and system (package) level. A comparison of the actual idleness of a real server and that from the theoretical models reveals significant missed opportunities to enter deep idle states. This inefficiency is attributed to the idle-governor inaccuracy and the high latency to transition to/from legacy deep-idle states. The proposed methodology offers the means for an early-stage design exploration and insights into idle time behavior and opportunities for varying server system configurations and load.

Modern datacenter applications commonly adopt a microservice software architecture, where an application is decomposed into smaller interconnected microservices communicating via the network. These microservices often operate under strict latency requirements, rendering them particularly vulnerable to microarchitectural cold effects that may arise from the interleaved execution of services on cores or power-gating cores between invocations. Previous analyses of microservices find branch mispredictions due to cold predictor resources to be a significant contributor to performance degradation, indicating that the dynamic control flow must be very similar in the set and order of executed instructions across different requests. Our analysis of control-flow similarity across requests, using static and dynamic control flow information to determine dynamic control flow reconvergence, confirms that, indeed, a large portion of requests follow similar paths. Motivated by the above findings, we propose Similarity-based Branch Prediction (SBP), a hybrid predictor architecture that enhances conventional predictors with a similarity component. SBP leverages the control-flow similarity across microservice requests to predict control flow (branch direction and target) by utilizing the control flow of past executions encoded in a reference execution trace. We realize a specific instantiation of SBP, called CHESS, which combines a conventional history-based fetch predictor, a static-hint predictor, and a similarity predictor. CHESS judiciously applies similarity prediction for branches identified as hard-to-predict through conventional prediction techniques, effectively mitigating branch predictor cold-start effects while keeping the length of the reference trace practical. Evaluation through a suite of microservices shows that CHESS reduces branch MPKI by 94% over a cold fetch predictor and 78% over a state-of-the-art predictor, while requiring a modest 18.1KB of additional storage space. This enables CHESS to deliver performance that is, on average, within 95% of a warm baseline system.

As memory technologies continue to shrink and memory error rates increase, the demand for stronger reliability becomes increasingly critical. Fine-grain memory replication has emerged as an appealing approach to improving memory fault tolerance by augmenting conventional memory protection based on error-correcting codes with an additional layer of redundancy that replicates data across independent failure domains, such as replicating memory pages across different NUMA sockets. This method can tolerate a broad spectrum of memory errors, from individual memory cell failures to more complex memory controller failures. However, applying memory replication without a holistic consideration of the interaction between error-correcting codes and replication can result in redundant duplication and unnecessary storage overhead. We propose Replication-Aware Memory-error Protection (RAMP), a model that helps explore error protection strategies to improve the storage efficiency of memory protection in memory systems that utilize memory replication for performance and availability. We use RAMP to determine a protection strategy that can lower the storage cost of individual replicas while still ensuring robust protection through the collective protection conferred by multiple replicas. Our evaluation shows that a solution derived with RAMP enhances the storage efficiency of a state-of-the-art memory protection mechanism when paired with rack-level replication for disaggregated memory. Specifically, we can reduce the storage cost of memory protection from 27 performance overhead.

Latency critical applications running in modern datacenters exhibit irregular request arrival patterns and are implemented using multiple services with strict latency requirements (30μs–250μs). These characteristics render existing energy saving idle CPU sleep states ineffective due to the performance overhead caused by the state’s transition latency. Besides the state transition latency, another important contributor to the performance overhead of sleep states is the cold-start latency, or in other words, the time required to warm-up microarchitectural state (e.g., cache contents, branch predictor metadata) that is flushed or discarded when transitioning to a lower-power state. Both the transition latency and cold-start latency can be particularly detrimental to the performance of latency critical applications with short execution times. While prior work focuses on mitigating the effects of transition and cold-start latency by optimizing request scheduling, in this work, we propose a redesign of the Core C-state architecture for latency-critical applications. In particular, we introduce C6Awarm a new Agile Core C-state that drastically reduces the performance overhead caused by idle sleep state transition latency and cold-start latency, while maintaining significant energy savings. C6Awarm achieves its goals by implementing 1) medium-grained power gating, 2) preserving the microarchitectural state of the core and 3) by keeping the clock generator and PLL active and locked. Our analysis for a set of microservices based on an Intel Skylake server, shows that C6Awarm manages to reduce the energy consumption by up to \(70\% \) with limited performance degradation (at-most \(2\% \) ).

Many online services running in datacenters are implemented using a microservice software architecture characterized by strict latency requirements. Consequently, this popular software paradigm is increasingly used for the performance evaluation of server systems. Due to the scale and complexity of datacenters, the evaluation of server optimization techniques is usually done on a smaller scale using a client-server model. Although the experimental details of the server side are excessively described in most publications, the client side is often ignored. This paper identifies the hardware configuration of the client side as an important source of performance variation that can affect the accuracy and the correctness of the conclusions of a study that analyzes the performance of microservices. This is partially attributed to the strict latency requirements of microservices and the small scale of the experimental environment.In this work we present, using a widely used online-service, several examples where the accuracy and the trends of the conclusions differ based on the configuration of the client-side. At the same time we show that the experimental evaluation time can be significantly affected by the hardware configuration of the client. All these provoke the discussion of the right way to configure the experimental environment for assessing the performance of microservices.

User-facing applications running in modern datacenters exhibit irregular request patterns and are implemented using a multitude of services with tight latency requirements. These characteristics render ineffective existing energy conserving techniques when processors are idle due to the long transition time from a deep idle power state (C-state). While prior works propose management techniques to mitigate this inefficiency, we tackle it at its root with AgileWatts (AW): a new deep C-state architecture optimized for datacenter server processors targeting latency-sensitive applications. AW is based on three key ideas. First, AW eliminates the latency overhead of saving/restoring the core context (i.e., micro-architectural state) when powering-off/-on the core in a deep idle power state by i) implementing medium-grained power-gates, carefully distributed across the CPU core, and ii) retaining context in the power-ungated domain. Second, AW eliminates the flush latency overhead (several tens of microseconds) of the L1/L2 caches when entering a deep idle power state by keeping L1/L2 cache content power-ungated. A minimal control logic also remains power-ungated to serve cache coherence traffic (i.e., snoops) seamlessly. AW implements sleep-mode in caches to reduce caches leakage power consumption and lowers a core voltage to the minimum operational voltage level to minimize the leakage power of the power-ungated domain. Third, using a state-of-the-art power efficient all-digital phase-locked loop (ADPLL) clock generator, AW keeps the PLL active and locked during the idle state, further cutting precious microseconds of wake-up latency at a negligible power cost. Our evaluation with an accurate simulator calibrated against an Intel Skylake server shows that AW reduces the energy consumption of Memcached by up to 71% (35% on average) with up to 1% performance degradation.

Disaggregated memory architectures provide benefits to applications beyond traditional scale out environments, such as independent scaling of compute and memory resources. They also provide an independent failure model, where computations or the compute nodes they run on may fail independently of the disaggregated memory; thus, data that's resident in the disaggregated memory is unaffected by the compute failure. Blind application of traditional techniques for resilience (e.g., checkpoints or data replication) does not take advantage of these architectures. To demonstrate the potential benefit of these architectures for resilience, we develop Memory-Oriented Distributed Computing (MODC), a framework for programming disaggregated architectures that borrows and adapts ideas from task-based programming models, concurrent programming techniques, and lock-free data structures. This framework includes a task-based application programming model and a runtime system that provides scheduling, coordination, and fault tolerance mechanisms. We present highlights of our MODC prototype and experimental results demonstrating that MODC-style resilience outperforms a checkpoint-based approach in the face of failures.

Disaggregated memory leverages recent technology advances in high-density, byte-addressable non-volatile memory and high-performance interconnects to provide a large memory pool shared across multiple compute nodes. Due to higher memory density, memory errors may become more frequent. Unfortunately, tolerating memory errors through existing memory-error protection techniques becomes impractical due to increasing storage cost. This letter proposes replication-aware memory-error protection to improve storage efficiency of protection in data-centric applications that already rely on memory replication for performance and availability. It lets such applications lower protection storage cost by weakening the protection of each individual replica, but still realize a strong protection target by relying on the collective protection conferred by multiple replicas.

To process real-world datasets, modern data-parallel systems often require extremely large amounts of memory, which are both costly and energy inefficient. Emerging non-volatile memory (NVM) technologies offer high capacity compared to DRAM and low energy compared to SSDs. Hence, NVMs have the potential to fundamentally change the dichotomy between DRAM and durable storage in Big Data processing. However, most Big Data applications are written in managed languages and executed on top of a managed runtime that already performs various dimensions of memory management. Supporting hybrid physical memories adds a new dimension, creating unique challenges in data replacement. This article proposes Panthera, a semantics-aware, fully automated memory management technique for Big Data processing over hybrid memories. Panthera analyzes user programs on a Big Data system to infer their coarse-grained access patterns, which are then passed to the Panthera runtime for efficient data placement and migration. For Big Data applications, the coarse-grained data division information is accurate enough to guide the GC for data layout, which hardly incurs overhead in data monitoring and moving. We implemented Panthera in OpenJDK and Apache Spark. Based on Big Data applications’ memory access pattern, we also implemented a new profiling-guided optimization strategy, which is transparent to applications. With this optimization, our extensive evaluation demonstrates that Panthera reduces energy by 32–53% at less than 1% time overhead on average. To show Panthera’s applicability, we extend it to QuickCached, a pure Java implementation of Memcached. Our evaluation results show that Panthera reduces energy by 28.7% at 5.2% time overhead on average.

Mobile edge computing (MEC) is a key enabler of delay-sensitive vehicle-to-everything (V2X) applications. Determining where to execute a task necessitates accurate estimation of the offloading latency. In this paper, we propose a latency prediction framework that integrates machine learning and statistical approaches. Aided by extensive latency measurements collected during driving, we first preprocess the data and divide it into two components: one that follows a trackable trend over time and the other that behaves like random noise. We then develop a Long Short-Term Memory (LSTM) network to predict the first component. This LSTM network captures the trend in latency over time. We further enhance the prediction accuracy of this technique by employing a k-medoids classification method. For the second component, we propose a statistical approach using a combination of Epanechnikov Kernel and moving average functions. Experimental results show that the proposed prediction approach reduces the prediction error to half of a standard deviation (STD) of the raw data.

Modern data-parallel systems such as Spark rely increasingly on in-memory computing that can significantly improve the efficiency of iterative algorithms. To process real-world datasets, modern data-parallel systems often require extremely large amounts of memory, which are both costly and energy-inefficient. Emerging non-volatile memory (NVM) technologies offers high capacity compared to DRAM and low energy compared to SSDs. Hence, NVMs have the potential to fundamentally change the dichotomy between DRAM and durable storage in Big Data processing. However, most Big Data applications are written in managed languages (e.g., Scala and Java) and executed on top of a managed runtime (e.g., the Java Virtual Machine) that already performs various dimensions of memory management. Supporting hybrid physical memories adds in a new dimension, creating unique challenges in data replacement and migration. This paper proposes Panthera, a semantics-aware, fully automated memory management technique for Big Data processing over hybrid memories. Panthera analyzes user programs on a Big Data system to infer their coarse-grained access patterns, which are then passed down to the Panthera runtime for efficient data placement and migration. For Big Data applications, the coarse-grained data division is accurate enough to guide GC for data layout, which hardly incurs data monitoring and moving overhead. We have implemented Panthera in OpenJDK and Apache Spark. An extensive evaluation with various datasets and applications demonstrates that Panthera reduces energy by 32 – 52% at only a 1 – 9% execution time overhead.

Fog computing has been advocated as an enabling technology for computationally intensive services in connected smart vehicles. Most existing works focus on analyzing and optimizing the queueing and workload processing latencies, ignoring the fact that the access latency between vehicles and fog/cloud servers can sometimes dominate the end-to-end service latency. This motivates the work in this paper, where we report a five-month urban measurement study of the wireless access latency between a connected vehicle and a fog computing system supported by commercially available multi-operator LTE networks. We propose AdaptiveFog, a novel framework for autonomous and dynamic switching between different LTE operators that implement fog/cloud infrastructure. The main objective here is to maximize the service confidence level, defined as the probability that the tolerable latency threshold for each supported type of service can be guaranteed. AdaptiveFog has been implemented on a smart phone app, running on a moving vehicle. The app periodically measures the round-trip time between the vehicle and fog/cloud servers. An empirical spatial statistic model is established to characterize the spatial variation of the latency across the main driving routes of the city. To quantify the performance difference between different LTE networks, we introduce the weighted Kantorovich-Rubinstein (K-R) distance. An optimal policy is derived for the vehicle to dynamically switch between LTE operators’ networks while driving. Extensive analysis and simulation are performed based on our latency measurement dataset. Our results show that AdaptiveFog achieves around 30% and 50% improvement in the confidence level of fog and cloud latency, respectively.

Low latency is required for connected and intelligent vehicle applications. For instance, safety applications have strict latency requirements. Mobile edge computing (MEC) improves the latency by bringing the computational resources closer to the data source. However, despite the improvements in latency by MEC, the latency will vary based on traffic load and signal conditions. We seek to answer the question: is the current latency environment acceptable for our application? This paper presents our Reliable Latency Decision (ReLaDec) algorithm. ReLaDec uses prior information and the last latency samples to decide whether the latency will be acceptable to our application. The decisions are done with predictable confidence and with ReLaDec the false positive decision rate never exceeds the set maximum value. We demonstrate the performance of the algorithm using 860000 latency samples that include stationary and driving data over multiple USA states.

Introduction: Recent advances provide the building blocks for constructing rack-scale architectures with a large pool of disaggregated non-volatile memory (NVM) that can be shared across a high-performance system interconnect by decentralized compute resources. NVDIMMs and new NVM technologies provide byte-addressable persistent storage accessible through loads and stores, rather than the block I/O path used today. High-performance system interconnects, such as Gen-Z, OmniPath, and RDMA over InfiniBand, provide low-latency access from compute nodes to fabricattached memory (e.g., microsecond-scale remote memory latencies are already possible with RDMA). Disaggregated memory architectures1 share several characteristics: 1) a high-capacity pool of memory that can be shared by heterogeneous computing resources at low latency; 2) a partially disaggregated architecture that treats node-local memory as private and disaggregated memory as shared; 3) a heterogeneous memory system containing both volatile DRAM and NVM; 4) unmediated access from a compute node to disaggregated memory provided by one-sided loads/stores or gets/puts and facilitated through atomic operations (e.g., compare-and-swap as in RDMA or Gen-Z); 5) hardwareenforced cache coherence domains limited to a single compute node; and 6) a separation of fault domains between processing and disaggregated memory. MODC: Our goal is to investigate how to program this emerging class of system architectures. We propose memoryoriented distributed computing (MODC), an approach for building system runtimes that exploits disaggregated memory to facilitate work distribution, coordination and fault tolerance. Global state is maintained as shared data structures in disaggregated memory that are visible to all participating processes, rather than being physically partitioned. Because processes on all nodes have direct access to global data structures, data can be efficiently shared, without the need for message overheads. Processes are equally able to analyze and service requests for any part of the dataset, which provides better load balancing and more robust performance for skewed workloads. Shared access to global data also

Smart metes used in electric grids need a dedicated network that should be highly reliable & cost effective. Various techniques like 3G cellular have been proposed to improve efficiency of this smart grid electric meter network. For distribution of proper information in smart grid system Hybrid Spread Spectrum using slow frequency technology is also better choice. To improve the performance of this network in the terms of throughput and number of smart meter per data aggregation point (DAP), we have proposed HSS-FFH (to implement AMI) method. These techniques give better result in terms of coverage high density population area & interference immunity.

A cognitive radio engine (CE) is where the advanced adaptation algorithms for a cognitive radio is implemented. A CE is an intelligent agent which observes the radio environment and chooses the best communication settings that meet the application’s goal. In this process, providing reliable performance is one of the major challenges faced by a CE. Therefore, one of the most important issues in designing CEs is the ability to characterize and reliably predict performance of the CE in different operating scenarios. An operating scenario is defined as the set of the operating objective, channel availability, and channel quality metrics. In this paper, we develop several performance evaluation and prediction indices to quantify the amount of knowledge of different CE algorithms independently of the implementation approach and/or their operating scenarios. Using these new indices, we are able to provide a more accurate estimation of the learning process and future performance of each individual CE algorithm. A number of simulation-based experiments was conducted. Our results show that proposed contextual CE algorithms based on the developed knowledge indicators is able to improve the wireless communication system’s objective rewards significantly. In effect, the contextual CE is able to deliver about 10% more data than the CE with the fixed exploration rate.

Modulation Classification (MC) is a difficult task that can increase awareness in Cognitive Radio (CR) applications. Much of the research in MC has been for single antenna and single user scenarios. With multiple users, blind source separation (BSS) techniques have successfully been used to separate a linear mixture of signals. This work demonstrates that results for MC in a single -user MIMO communications system can be extended to MC in a multi -user scenario with the use of blind source separation techniques. However, a number of difficulties exist with the use of blind source separation techniques that make a simple extension (difficult) possible. First, since the number of users is unknown, BSS techniques must attempt to separate signals with the assumption that a larger number of users exist (than are actually present). Second, BSS techniques can separate signals up to an ambiguity in phase, order, and magnitude—further complicating an extension of common classification methods. Lastly, well-known BSS techniques sometimes fail to properly separate even common digital modulations. The proposed approach to solve these issues comprises of the fastICA BSS technique for signal separation Hyvärinen and Oja (Neural Netw Off J Int Neural Netw Soc 13(4–5):411–430, 2000 ), fourth and sixth-order cumulants as distinguishing features for several digital modulations, and support vector machines with a radial basis function for classification. Given four common modulation schemes BPSK, QPSK, 8-PSK, and 16-QAM, the proposed approach classifies correctly more than 50% of the time for signal to noise ratios higher than 0 dB.