This work presents a memory-architecture-algorithm framework that explores neural network reversibility and embedded DRAM (eDRAM) to enable efficient on-device training under tight energy and area constraints. Reversible architectures recompute intermediate activations instead of storing them, minimizing off-chip transfers to DRAM. Applied to transformer models, reversible adapter branches enable low-overhead fine-tuning of large language models such as GPT-2. We fabricated 4T eDRAM arrays for intermediate activation storage that exhibit 12.4 microsecond data retention with less than 0.01 percent bit errors, a unique implicit refresh property to preserve data integrity, and over 11 percent and 20 percent reductions in access energy and static power versus SRAM, respectively. Our results establish eDRAM as a practical storage medium for transient data in LLM workloads and show that reversibility enables scalable, memory-efficient edge training.

3D die-stacked DRAM has emerged as a key technology for delivering high bandwidth and high density for applications such as high-performance computing, graphics, and machine learning. However, different applications place diverse and sometimes diverging demands on power, performance, and area that cannot be universally satisfied with fixed commodity DRAM designs. Die stacking creates the opportunity for a large DRAM design space through 3D integration and expanded total die area. To open and navigate this expansive design space of customized memory architectures that cater to application-specific needs, we introduce DreamRAM, a configurable bandwidth, capacity, energy, latency, and area modeling tool for custom 3D die-stacked DRAM designs. DreamRAM exposes fine-grained design customization parameters at the MAT, subarray, bank, and inter-bank levels, including extensions of partial page and subarray parallelism proposals found in the literature, to open a large previously-unexplored design space. DreamRAM analytically models wire pitch, width, length, capacitance, and scaling parameters to capture the performance tradeoffs of physical layout and routing design choices. Routing awareness enables DreamRAM to model a custom MAT-level routing scheme, Dataline-Over-MAT (DLOMAT), to facilitate better bandwidth tradeoffs. DreamRAM is calibrated and validated against published industry HBM3 and HBM2E designs. Within DreamRAM's rich design space, we identify designs that achieve each of 66

The transformer neural architecture has catalyzed a series of AI breakthroughs in the field of natural language processing, fueling the growth of AI conversational interfaces. The quality of service for such AI-based chatbots is typically defined by the round-trip latency following a user’s prompt and the energy consumption of the computing hardware. We present a sparse transformer processor, named Atropos, that tailors its latency and energy expenditures according to the complexity of the input prompt it processes. The 12nm processor introduces three key innovations: (1) a specialized datapath for entropy-based early exit assessment that reduces BERT inference latency by up to 6.13×, (2) a mixed-precision FP4/FP8 MAC unit with per-vector exponent biases that doubles throughput while maintaining accuracy, and (3) a prompt-level power management scheme that dynamically scales voltage and frequency based on input prompt complexity. These innovations together enable the STP to achieve a peak efficiency of 18.1 TFLOPs/W and 65mJ per inference, representing a 7.14× energy improvement over conventional BERT inference.

Large language model (LLM) inference performance is increasingly bottlenecked by the memory wall. While GPUs continue to scale raw compute throughput, they struggle to deliver scalable performance for memory bandwidth bound workloads. This challenge is amplified by emerging reasoning LLM applications, where long output sequences, low arithmetic intensity, and tight latency constraints demand significantly higher memory bandwidth. As a result, system utilization drops and energy per inference rises, highlighting the need for an optimized system architecture for scalable memory bandwidth. To address these challenges we present the Reasoning Processing Unit (RPU), a chiplet-based architecture designed to address the challenges of the modern memory wall. RPU introduces: (1) A Capacity-Optimized High-Bandwidth Memory (HBM-CO) that trades capacity for lower energy and cost; (2) a scalable chiplet architecture featuring a bandwidth-first power and area provisioning design; and (3) a decoupled microarchitecture that separates memory, compute, and communication pipelines to sustain high bandwidth utilization. Simulation results show that RPU performs up to 45.3x lower latency and 18.6x higher throughput over an H100 system at ISO-TDP on Llama3-405B.

The inherent uncertainty in quantifying carbon footprint is a major challenge for designing environmentally sustainable computing systems. While existing efforts propose a variety of approaches to address this uncertainty, optimizing carbon footprint and energy efficiency together, while also accounting for uncertainty in carbon footprint, remains challenging. We identify two types of uncertainty in $\mathbf{C O}_{2}$-aware hardware design: data value $\left(\mathcal{D}_{\mathbf{V}}\right)$ uncertainty, where probability distributions of $\mathbf{C O}_{\mathbf{2}}$-related parameters exhibit high standard deviation (e.g., fluctuations in CO2 emissions of the power grid), and data availability $\left(\mathcal{D}_{\mathbf{A}}\right)$ uncertainty, in which probability distributions of $\mathbf{C O}_{\mathbf{2}}$-related parameters cannot even be reliably estimated due to a lack of data (e.g., carbon emissions during integrated circuit fabrication, with very few data points available). $\mathcal{D}_{A}$ uncertainty is especially challenging. To address this challenge, we leverage mathematical Robust Optimization (RO) techniques that enable uncertainty-aware decision-making without requiring explicit probability distributions. We present RHODES, a robust optimization framework for designing $\mathbf{C O}_{\mathbf{2}}$-efficient computing systems under carbon footprint data uncertainty. RHODES jointly models $\mathcal{D}_{\mathrm{v}}$ and $\mathcal{D}_{\mathrm{A}}$ uncertainties at multiple layers of the computing stack to optimize total carbon (tC) under uncertainty, and can target 3 different primary objectives (we include example results for specific systems, details in the full paper): (1) minimizing execution time given a performance constraint: designs that do not account for uncertainty incur 1.7× worse tC for the same performance constraint; (2) minimizing execution time given a tC constraint: RHODES produces robust designs with $\mathbf{1. 3 6} \times$ lower tC and only 0.2% execution time degradation, for complex heterogeneous system-on-chip (SoC), considering domain specific accelerators and workload level parallelism; and (3) minimizing total carbon delay product (tCDP, a metric of $\boldsymbol{C} O_{2}$ efficiency): improves tCDP by 1.3-3.17× vs. state-of-the-art $\mathbf{C O}_{2}$-aware optimization frameworks.

The push for greater efficiency in AI computation has given rise to an array of accelerator architectures that increasingly challenge the GPU's long-standing dominance. In this work, we provide a quantitative view of this evolving landscape of AI accelerators, including the Cerebras CS-3, SambaNova SN-40, Groq, Gaudi, and TPUv5e platforms, and compare against both NVIDIA (A100, H100) and AMD (MI-300X) GPUs. We evaluate key trade-offs in latency, throughput, power consumption, and energy-efficiency across both (i) end-to-end workloads and (ii) benchmarks of individual computational primitives. Notably, we find the optimal hardware platform varies across batch size, sequence length, and model size, revealing a large underlying optimization space. Our analysis includes detailed power measurements across the prefill and decode phases of LLM inference, as well as quantification of the energy cost of communication. We additionally find that Cerebras, SambaNova, and Gaudi have 10-60

Large model training beyond tens of thousands of GPUs is an uncharted territory. At such scales, disruptions to the training process are not a matter of if, but a matter of when – a stochastic process degrading training productivity. Dynamic runtime variation will become increasingly more frequent as training scales up and GPUs are operated in increasingly power-limited and thermally-stressed environments. At the 64k GPU scale, we already observed 9 Motivated by our analysis and the large design space around performance variability, we present PRISM – a performance modeling framework that considers the stochastic nature of the large-scale distributed training. The core of PRISM is the statistical method that provides a quantifiable measure for probabilistic guarantees on training time. Using PRISM, we explore the design and optimization space of distributed training, from parallelization methods to next-generation training systems. PRISM is validated with real-system measurement, showing training time prediction accuracy with 20.8

Deploying LLMs on edge devices presents serious technical challenges. Memory elasticity is crucial for edge devices with unified memory, where memory is shared and fluctuates dynamically. Existing solutions suffer from either poor transition granularity or high storage costs. We propose FlexQuant, a novel elasticity framework that generates an ensemble of quantized models, providing an elastic hosting solution with 15x granularity improvement and 10x storage reduction compared to SoTA methods. FlexQuant works with most quantization methods and creates a family of trade-off options under various storage limits through our pruning method. It brings great performance and flexibility to the edge deployment of LLMs.

Scaling large language models (LLMs) has driven significant advancements, yet it faces diminishing returns and escalating energy demands. This work explores how test-time compute (TTC) can serve as an energy-efficient complement to conventional scaling strategies by allocating additional computational resources at inference time rather than during training. Specifically, we investigate whether employing TTC can achieve superior accuracy-energy trade-offs compared to simply increasing model size. Our empirical analysis reveals that TTC surpasses traditional model scaling in accuracy/energy efficiency, with notable gains in tasks demanding complex reasoning rather than mere factual recall. Further, we identify a critical interaction between TTC performance and output sequence length, demonstrating that strategically adjusting compute resources at inference time according to query complexity can substantially enhance efficiency. Our findings advocate for TTC as a promising direction, enabling more sustainable, accurate, and adaptable deployment of future language models.

The electronics and semiconductor industry is a prominent consumer of per- and poly-fluoroalkyl substances (PFAS), also known as forever chemicals. PFAS are persistent in the environment and can bioaccumulate to ecological and human toxic levels. Computer designers have an opportunity to reduce the use of PFAS in semiconductors and electronics manufacturing, including integrated circuits (IC), batteries, displays, etc., which currently account for a staggering 10 of the total PFAS fluoropolymers usage in Europe alone. In this paper, we present a framework where we (1) quantify the environmental impact of PFAS in computing systems manufacturing with granular consideration of the metal layer stack and patterning complexities in IC manufacturing at the design phase, (2) identify contending trends between embodied carbon (carbon footprint due to hardware manufacturing) versus PFAS. For example, manufacturing an IC at a 7 nm technology node using EUV lithography uses 18 compared to manufacturing the same IC at a 7 nm technology node using DUV immersion lithography (instead of EUV) unlike embodied carbon trends, and (3) conduct case studies to illustrate how to optimize and trade-off designs with lower PFAS, while meeting power-performance-area constraints. We show that optimizing designs to use less back-end-of-line (BEOL) metal stack layers can save 1.7× PFAS-containing layers in systolic arrays.

We present EPOCHS-1, a 12 nm, 64 mm(2) system-on-chip (SoC) with a high degree of heterogeneity. It features four Linux-SMP-capable RISC-V cores, 14 different types of accelerators, a distributed memory hierarchy, and various peripherals. EPOCHS-1's memory hierarchy has the flexibility to support a diverse set of accelerators and can scale to support complex applications with 34% and 25% reduction in latency and energy, respectively. A subset of the SoC's 23 power and 35 clock domains is regulated with a fully-decentralized power-allocation scheme and hybrid unified voltage and frequency scaling (HUVFS) that combines an in-package switched regulator with a per-tile low dropout (LDO). Combined, these techniques achieve up to a 1.57x speedup versus a centralized power management baseline. Designed with an agile methodology, EPOCHS-1 is based on an open-source SoC architecture and features only open-source components, either third-party or newly designed, thus enabling design reuse for future research projects.

PFAS (per-and poly-fluoroalkyl substances), also known as forever chemicals, are widely used in electronics and semiconductor manufacturing. PFAS are environmentally persistent and bioaccumulative synthetic chemicals, which have recently received considerable regulatory attention. Manufacturing semiconductors and electronics, including integrated circuits (IC), batteries, displays, etc., currently accounts for a staggering 10% of the total PFAS-containing fluoropolymers used in Europe alone. Now, computer system designers have an opportunity to reduce the use of PFAS in semiconductors and electronics at the design phase. In this work, we quantify the environmental impact of PFAS in computing systems, and outline how designers can optimize their designs to use less PFAS. We show that manufacturing an IC design at a 7 nm technology node using Extreme Ultraviolet (EUV) lithography uses 20% less volume of PFAS-containing chemicals versus manufacturing the same design at a 7 nm node using Deep Ultraviolet (DUV) immersion lithography (instead of EUV). We also show that manufacturing an IC design at a 16 nm technology node results in 15% less volume of PFAS than manufacturing the same design at a 28 nm node due to its smaller area.

This article presents a holistic research agenda to address the significant environmental impact of information and communication technology (ICT), which accounts for 2.1%-3.9% of global greenhouse gas emissions. It proposes several research thrusts to achieve sustainable computing: accurate carbon accounting models, life cycle design strategies for hardware, efficient use of renewable energy, and integrated design and management strategies for next-generation hardware and software systems. If successful, the research would flatten and reverse growth trajectories for computing power and carbon, especially for rapidly growing applications like artificial intelligence. The research takes a holistic approach because strategies that reduce operational carbon may increase embodied carbon, and vice versa. Achieving these goals will require interdisciplinary collaboration between computer scientists, electrical engineers, environmental scientists, and economists.

The rapid rise of Large Language Models (LLMs) has prompted a re-evaluation of system architecture design, making energy efficiency and sustainability more crucial than ever. Recently, wafer-scale architectures have emerged as a viable alternative for LLM training and inference, as evidenced by the success of Cerebras Systems. In this work, we examine the carbon implications of wafer-scale architectures as compared to traditional GPUs. As a case study, we examine LLMs on a Cerebras CS-3 system in order to quantify power and total carbon. Then, we analyze total carbon delay product (tCDP) to evaluate the carbon efficiency and performance potential of these systems. We take the first step towards exploring this trade-off for wafer-scale versus traditional GPU architectures - and ultimately find there exists a rich design space, depending on workload and hardware configuration.

The demand for efficient machine learning in edge devices is challenging the capabilities of general-purpose computing systems. While domain-specific System on Chip (SoCs) are efficient, they are often prohibitively expensive due to long design times and high design costs. To address these limitations, the community has begun to explore System in Package (SiP) designs for low-cost assembly of reusable accelerators, available as chiplets, to democratize customization. This presents a new challenge of macro-architecture design space exploration (DSE). Prior works do not address this problem, having only investigated micro-architecture design and optimization of homogeneous SiPs. To address this need, and unlock the potential of assembling custom SiPs, comprising heterogeneous chiplets, we introduce an early DSE framework, CASCADE - A. CASCADE employs fast, first-order performance models to capture the tradeoffs of composable compute chiplets, leveraging tool-generated traces to comprehend dataflow patterns in the context of state-of-the-art machine learning tasks. Using CASCADE, we assess the performance benefits of composable SiPs comprising hetero-chiplets for single-tenant and two-tenant scenarios. Notably, we demonstrate that hetero-chiplet systems can deliver speedups in the range of 3-5x, depending on the application, compared to a baseline GPU chiplet system.

The world's push toward an environmentally sustainable society is highly dependent on the semiconductor industry. Despite existing carbon modeling efforts to quantify carbon footprint of computing systems, optimizing carbon footprint in large design spaces-while also considering trade-offs in power, performance, and area-is especially challenging. To address this need, we present CORDOBA, a carbon-aware optimization framework that optimizes carbon efficiency. We quantify carbon efficiency using the total Carbon Delay Product metric (tCDP): the product of total carbon and application execution time. We justify why tCDP is an effective metric for quantifying carbon efficiency. We use CORDOBA to explore the large design space for carbon-efficient specialized hardware, and identify distinct carbonefficient optimal designs across operational use (eliminating up to 98% of the design space) despite uncertainty in carbon footprint parameters. We quantify opportunities to improve tCDP for real system case studies: (a) optimizing hardware provisioning from 8 to 4 cores in real system CPUs improves tCDP by 1.25x; and (b) leveraging advanced three-dimensional (3D) integration techniques (3D stacking of separately-fabricated logic and memory chips) improves tCDP by 6.9x versus conventional systems.

Given billions of mobile users, the environmental impact of edge computing is significant. To address this, future applications need to execute computations on a green component which is fueled by renewable energy sources. However, because of the intermittent nature of the renewable energy sources, the carbon intensity of computing components can significantly vary with location and time of use. This poses a new challenge for edge applications-deciding when and where to run computations across consumer devices at the edge and servers in the cloud. Such scheduling decisions become more complicated with the amortization of the rising embodied emissions and stochastic runtime variance. This work proposes GreenScale, an intelligent execution scaling engine that accurately selects the carbon-optimal execution target for edge applications in different runtime environments. Our evaluation with three representative categories of applications (i.e., AI, Game, and AR/VR) demonstrate that the carbon emissions of the applications can be reduced by 36.5%, on average, with GreenScale.

The increase in both the number and the types of accelerators in modern systems on chip (SoCs) necessitates a rethinking of power management (PM) strategies. To overcome the scalability shortcomings of current methods, we propose BlitzCoin, a fully decentralized hardware-based PM coupled with optimized unified voltage and frequency regulation. We evaluated BlitzCoin through register transfer-level simulations of multiple SoCs targeted toward different application domains. The results are further validated through silicon measurements of a fabricated 12-nm many-accelerator SoC that includes BlitzCoin. Our evaluations show that BlitzCoin is markedly faster than state-of-the-art centralized PM strategies, with 8 x to 12 x lower response times. This results in 25%-34% throughput improvement and allows for scaling to 7 x to 13 x larger SoCs, all with a small area overhead of <1%. BlitzCoin is an addition to the open source ESP SoC platform, offering a foundation for further exploration of PM strategies.

Trapped-ion (TI) qubit architectures have recently become a promising candidate for designing and building quantum computers. In the current noisy-intermediate scale quantum (NISQ) era, TI qubits stand out for their connectivity and reliability over other candidates such as superconducting qubits. However, physical constraints stemming from fine-grained frequency control of TI qubits introduce limitations to the maximum number of trapped-ions in a quantum computing system. This fundamentally challenges the design of large TI-based quantum computers, with various quantum applications requiring a large number of qubits for practical realization.Recent work has proposed TI Quantum Charge Coupled Devices (QCCD) which provides mechanisms to link multiple ion-chains together to address the issue of scalability. While such advances help increase the total qubit count in a TI system, the weak links between ion chains introduce a performance bottleneck and gate-latency penalty. Prior TI modeling toolflows have not explored the performance and scalability implications introduced by weak links on the design of future TI systems; in this work, we directly elevate the weak link as an architectural knob, and present an architecture-level performance modeling framework called VelociTI. We use VelociTI to study the performance trade-offs in a trapped-ion quantum computing design and find that optimal scheduling of qubits can provide a 6.2× speedup in performance.

As cutting-edge large language models (LLMs) continue to transform various industries, their fast-growing model size and sequence length have led to memory traffic and capacity challenges. Recently, AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm have proposed a Microscaling standard (Mx), which augments block floating-point with microexponents to achieve promising perplexity-to-footprint trade-offs. However, the Microscaling suffers from significant perplexity degradation on modern LLMs with less than six bits. This paper profiles modern LLMs and identifies three main challenges of low-bit Microscaling format, i.e., inaccurate tracking of outliers, vacant quantization levels, and wasted binary code. In response, Nanoscaling (NxFP) proposes three techniques, i.e., NanoMantissa, Adaptive Microexponent, and Code Recycling to enable better accuracy and smaller memory footprint than state-of-the-art MxFP. Experimental results on direct-cast inference across various modern LLMs demonstrate that our proposed methods outperform state-of-the-art MxFP by up to 0.64 in perplexity and by up to 30 Furthermore, NxFP reduces memory footprint by up to 16 comparable perplexity as MxFP.