A 4nm-based quad-chiplet with an advanced packaged LLM accelerator achieving 56.8TPS on LLaMA v3.3 70B with single-batch 2k/2k input/output sequences. The architecture combines chiplet-based design, low-latency die-to-die interfaces, unified mixed-precision compute, holistic synchronization, and HBM3E with advanced power schemes to sustain bandwidth, capacity and thermal stability. The SoC integrates four NPU chiplets, four HBM3E modules, and four ISCs for stable power delivery.

The growing computational demands of AI inference have led to widespread use of hardware accelerators for different platforms, spanning from edge to the datacenter/cloud. Certain AI application areas, such as in high-frequency trading (HFT) [1–2], have a hard inference latency deadline for successful execution. We present our new AI accelerator which achieves high inference capability with outstanding single-stream responsiveness for demanding service-layer objective (SLO)-based AI services and pipelined inference applications, including large language models (LLM). Owing to low thermal design power (TDP), the scale-out solution can support multi-stream applications, as well as total cost of ownership (TCO)-centric systems effectively.

72.5GFLOPS GPGPU computing, 240 Mpixel/s sustainable image signal processing and 60fps 1080p multi-format video codec (MFC) capabilities are integrated with an 1.7GHz out-of-order-execution dual-core ARMv7A architecture CPU and 12.8GB/s memory subsystem for a next-generation application processor. The GPU-based general-purpose computing capability can deliver 10× higher energy efficiency in compute-intensive multimedia applications, compared with a CPU solution on the same die. The improved energy efficiency with GPGPU computing enables next-generation fused multimedia applications, with the assistance of dedicated high-performance low-power multimedia accelerators, as well as with low-power design and process technology, as shown in Fig. 9.4.1.

The Unified Early Z-Test (U-EZT) is proposed to examine the visibility of pixels during tile-based rasterization in a mobile 3D graphics processor. U-EZT combines the advantages of the Z-max and Z-min EZT algorithms: the Z-max algorithm is improved by the independently updatable z-max tiles and the use of mask bits; and the Z-min algorithm is improved by reusing the mask bits from the z-max test to update the z-min tiles after tile rasterizing. As a result, storage requirements are reduced to 3 bits per pixel, and simulations suggest that U-EZT requires 20 percent to 57 percent less memory bandwidth than previous EZT algorithms.

A 3D graphics system integrating two symmetric unified shader cores for mobile application is presented. To utilize instruction, data, and task level parallelism, a dual-core, dual-issue VLIW and multi-threading method is adopted. For efficient processing, an IEEE-754 compliant fast 4D vector inner product arithmetic unit for matrix multiplication, an internal bus system and a configurable texture cache technique to reduce power consumption in texture unit are proposed. By these methods, the proposed processor achieves 143 Mvertices/s and 2.3 Gtexels/s consuming the power of 367 mW. Also, 45% performance improvement and 26% increase in performance per power ratio are achieved.

In this paper, we propose an architecture of tessellation hardware to save memory bandwidth in a mobile multimedia processor. To reduce the implementation overhead, floating-point computations of tessellation are accelerated by the conventional GPU pipeline, and only tessellation-specific control logic is handled by an additional hardware unit. Tightly coupled with a vertex shader, the additional unit dynamically produces topological configurations and parametric coordinates of refinement patterns in the type of indexed triangle strips for object-level adaptive tessellation. The topological configurations improve the efficiency of the vertex cache so as to avoid redundant shader operations. Since the proposed tessellator is area-efficient and does not require intermediate memory accesses, its architecture is especially appropriate for the mobile environment, which adopts a shared bus and unified external memory architecture. The proposed tessellator is fabricated on a chip using 0.18μm CMOS technology. With 6.2% additional hardware for a dual-core vertex shader, the implemented chip performs 120Mvertices/s vertex shading and saves memory bandwidth up to 250 times in tessellation.

In this paper, a power efficient vertex processor for mobile graphics applications is presented. A four-threaded and four-issue expanded VLIW datapath with a quad-float vertex texture fetcher is proposed by exploiting graphics specific characteristics after evaluation of several candidate architectures. Instruction-level power control methods such as operand sharing and writeback re-allocation along with operand isolations and gated clocks result in 40.4% and 82% reduction in energy dissipation and energy delay product compared to the most widely used single threaded SIMD. The proposed processor with the optimized datapath and vertex caches implemented in a 0.18- mum 1P4M CMOS process achieves 186-Mvertices/s geometry performance which is the best result among the processors that are IEEE-754 compliant.

Hardware architecture of a tessellator based on a vertex shader is proposed to save memory bandwidth for a mobile 3D graphics engine. According to the appropriate separation of the tessellation process, the powerful performance of a vertex shader, 120 Mvertices/s, is utilised for floating point computations. The remaining part of the process is handled by a dedicated control unit, implementation of which requires 6.2% additional logic gates. The proposed tessellator reduces the bandwidth consumed to transfer 3D geometry data up to 1/250.