The rapid advancement of artificial-intelligence (Al) models has increased demand for high-precision and energy-efficient edge-Al chips. Floating-point (FP) support is essential for high-precision neural-network (NN) training and inference; yet FP incurs higher energy and area overhead due to complex FP multiplication and accumulation (MAC) operations. Digital compute-in-memory (DCIM) and floating-point CIM (FP-CIM) [1]–[10] have emerged as promising techniques to improve energy efficiency with higher accuracy. Previous FP-CIM implementations [1]–[7] achieved good performance through various alignment schemes and computing processes. However, as illustrated in Figure 14.3.1, the implementation of a digital-domain FP-CIM faces several challenges: (1) the difficulty of balancing FP-computation precision and input reusability, as alignment operations are unfriendly to CIM structure; (2) a large performance loss or area overhead due to peripheral parallel-alignment schemes; and (3) huge digital-MAC dynamic-energy consumption due to low 2's-complement (2C) negative-weight sparsity, coupled with an additional sign-bit computation overhead in digital CIM. This work presents a hierarchical broadcast-alignment non-2's-complement-MAC (B-A-N2CMAC) FP-CIM macro, featuring (1) a broadcast input and embedded lightweight convertor structure to enable BF16/LNT8 MAC operations with an improved input reusability; (2) an embedded area-efficient adaptive-alignment scheme with a dual-bit serial MAC; and (3) a format-mixed N2CMAC to reduce dynamic circuit activity and signed computation overhead. A 28nm 64kb B-A-N2CMAC FP-CIM macro is fabricated to support FP-MAC operations using BF16 and INT8 representations. This CIM macro achieved an energy efficiency of 62.84TFLOPS/W for BF16 and 90.15TOPS/W for LNT8.

Hybrid-domain CIMs [1]–[3] are attracting increasing attention nowadays as they combine the advantages of both digital CIMs (DCIM) [4]–[6] and analog CIMs (ACIM) [7]–[10], offering a more balanced choice. As depicted in Figure 14.6.1, when designing a hybrid CIM (HCIM), the first challenge to address is (1) the definition of boundary between the digital and analog part, which is determined by feature input mode and weight mapping. Prior HCIMs either utilized a bit-parallel in features to obtain high accuracy at the cost of hardware (HW) overhead [1] or realized a low HW overhead but suffered from accuracy loss due to error pollution problem (error of the analog partial sum makes the high precision digital partial sum of the same bit weight meaningless) with bit-serial scheme and specific weight mapping strategy [2], [5], neither of which provided a perfect solution to the boundary question. Therefore, we propose a bit-rotated feature-in scheme to address this challenge. To compress multiple multiplication results into one MAC result, several levels of shifters and adders are usually required, and the sign-bit needs to be considered when compressing signed and unsigned information. Unlike the bit-parallel/serial approach [1], [2], the bit-rotated scheme considers the sign-bit in the first product-wise adder, which induces (2) more digital overhead for sign-bit processing. Another challenge that continues to plague bit-rotated HCIM is (3) excessive energy wasted on low accuracy-contributed low-bit quantization in analog parts.