Energy and Performance Optimized Computation Offloading for Near-Memory Computing

The memory wall problem significantly hampers the performance of modern data-intensive applications (such as AI/ML) in conventional computing architectures. Near-Memory Computing (NMC) offers a promising alternative by embedding computation units within main memory; however, it necessitates an effective offloading approach. Current offloading approaches primarily concentrate on performance metrics, often neglecting energy considerations during decision-making. Furthermore, many existing approaches require conducting preliminary trials on both the host and NMC sides, which can lead to considerable overhead. To address these challenges, we propose EP-Off, an estimation-based dynamic offloading approach. Initially, EP-Off executes a few iterations of the offloadable region exclusively on the host CPU, allowing for the estimation of the Energy-Delay Product (EDP) for both host and NMC sides. The remaining portion is then offloaded to the side with the lower estimated EDP. This approach ensures a well-balanced decision-making process that strikes a balance between performance and energy efficiency, while reducing overhead by eliminating unnecessary preliminary trials on the NMC. Through extensive experiments with diverse data-intensive applications, EP-Off exhibits substantial performance improvements ($2.1 x$ & $1.5 x$), energy saving (54% & 35%), and reductions in off-chip data transfers (65% & 27%) compared to both conventional and state-of-the-art approaches. Furthermore, EP-Off reduces EDP by 79% and 52% compared to conventional and state-of-the-art approaches, respectively.