Exploring Multi-Level Cache Prefetching for Fabric Attached Memory.

Memory disaggregation in data centers has been approaching practicality, owing to the maturity of interconnect standards like Compute Express Link (CXL) [3]. CXL presents a hardware centric approach for multiple compute nodes to pool memory capacities from a shared Fabric Attached Memory (FAM) node, on a per need basis. Using FAM for memory provisioning can potentially mitigate resource underutilization and yield in cost savings, but can cost the application it's performance due to relatively longer access latency. Modern processors attempt to hide memory access latency by employing sophisticated cache prefetchers. While resourceful, current cache prefetching techniques can be further optimized, in light of the long access latency of CXL FAM. To that end, we consider multi-level cache prefetcher that adds additional layer of prefetching at Last Level Cache (LLC). Our multi-level prefetching scheme increases the fraction of requests that hit in LLC, potentially decreasing the sensitivity of workload to FAMlatency. We implemented our multi-level cache prefetcher using SST simulation components [29], and evaluated it with workloads from standard benchmarks suites in single and multi-node system configuration. Our evaluation reveals that, comparing to using only per-core prefetcher, multi-level prefetcher resulted in performance improvement of 2-7%, with LLC hit fraction increasing by 13%.