Prescient Instruction Prefetch

This paper introduces prescient instruction prefetch, a technique that uses helper threads to improve single-threaded application performance by performing judicious and timely instruction prefetch. A helper thread is initiated when the main thread encounters a spawn point. The execution of the helper thread prefetches instructions starting at a distant target point that identifies a code region that the main thread is likely to execute soon and tends to incur I-cache misses. This paper formulates the identification of appropriate spawn and target points for the associated helper threads as an optimization problem by modeling runtime program behavior as a Markov chain with statistics derived from profile data. This formulation enables the accurate estimation of important statistical quantities related to helper thread execution via a simple and efficient computational strategy based upon Tarjan’s fast path expression algorithm. Using this formulation we propose a spawn-target pair selection algorithm. This algorithm has been implemented for the Itanium Processor Family (IPF) architecture. As an initial limit study we present simulation results indicating that helper threads given perfectly predicted register and memory live-in values at the target can achieve speedups in the range of 6% to 63% on an in-order SMT machine with four hardware thread contexts for a select set of benchmarks that have high I-cache miss rates. With increasing levels of realism, we find speedups ranging from 0.5% to 54% are feasible when taking into account the overhead of live-in precomputation. The approach presented in this paper is potentially applicable to other thread speculation techniques.