A virtual tile approach to raster-based calculations of large digital elevation models in a shared-memory system

Grid digital elevation models (DEMs) are commonly used in hydrology to derive information related to topographically driven flow. Advances in technology for creating DEMs have increased their resolution and data size with the result that algorithms for processing them are frequently memory limited. This paper presents a new approach to the management of memory in the parallel solution of hydrologic terrain processing using a user-level virtual memory system for shared-memory multithreaded systems. The method includes tailored virtual memory management of raster-based calculations for datasets that are larger than available memory and a novel order-of-calculations approach to parallel hydrologic terrain analysis applications. The method is illustrated for the pit filling algorithm used first in most hydrologic terrain analysis workflows. HighlightsWe presented a tiled virtual memory system to process large DEM datasets on a machine with limited memoryEfficiencies were achieved using algorithm refinements such as multi-threading, load balancing and the pre-fetching approaches.We found optimal tile sizes for the cases of best performance and better memory usage efficiency.