EOSDIS Petabyte Archives: Tenth Anniversary

One of the world's largest scientific data systems, NASA's Earth Observing System Data and Information System (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASA's collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the Hubble Space Telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS archives focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.