Supercomputing's Role in Data Problems and Its Contribution to Solutions

High Performance Computing (HPC) has proven to be a blessing that unfortunately generates a conundrum, the solution to which is of major importance: Will the resultant glut of data overwhelm the user to the point of destroying the benefit of the advances in computation? The DoD can look forward with certainty to significant steady increases in computer performance and may soon experience an even more expansive revolution in this area. This article sets forth actual experiences with several instantiations of HPC battlefield simulations, discussing the potential value contained in the flood of data that they generated. The impediments to the effective use of that data are identified and alternative methods of reducing these impediments are surveyed. A solution, which is now being implemented and tested, is presented. Further subjects such as data mining and data visualization, both needed to enhance insights, are described and discussed. The article ends with an outline of the future and an over-arching vision for the appropriate response to the threat of data inundation.