NEX AI: A Cloud and HPC Agnostic Framework for Scaling Deep Learning and Machine Learning Applications for Earth Science

Training deep learning (DL) and machine learning (ML) models is data intensive and computationally demanding. As the application of DL/ML surges in academia and industry, so does the demand for high performance computing (HPC) resources. Cloud Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Function as a Service (FaaS) are increasingly being used for real-time inferences based on events and user requirements for time to completion and cost for running these DL/ML workflows over large spatio-temporal data sets. The NASA Earth Exchange (NEX)-AI has established an end-to-end framework for performing inferences and predictions from Earth Science data using a benchmarked suite of deep learning models, unique labeled training datasets and creating a serverless stack for workflow automation and scaling on both HPC and Cloud. NEX-AI has long invested in deep learning …