Reducing Southern Ocean shortwave radiation errors in the ERA5 reanalysis with machine learning and 25 years of surface observations

Abstract Earth System models struggle to simulate clouds and their radiative effects over the Southern Ocean, partly due to a lack of measurements and targeted cloud microphysics knowledge. We have evaluated biases of downwelling shortwave radiation in the ERA5 climate reanalysis using 25 years (1995 - 2019) of summertime surface measurements, collected on the RSV Aurora Australis, the RV Investigator, and at Macquarie Island. During October - March daylight hours, the ERA5 simulation of SWdown exhibited large errors (mean bias = 54 Wm−2, mean absolute error = 82 Wm−2, root mean squared error = 132 Wm-2, R2 = 0.71). To determine whether we could improve these statistics, we bypassed ERA5’s radiative transfer model for SWdown with machine learning-based models using a number of ERA5’s grid-scale meteorological variables as predictors. These models were trained and tested with the surface measurements of SWdown using a 10-fold shuffle split. An XGBoost and a random forest-based model setup had the best performance relative to ERA5, both with a near complete reduction of the mean bias error, a decrease in the mean absolute error and root mean squared error by 25% ± 3 %, and an increase in the R2 value of 5% ± 1% over the 10 splits. Large improvements occurred at higher latitudes and cyclone cold-sectors, where ERA5 performed most poorly. We further interpret our methods using SHapley Additive exPlanations. Our results indicate that data-driven techniques could have an important role in simulating surface radiation fluxes and in improving reanalysis products.