Abstract. Radiances from the Advanced Very High Resolution Radiometer (AVHRR) onboard the NOAA-19 satellite were successfully simulated from Suomi-NPP Visible Infrared Imaging Radiometer Suite (VIIRS) radiances using collocated AVHRR/VIIRS datasets from 2012–2013. Spectral Band Adjustment Factors (SBAFs) were derived using linear regression and neural networks (NNs). The NN approach produced the best results, and separating daytime from night-time conditions when simulating AVHRR channel 3B at 3.7 µm was key. Furthermore, daytime radiance corrections in this channel must depend on actual surface and cloud reflectances to be realistic, which was only achieved by the NN approach. The cloud mask, cloud top height, and cloud phase products were produced from the simulated AVHRR radiances using the same retrieval methods for NOAA-19 data used to compile the CLARA-A3 climate data record (CDR). CLARA-A3 is the third edition of the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF) CDR with cloud parameters, surface albedo, surface radiation, and Top of Atmosphere (TOA) radiation products from AVHRR. Products were validated using CALIPSO cloud products and agreed well with original CLARA-A3 products, with the best results provided by the NN simulation approach. The NN-based approach best reproduced the corresponding products for cloud optical thickness (COT), cloud effective radius (CRE), liquid water path (LWP), and ice water path (IWP). The CLARA-A3 CDR will be complemented and extended with VIIRS-based products to cover the period 1979–2024 (46 years). This edition will be known as CLARA-A3.5. Future extensions and editions can follow a similar approach by applying the same radiance simulation method to collocated data from the Metop-C AVHRR and the Metop-SG METimage sensors, the first version of the latter scheduled for launch in August 2025. Successful simulation of AVHRR radiances from METimage data enables the extension of the CLARA CDR for several decades using observations from VIIRS and METimage.

Clouds are characterized - among other things - by their intense variability in time, space and optical thickness. These variables impact the modulation of solar radiation (reflection, transmission and absorption) and may distort the signal from the surface beneath. This in turn makes it important to detect even optically thin clouds using remote sensing methods, even if the focus is on earth observation. This study has been initiated by the Swedish Forest Agency (SFA). In order to reduce the proliferation of bark beetles, SFA needs to identify stressed trees at an early stage. To this end, high-resolution scenes from the Multi-Spectral Imager (MSI) on board the Sentinel-2 platforms were analyzed. Unfortunately, the quality of ESA's scene classification layer (SCL) does not meet the requirements for reliably sorting out scenes contaminated with thin clouds. To overcome this problem, it was decided to make use of the fact that the integration of machine learning (ML) methods within the remote sensing domain has significantly improved performance on remote sensing tasks. But a common difficulty is that ML methods typically depend on large amounts of annotated data for training. Annotation or classification is usually done manually or by a superior instrument (i.e. active LIDAR). Since such a data basis is missing, a synthetic database (based on simulations instead of observations) was generated to train a Multi Layer Perceptron (MLP). The dataset consists of 200,000 data points, which have been simulated taking into consideration different cloud types, cloud optical thicknesses (COT), cloud geometrical thickness, cloud heights, as well as ground surface and atmospheric profiles. The MLP is trained to predict COT as a proxy for the cloud/clear decision. The performance of the proposed algorithm using both synthetic data (as used during training) and real satellite observations (never presented to the algorithm before) will be discussed in detail. It was found that the MLP approach trained on 1D synthetic data can seamlessly transition to real datasets without requiring additional training. Furthermore it outperforms the ESA-SCL.

Cloud formations often obscure optical satellite-based monitoring of the Earth’s surface, thus limiting Earth observation (EO) activities such as land cover mapping, ocean color analysis, and cropland monitoring. The integration of machine learning (ML) methods within the remote sensing domain has significantly improved performance for a wide range of EO tasks, including cloud detection and filtering, but there is still much room for improvement. A key bottleneck is that ML methods typically depend on large amounts of annotated data for training, which are often difficult to come by in EO contexts. This is especially true when it comes to cloud optical thickness (COT) estimation. A reliable estimation of COT enables more fine-grained and application-dependent control compared to using pre-specified cloud categories, as is common practice. To alleviate the COT data scarcity problem, in this work, we propose a novel synthetic dataset for COT estimation, which we subsequently leverage for obtaining reliable and versatile cloud masks on real data. In our dataset, top-of-atmosphere radiances have been simulated for 12 of the spectral bands of the Multispectral Imagery (MSI) sensor onboard Sentinel-2 platforms. These data points have been simulated under consideration of different cloud types, COTs, and ground surface and atmospheric profiles. Extensive experimentation of training several ML models to predict COT from the measured reflectivity of the spectral bands demonstrates the usefulness of our proposed dataset. In particular, by thresholding COT estimates from our ML models, we show on two satellite image datasets (one that is publicly available, and one which we have collected and annotated) that reliable cloud masks can be obtained. The synthetic data, the newly collected real dataset, code and models have been made publicly available.

Cloud formations often obscure optical satellite-based monitoring of the Earth's surface, thus limiting Earth observation (EO) activities such as land cover mapping, ocean color analysis, and cropland monitoring. The integration of machine learning (ML) methods within the remote sensing domain has significantly improved performance on a wide range of EO tasks, including cloud detection and filtering, but there is still much room for improvement. A key bottleneck is that ML methods typically depend on large amounts of annotated data for training, which is often difficult to come by in EO contexts. This is especially true when it comes to cloud optical thickness (COT) estimation. A reliable estimation of COT enables more fine-grained and application-dependent control compared to using pre-specified cloud categories, as is commonly done in practice. To alleviate the COT data scarcity problem, in this work we propose a novel synthetic dataset for COT estimation, that we subsequently leverage for obtaining reliable and versatile cloud masks on real data. In our dataset, top-of-atmosphere radiances have been simulated for 12 of the spectral bands of the Multispectral Imagery (MSI) sensor onboard Sentinel-2 platforms. These data points have been simulated under consideration of different cloud types, COTs, and ground surface and atmospheric profiles. Extensive experimentation of training several ML models to predict COT from the measured reflectivity of the spectral bands demonstrates the usefulness of our proposed dataset. In particular, by thresholding COT estimates from our ML models, we show on two satellite image datasets (one that is publicly available, and one which we have collected and annotated) that reliable cloud masks can be obtained. The synthetic data, the collected real dataset, code and models have been made publicly available at https://github.com/aleksispi/ml-cloud-opt-thick.

The impact of multiple scattering (MS) by aerosols on satellite-borne lidar measurements is studied by Monte-Carlo radiative transfer simulations.A total of 48 aerosol scenarios are considered.We find that the frequently used MS correction factor can be parameterized as a function of aerosol size and aerosol optical depth.Its dependencies on vertical distribution and total optical depth can be treated as a random error.We illustrate the use of our parameterization by considering an episode of high sea salt concentrations over the ocean.Neglecting MS, or using a constant value of the MS correction factor, can introduce a negative bias in the computed backscattered power that exceeds the random error in our approach.

Satellite Spectral Response functions for a number of imaging sensors. Currently supports: Himawari-8 AHI GOES-16 ABI GOES-17 ABI NOAA AVHRR/1, AVHRR/2, AVHRR/3 Metop AVHRR/3 TIROS-N AVHRR/1 Envisat AATSR Sentinel-3A SLSTR Sentinel-3A OLCI - mean rsr Meteosat SEVIRI Terra/Aqua MODIS Suomi-NPP VIIRS JPSS-1 (NOAA-20) VIIRS Sentinel-2A MSI Sentinel-2B MSI Landsat-8 OLI HY-1C COCTS Metop-SG-A1 MetImage - multiple detectors Sentinel-3B OLCI - mean rsr FY-3D MERSI-2 FY-4A AGRI FY-3B VIRR FY-3C VIRR

Cloud top height retrieval from imager instruments is important for nowcasting and for satellite climate data records. A neural network approach for cloud top height retrieval from the imager instrument MODIS (Moderate Resolution Imaging Spectroradiometer) is presented. The neural networks are trained using cloud top layer pressure data from the CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) dataset. Results are compared with two operational reference algorithms for cloud top height: the MODIS Collection 6 Level 2 height product and the cloud top temperature and height algorithm in the 2014 version of the NWC SAF (EUMETSAT (European Organization for the Exploitation of Meteorological Satellites) Satellite Application Facility on Support to Nowcasting and Very Short Range Forecasting) PPS (Polar Platform System). All three techniques are evaluated using both CALIOP and CPR (Cloud Profiling Radar for CloudSat (CLOUD SATellite)) height. Instruments like AVHRR (Advanced Very High Resolution Radiometer) and VIIRS (Visible Infrared Imaging Radiometer Suite) contain fewer channels useful for cloud top height retrievals than MODIS, therefore several different neural networks are investigated to test how infrared channel selection influences retrieval performance. Also a network with only channels available for the AVHRR1 instrument is trained and evaluated. To examine the contribution of different variables, networks with fewer variables are trained. It is shown that variables containing imager information for neighboring pixels are very important. The error distributions of the involved cloud top height algorithms are found to be non-Gaussian. Different descriptive statistic measures are presented and it is exemplified that bias and SD (standard deviation) can be misleading for non-Gaussian distributions. The median and mode are found to better describe the tendency of the error distributions and IQR (interquartile range) and MAE (mean absolute error) are found to give the most useful information of the spread of the errors. For all descriptive statistics presented MAE, IQR, RMSE (root mean square error), SD, mode, median, bias and percentage of absolute errors above 0.25, 0.5, 1 and 2 km the neural network perform better than the reference algorithms both validated with CALIOP and CPR (CloudSat). The neural networks using the brightness temperatures at 11 and 12 µm show at least 32 % (or 623 m) lower MAE compared to the two operational reference algorithms when validating with CALIOP height. Validation with CPR (CloudSat) height gives at least 25 % (or 430 m) reduction of MAE.

Atmospheric interaction distorts the surface signal received by a space-borne instrument. Images derived from visible channels appear often too bright and with reduced contrast. This hampers the use of RGB imagery otherwise useful in ocean color applications and in forecasting or operational disaster monitoring, for example forest fires. In order to correct for the dominant source of atmospheric noise, a simple, fast and flexible algorithm has been developed. The algorithm is implemented in Python and freely available in PySpectral which is part of the PyTroll family of open source packages, allowing easy access to powerful real-time image-processing tools. Pre-calculated look-up tables of top of atmosphere reflectance are derived by off-line calculations with RTM DISORT as part of the LibRadtran package. The approach is independent of platform and sensor bands, and allows it to be applied to any band in the visible spectral range. Due to the use of standard atmospheric profiles and standard aerosol loads, it is possible just to reduce the background disturbance. Thus signals from excess aerosols become more discernible. Examples of uncorrected and corrected satellite images demonstrate that this flexible real-time algorithm is a useful tool for atmospheric correction.

Look Up Tables for removing atmospherical signal due to Rayleigh scattering and (marine tropical) aerosols in visible satellite imagery LibRadTran simulations for various standard atmospheres for the correction of Rayleigh scattering and marine tropical aerosol composition (Hess et al., 1998) within satellite images of channels in the visible spectral range.

Cloud property retrievals from 3 decades of the Advanced Very High Resolution Radiometer (AVHRR) measurements provide a unique opportunity for a long-term analysis of clouds. In this study, the accuracy of AVHRR-derived cloud properties cloud mask, cloud-top height, cloud phase and cloud liquid water path is assessed using three state-of-the-art retrieval schemes. In addition, the same retrieval schemes are applied to the AVHRR heritage channels of the Moderate Resolution Imaging Spectroradiometer (MODIS) to create AVHRR-like retrievals with higher spatial resolution and based on presumably more accurate spectral calibration. The cloud property retrievals were collocated and inter-compared with observations from CloudSat, CALIPSO and AMSR-E The resulting comparison exhibited good agreement in general. The schemes provide correct cloud detection in 82 to 90% of all cloudy cases. With correct identification of clear-sky in 61 to 85% of all clear areas, the schemes are slightly biased towards cloudy conditions. The evaluation of the cloud phase classification shows correct identification of liquid clouds in 61 to 97% and a correct identification of ice clouds in 68 to 95%, demonstrating a large variability among the schemes. Cloud-top height (CTH) retrievals were of relatively similar quality with standard deviations ranging from 2.1km to 2.7km. Significant negative biases in these retrievals are found in particular for cirrus clouds. The biases decrease if optical depth thresholds are applied to determine the reference CTH measure. Cloud liquid water path (LWP) is also retrieved well with relative low standard deviations (20 to 28g/m2), negative bias and high correlations. Cloud ice water path (IWP) retrievals of AVHRR and MODIS exhibit a relative high uncertainty with standard deviations between 800 and 1400g/m2, which in relative terms exceed 100% when normalized with the mean IWP. However, the global histogram distributions of IWP were similar to the reference dataset.MODIS retrievals are for most comparisons of slightly better quality than AVHRR-based retrievals. Additionally, the choice of different near-infrared channels, 3.7μm as opposed to 1.6μm, can have a significant impact on the retrieval quality, most pronounced for IWP, with better accuracy for the 1.6μm channel setup. This study presents a novel assessment of the quality of cloud properties derived from AVHRR channels, which quantifies the accuracy of the considered retrievals based on common approaches and validation data. Furthermore, it assesses the capabilities of AVHRR-like spectral information for retrieving cloud properties in the light of generating climate data records of cloud properties from three decades of AVHRR measurements.

Cloud masking remains as one of the most fundamental steps for any attempt to retrieve geophysical parameters for Earth surfaces or for cloud-free portions of the atmosphere from satellite imagery. It also forms the basis for specific studies of clouds and their properties in climatological applications. Because of its fundamental importance it also means that potential errors in cloud masking may lead to serious errors in retrieved geophysical parameters. It is therefore important to try to minimise these errors and to make attempts to understand and describe the error propagation further downstream in the processing chain. Many of today’s operational cloud masking methods are non-parametric, i.e., make the decision of if a pixel is cloudy or cloud-free based on a multispectral sorting (or thresholding) of data rather than by a numerical calculation process directly estimating the likelihood of cloudiness. This means that there is no direct measure of how certain the cloud mask estimation is. On the other hand, methods that offer such uncertainty estimation (e.g., Bayesian Maximum Likelihood schemes and Optimal Estimation schemes) require extensive training and very accurate knowledge of cloudy properties in radiance space for being able to describe realistic error estimates. Furthermore, these methods are computationally very expensive, especially if handling information from many spectral channels. This presentation gives some ideas on how to extend results from thresholding schemes with parametric uncertainty estimations. The main concept is based on the fact that many current thresholding schemes have undergone extensive tuning and validation which means that the identification of cloudy regimes in radiance space is relatively well understood. Consequently, a method based on estimating the distance from chosen thresholds values in radiance space should have a potential of describing the uncertainty of the cloud mask decision. Previous attempts here have been restricted to defining some specific quality flags but here a more continuous quality parameter is considered. Examples and applications of this method are demonstrated using cloud mask results from the NWCSAF PPS cloud masking scheme applied to global AVHRR high-resolution (LAC) and coarse resolution (GAC) data.

Using three cloud generators, three-dimensional (3D) cloud fields are reproduced from microphysical cloud data measured in situ by aircraft. The generated cloud fields are used as input to a 3D radiative transfer model to calculate the corresponding fields of downward and upward irradiance, which are then compared with airborne and ground-based radiation measurements. One overcast stratocumulus scene and one broken cumulus scene were selected from the European INSPECTRO field experiment, which was held in Norwich, UK, in September 2002. With these data, the characteristics of the three different cloud reproduction techniques are assessed. Besides vertical profiles and histograms of measured and modelled liquid water content and irradiance, the horizontal structure of these quantities is examined in terms of power spectra and autocorrelation lengths. 3D radiative transfer calculations are compared with the independent pixel approximation, and their differences with respect to domain-averaged quantities and 3D fields are interpreted. Copyright (C) 2007 Royal Meteorological Society.