Nowcasting with Transformer-based Models using Multi-Source Data 

Rapid advancements in data-driven weather prediction have shown notable success, particularly in nowcasting, where forecast lead times span just a few hours. Transformer-based models, in particular, have proven effective in learning spatiotemporal connections of varying scales by leveraging the attention mechanism with efficient space-time patching of data. This offers potential improvements over traditional nowcasting techniques, enabling early detection of convective activity and reducing computational costs.  In this presentation, we demonstrate the effectiveness of a modified Earthformer model, a space-time Transformer framework, in addressing two specific nowcasting challenges. First, we introduce a nowcasting model that predicts ground-based 2D radar mosaics up to 2-hour lead time with 5-minute temporal resolution, using geostationary satellite data from the preceding two hours. Trained on a benchmark dataset sampled across the United States, our model exhibits robust performance against various impactful weather events with distinctive features. Through permutation tests, we interpret the model to understand the effects of input channels and input data length. We found that the infrared channel centered at 10.3 µm contains skillful information for all weather conditions, while, interestingly, satellite-based lightning data is the most skilled at predicting severe weather events in short lead times. Both findings align with existing literature, enhancing confidence in our model and guiding better usage of satellite data for nowcasting. Moreover, we found the model is sensitive to input data length in predicting severe weather events, suggesting early detection of convective activity by the model in rapidly growing fields.  Second, we present the initial attempts to develop a multi-source precipitation nowcasting model for Austria, tailored to predict impactful events with convective activities. This model integrates satellite- and ground-based observations with analysis and numerical weather prediction data to predict precipitation up to 2-hour lead time with 5-minute temporal resolution.   We conclude by discussing the broad spectrum of applications for such models, ranging from enhancing operational nowcasting systems to providing synthetic data to data-scarce regions, and the challenges therein.