Seismic data often face challenges in their utilization due to noise contamination, incomplete acquisition, and limited low-frequency information, which hinder accurate subsurface imaging and interpretation. Traditional processing methods rely heavily on task-specific designs to address these challenges and fail to account for the variability of data. To address these limitations, we present a generative seismic foundation model (GSFM), a unified framework based on generative diffusion models (GDMs), designed to tackle multi-task seismic processing challenges, including denoising, backscattered noise attenuation, interpolation, and low-frequency extrapolation. GSFM leverages a pre-training stage on synthetic data to capture the features of clean, complete, and broadband seismic data distributions and applies an iterative fine-tuning strategy to adapt the model to field data. By adopting a target-oriented diffusion process prediction, GSFM improves computational efficiency without compromising accuracy. Synthetic data tests demonstrate GSFM surpasses benchmarks with equivalent architectures in all tasks and achieves performance comparable to traditional pre-training strategies, even after their fine-tuning. Also, field data tests suggest that our iterative fine-tuning approach addresses the generalization limitations of conventional pre-training and fine-tuning paradigms, delivering significantly enhanced performance across diverse tasks. Furthermore, GSFM's inherent probabilistic nature enables effective uncertainty quantification, offering valuable insights into the reliability of processing results.

Recent advancements in deep learning have led to the emergence of various seismic processing methodologies, leveraging the adaptability of modern neural network models to tackle complex seismic tasks. Marine seismic data, with its unique characteristics and challenges, has been a focal point in recent research efforts. However, land seismic data, which recently drew attention among researchers, poses different, often more complex, challenges. One of the intricate challenges in land seismic data is the presence of ground roll noise, which demands specialized processing approaches. In this study, we introduce a Transformer-based network to efficiently suppress ground roll noise in land seismic data. We demonstrate the applicability of the proposed model to land data from Saudi Arabia, emphasizing its efficacy in suppressing ground roll noise. Moreover, we adopt a focal modulation mechanism to enhance the denoising performance, supported by comparative analyses against benchmark denoising techniques.

Building subsurface velocity models is essential to our goals in utilizing seismic data for Earth discovery and exploration, as well as monitoring. With the dawn of machine learning, these velocity models or more precisely their distribution can be stored in a generative model. These stored velocity model distributions can be utilized to regularize inverse problems, like full waveform inversion. However, most generators, like normalizing flows or diffusion models, treat the image (velocity model) uniformly, disregarding spatial dependencies and varying resolution. To address this weakness, we introduce VelocityGPT, a novel implementation that utilizes Transformer decoders trained autoregressive to generate a velocity model from shallow to deep. Owing to the fact that seismic data are often recorded on the Earth’s surface, a top-down generator can utilize the inverted information in the shallow as guidance (prior) to the deep. To facilitate the implementation, we use an additional network to compress the velocity model. We also inject prior information, like well or dip (extracted from an image) to generate the velocity model. Using synthetic data, we demonstrate the effectiveness of VelocityGPT as a promising approach in generative model applications for seismic velocity model building. We will share more realistic velocity models including field data in the presentation of this work.

The performance of full‐wave inversion (FWI) depends highly on how we compare the simulated data to observed ones. The simplified assumptions used to generate the simulated data make such comparison even harder. To address this challenge, we introduce SiameseFWI, a novel approach to FWI that plays a critical role in the comparative analysis of simulated and observed seismic data. Employing a Siamese network, this methodology transforms the data into a shared latent space, enabling a robust and effective comparison of data representations. SiameseFWI leverages two identical Convolutional Neural Networks with shared weights trained in a self‐supervised framework, eliminating the necessity for labeled data. In each FWI iteration, the Siamese network and the velocity model are updated to minimize Euclidean distance loss between the latent representations of the data. Empirical evaluation conducted on the Marmousi2 and Overthrust models affirms the robust inversion performance of SiameseFWI compared to traditional FWI methodologies. Furthermore, its application to field data from Western Australia demonstrates its strength and efficacy in inversion. Notably, SiameseFWI exhibits robust inversion performance even in the presence of noise or when employing a linear initial model.

Full waveform inversion (FWI) is capable of generating high-resolution subsurface parameter models, but it is susceptible to cycle-skipping when the data lack low-frequency. Unfortunately, the low-frequency components (< 5.0 Hz) are often tainted by noise in real seismic exploration, which hinders the application of FWI. To address this issue, we develop a novel self-supervised low-frequency extrapolation method that does not require labeled data, enabling neural networks to be trained directly on real data. This paradigm effectively addresses the significant generalization gap often encountered by supervised learning techniques, which are typically trained on synthetic data. We validate the effectiveness of our method on both synthetic and field data. The results demonstrate that our method effectively extrapolates low-frequency components, aiding in circumventing the challenges of cycle-skipping in FWI. Meanwhile, by integrating a self-supervised denoiser, our method effectively performs simultaneously denoising and low-frequency extrapolation on noisy data. Furthermore, we showcase the potential application of our method in extending the ultra-low frequency components of the large-scale collected earthquake seismogram.

StorSeismic is a recently introduced model based on the Transformer to adapt to various seismic processing tasks through its pretraining and fine-tuning training strategy. In the original implementation, StorSeismic utilized a sinusoidal positional encoding and a conventional self-attention mechanism, both borrowed from the natural language processing (NLP) applications. For seismic processing they admitted good results, but also hinted to limitations in efficiency and expressiveness. We propose modifications to these two key components, by utilizing relative positional encoding and low-rank attention matrices as replacements to the vanilla ones. The proposed changes are tested on processing tasks applied to a realistic Marmousi and offshore field data as a sequential strategy, starting from denoising, direct arrival removal, multiple attenuation, and finally root-mean-squared velocity (V_RMS) prediction for normal moveout (NMO) correction. We observe faster pretraining and competitive results on the fine-tuning tasks and, additionally, fewer parameters to train compared to the vanilla model.

Machine learning-based seismic processing models are typically trained separately to perform specific seismic processing tasks (SPTs), and as a result, require plenty of training data. However, preparing training data sets is not trivial, especially for supervised learning (SL). Nevertheless, seismic data of different types and from different regions share generally common features, such as their sinusoidal nature and geometric texture. To learn the shared features, and thus, quickly adapt to various SPTs, we develop a unified paradigm for neural network-based seismic processing, called Meta-Processing, that uses limited training data for meta learning a common network initialization, which offers universal adaptability features. The proposed Meta-Processing framework consists of two stages: meta-training and meta-testing. In the meta-training stage, each SPT is treated as a separate task and the training dataset is divided into support and query sets. Unlike conventional SL methods, here, the neural network (NN) parameters are updated by a bilevel gradient descent from the support set to the query set, iterating through all tasks. In the meta-testing stage, we also utilize limited data to fine-tune the optimized NN parameters in an SL fashion to conduct various SPTs, such as denoising, interpolation, ground-roll attenuation, image enhancement, and velocity estimation, aiming to converge quickly to ideal performance. Comprehensive numerical examples are performed to evaluate the performance of Meta-Processing on both synthetic and field data. The results demonstrate that our method significantly improves the convergence speed and prediction accuracy of the NN.

Building subsurface velocity models is essential to our goals in utilizing seismic data for Earth discovery and exploration, as well as monitoring. With the dawn of machine learning, these velocity models (or, more precisely, their distribution) can be stored accurately and efficiently in a generative model. These stored velocity model distributions can be utilized to regularize or quantify uncertainties in inverse problems, like full waveform inversion. However, most generators, like normalizing flows or diffusion models, treat the image (velocity model) uniformly, disregarding spatial dependencies and resolution changes with respect to the observation locations. To address this weakness, we introduce VelocityGPT, a novel implementation that utilizes Transformer decoders trained autoregressively to generate a velocity model from shallow subsurface to deep. Owing to the fact that seismic data are often recorded on the Earth's surface, a top-down generator can utilize the inverted information in the shallow as guidance (prior) to generating the deep. To facilitate the implementation, we use an additional network to compress the velocity model. We also inject prior information, like well or structure (represented by a migration image) to generate the velocity model. Using synthetic data, we demonstrate the effectiveness of VelocityGPT as a promising approach in generative model applications for seismic velocity model building.

Full-waveform inversion (FWI) hinges on accurately comparing the observed and simulated seismic data, a task complicated by the sinusoidal nature of the wavefield and the simplified assumptions (like acoustic) used to simulate data. To overcome this challenge, we introduce SiameseFWI, a framework that incorporates a Siamese network to transform data into a shared latent representation, enhancing comparative analysis. The Siamese network employs two identical Convolutional Neural Networks (CNNs) with shared weights to ensure consistent feature extraction from observed and simulated data. The primary goal of SiameseFWI is to minimize Euclidean distance loss between the latent representations of observed and simulated data. The parameters of the Siamese network are optimized in an unsupervised manner during the FWI process. This integration leads to improved FWI performance without imposing a significant computational cost. Empirical assessments, including the Overthrust model, consistently demonstrate SiameseFWI’s superiority over traditional FWI methods. Furthermore, practical validation with real field data from Western Australia emphasizes the robust inversion performance of the SiameseFWI.

Every seismic dataset has its particular characteristics guided mainly by the properties of the subsurface, the data acquisition parameters (the survey), and the often unique noise conditions it experiences. Capturing such characteristics in a neural network model for the efficient application of processing tasks, like denoising, first arrival picking, velocity estimation, and so on, offer a more effective approach to incorporating machine learning in processing than training neural networks for specific tasks that may or may not transfer well to new data. We introduce a framework for seismic processing that allows us to pre-train a neural network to learn the features of a seismic dataset, and then fine-tune that network for any downstream processing task. We take advantage of the fact that most processing tasks utilize the same features embedded in the seismic dataset, and thus, these features can be stored in a common pre-trained network in a self-supervised manner, we refer to as StorSeismic. In this framework, we utilize a Bidirectional Encoder Representations from Transformers (BERT) model to promote pre-training for storing the features of a seismic dataset and then efficiently fine-tune it to adapt to a wide spectrum of seismic processing tasks. We apply this framework on field data, along with synthetically generated data, in the self-supervised pre-training step to store the seismic features. Then, we use the labeled synthetic data to fine-tune the pre-trained network in a supervised fashion to perform various seismic processing tasks, like denoising, low frequency extrapolation, first arrival picking, and velocity estimation, and finally obtain satisfactory inference results on the field data.

Summary Machine Learning (ML) has recently been helpful for many seismic processing and imaging tasks. However, these tasks are often handled separately with their own neural network model and training. We propose StorSeismic, a unified framework to store the features in seismic data and use them later for varying seismic processing tasks. Through the help of the self-attention mechanism embedded in the Bidirectional Encoder Representation from Transformers (BERT), a Transformer-based network architecture, we capture and store the local and global features of seismic data in the pre-training stage, then utilize them in various seismic processing tasks in the fine-tuning stage. Using this framework, we could achieve a more efficient and flexible training process than existing approaches. Two applications on denoising and velocity estimation demonstrate the flexibility and the potential of this proposed framework in adapting to various seismic processing tasks.

We often apply a number of processing modules to a field dataset to perform various tasks, like denoising, first arrival picking, velocity estimation, and so on. Selecting the architecture, collecting labels, and tuning a deep-learning network for a specific seismic processing task have been common practice in the field, yet these often independent networks do not take advantage of the fact that most processing tasks utilize the same features embedded in the seismic data that actually can be stored in a common pretrained network in a self-supervised manner; we refer to it as StorSeismic. In this framework, we utilize a Bidirectional Encoder Representations from Transformers (BERT) model to promote pretraining for storing the features of a seismic dataset, and then efficiently fine-tune it to adapt to a wide spectrum of seismic processing tasks. Here, we combine field data with synthetic generated data in the self-supervised pretraining step to store the seismic features. Then, we use the labeled synthetic data to fine-tune the pretrained network in a supervised fashion to perform various seismic processing tasks, like denoising, low-frequency extrapolation, first-arrival picking, and velocity estimation, with satisfactory results.

Machine learning has already made many inroads in developments related to acquisition, processing, imaging, inverting, and interpreting seismic data. In spite of the many success stories, its commercial use has been limited as the challenges mount. These challenges include cost of training, availability of training samples, the applicability of the trained model to real data (generalization), and more importantly, the availability of practitioners who actually know what the neural networks (NNs) are doing. Taking a step back, I will review what worked in deep learning and what we are still waiting on to work. We will look into the various ML algorithms, from supervised to unsupervised, transformers to contrastive learning, and identify the potential role of these various algorithms on seismic data, with examples. The examples include seismic data denoising, data extrapolation, first arrival picking, microseismic location, velocity inversion all on real data.

Machine learned tasks on seismic data are often trained sequentially and separately, even though they utilize the same features (i.e., geometrical) of the data. We present StorSeismic as a dataset-centric framework for seismic data processing, which consists of neural network (NN) pretraining and fine-tuning procedures. We, specifically, utilize a NN as a preprocessing tool to extract and store seismic data features of a particular dataset for any downstream tasks. After pretraining, the resulting model can be utilized later, through a fine-tuning procedure, to perform different tasks using limited additional training. Used often in natural language processing (NLP) and lately in vision tasks, bidirectional encoder representations from transformer (BERT), a form of a transformer model, provides an optimal platform for this framework. The attention mechanism of BERT, applied here on a sequence of traces within the shot gather, is able to capture and store key geometrical features of the seismic data. We pretrain StorSeismic on field data, along with synthetically generated ones, in the self-supervised step. Then, we use the labeled synthetic data to fine-tune the pretrained network in a supervised fashion to perform various seismic processing tasks, such as denoising, velocity estimation, first arrival picking, and normal moveout (NMO). Finally, the fine-tuned model is used to obtain satisfactory inference results on the field data.

Because of its robustness and practicality, the Horizontal-to-Vertical Spectral Ratio (HVSR) method has been widely used to obtain subsurface structure, mainly the sediment thickness that resides over bedrock. The method uses Fourier Transform to obtain frequency spectrum and calculate the H/V ratio. However, the conventional Fourier Transform method has some limitations; e.g., the inability to remove local noises that are very common in microtremor recordings. In this study, we investigate the application of the Hilbert–Huang Transform (HHT), in obtaining the HVSR curve, and compare it to the conventional method in terms of its effectiveness in removing local noise through the inversion results of HVSR curves. Such a comparison has never been conducted before. We used data from a microtremor survey in Mataram City, Lombok Island, Indonesia which experienced a series of destructive earthquakes in 2018. The results show that the S-wave velocity structure derived from the inversion process of the HHT-HVSR curves is in better agreement with the previous study of Mataram City than the conventional method. Furthermore, the resulting S-wave velocity structure is also interpreted based on geological reference, giving new insights into the subsurface structure beneath Mataram City.

In 2018, Lombok Island was hit by a series of destructive earthquakes. According to Indonesian Meteo- rological, Climatological, and Geophysical Agency data, about 1,973 felt earthquakes (M > 3) which shaken Lombok were recorded during August 2018 with three earthquakes with the largest magnitude of 6.9 Mw, 6.8 Mw, and 6.2 Mw. National Board for Disaster Management reported about 555 deaths, 1,833 people injured, and 186,010 houses damaged as a result of Lombok earthquake on August 5th, 2018. A number of seismometers were placed on Lombok Island from August 3rd, 2018 to October 19th, 2018 to monitor the aftershock events. There are 17 stations that record seismicity in Lombok consisting of 10 broadband sensors and 7 short period sensors. In this study, we used the Horizontal-to-Vertical Spectral Ratio (HVSR) method to analyze the risk of earthquake in the Lombok region. The basic concept of this method is to do a comparison between the horizontal component spectrum and the vertical component spectrum of a wave, where theoretically the particle movement of the horizontal component is greater than the particle movement of the vertical component on soft ground, whereas on the hard ground both components (horizontal and vertical) will be similar. H/V curve obtained from earthquake record- ings (Earthquake Horizontal-to-Vertical Ratio/EHVR) and H/V curve obtained from microtremor recordings (Microtremor Horizontal-to-Vertical Ratio/MHVR) shows good agreement and high correlation. Empirical correction of EMR (Earthquake-to-Microtremor Ratio) managed to decrease the difference of estimation of predominant frequency and amplification factor between EHVR and MHVR. Predominant frequency, amplification, and seismic vulnerability map agree with the geological condition of Lombok Island, where high value of amplification and seismic vulnerability was found on soft and thick ground. This study conclude that the maximum ground acceleration and the construction of the building should also considered when one wants to investigate the effect of an earthquake to the damage occurred, beside the site effects.