Hybrid approaches combining machine learning with traditional inverse problem solution methods represent a promising direction for the further development of inverse modeling algorithms. The paper proposes an approach to emission source identification from measurement data for advection–diffusion–reaction models. The approach combines general-type source identification and post-processing refinement: first, emission source identification by measurement data is carried out by a sensitivity operator-based algorithm, and then refinement is done by incorporating a priori information about unknown sources. A general-type distributed emission source identified at the first stage is transformed into a localized source consisting of multiple point-wise sources. The second, refinement stage consists of two steps: point-wise source localization and emission rate estimation. Emission source localization is carried out using deep learning with convolutional neural networks. Training samples are generated using a sensitivity operator obtained at the source identification stage. The algorithm was tested in regional remote sensing emission source identification scenarios for the Lake Baikal region and was able to refine the emission source reconstruction results. Hence, the aggregates used in traditional inverse problem solution algorithms can be successfully applied within machine learning frameworks to produce hybrid algorithms.
Large-scale inverse problems that require high-performance computing arise in various fields, including regional air quality studies. The paper focuses on parallel solutions of an emission source identification problem for a 2D advection–diffusion–reaction model where the sources are identified by heterogeneous measurement data. In the inverse modeling approach we use, a source identification problem is transformed to a quasi-linear operator equation with a sensitivity operator, which allows working in a unified way with heterogeneous measurement data and provides natural parallelization of numeric algorithms by concurrent calculation of the rows of a sensitivity operator matrix. The parallel version of the algorithm implemented with a message passing interface (MPI) has shown a 40× speedup on four Intel Xeon Gold 6248R nodes in an inverse modeling scenario for the Lake Baikal region.
The organization of computations in cloud Web services for satellite data processing is considered. Computing component of almost every service is a batch version of the corresponding technology of the PlanetaMonitoring software for processing remote sensing data. The exceptions are the technologies that require interactive communication with user, i.e., supervised classification of remote sensing data and movement tracking of natural environments by the coordinates of identifiable objects, each of which consists of two parts, i.e., an interactive Windows application running on the user's computer and the part hidden in the cloud.
Framework-based approach to the implementation of high-performance image processing library is suggested. The implementation of prototype libraries for doing processing on computational cluster and GPU is described.
We consider a distributed network of cloud web services for processing satellite data, which provides data processing facilities for Earth remote sensing within SaaS model. In fact, this is a set of web services that implement the functional modules of the PlanetaMonitoring remote sensing data processing system.
The first Russian advanced infrared (IR) atmospheric sounder IKFS-2 was launched in July 2014 on the "Meteor-M" No. 2 meteorological satellite. It is planned that this instrument and similar devices will continue to operate until 2025 aboard the current and subsequent satellites of "Meteor-M" series. IKFS-2 is a Fourier transform spectrometer covering the spectral domain of 5-15 mu m. It belongs to a class of hyper-spectral IR sounders, designed to measure the outgoing IR radiance spectra and to provide information on the thermodynamic parameters and the composition of the atmosphere such as vertical temperature and humidity profiles, estimates of the ozone and other trace gases total column amounts. In the paper, the IKFS-2 operation on board "Meteor-M" No. 2 is analyzed, including the assessment of the measurements' quality (the errors of radiometric and spectral calibration) and their information content. Since launch, the instrument performance has been remained stable, and the actual IKFS-2 characteristics (threshold value of NESR, uncertainty of onboard radiometric and spectral calibrations, spectral resolution) meet the planned requirements. There is a good agreement between IKFS-2 measurements and measurements of independent satellite instruments (SEVIRI, IASI). The paper also provides an overview of the developed scientific basis for simulation and interpretation ("inversion") of satellite measurements. The examples of output IKFS-2 level 2 products (vertical profiles of temperature and humidity, total ozone content) are given together with the error analysis. The performance of atmospheric (temperature and humidity) profile retrievals and the feasibility of total ozone content estimates are evaluated by comparison with independent ground-based or satellite measurements. (C) 2019 The Authors. Published by Elsevier Ltd.
Preliminary results of a space experiment using the IKFS-2 infrared sounder (Meteor-M2 satellite) showed high-quality of measurements of spectra of the outgoing thermal radiation of the atmosphere–surface system and the adequacy of developed IR radiation atmospheric models in the 15-μm carbon gas absorption band used to recover the vertical profiles of the atmospheric temperature. Outgoing radiation spectra measured by IKFS-2 instruments make it possible to restore vertical temperature profiles with errors close to 1K in most of the 0–30 km high-altitude region, except for the lower troposphere and altitudes above 30 km, where these errors are close to 2–3K.
The methodological and computational aspects of Fast Radiative Transfer Model (FRTM) development designed for the analysis and validation of the data of measurements using satellite-based instrument-hyperspectral IR sounders of high spectral resolution—are considered. A description of the FRTM is given for the analysis and modeling of the measurements by the IRFS-2 IR Fourier spectrometer for polarorbiting meteorological satellites of the Meteor-M series based on the known RTTOV FRTM. Computational efficiency is estimated and the results of the verification of developed FRTM are presented. They were obtained from a comparison of model simulations with exact line-by-line calculations for the IRFS-2 IR sounder. The increase in computational performance and the accuracy of the FRTM, caused by the application of the algorithms of the principal components method, are discussed. The construction of radiative models, which use the algorithm of the Monte Carlo method and are applicable for the analysis and modeling of the data of IR sounders under conditions of cloudiness in the instrument field of view, is considered.
Рассмотрены методические и вычислительные аспекты создания быстрых радиационных моделей (БРМ), предназначенных для анализа и валидации данных измерений спутниковой гиперспектральной аппаратуры ИК-зондировщиков высокого спектрального разрешения. Приведено описание БРМ для анализа и моделирования измерений ИК-Фурье-спектрометра ИКФС-2 полярно-орбитальных метеоспутников серии “Метеор-М”, созданной на основе известной БРМ RTTOV. Оценивается вычислительная производительность и приводятся результаты верификации созданной БРМ, полученные путем сравнения модельных расчетов с точными полинейными расчетами для ИК-зондировщика ИКФС-2. Обсуждается повышение вычислительной производительности и точности БРМ за счет использования алгоритмов метода главных компонент. Рассмотрено построение радиационных моделей, использующих алгоритм метода Монте-Карло и пригодных для анализа и моделирования данных ИК-зондировщиков при наличии облачности в поле зрения прибора.
An experimental framework SSCCIP for high-performance image processing on multiprocessor computer is described. The basic principles of the system construction, various architectural solutions, and an example of implementing a specific technology of the image processing are discussed.
The software technologies for processing of Earth remote sensing (ERS) data is discussed in the paper. Since the ERS data analysis and processing require mobilizing all available hardand software tools, the new software technologies are developed to solve these tasks. The preliminary results of these technologies implementation are presented.
The paper describes the experimental library SSCC_PIPL for image processing on multicomputers. Basic principles of library building, some architectural solutions, and test results are given.
A multi-component technology integrating multiprocessor computer into remote sensing data processing is discussed.
An experimental library PLVIP for parallel image processing, based on the principle of vertical processing, is described. The library was elaborated and implemented in the Image Processing Laboratory of the Institute of Computational Mathematics and Mathematical Geophysics SB RAS and installed on the 32-processor Linux cluster MVS-1000/M of the Siberian Supercomputer Center. The basic characteristics of the library (the data formats supported, organization of computational process, and subprograms implemented) and examples of its application are considered.
PLVIP, experimental library for parallel image processing, designed and implemented in the Image Processing Laboratory of the Institute of Computational Mathematics and Mathematical Geophysics SB RAS, is described. The library is built on the principle of vertical processing and is installed on two multiprocessor computers of the Siberian Supercomputer Center, the 32-processor Linux cluster MVS–1000/M and the 8-processor SMP server RM600–E30. Basic characteristics of the library (supported data formats, organization of computational process, and implemented subprograms) and an example of its application are considered.
The paper describes the experimental framework for distributed image processing with the use of multicomputer providing fast development of high-performance remote sensing data processing technologies. Basic principles of system building, some architectural solutions, and sample implementation of concrete processing technologies are given.