A Machine Learning Approach For Data Quality Control Of Earth Observation Data Management System

In the big data era, innovative technologies like cloud computing, artificial intelligence, and machine learning are increasingly utilized in the large-scale data management systems of many industry sectors to make them more scalable and intelligent. Applying them to automate and optimize earth observation data management is a hot topic. To improve data quality control mechanisms, a machine learning method in combination with built-in quality rules is presented in this paper to evolve processes around data quality and enhance management of earth observation data. The rules of quality check are set up to detect the common issues, including data completeness, data latency, bad data, and data duplication, and the machine learning model is trained, tested, and deployed to address these quality issues automatically and reduce manual efforts.