With the development of science and technology, the status of the water environment has received more and more attention. In this paper, we propose a deep learning model, named a Joint Auto-Encoder network, to solve the problem of outlier detection in water supply data. The Joint Auto-Encoder network first expands the size of training data and extracts the useful features from the input data, and then reconstructs the input data effectively into an output. The outliers are detected based on the network's reconstruction errors, with a larger reconstruction error indicating a higher rate to be an outlier. For water supply data, there are mainly two types of outliers: outliers with large values and those with values closed to zero. We set two separate thresholds, tau(1) and tau(2) , for the reconstruction errors to detect the two types of outliers respectively. The data samples with reconstruction errors exceeding the thresholds are voted to be outliers. The two thresholds can be calculated by the classification confusion matrix and the receiver operating characteristic (ROC) curve. We have also performed comparisons between the Joint Auto-Encoder and the vanilla Auto-Encoder in this paper on both the synthesis data set and the MNIST data set. As a result, our model has proved to outperform the vanilla Auto-Encoder and some other outlier detection approaches with the recall rate of 98.94 percent in water supply data.
For the problems of abnormal values existing in the water intake monitoring data and centralized uploaded report, the abnormal data region discrimination (ADRD) algorithm and the cross-monitoring points historical correlation repair (CMHCR) method are proposed to discriminate and repair the abnormal data. The characteristics of abnormal data distribution are analyzed, and the ADRD algorithm is proposed. ADRD uses the relationship between 0 values and the abnormal large value, and the ratio of the abnormal large value to the expectation to distinguish the abnormal data region. The correlation between the monitoring data of current detection points and the historical data of different detection points is analyzed. The results show that the data of current monitoring point and the historical data of corresponding point do not fully conform to the maximum correlation. Therefore, the CMHCR method is proposed to repair abnormal data. Experiments based on actual half year water intake data of 2016 and 2017 are performed by using ADRD. The experimental results show that the proposed algorithm and method can correctly distinguish the abnormal data region and repair the abnormal data properly.