Growing and Pruning Selective Ensemble Regression for Nonlinear and Nonstationary Systems

For a selective ensemble regression (SER) scheme to be effective in online modeling of fast-arriving nonlinear and nonstationary data, it must not only be capable of maintaining a most up to date and diverse base model set but also be able to forget old knowledge no longer relevant. Based on these two important principles, in this paper, we propose a novel growing and pruning SER (GAP-SER) for time-varying nonlinear data. Specifically, during online operation, newly emerging process state is automatically identified and a local linear model is fitted to it. This adaptive growing strategy therefore maintains a most up to date and diverse local model set. The online prediction model is then constructed as a selective ensemble from the local linear model set based on a probability metric. Moreover, a pruning strategy is derived to remove 'unwanted' out of date local linear models in order to achieve low online computational complexity without sacrificing online modeling accuracy. A chaotic time series prediction and two real-world data sets are used to demonstrate the superior online modeling performance of the proposed GAP-SER over a range of benchmark schemes for nonlinear and nonstationary systems, in terms of online prediction accuracy and computational complexity.