A Feature Selection Method Based on Shapley Values Robust for Concept Shift in Regression

Feature selection is one of the most relevant processes in any methodologyfor creating a statistical learning model. Usually, existing algorithmsestablish some criterion to select the most influential variables, discardingthose that do not contribute to the model with any relevant information. Thismethodology makes sense in a static situation where the joint distribution ofthe data does not vary over time. However, when dealing with real data, it iscommon to encounter the problem of the dataset shift and, specifically, changesin the relationships between variables (concept shift). In this case, theinfluence of a variable cannot be the only indicator of its quality as aregressor of the model, since the relationship learned in the training phasemay not correspond to the current situation. In tackling this problem, ourapproach establishes a direct relationship between the Shapley values andprediction errors, operating at a more local level to effectively detect theindividual biases introduced by each variable. The proposed methodology isevaluated through various examples, including synthetic scenarios mimickingsudden and incremental shift situations, as well as two real-world casescharacterized by concept shifts. Additionally, we perform three analyses ofstandard situations to assess the algorithm's robustness in the absence ofshifts. The results demonstrate that our proposed algorithm significantlyoutperforms state-of-the-art feature selection methods in concept shiftscenarios, while matching the performance of existing methodologies in staticsituations.