Non-Data-Splitting Estimator Selection for Regression in Exponential Families

We observe n independent pairs of random variables (Wi,Yi), where the conditional distribution of Yi given Wi=wi follows a one-parameter exponential family with parameter γ∗(wi)∈R. The goal is to estimate the regression function γ∗. We start with an arbitrary collection of piecewise constant candidate estimators based on the observations and, using the same data, select an estimator from this collection. The approach is agnostic to the dependencies of the candidate estimators on the data, differing from methods like data splitting, cross-validation, and hold-out. To demonstrate the theoretical performance of this approach, a non-asymptotic risk bound is established for the selected estimator. We then discuss the application of the procedure to changepoint detection in exponential families. The practical performance of the proposed approach is illustrated using comparative simulations across different scenarios and real datasets.