# Optimal score estimation via empirical Bayes smoothing

Annual Conference Computational Learning Theory（2024）

Abstract

We study the problem of estimating the score function of an unknown
probability distribution ρ^* from n independent and identically
distributed observations in d dimensions. Assuming that ρ^* is
subgaussian and has a Lipschitz-continuous score function s^*, we establish
the optimal rate of Θ̃(n^-2/d+4) for this estimation
problem under the loss function ŝ - s^*^2_L^2(ρ^*) that is
commonly used in the score matching literature, highlighting the curse of
dimensionality where sample complexity for accurate score estimation grows
exponentially with the dimension d. Leveraging key insights in empirical
Bayes theory as well as a new convergence rate of smoothed empirical
distribution in Hellinger distance, we show that a regularized score estimator
based on a Gaussian kernel attains this rate, shown optimal by a matching
minimax lower bound. We also discuss the implication of our theory on the
sample complexity of score-based generative models.

MoreTranslated text

AI Read Science

Must-Reading Tree

Example

Generate MRT to find the research sequence of this paper

Chat Paper

Summary is being generated by the instructions you defined