Revisiting Differentially Private Hyper-parameter Tuning
CoRR(2024)
摘要
We study the application of differential privacy in hyper-parameter tuning, a
crucial process in machine learning involving selecting the best
hyper-parameter from several candidates. Unlike many private learning
algorithms, including the prevalent DP-SGD, the privacy implications of tuning
remain insufficiently understood or often totally ignored. Recent works propose
a generic private selection solution for the tuning process, yet a fundamental
question persists: is this privacy bound tight?
This paper provides an in-depth examination of this question. Initially, we
provide studies affirming the current privacy analysis for private selection is
indeed tight in general. However, when we specifically study the
hyper-parameter tuning problem in a white-box setting, such tightness no longer
holds. This is first demonstrated by applying privacy audit on the tuning
process. Our findings underscore a substantial gap between current theoretical
privacy bound and the empirical bound derived even under strong audit setups.
This gap motivates our subsequent investigations. Our further study provides
improved privacy results for private hyper-parameter tuning due to its distinct
properties. Our results demonstrate broader applicability compared to prior
analyses, which are limited to specific parameter configurations.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要