Stochastic multi-objective optimization (SMOO) has attracted increasing interest as a framework for machine learning problems with multiple objectives. The nonlinearity of the subproblem solution mapping introduces bias in the weighting parameters, which complicates the convergence analysis of multi-gradient algorithms. In this paper, we propose the Multi-gradient Stochastic Mirror Descent (MSMD) algorithm, which solves the SMOO subproblem via the stochastic mirror descent method and provides convergence guarantees. By selecting an appropriate Bregman distance-generating function, the algorithm admits a closed-form solution for the weighting vector and requires only a single stochastic gradient sample per iteration. We establish sublinear convergence rates for MSMD under four combinations of inner and outer step size strategies. A variant of MSMD for SMOO with preferences is also proposed and analyzed. Numerical experiments on benchmark test functions and neural network training tasks show that MSMD achieves competitive performance.
原文链接
分享
引用
