Boosting and Margin Theory

Many researchers have worked on the explanation of AdaBoost’s good experimental results in theory. Some work give an upper bound of generalization error in terms of the margin distribution function, while Breiman gave a sharper generalization error bound based on minimum margin. He also developed the arcgv algorithm to maximize the minimum margin, then made the minimum margin larger than AdaBoost. However, its empirical results are even worse than AdaBoost. Therefore, is the minimum margin bound not practical? This paper gives a new concept called Equilibrium margin (Emargin) and proves a new generalization error bound using Emargin, which is always better than minimum margin bound. In addition, we show Emargin is a good indicator of generalization. Then, we conduct experiments showing that the Emargin of AdaBoost is larger than arc-gv, but the generalization error of Ada-Boost is usually better.