Abstract：Margin distribution is critical to Boosting. However, the existing margin-based generalization error bounds are too complicated to be used for the design of new Boosting algorithms. In this paper, a moment-optimized Boosting (MOBoost) algorithm is proposed with direct optimization of the margin distribution. Firstly, a generalization error bound for Boosting based on first and secondary moments of the margin distribution is derived to reveal the close relationship between margin distribution and generalization error. Then, a moment criterion for Boosting model selection is presented based on the moment generalization bound. The criterion maximizes the first moment and minimizes the second moment of the margin distribution simultaneously. Consequently, the primary and dual forms are formulated for solving the convex quadratic program of the moment criterion for Boosting. Thus, an efficient computing method for the moment criterion is proposed. Theoretical analysis and experimental results show that MOBoost is effective and reliable.
2015, Vol. 28 
