A novel approach to optimize the objective function based on VC dimension and structural risk minimization

So far, lots of algorithms and learning methods are taking the empirical risk for the optimization goal. In this paper, we propose a new way to optimize the objective function based on VC dimension and structural risk minimization. The optimized function F is firstly defined by us, and some effective design forms of it will also be given. Then we fulfill a useful criterion—the balance minimum optimization principle. This principle considers not only the empirical risk, but also considers the VC dimension of learning machine. Balancing the two factors can avoid the underfitting problem and overfitting problem. Experimental results show that the method we proposed is effective to improve the property of algorithm efficiency, convergence and generalization of the learning machine. Also, the proposed principle for optimization is a new criterion, which is not a practical method for a particular problem of improvement. Therefore, this method is suitable to be applied in many practical situations, which may bring a good generalization performance and efficiency in some learning problems.