Adaptive Kernel Methods Using the Balancing Principle

De Vito E., Pereverzyev S., Rosasco L.,

Foundations of Computational Mathematics , Volume 10, Number 4, 455-479, [doi: 10.1007/s10208-010-9064-2] , 2010

Foundations of Computational Mathematics
Volume 10, Number 4, 455-479, [doi: 10.1007/s10208-010-9064-2]
August 2010

Abstract The regularization parameter choice is a fundamental problem in Learning Theory since the performance of most supervised algorithms crucially depends on the choice of one or more of such parameters. In particular a main theoretical issue regards the amount of prior knowledge needed to choose the regularization parameter in order to obtain good learning rates. In this paper we present a parameter choice strategy, called the balancing principle, to choose the regularization parameter without knowledge of the regularity of the target function. Such a choice adaptively achieves the best error rate. Our main result applies to regularization algorithms in reproducing kernel Hilbert space with the square loss, though we also study how a similar principle can be used in other situations. As a straightforward corollary we can immediately derive adaptive parameter choices for various kernel methods recently studied. Numerical experiments with the proposed parameter choice rules are also presented.

Download