Spectral Algorithms for Supervised Learning

Lo Gerfo L., Rosasco L., Odone F., De Vito E., Verri A.,

Neural Computation (NECO) , 20: 1873-1897 , Massachusetts Institute of Technology, Cambridge, MA , 2008

Abstract: We discuss how a large class of regularization methods, collectively known as spectral regularization and originally designed for solving ill- posed inverse problems, gives rise to regularized learning algorithms. All these algorithms are consistent kernel methods which can be easily implemented. The intuition behind their derivation is that the same principle allowing to numerically stabilize a matrix inversion problemis crucial to avoid over- tting. The various methods have a common derivation, but di erent computational and theoretical properties. We describe examples of such algorithms, analyzing their classi cation performance on several datasets and discussing their applicability to real world problems.

Download