Ross A Lippert

US Patent:

7685080, Mar 23, 2010

Filed:

Sep 27, 2006

Appl. No.:

11/535921

Inventors:

Ryan Rifkin - Cambridge MA, US
Ross Lippert - Somerville MA, US

Assignee:

Honda Motor Co., Ltd. - Tokyo

International Classification:

G06E 1/00

US Classification:

706 20, 706 21

Abstract:

Techniques are disclosed that implement algorithms for rapidly finding the leave-one-out (LOO) error for regularized least squares (RLS) problems over a large number of values of the regularization parameter λ. Algorithms implementing the techniques use approximately the same time and space as training a single regularized least squares classifier/regression algorithm. The techniques include a classification/regression process suitable for moderate sized datasets, based on an eigendecomposition of the unregularized kernel matrix. This process is applied to a number of benchmark datasets, to show empirically that accurate classification/regression can be performed using a Gaussian kernel with surprisingly large values of the bandwidth parameter σ. It is further demonstrated how to exploit this large σ regime to obtain a linear-time algorithm, suitable for large datasets, that computes LOO values and sweeps over λ.

US Patent:

20080162386, Jul 3, 2008

Filed:

Nov 16, 2007

Appl. No.:

11/941729

Inventors:

Ryan M. Rifkin - Cambridge MA, US
Ross A. Lippert - Somerville MA, US

Assignee:

Honda Motor Co., Ltd. - Tokyo

International Classification:

G06F 15/18

US Classification:

706 12

Abstract:

A simple yet powerful Bayesian model of linear regression is disclosed for methods and systems of machine learning. Unlike previous treatments that have either considered finding hyperparameters through maximum likelihood or have used a simple prior that makes the computation tractable but can lead to overfitting in high dimensions, the disclosed methods use a combination of linear algebra and numerical integration to work a full posterior over hyperparameters in a model with a prior that naturally avoids overfitting. The resulting algorithm is efficient enough to be practically useful. The approach can be viewed as a fully Bayesian version of the discriminative regularized least squares algorithm.