CSAIL Event Calendar: Previous Series

Two Easy Classification Tricks To Simplify the Practitioner

Speaker: Ali Rahimi , Intel Research, Seattle
Date: October 29 2007
Time: 4:00PM to 5:00PM
Location: 32-D463 Stata Center - Star Conference Room
Host: Trevor Darrell, MIT CSAIL

Contact: Marcia Davidson, 617-253-3049, marcia@csail.mit.edu
Relevant URL:

A powerful and relatively recent observation in computer vision is that generic machine techniques trained on very large data sets can outperform more complicated domain-specific methods. Nearest neighbors is a preferred classification workhorse for demonstrating this largely because more sophisticated classifiers, such as Support Vector Machines, could not until recently be trained on more than tens of thousands of examples. I introduce Random Features, an alternative to the Kernel Trick that provides one way to train SVMs on millions of data points in minutes or seconds. Unlike the Kernel Trick, which implicitly maps the data to an infinite-dimensional space, Random Features map the data to a relatively low-dimensional random feature space in which a linear SVM can be trained very efficiently. The resulting decision boundary is guaranteed to be close to that obtained by the kernelized SVM. The Random Features trick also significantly speeds up the evaluation of kernel machines, making it competitive in speed with fast nearest neighbors methods.

Another shortcoming of kernel machines is that they require positive definite kernels. Since most intuitive domain-specific similarity measures are not positive definite kernels, this leaves practitioners with three poor options: 1) Train an SVM with a sub-optimal positive definite similarity measure, 2) Use less powerful classification algorithms that admit indefinite similarity measures, such as nearest neighbors, 3) Train an SVM with a domain-specific indefinite similarity measure and hope things work out in practice. I show that a minor modification to the SVM permits it to operate with indefinite similarity functions, while still retaining good generalization bounds. I demonstrate this technique in a real-time object instance recognition system.

This is joint work with Ben Recht, Eric Garcia, and Shen-Hui Lee.

See other events that are part of Artificial Intelligence Seminar Series 2007/2008

See other events happening in October 2007


About Us Research News Resources Directory