Two Easy Classification Tricks To Simplify the Practitioner
Speaker: Ali Rahimi , Intel Research, Seattle
A powerful and relatively recent observation in computer vision is that generic machine techniques trained on very large data sets can outperform more complicated domain-specific methods. Nearest neighbors is a preferred classification workhorse for demonstrating this largely because more sophisticated classifiers, such as Support Vector Machines, could not until recently be trained on more than tens of thousands of examples. I introduce Random Features, an alternative to the Kernel Trick that provides one way to train SVMs on millions of data points in minutes or seconds. Unlike the Kernel Trick, which implicitly maps the data to an infinite-dimensional space, Random Features map the data to a relatively low-dimensional random feature space in which a linear SVM can be trained very efficiently. The resulting decision boundary is guaranteed to be close to that obtained by the kernelized SVM. The Random Features trick also significantly speeds up the evaluation of kernel machines, making it competitive in speed with fast nearest neighbors methods.