Maximum entropy and applications in natural and social sciences
Speaker: Miroslav Dudik , Carnegie Mellon University
Date: April 5 2010
Time: 4:00PM to 5:00PM
Location: 32-G449
Host: Pablo Parrilo and Tommi Jaakkola, CSAIL
Contact: Francis Doughty, 253-4602, doughty@mit.edu
Relevant URL: The maximum entropy approach (maxent), equivalent to maximum
likelihood, is a widely used density-estimation technique. However,
when trained on small datasets, maxent puts too much confidence on too
little data (a phenomenon known as "overfitting"), and when trained
over large sample spaces, naive implementations of maxent are
intractable. To prevent overfitting, we propose a relaxed version of
maxent, which turns out to be equivalent to L1-regularized log
likelihood. We prove strong statistical guarantees for L1-regularized
maxent, and show how it can be generalized to the problem of
estimation in the presence of sample-selection bias, and to the
problem of simultaneous estimation of multiple densities. To address
computational challenges, we propose an approach based on sampling and
coordinate descent.
I discuss two applications of maxent: statistical modeling of
distributions of biological species and game-theoretic modeling of
human negotiation, focusing mainly on the former. In species
distribution modeling, statistical properties of regularized maxent
are key in obtaining state-of-the-art performance on small data sets.
In game-theoretic modeling, the coordinate descent algorithm and
sampling allow our approach to solve negotiation scenarios an order of
magnitude larger than previous techniques.
Based on joint work with Rob Schapire, Steven Phillips, Geoff Gordon,
Dave Blei and others.
BIO: Miroslav Dudik received his PhD in Computer Science from
Princeton University in 2007. Currently, he is a postdoctoral fellow
at Carnegie Mellon University. His interests are in theoretical and
applied aspects of machine learning, both statistical and algorithmic.
He focuses on small-sample density estimation and game-theoretic
modeling.
See other events that are part of CS Special Seminar Series Spring 2010
See other events happening in April 2010
