Jacob Eisenstein

I'm an outgoing graduate student at MIT's Computer Science and Artificial Intelligence Lab. My advisors are Randall Davis and Regina Barzilay. I completed my Ph.D. in May 2008.

After MIT, I will be a Postdoctoral Fellow at the Beckman Institute of UIUC.

Research Interests

My primary research interest is learning computational models of the communicative properties of hand gestures. I work on machine learning approaches that identify patterns of gesture that can be leveraged to improve performance on natural language understanding tasks. I'm especially interested in designing structured models to exploit linguistic properties of gesture. Other research interests include intelligent and multimodal user interfaces.

For non-specialists, here is a short description of my research that I wrote for my grandparents.

Selected Recent Publications

J. Eisenstein, R. Barzilay, and R. Davis. Discourse Topic and Gestural Form. AAAI 2008.: A quantitative analysis of the influence of speaker and topic on gestural form. Using low-level interest point features, it is possible to show that multiple speakers use similar gestures when describing the same topic.
J. Eisenstein, R. Barzilay, and R. Davis. Gestural Cohesion for Topic Segmentation. ACL 2008.: Coherent discourse topics contain internally consistent gestural forms, paralleling similar work on the distribution of lexical items. Automatically extracted gesture features are used to improve unsupervised topic segmentation on dialogues.
S. R. K. Branavan, H. Chen, J. Eisenstein, and R. Barzilay. Learning Document-Level Semantic Properties from Free-text Annotations. ACL 2008.: Unstructured text and keyphrase annotations predict document-level semantics in a joint Bayesian framework. This can be used to automatically generate pro/con lists from product reviews.
J. Eisenstein, R. Barzilay, and R. Davis. Modeling Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction. Journal of Artificial Intelligence Research, volume 31, 353-398.: Describes the use of a conditional hidden variable model for gesture salience in coreference resolution. The model improves performance on coreference resolution, and the estimates of gesture salience can be transferred directly to select keyframes containing interesting gestures.
Here is a full list of my publications.
And here is my PhD thesis

Software

Some code for machine learning, computer vision, video annotation, and other random stuff.

Election Prediction

A side interest of mine is statistical models for predicting election results based on polls. I did pretty well predicting the US Senate elections in 2006.

Contact

The Stata Center
Massachusetts Institute of Technology
32 Vassar Street
Room 235
Cambridge, MA 02139
617-253-2663
jacobe | csail mit edu