Learning Flexible Sprites in Video Layers
Speaker: Brendan Frey , University of Toronto
Date: November 15 2001
Time: 4:00PM
Location: NE43-941
In 1990, Adelson and Anandan proposed a mathematical representation of visual scenes as an ordered set of 2-D layers, each with an appearance map and a transparency map. This representation significantly simplifies the geometry of vision and graphics, and offers the hope of tremendously simplifying computational vision. Layers have been used quite successfully to model changes between nearby images in a video sequence, with very impressive applications in video coding.
We are working on the problem of automatically learning representations of multiple objects from a video sequence, using a layered representation. We call a probability density model over the appearance and transparency map for an object a "flexible sprite". Properly estimating a flexible sprite from a video sequence is a computationally difficult task, since all frames in which the object appears must be aligned, despite occlusions, changes in position, changes in lighting, deformations, etc. Then, each pixel in each frame must be judged to be appropriate or inappropriate for the purpose of estimating the corresponding pixel in the sprite. For example, if part of a sprite is occluded in a particular frame (explained by another sprite), the occluded pixels should not be used to estimate the sprite.
In this talk, I'll describe a variational technique for probabilistic inference and learning in a hierarchical probability model of flexible sprites. The learning algorithm is completely automatic and decomposes a video sequence into an ordered set of video layers. I'll also demonstrate some fun, low-quality computer graphics applications, such as removing objects from scenes and turning specific objects into cartoons.
Joint work with Nebojsa Jojic, Microsoft Research.
Brendan Frey's group (www.psi.toronto.edu) invents algorithms for probabilistic inference and machine learning, with applications in computer vision, speech processing, bioinformatics, iterative error-correcting decoding, and data compression. Members of the group participate in joint projects with Microsoft Research, Redmond, the University of Illinois at Urbana-Champaign, and the University of Waterloo. Frey has given over 40 invited talks and published over 80 papers in his areas of research and has received several awards, including the Beckman Fellowship and the Premier's Research Excellence Award. He serves on the program committees of CVPR, ICCV, ISIT, NIPS and UAI and is General Co-Chair of both the 2003 Workshop on Artificial Intelligence and Statistics, and the 2003 Canadian Workshop on Information Theory. Frey is Co-Editor-in-Chief of the February 2001 special issue of the IEEE Transactions on Information Theory, Codes on Graphs and Iterative Algorithms, and Associate Editor, IEEE Transactions on Pattern Analysis and Machine Intelligence.
See other events that are part of AI Colloquium Series Fall 2001
See other events happening in November 2001
