CSAIL Event Calendar: Previous Series

Less is More: Coded Computational Photography

Speaker: Ramesh Raskar , Associate Professor, MIT Media Lab
Date: February 20 2008
Time: 3:00PM to 4:00PM
Location: Star Seminar Room (32-D463)
Host: C. Mario Christoudias, Gerald Dalley, MIT CSAIL

Less is More: Coded Computational Photography

Ramesh Raskar

Associate Professor, Media Lab, MIT and
Senior Research Scientist, Mitsubishi Electric Research Labs (MERL), Cambridge, MA

Computational Photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics+vision, electronics, art, and online sharing in social networks. The field is evolving through three phases. The first phase was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view or depth of field. I call this Epsilon Photography. Due to limited capabilities of a camera, the scene is sampled via multiple photos, each captured by epsilon variation of the camera parameters. It corresponds to the low-level vision: estimating pixels and pixel features. The second phase is building tools that go beyond capabilities of this super-camera. I call this Coded Photography. The goal here is to reversibly encode information about the scene in a single photograph (or a very few photographs) so that the corresponding decoding allows powerful decomposition of the image into light fields, motion deblurred images, global/direct illumination components or distinction between geometric versus material discontinuities. This corresponds to the mid-level vision: segmentation, organization, inferring shapes, materials and edges. The third phase will be about going beyond the radiometric quantities and challenging the notion that a camera should mimic a single-chambered human eye. Instead of recovering physical parameters, the goal will be to capture the visual essence of the scene and analyze the perceptually critical components. I call this Essence Photography and it may loosely resemble depiction of the world after high level vision processing. It will spawn new forms of visual artistic expression and communication.

In this talk, I will focus on Coded Photography. 'Less is more' in Coded Photography. By blocking light over time or space, we can preserve more details about the scene in the recorded single photograph.

1. Coded Exposure: By blocking light in time, by fluttering the shutter open and closed in a carefully chosen binary sequence, we can preserve high spatial frequencies of fast moving objects to support high quality motion deblurring.
2. Coded Aperture Optical Heterodyning: By blocking light near the sensor with a sinusoidal grating mask, we can record 4D light field on a 2D sensor. And by blocking light with a mask at the aperture, we can extend the depth of field and achieve full resolution digital refocussing.
3. Coded Illumination: By observing blocked light at silhouettes, a multi-flash camera can locate depth discontinuities in challenging scenes without depth recovery.
4. Coded Sensors: By sensing intensities with lateral inhibition, a ‘Gradient Camera’ can record large as well as subtle changes in intensity to recover a high-dynamic range image.
5. Coded Spectrum: By blocking parts of a ‘rainbow’, we can create cameras with digitally programmable wavelength profile.



I will show several applications and describe emerging techniques to recover scene parameters from coded photographs.

Recent joint work with Jack Tumblin, Amit Agrawal, Ashok Veeraraghavan and Ankit Mohan

Bio
Ramesh Raskar will join the Media Lab, MIT in Spring 2008 as head of the Camera Culture research group. Dr. Raskar received the TR100 Award, Technology Review's 100 Top Young Innovators Under 35 worldwide, 2004 and Global Indus Technovator Award 2003, instituted at MIT to recognize the top 20 Indian technology innovators on the globe. He holds 30 US patents and has received Mitsubishi Electric Invention Awards in 2003, 2004 and 2006. He is currently co-authoring a book on computational photography (with Jack Tumblin).

http://raskar.info

See other events that are part of MIT Machine Vision Colloquium 2007/2008

See other events happening in February 2008