Defense: Shape, Reflectance, and Illumination From Appearance
Host
William T. Freeman
MIT CSAIL
PLEASE CONTACT xiuming at mit dot edu FOR THE MEETING PASSCODE.
Abstract:
The image formation process describes how light interacts with the objects in a scene and eventually reaches the camera, forming an image that we observe. Inverting this process is a long-standing, ill-posed problem in computer vision, which involves estimating the shape, material properties, and illumination passively from the object's appearance.
In this dissertation defense, we discuss "inverse rendering" by recovering three-dimensional (3D) shape, reflectance, illumination, or everything jointly under different setups. The input in these different setups can vary from single images to multi-view images lit by multiple known lighting conditions, then to multi-view images under one unknown illumination. Across the setups, we explore learning-based reconstruction that heavily relies on data-driven priors, optimization-based recovery that exploits multiple observations of just one object, and a mixture of both. As inverse rendering essentially decomposes the final appearance into intermediate representations, we examine different levels of the decomposition: from "blind" lighting recovery (high-level abstraction) to light transport function modeling (mid-level abstraction), finally to the full decomposition into shape, reflectance, and illumination (low-level abstraction); we also demonstrate how higher-level abstraction leads to wider applicability (e.g., to single images) but also more limited capabilities.
Abstract:
The image formation process describes how light interacts with the objects in a scene and eventually reaches the camera, forming an image that we observe. Inverting this process is a long-standing, ill-posed problem in computer vision, which involves estimating the shape, material properties, and illumination passively from the object's appearance.
In this dissertation defense, we discuss "inverse rendering" by recovering three-dimensional (3D) shape, reflectance, illumination, or everything jointly under different setups. The input in these different setups can vary from single images to multi-view images lit by multiple known lighting conditions, then to multi-view images under one unknown illumination. Across the setups, we explore learning-based reconstruction that heavily relies on data-driven priors, optimization-based recovery that exploits multiple observations of just one object, and a mixture of both. As inverse rendering essentially decomposes the final appearance into intermediate representations, we examine different levels of the decomposition: from "blind" lighting recovery (high-level abstraction) to light transport function modeling (mid-level abstraction), finally to the full decomposition into shape, reflectance, and illumination (low-level abstraction); we also demonstrate how higher-level abstraction leads to wider applicability (e.g., to single images) but also more limited capabilities.