Statistical Acquisition of Texture Appearance
We propose a simple method to acquire and reconstruct material appearance with sparsely sampled data. Our technique renders elaborate view- and light-dependent effects and faithfully reproduces materials such as fabrics and knitwears. Our approach uses sparse measurements to reconstruct a full six-dimensional Bidirectional Texture Function (BTF). Our reconstruction only require input images from the top view to be registered, which is easy to achieve with a fixed camera setup. Bidirectional properties are acquired from a sparse set of viewing directions through image statistics and therefore precise registrations for these views are unnecessary. Our technique is based on multi-scale histograms of image pyramids.
A Frequency Analysis of Light Transport
We present a signal-processing framework for light transport. We study the frequency content of radiance and how it is affected by phenomena such as shading, occlusion, and travel in free space. This extends previous work that considered either spatial or angular dimensions, and offers a comprehensive treatment of both space and angle. We characterize how the radiance signal is modified as light propagates and interacts with objects. In particular, we show that occlusion (a multiplication in the primal space) amounts in the Fourier domain to a convolution by the frequency content of the blocker. Propagation in free space corresponds to a shear in the space-angle frequency domain, while reflection on curved objects performs a different shear along the angular frequency axis.
We present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, we need to accurately measure visual motions, and group the pixels to be modified. After an initial image registration step, we measure motion by a robust analysis of feature point trajectories, and segment pixels based on similarity of position, color, and motion. A novel measure of motion similarity groups even very small motions according to correlation over time, which often relates to physical cause.
Texture Design Using a Simplicial Complex of Morphable Textures
We present a system for designing novel textures in the space of textures induced by an input database. We capture the structure of the induced space by a simplicial complex where vertices of the simplices represent input textures. A user can generate new textures by interpolating within individual simplices. We propose a morphable interpolation for textures, which also defines a metric used to build the simplicial complex. To guarantee sharpness in interpolated textures, we enforce histograms of high-frequency content using a novel method for histogram interpolation. We allow users to continuously navigate in the simplicial complex and design new textures using a simple and efficient user interface.
Defocus Video Matting
Video matting is the process of pulling a high-quality alpha matte and foreground from a video sequence. Current techniques require either a known background (e.g., a blue screen) or extensive user interaction (e.g., to specify known foreground and background elements). The matting problem is generally under-constrained, since not enough information has been collected at capture time. We propose a novel, fully autonomous method for pulling a matte using multiple synchronized video streams that share a point of view but differ in their plane of focus. The solution is obtained by directly minimizing the error in filter-based image formation equations, which are over-constrained by our rich data stream.