CSAIL Event Calendar: Previous Series
On Building Flexible and Robust Multimodal Interfaces
Speaker: Phil Cohen , Oregon Health and Science University
Multimodal interfaces are all about flexibility and robustness. They allow users to employ the most appropriate communication modalities for the immediate circumstances, and the most appropriate form-factor for the activity being performed. Because of this flexibility, such system will be confronted with more adverse conditions than graphical user interfaces (GUIs) operating in standard office-based environments, or unimodal interfaces that have been developed and tested under benign conditions. The natural result is that without an architecture that is designed for error correction and the resolution of uncertainty, the processing of the users' inputs is more susceptible to failure. This talk will discuss and illustrate how the QuickSet multimodal interface architecture supports flexibility and robustness through its symbolic and statistical multimodal fusion algorithms. I will also discuss how we have extended the algorithms that were designed for 2D environments to 3D interaction in 3D worlds, resulting in a system that can simultaneously correct individual interpretations from three modalities/information sources in order to arrive at a sensible overall interpretation.