Oliva Explores New Ground in Computational Perception
7 November 2011
Traversing a busy street, entering a new building and locating the elevator may seem like simple tasks, mainly because you don’t notice all the heavy lifting your brain undertakes when absorbing and reacting to your visual surroundings so that you don’t crash into walls or get hit by a car. For years CSAIL Principal Investigator Aude Oliva has studied human visual intelligence, how the brain tackles scene and object recognition and visual memory, by closely examining the functions of the human brain. Joining CSAIL’s Computer Vision Research Group, Oliva feels, is an opportunity to make further progress with her work in computational perception with the ultimate goal of gaining a better understanding of the brain’s visual processes, and creating machines and systems that can see and understand the visual world just like humans do.
Oliva, an associate professor in the Department of Brain and Cognitive Sciences (BCS), received two M. Sc. Degrees, in Experimental Psychology and in Cognitive Science, and a Ph.D. from the Grenoble Institute of Technology. Her work focuses on high-level recognition tasks like understanding scenes and events, perceiving space, finding objects in cluttered scenes, modeling attention and visual memory, as well as predicting subjective properties of images as humans do. Inspired by human vision, she has created a new visual technology called hybrid images, composites of more than one image that illustrate how we see. She has also developed computational methods for automatic recognition of visual environments and for predicting where people look when searching for objects in a picture.
Recently she has focused on the concept of memorability, examining what qualities and characteristics make an image memorable. Oliva’s results (documented in two recent papers at the 2011 Computer Vision and Pattern Recognition Conference and the 2011 Neural Information Processing Systems Conference, in collaboration with CSAIL Principal Investigator Antonio Torralba, BCS graduate student Phillip Isola, visiting researcher Devi Parikh and CSAIL graduate student Jianxiong Xiao) debunked the long-held belief that visual memory is subjective, showing instead that humans are consistent in which types of images they will remember and that the probability that a viewer will remember a certain image can be predicted quite accurately.
“What we did was to look for visual features in the image that could predict which type of image was memorable or not memorable, and we developed an algorithm that can take a new image, and automatically predict how likely it is to be remembered,” said Oliva. “This defines a new application in computer vision because people never thought that it would be possible to predict what makes an image memorable because they always thought that people could not agree on what makes images memorable.”
Oliva feels this finding is applicable in a number of different areas, from helping users create more memorable image galleries on Facebook to designing more memorable logos and websites. The work could prove particularly useful in education, as the technique could be used to help create textbooks with images that will prove memorable and hopefully helpful in teaching students new concepts. The technique could also be used as a diagnostic tool for memory loss.
Oliva hopes to expand on this area of research at CSAIL, and feels that joining the lab’s Computer Vision Research Group will be particularly helpful in exploring a new frontier of research, computational perception. By working with experts in computer vision, computational photography and computer graphics, Oliva feels she and her colleagues at CSAIL can break new ground in this field and find many new applications for her work.
The Vision Group “is already a multi-disciplinary research group, so for me joining that group is like getting the best of all worlds because I can add knowledge of the perception, cognition and neuroscience side and they can bring more about the computer vision, computer graphics and application side,” said Oliva.
Additionally, Oliva feels that her new research findings that aesthetic and beauty are not correlated with memorability could prove useful in the realm of computer graphics and design. Drawing on knowledge of image memorability, Oliva and her team envision creating programs that will allow users to make their images more memorable with a simple click of a button, a technique that could prove especially useful to graphic designers.
With funding from the National Institute of Health, Oliva also plans to delve into neuroimaging and brain studies of human memory and space perception at CSAIL.
“If a computer vision algorithm was able to determine whether an image or an event in a video is memorable or not, this could give us new insights about how memory might be represented in the brain, and bridge computer vision with neuroscience,” said Oliva. “This is where I think people in computer science like Polina Golland [a principal investigator at CSAIL] can be great collaborators for creating algorithms that analyze the humongous data sets produced by neuroimaging studies of the human brain.”