Back to Events

Seminar Series

November 05

Feedforward and feedback processes in visual recognition

Prof. Thomas Serre
Brown University

Part Of

4:00P
- 5:00P

Location

46-3002
Singleton Auditorium (MIT #46-3002)
Add to Calendar 2019-11-05 16:00:00 2019-11-05 17:00:00 America/New_York Feedforward and feedback processes in visual recognition Abstract: Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. In this talk, however, I will show that these neural networks and their recent extensions exhibit a limited ability to solve seemingly simple visual reasoning problems involving incremental grouping, similarity and spatial relation judgments. Our group has developed a recurrent network model of classical and extra-classical receptive fields that is constrained by the anatomy and physiology of the visual cortex. The model was shown to account for diverse visual illusions providing computational evidence for a novel canonical circuit that is shared across visual modalities. I will show that this computational neuroscience model can be turned into a modern end-to-end trainable deep recurrent network architecture which addresses some of the shortcomings exhibited by state-of-the-art feedforward networks for solving complex visual reasoning tasks. This suggests that neuroscience may contribute powerful new ideas and approaches to computer science and artificial intelligence. 46-3002

October 29

Calibrating Generative Models: The Probabilistic Chomsky-Schützenberger Hierarchy

Prof. Thomas Icard
Stanford University

Part Of

4:00P
- 5:00P

Location

MIT D463
Star Seminar Room (Stata D463)
Add to Calendar 2019-10-29 16:00:00 2019-10-29 17:00:00 America/New_York Calibrating Generative Models: The Probabilistic Chomsky-Schützenberger Hierarchy Abstract: How might we assess the expressive capacity of different classes of probabilistic generative models? The subject of this talk is an approach that appeals to machines of increasing strength (finite-state, recursive, etc.), or equivalently, by probabilistic grammars of increasing complexity, giving rise to a probabilistic version of the familiar Chomsky hierarchy. Many common probabilistic models — hidden Markov models, generative neural networks, probabilistic programming languages, etc. — naturally fit into the hierarchy. The aim of the talk is to give as comprehensive a picture as possible of the landscape of distributions that can be expressed at each level in the hierarchy. Of special interest is what this pattern of results might mean for cognitive modeling.---*Please note the change in title and abstract. MIT D463

October 28

Beyond Empirical Risk Minimization: the Lessons of Deep Learning

Prof. Mikhail Belkin
The Ohio State University

Part Of

4:00P
- 5:00P

Location

46-3002
Singleton Auditorium (MIT #46-3002)
Add to Calendar 2019-10-28 16:00:00 2019-10-28 17:00:00 America/New_York Beyond Empirical Risk Minimization: the Lessons of Deep Learning Abstract: "A model with zero training error is overfit to the training data and will typically generalize poorly" goes statistical textbook wisdom. Yet, in modern practice, over-parametrized deep networks with near perfect fit on training data still show excellent test performance. This apparent contradiction points to troubling cracks in the conceptual foundations of machine learning. While classical analyses of Empirical Risk Minimization rely on balancing the complexity of predictors with training error, modern models are best described by interpolation. In that paradigm a predictor is chosen by minimizing (explicitly or implicitly) a norm corresponding to a certain inductive bias over a space of functions that fit the training data exactly. I will discuss the nature of the challenge to our understanding of machine learning and point the way forward to first analyses that account for the empirically observed phenomena. Furthermore, I will show how classical and modern models can be unified within a single "double descent" risk curve, which subsumes the classical U-shaped bias-variance trade-off.Finally, as an example of a particularly interesting inductive bias, I will show evidence that deep over-parametrized autoencoders networks, trained with SGD, implement a form of associative memory with training examples as attractor states. 46-3002

October 02

Quantum Computing: Current Approaches and Future Prospects

Jack Hidary
Alphabet X, formerly Google X

Part Of

11:00A
- 12:00P

Location

46-3002
Singleton Auditorium (MIT #46-3002)
Add to Calendar 2019-10-02 11:00:00 2019-10-02 12:00:00 America/New_York Quantum Computing: Current Approaches and Future Prospects CBMM Special SeminarAbstract: Jack Hidary will take us through the nascent, but promising field of quantum computing and his new book, Quantum Computing: An Applied Approach.Bio: Jack D. Hidary is a research scientist in quantum computing and in AI at Alphabet X, formerly Google X. He and his group develop and research algorithms for NISQ-regime quantum processors as well as create new software libraries for quantum computing. In the AI field, Jack and his group focus on fundamental research such as the generalization of deep networks as well as applied AI technologies. 46-3002

September 17

A distributional point of view on hierarchy

Prof. Maia Fraser
University of Ottawa

Part Of

4:00P
- 5:00P

Location

46-3002
Singleton Auditorium (MIT Building 46-3002)
Add to Calendar 2019-09-17 16:00:00 2019-09-17 17:00:00 America/New_York A distributional point of view on hierarchy Abstract: Hierarchical learning is found widely in biological organisms. There are several compelling arguments for advantages of this structure. Modularity (reusable components) and function approximation are two where theoretical support is readily available. Other, more statistical, arguments are surely also relevant, in particular there's a sense that "hierarchy reduces generalization error". In this talk, I will bolster this from a distributional point of view and show how this gives rise to deep vs. shallow regret bounds in semi-supervised learning that can also be carried over to some reinforcement learning settings. The argument in both paradigms deals with partial observation, namely partially labeled data resp. partially observed states, and useful representations that can be learned therefrom. Examples include manifold learning and group-invariant features. 46-3002