Back to Events

Seminar Series

December 05

The Non-Stochastic Control Problem

Elad Hazan
Princeton University

Part Of

4:00P
- 5:00P

Location

32-G449 (Stata Center, Patil/Kiva Conference Room)
Add to Calendar 2019-12-05 16:00:00 2019-12-05 17:00:00 America/New_York The Non-Stochastic Control Problem Abstract:Linear dynamical systems are a continuous subclass of reinforcement learning models that are widely used in robotics, finance, engineering, and meteorology. Classical control, since the work of Kalman, has focused on dynamics with Gaussian i.i.d. noise, quadratic loss functions and, in terms of provably efficient algorithms, known statespace realization and observed state. We'll discuss how to apply new machine learning methods which relax all of the above: efficient control with adversarial noise, general loss functions, unknown systems, and partial observation.Bio:Elad Hazan is a professor of computer science at Princeton University. His research focuses on the design and analysis of algorithms for basic problems in machine learning and optimization. Amongst his contributions are the co-development of the AdaGrad optimization algorithm, and the first sublinear-time algorithms for convex optimization. He is the recipient of the Bell Labs prize, (twice) the IBM Goldberg best paper award in 2012 and 2008, a European Research Council grant, a Marie Curie fellowship and Google Research Award (twice). He served on the steering committee of the Association for Computational Learning and has been program chair for COLT 2015. In 2017 he co-founded In8 inc. focusing on efficient optimization and control, acquired by Google in 2018. He is the co-founder and director of Google AI Princeton. 32-G449 (Stata Center, Patil/Kiva Conference Room)

November 21

Synthetic Control (NeurIPS 2019 tutorial)

Devavrat Shah and Alberto Abadie
MIT

Part Of

3:00P
- 5:00P

Location

32-G449 (Stata Center - Patil/Kiva Conference Room)
Add to Calendar 2019-11-21 15:00:00 2019-11-21 17:00:00 America/New_York Synthetic Control (NeurIPS 2019 tutorial) Abstract:The synthetic control method, introduced in Abadie and Gardeazabal(2003), has emerged as a popular empirical methodology for estimating a causal effects with observational data, when the “gold standard” of a randomized control trial is not feasible. In a recent survey on causal inference and program evaluation methods in economics, Athey and Imbens (2015) describe the synthetic control method as “arguably the most important innovation in the evaluation literature in the last fifteen years”. While many of the most prominent application of the method, as well as its genesis, were initially circumscribed to the policy evaluation literature, synthetic controls have found their way more broadly to social sciences, biological sciences, engineering and even sports. However, only recently, synthetic controls have been introduced to the machine learning community through its natural connection to matrix and tensor estimation in Amjad, Shah and Shen (2017) as well as Amjad, Misra, Shah and Shen (2019).In this tutorial, we will survey the rich body of literature on methodical aspects, mathematical foundations and empirical case studies of synthetic controls. We willprovide guidance for empirical practice, with special emphasis on feasibility and data requirements, and characterize the practical settings where synthetic controls may be useful and those where they may fail. We will describe empirical case studies from policy evaluation, retail, and sports. Moreover, we will discuss mathematical connections of synthetic controls to matrix and tensor estimation, high dimensional regression, and time series analysis. Finally, we will discuss how synthetic controls are likely to be instrumental in the next wave of development in reinforcement learning using observational data.Bios:Devavrat Shah is a Professor with the department of Electrical Engineering and Computer Science and Director of Statistics and Data Science at the Massachusetts Institute of Technology. His current research interests are at the interface of Statistical Inference and Social Data Processing. His work has been recognized through prize paper awards in Machine Learning, Operations Research and Computer Science, as well as career prizes 2008 ACM Sigmetrics Rising Star Award, 2010 Erlang prize from the INFORMS Applied Probability Society and 2019 ACM Sigmetrics Test of Time Paper Award. He is a distinguished young alumni of his alma mater IIT Bombay. He has authored monographs “Gossip algorithms” and “Explaining the success of nearest neighbors in prediction’’. He co-founded machine learning start-up Celect, Inc. which is part of Nike, Inc. since August 2019. Alberto Abadie is an econometrician and empirical microeconomist, with broad disciplinary interests that span economics, political science and statistics. Professor Abadie received his Ph.D. in Economics from MIT in 1999. Upon graduating, he joined the faculty at the Harvard Kennedy School, where he was promoted to full professor in 2005. He returned to MIT in 2016, where he is Professor of Economics and Associate Director of the Institute for Data, Systems, and Society (IDSS).His research areas are econometrics, statistics, causal inference, and program evaluation. Professor Abadie’s methodological research focuses on statistical methods to estimate causal effects and, in particular, the effects of public policies, such as labor market, education, and health policy interventions. He is Associate Editor of Econometrica and AER: Insights, and has previously served as Editor of the Review of Economics and Statistics and Associate Editor of the Journal of Business and Economic Statistics. He is a Fellow of the Econometric Society. 32-G449 (Stata Center - Patil/Kiva Conference Room)

November 07

Advancements in Graph Neural Networks

Jure Leskovec
Stanford University

Part Of

4:15P
- 5:15P

Location

32-G449 (Patil/Kiva Conference Room)
Add to Calendar 2019-11-07 16:15:00 2019-11-07 17:15:00 America/New_York Advancements in Graph Neural Networks ABSTRACT: Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. In this talk I will discuss recent advancements in the field of Graph Neural Networks that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning. I will provide a conceptual overview of key advancements in this area of representation learning on graphs, including graph convolutional networks and their representational power. We will also discuss applications to web-scale recommender systems, healthcare, and knowledge representation and reasoning.BIO:Jure Leskovec is Associate Professor of Computer Science at Stanford University, Chief Scientist at Pinterest, and investigator at Chan Zuckerberg Biohub. His research focuses on machine learning and data mining with graphs, a general language for describing social, technological and biological systems. Computation over massive data is at the heart of his research and has applications in computer science, social sciences, marketing, and biomedicine. This research has won several awards including a Lagrange Prize, Microsoft Research Faculty Fellowship, the Alfred P. Sloan Fellowship, and numerous best paper and test of time awards. Leskovec received his bachelor's degree in computer science from University of Ljubljana, Slovenia, PhD in machine learning from Carnegie Mellon University and postdoctoral training at Cornell University. 32-G449 (Patil/Kiva Conference Room)

October 31

Generative Modeling by Estimating Gradients of the Data Distribution

Stefano Ermon
Stanford University

Part Of

4:30P
- 5:30P

Location

35-225
Add to Calendar 2019-10-31 16:30:00 2019-10-31 17:30:00 America/New_York Generative Modeling by Estimating Gradients of the Data Distribution Abstract:Existing generative models are typically based on explicit representations of probability distributions (e.g., autoregressive or VAEs) or implicit sampling procedures (e.g., GANs). We propose an alternative approach based on modeling directly the vector field of gradients of the data distribution (scores). Our framework allows flexible energy-based model architectures, requires no sampling during training or the use of adversarial training methods. Using annealed Langevin dynamics, we produces samples comparable to GANs on MNIST, CelebA and CIFAR-10 datasets, achieving a new state-of-the-art inception score of 8.91 on CIFAR-10. Finally, I will discuss challenges in evaluating bias and generalization in generative models.Bio:Stefano Ermon is an Assistant Professor of Computer Science in the CS Department at Stanford University, where he is affiliated with the Artificial Intelligence Laboratory, and a fellow of the Woods Institute for the Environment. His research is centered on techniques for probabilistic modeling of data, inference, and optimization, and is motivated by a range of applications, in particular ones in the emerging field of computational sustainability. He has won several awards, including four Best Paper Awards (AAAI, UAI and CP), a NSF Career Award, ONR and AFOSR Young Investigator Awards, a Sony Faculty Innovation Award, an AWS Machine Learning Award, a Hellman Faculty Fellowship, Microsoft Research Fellowship, and the IJCAI Computers and Thought Award. Stefano earned his Ph.D. in Computer Science at Cornell University in 2015. 35-225

October 29

David Spiegelhalter: Communicating uncertainty about facts, numbers and science

David Spiegelhalter
University of Cambridge, Winton Center for Risk and Evidence Communication

Part Of

11:00A
- 12:00P

Location

Seminar Room D463 (Star)
Add to Calendar 2019-10-29 11:00:00 2019-10-29 12:00:00 America/New_York David Spiegelhalter: Communicating uncertainty about facts, numbers and science "Communicating uncertainty about facts, numbers and science"The claim of a ‘post-truth’ society, in which emotional responses trump balanced consideration of evidence, presents a strong challenge to those who value quantitative and scientific evidence: how can we communicate risks and unavoidable scientific uncertainty in a transparent and trustworthy way?Communication of quantifiable risks has been well-studied, leading to recommendations for using an expected frequency format. But deeper uncertainty about facts, numbers, or scientific hypotheses needs to be communicated without losing trust and credibility. This is an empirically researchable issue, and I shall describe some current randomised experiments concerning the impact on audiences of alternative verbal, numerical and graphical means of communicating uncertainty. Available evidence may often not permit a quantitative assessment of uncertainty, and I will also examine scales being used to summarise degrees of ‘confidence’ in conclusions, in terms of the quality of the research underlying the whole assessment.Professor Sir David Spiegelhalter is Chair of the Winton Centre for Risk and Evidence Communication in the University of Cambridge, which aims to improve the way that statistical evidence is used by health professionals, patients, lawyers and judges, media and policy-makers. He advises organisations and government agencies on risk communication and is a regular media commentator on statistical issues, with a particular focus on communicating uncertainty. His background is in medical statistics, and he has over 200 refereed publications and is co-author of 6 textbooks, as well as The Norm Chronicles (with Michael Blastland), and Sex by Numbers. He works extensively with the media, and presented the BBC4 documentaries “Tails you Win: the Science of Chance”, the award-winning “Climate Change by Numbers”, and in 2011 came 7 th in an episode of BBC1’s Winter Wipeout. He was elected Fellow of the Royal Society in 2005, and knighted in 2014 for services to medical statistics. He was President of the Royal Statistical Society for 2017-2018. His bestselling book, The Art of Statistics, was published in March 2019. He is @d_spiegel on Twitter, and his home page is http://www.statslab.cam.ac.uk/~david/http://evite.me/xydVhP9GHQ Seminar Room D463 (Star)

October 24

Cancelled

TBD

This event has been cancelled

4:30P
- 5:30P

Neural Stochastic Differential Equations for Sparsely-sampled Time Series

David Duvenaud
University of Toronto

Part Of

4:30P
- 5:30P

Location

35-225
Add to Calendar 2019-10-24 16:30:00 2019-10-24 17:30:00 America/New_York Neural Stochastic Differential Equations for Sparsely-sampled Time Series Abstract: Much real-world data is sampled at irregular intervals, but most time series models require regularly-sampled data. Continuous-time latent variables models can handle address this problem, but until now only deterministic models, such as latent ODEs, were efficiently trainable by backprop. We generalize the adjoint sensitivities method to SDEs, constructing an SDE that runs backwards in time and computes all necessary gradients, along with a general algorithm that allows SDEs to be trained by backpropgation with constant memory cost. We also give an efficient algorithm for gradient-based stochastic variational inference in function space, all with the use of adaptive black-box SDE solvers. Finally, we'll show initial results of applying latent SDEs to time series data, and discuss prototypes of infinitely-deep Bayesian neural networks.Bio:David Duvenaud is an assistant professor in computer science and statistics at the University of Toronto. He holds a Canada Research Chair in generative models. His postdoctoral research was done at Harvard University, where he worked on hyperparameter optimization, variational inference, and chemical design. He did his Ph.D. at the University of Cambridge, studying Bayesian nonparametrics with Zoubin Ghahramani and Carl Rasmussen. David spent two summers in the machine vision team at Google Research, and also co-founded Invenia, an energy forecasting and trading company. David is a founding member of the Vector Institute and a Faculty Fellow at ElementAI. 35-225

October 17

Challenges in Reliable Machine Learning

Kamalika Chaudhuri
University of California, San Diego

Part Of

4:15P
- 5:15P

Location

32-G449 (Stata Center, Patil/Kiva Conference Room)
Add to Calendar 2019-10-17 16:15:00 2019-10-17 17:15:00 America/New_York Challenges in Reliable Machine Learning Abstract:As machine learning is increasingly used in real applications, there is a need for reliable and robust methods. In this talk, we will discuss two such challenges that arise in reliable machine learning. The first is sample selection bias, where training data is available from a distribution conditioned on a sample selection policy, but the resultant classifier needs to be evaluated on the entire population. We will show how we can use active learning to get a small amount of labeled data from the entire population that can be used to correct this kind of sample selection bias. The second is robustness to adversarial examples -- slight strategic perturbations of legitimate test inputs that cause misclassification. We next look at adversarial examples in the context of a simple non-parametric classifier -- the k-nearest neighbor classifier, and look at its robustness properties. We provide bounds on its robustness as a function of k, and propose a more robust 1-nearest neighbor classifier.Joint work with Songbai Yan, Tara Javidi, Yaoyuan Yang, Cyrus Rastchian, Yizhen Wang and Somesh Jha.Bio:Kamalika Chaudhuri received a Bachelor of Technology from the Indian Institute of Technology, Kanpur, and a PhD in Computer Science from the University of California, Berkeley. Currently, she is an Associate Professor at the University of California, San Diego. She is a recipient of the NSF Career Award, Hellman Faculty Fellowship, and Google and Bloomberg Faculty Awards. Kamalika's research is on the foundations of trustworthy machine learning -- which includes problems such as learning from sensitive data while preserving privacy, learning under sampling bias, and in the presence of an adversary. She is also broadly interested in a number of topics in learning theory, such as non-parametric methods, online learning, and active learning. 32-G449 (Stata Center, Patil/Kiva Conference Room)

October 03

Does Learning Require Memorization? A Short Tale about a Long Tail

Vitaly Feldman
Google Research

Part Of

4:00P
- 5:00P

Location

32-G449 (Stata Center - Patil/Kiva Conference Room)
Add to Calendar 2019-10-03 16:00:00 2019-10-03 17:00:00 America/New_York Does Learning Require Memorization? A Short Tale about a Long Tail Abstract:Learning algorithms based on deep neural networks are well-known to (nearly) perfectly fit the training set and fit well even the random labels. The reasons for this tendency to memorize the labels of the training data are not well understood.We provide a simple model for prediction problems in which such memorization is necessary for achieving close-to-optimal generalization error. In our model, data is sampled from a mixture of subpopulations and the frequencies of these subpopulations are chosen from some prior. Our analysis demonstrates that memorization becomes necessary whenever the frequency prior is long-tailed. Image and text data are known to follow such distributions and therefore our results establish a formal link between these empirical phenomena. We complement the theoretical results with experiments on several standard benchmarks showing that memorization is an essential part of deep learning.Based on https://arxiv.org/abs/1906.05271 and an ongoing work with Chiyuan Zhang.Bio:Vitaly Feldman is a research scientist at Google working on design and theoretical analysis of machine learning algorithms. His recent research interests include stability-based and information-theoretic tools for analysis of generalization, privacy-preserving learning, and adaptive data analysis. Vitaly holds a PhD from Harvard (2006) and was previously a research scientist at IBM Research - Almaden (2007-2017). He serves as a director on the steering committee of the Association for Computational Learning and was a program co-chair for COLT 2016. 32-G449 (Stata Center - Patil/Kiva Conference Room)

May 01

Deep Generative Models in the Diffusion Limit

Maxim Raginsky
University of Illinois, Urbana-Champaign

Part Of

4:00P
- 5:00P

Location

32-D463 (Stata Center, Star Conference Room)
Add to Calendar 2019-05-01 16:00:00 2019-05-01 17:00:00 America/New_York Deep Generative Models in the Diffusion Limit Abstract: In deep generative models, the latent variable is generated by a time-inhomogeneous Markov chain, where at each time step we pass the current state through a parametric nonlinear map, such as a feedforward neural net, and add a small independent Gaussian perturbation. In this talk, based on joint work with Belinda Tzen, I will discuss the diffusion limit of such models, where we increase the number of layers while sending the step size and the noise variance to zero. I will first provide a unified viewpoint on both sampling and variational inference in such generative models through the lens of stochastic control. Then I will show how we can quantify the expressiveness of diffusion-based generative models. Specifically, I will prove that one can efficiently sample from a wide class of terminal target distributions by choosing the drift of the latent diffusion from the class of multilayer feedforward neural nets, with the accuracy of sampling measured by the Kullback-Leibler divergence to the target distribution. Finally, I will briefly discuss a scheme for unbiased, finite-variance simulation in such models. This scheme can be implemented as a deep generative model with a random number of layers.Bio: Maxim Raginsky received the B.S. and M.S. degrees in 2000 and the Ph.D. degree in 2002 from Northwestern University, all in Electrical Engineering. He has held research positions with Northwestern, the University of Illinois at Urbana-Champaign (where he was a Beckman Foundation Fellow from 2004 to 2007), and Duke University. In 2012, he has returned to the UIUC, where he is currently an Associate Professor with the Department of Electrical and Computer Engineering, the Coordinated Science Laboratory, and the Department of Computer Science. His research interests cover probability and stochastic processes, deterministic and stochastic control, machine learning, optimization, and information theory. Much of his recent research is motivated by fundamental questions in modeling, learning, and simulation of nonlinear dynamical systems, with applications to advanced electronics, autonomy, and artificial intelligence. 32-D463 (Stata Center, Star Conference Room)

April 24

Gauge Fields in Deep Learning

Max Welling
University of Amsterdam

Part Of

2:00P
- 3:00P

Location

32-G449 (Stata Center - Patil/Kiva Conference Room)
Add to Calendar 2019-04-24 14:00:00 2019-04-24 15:00:00 America/New_York Gauge Fields in Deep Learning Abstract:Gauge field theory is the foundation of modern physics, including general relativity and the standard model of physics. It describes how a theory of physics should transform under symmetry transformations. For instance, in electrodynamics, electric forces may transform into magnetic forces if we transform a static observer to one that moves at constant speed. Similarly, in general relativity acceleration and gravity are equated to each other under symmetry transformations. Gauge fields also play a crucial role in modern quantum field theory and the standard model of physics, where they describe the forces between particles that transform into each other under (abstract) symmetry transformations. In this work we describe how the mathematics of gauge groups becomes inevitable when you are interested in deep learning on manifolds. Defining a convolution on a manifold involves transporting geometric objects such as feature vectors and kernels across the manifold, which due to curvature become path dependent. As such it becomes impossible to represent these objects in a global reference frame and one is forced to consider local frames. These reference frames are arbitrary and changing between them is called a (local) gauge transformation. Since we do not want our computations to depend on the specific choice of frames we are forced to consider equivariance of our convolutions under gauge transformations. These considerations result in the first fully general theory of deep learning on manifolds, with gauge equivariant convolutions as the key ingredient.Joint work with Taco Cohen, Maurice Weiler and Berkay KicanaogluBio:Prof. Dr. Max Welling is a research chair in Machine Learning at the University of Amsterdam and a VP Technologies at Qualcomm. He has a secondary appointment as a senior fellow at the Canadian Institute for Advanced Research (CIFAR). He is co-founder of “Scyfer BV” a university spin-off in deep learning which got acquired by Qualcomm in summer 2017. In the past he held postdoctoral positions at Caltech (’98-’00), UCL (’00-’01) and the U. Toronto (’01-’03). He received his PhD in ’98 under supervision of Nobel laureate Prof. G. ‘t Hooft. Max Welling has served as associate editor in chief of IEEE TPAMI from 2011-2015 (impact factor 4.8). He serves on the board of the NIPS foundation since 2015 (the largest conference in machine learning) and has been program chair and general chair of NIPS in 2013 and 2014 respectively. He was also program chair of AISTATS in 2009 and ECCV in 2016 and general chair of MIDL 2018. He has served on the editorial boards of JMLR and JML and was an associate editor for Neurocomputing, JCGS and TPAMI. He received multiple grants from Google, Facebook, Yahoo, NSF, NIH, NWO and ONR-MURI among which an NSF career grant in 2005. He is recipient of the ECCV Koenderink Prize in 2010. Welling is in the board of the Data Science Research Center in Amsterdam, he directs the Amsterdam Machine Learning Lab (AMLAB), and co-directs the Qualcomm-UvA deep learning lab (QUVA) and the Bosch-UvA Deep Learning lab (DELTA). 32-G449 (Stata Center - Patil/Kiva Conference Room)

April 17

Towards transparency in AI, Methods and Challenges

Timnit Gebru
Google

Part Of

4:00P
- 5:00P

Location

32-G449 (Stata Center - Patil/Kiva Conference Room)
Add to Calendar 2019-04-17 16:00:00 2019-04-17 17:00:00 America/New_York Towards transparency in AI, Methods and Challenges Abstract:Automated decision making tools are currently used in high stakes scenarios. From natural language processing tools used to automatically determine one’s suitability for a job, to health diagnostic systems trained to determine a patient’s outcome, machine learning models are used to make decisions that can have serious consequences on people’s lives. In spite of the consequential nature of these use cases, vendors of such models are not required to perform specific tests showing the suitability of their models for a given task. Nor are they required to provide documentation describing the characteristics of their models, or disclose the results of algorithmic audits to ensure that certain groups are not unfairly treated. I will show some examples to examine the dire consequences of basing decisions entirely on machine learning based systems, and discuss recent work on auditing and exposing the gender and skin tone bias found in commercial gender classification systems. I will end with the concept of AI datasheets for datasets and model cards for model reporting to standardize information for datasets and pre-trained models, in order to push the field as a whole towards transparency and accountability. Recently, we have seen many powerful entities in academia and industry announcing initiatives related to AI ethics. I will spend some time in this talk discussing how we can learn from the mistakes and evolutions of other disciplines who have/continue to perform what some call parachute research: one that uses the pain of marginalized communities without centering their voices or benefiting them.Bio:Timnit Gebru is a research scientist in the Ethical AI team at Google and just finished her postdoc in the Fairness Accountability Transparency and Ethics (FATE) group at Microsoft Research, New York. Prior to that, she was a PhD student in the Stanford Artificial Intelligence Laboratory, studying computer vision under Fei-Fei Li. Her main research interest is in data mining large-scale, publicly available images to gain sociological insight, and working on computer vision problems that arise as a result, including fine-grained image recognition, scalable annotation of images, and domain adaptation. She is currently studying the ethical considerations underlying any data mining project, and methods of auditing and mitigating bias in sociotechnical systems. The New York Times, MIT Tech Review and others have recently covered her work. As a cofounder of the group Black in AI, she works to both increase diversity in the field and reduce the negative impacts of racial bias in training data used for human-centric machine learning models. 32-G449 (Stata Center - Patil/Kiva Conference Room)

April 03

Making Deep Learning more Robust, Modular, and Efficient

Zico Kolter
Carnegie Mellon University

Part Of

4:30P
- 5:30P

Location

34-101
Add to Calendar 2019-04-03 16:30:00 2019-04-03 17:30:00 America/New_York Making Deep Learning more Robust, Modular, and Efficient Abstract: Deep learning is often seen as the "breakthrough" AI technology of recent years, revolutionizing areas spanning computer vision, natural language processing, and game playing. However, if we seek to deploy such systems in real-world, safety-critical domains, a starker reality emerges. Modern deep learning systems are brittle (sensitive to adversarial manipulation and a general lack of robustness), opaque (difficult to interpret and debug their components), and expensive (often requiring vastly more data than practical in real-world settings).These failings are sometimes billed as an argument against deep learning as a whole. But in this talk, I will argue instead for new methods that can address these challenges, while preserving the fundamental benefits of deep learning (namely, end-to-end training of composable, differentiable architectures). First, I will discuss our approaches to designing provably robust deep networks using tools from convex relaxations and duality. I also highlight recent work on scaling these methods to much larger domains, including some initial work on provable robustness at ImageNet scale. Second, I will present our work on integrating more complex modules as interpretable layers within deep learning architectures. I show how modules such as optimization solvers, physical simulation, model-based control, and game equilibrium solvers can all be integrated as layers within a deep network, enabling more intuitive architectures that can learn from vastly less data. Last, I will highlight some additional ongoing directions and open questions in both these areas.Bio:Zico Kolter is an Assistant Professor in the Computer Science Department at Carnegie Mellon University, and also serves as chief scientist of AI research for the Bosch Center for Artificial Intelligence. His work focuses on the intersection of machine learning and optimization, with a large focus on developing more robust, interpretable, and rigorous methods in deep learning. In addition, he has worked in a number of application areas, highlighted by work on sustainability and smart energy systems. He is a recipient of the DARPA Young Faculty Award, and best paper awards at KDD, PESGM, and IJCAI. 34-101

March 20

Planning from a Few Samples

Geoff Gordon
Microsoft Research Montreal

Part Of

3:00P
- 4:00P

Location

32-D463 (Stata Center - Star Conference Room)
Add to Calendar 2019-03-20 15:00:00 2019-03-20 16:00:00 America/New_York Planning from a Few Samples Abstract: We study the problem of planning when experience is expensive: we are given a few trajectories sampled from our target environment, and we wish to compute a good policy or value function. In such problems, we can’t afford to use “deep RL” methods like DQN or REINFORCE: these methods can need millions or even billions of samples to reach a useful solution. On the other hand, we are willing to assume that we can gather a representative sample of trajectories; e.g., we might have access to an expert who can steer the system into interesting regions of state space. Given these problem characteristics, we seek an algorithm that is reliable, fast, and data-efficient. To this end, we design a new approximate optimality criterion for planning: we frame the problem as a fixed-point iteration based on a variational inequality, and use projection and subsampling to reduce the problem to a tractable size. We demonstrate that our approach can solve simple planning problems quickly and reliably from moderate amounts of data.Bio: Dr. Gordon is Research Director of Microsoft Research Montreal. He is on leave from the Department of Machine Learning at Carnegie Mellon University, where he has served as Professor, Interim Department Head, and Associate Department Head for Education. His research interests include artificial intelligence, statistical machine learning, game theory, multi-robot systems, and planning in probabilistic, adversarial, and general-sum domains. His previous appointments include Visiting Professor at the Stanford Computer Science Department and Principal Scientist at Burning Glass Technologies in San Diego. Dr. Gordon received his B.A. in Computer Science from Cornell University in 1991, and his Ph.D. in Computer Science from Carnegie Mellon University in 1999. 32-D463 (Stata Center - Star Conference Room)