In contrast to traditional rule-based approaches to building spoken dialogue systems, recent research has shown that it is possible to implement all of the required functionality using statistical models trained using a combination of supervised learning and reinforcement learning. This new approach to spoken dialogue is based on the mathematics of partially observable Markov decision processes (POMDPs) in which user inputs are treated as observations of some underlying belief state, and system responses are determined by a policy which maps belief states into actions. Virtually all current spoken dialogue systems are designed to operate in a specific carefully defined domain such as restaurant information, appointment booking, product installation support, etc. However, if voice is to become a significant input modality for accessing web-based information and services, then techniques will be needed to enable spoken dialogue systems to operate within open domains. The first part of the talk will briefly review the basic ideas of POMDP dialogue systems as currently applied to closed-domains. Unlike many other areas of machine learning, spoken dialogue systems always have a user on-hand to provide supervision. Based on this idea, the second part of the talk describes a number of techniques by which implicit user supervision can allow a spoken dialogue system to adapt on-line to extended domains.