When humans and autonomous systems share control of a vehicle, there will be some explaining to do. When the autonomous system takes over suddenly, the driver will ask why. When an accident happens in a car that is co-driven by a person and a machine, police officials, insurance companies, and the people who are harmed will want to know who or what is accountable for the accident. Control systems in the vehicle should be able to give an accurate unambiguous accounting of the events. Explanations will have to be simple enough for users to understand even when subject to cognitive distractions. At the same time, given the need for legal accountability and technical integrity these systems will have to support their basic explanations with rigorous and reliable detail. In the case of hybrid human-machine systems, we will want to know how the human and mechanical parts contributed to final results such as accidents or other unwanted behaviors.The ability to provide coherent explanations of complex behavior is also important in the design and debugging of such systems, and it is essential to give us all confidence in the competence and integrity of our automatic helpers. But the mechanisms of merging measurements with qualitative models can also enable more sophisticated control strategies than are currently feasible. Our research explores the development of methodology and supporting technology for combining qualitative and semi-quantitative models with measured data to produce concise, understandable symbolic explanations of actions taken by a system built out of many parts (including the human operator).
Explaining explanations: An overview of interpretability of machine learning
IEEE 5th International Conference on data science and advanced analytics (DSAA)
Explaining Possible Futures for Robust Autonomous Decision-Making
Proceedings of the AAAI Fall Symposium on Anticipatory Thinking, 2019
Monitoring Opaque Learning Systems
ICLR 2019 Debugging ML Models Workshop, 2019
Learning From Explanations for Robust Autonomous Driving.
ICML Workshop on AI for Autonomous Driving
Getting Up to Speed on Vehicle Intelligence
AAAI Spring Symposium Series, 2017
AAAI Conference on Artificial Intelligence, 2018
Reasonable Perception: Connecting Vision and Language Systems for Validating Scene Descriptions
Proceedings of the Thirteenth Annual ACM/IEEE International Conference on Human Robot Interaction, 2018
An Adaptable Self-Monitoring Framework for Opaque Machines
Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 2019