Seamless communication between humans and robots is a critical condition for successful human-robot interaction; not only for safety purposes but to properly convey information, coordinate activities, and minimize the cost of collaborative actions. In these human-robot scenarios, humans are required to act as critics and communicate the decisions via an input device (i.e. keyboard, mouse), which alienates them (albeit briefly) from the real goal of the task. Our research focuses on using specific brain signals that respond to success or failure events in order to inform the other agents of such outcomes so that these can adapt their strategies and, in the long term, learn the optimal behavior.
Note: see Supervising Robots with Muscle and Brain Signals for the integration of this project with gesture-based control via muscle signals.
- MIT News article
- Publication PDF
- DRL website project page
- Media video (https://youtu.be/Zd9WhJPa2Ok)
- Conference Video (https://youtu.be/-iZbntmYoSE)
- Combination of this project with hand gesture detection via muscle signals
Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for real-time closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.