MIT CSAIL holds trustworthy AI event with Microsoft

Trustworthy AI 2

Last week MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) held a special workshop with Microsoft Research to explore key challenges in creating trustworthy and robust artificial intelligence (AI) systems. The effort focused on addressing concerns about the trustworthiness of AI systems, including rising concerns with the safety, fairness, and transparency of the technologies.

The workshop was led by CSAIL director Daniela Rus, MIT professor Aleksander Madry and Microsoft’s Eric Horvitz, who directs Microsoft Research Labs. Rus says that, in today’s climate, our fears of intelligent machines “taking over” have been misdirected. The real issue, she says, is one of trust: who is held accountable when a machine makes a decision, and how can we ensure that these machines will make fair ones?

“We’re excited about bringing together leading intellects at MIT CSAIL and Microsoft Research to collaborate on intriguing and important opportunities ahead -- and to develop trustworthy AI systems that are safe, reliable, understandable, and fair,” says Horvitz.

To better address challenges in robustness, reliability, and safety of AI, MIT CSAIL and Microsoft are placing a lens on creating a robust AI toolkit from the ground up. The ultimate goal is to develop AI systems that are guaranteed to be reliable, allowing them to free us from laborious tasks, improve how we detect and treat disease, and even one day drive our cars for us.

“Machine learning systems work well on average, but many issues remain - facial recognition systems can be hacked by special glasses, health-care datasets are not clean, to name a few,” says Madry. “We’re looking at the technical challenges to actually create models that can overcome specific barriers to make safer decisions.”

Trustworthy AI builds on MIT CSAIL’s existing initiatives that focus on machine learning, data systems and AI, financial technology and cybersecurity.

Today, modern deep learning systems contain millions of parameters and are largely considered “black boxes”,  meaning we have little knowledge of their internal workings. The opening discussion of the event centered on this issue, as well as the question at the crux of it all: can we truly rely on AI?

With the high expectations for these machine learning (ML) models, Madry says we must understand that they can be easily manipulated.

He further described the many vulnerabilities there are throughout the ML pipeline, including data collection, training, inference, and deployment. For example, one could manipulate a dataset to classify a dog as an ostrich, an autonomous vehicles’ software could confuse an unforeseen object and get derailed, or an unknown actor could access restricted data.

To address these issues, the researchers are taking a multifaceted approach to build robust AI. The team has a long-range vision to create the tools for human-AI teams that will ultimately enable better encryption for data integrity, protect interprocess communications, and ensure backward compatibility of AI models.

During the event’s many discussions, one in particular focused on a model called reliable reinforcement learning. CSAIL PhD student Ramya Ramakrishnan spoke of how AI systems that are trained in simulation can make errors in the real world, due to mismatches between training and execution environments.

Mistakes from an imperfectly learned model, therefore, can be risky: car accidents, incorrect diagnoses, and more can be the end result. To tackle this, Ramakrishnan is using human or oracle feedback to help reinforcement learning agents identify their blind spots, which can then lead to safer deployment of these systems.

The event concluded with a session aimed at solidifying the collaborative vision: building better models of human cognition to enable safety and robustness in real-world situations - for understanding capabilities and confidences, blind spots and biases, and resistance to adversarial attacks.

“By quantifying uncertainty, we’re getting closer to designing the novel transparent systems that can function in high-stakes environments”, says Rus. “Our goal here is to create a principled approach for robust machine learning -  in theory and practice.”