Inside the Brain of a Self-Driving Car: How Motional and MIT are Cracking Open the Black Box of AI

Imagine you are a driver behind the wheel of a car with self-driving capabilities, or a passenger in a robotaxi you hailed, or an engineer testing an autonomous driving system that you’re developing. The road is clear, the weather is perfect, and the ride is smooth. Suddenly, the car hits the brakes and comes to an unexpected halt.

Why did it stop? Is there a hazard the car detected but you don't see? Did the system malfunction?

This communication breakdown, where a vehicle's artificial intelligence (AI) makes a sudden decision without explaining why has been at the center of questions and skepticism surrounding self-driving technology. The AI in today’s self-driving cars increasingly relies on complex artificial intelligence; neural networks trained on massive amounts of driving data. Inspired by real biological neural networks in the brain, artificial neural networks can be incredibly capable. However, they are not readily available for introspection, which is why they’re often referred to as “black boxes.”

But what if your car could talk back? What if it could tell you exactly what it was thinking?

In a new paper, recently published in Nature, engineers from Motional, including CEO Laura Major, along with researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have proposed a new approach for making such black-box deep learning systems more interpretable: the Concept-Wrapper Network (CW-Net).

Understanding What Autonomous Vehicles are Thinking

Instead of relying on human drivers or engineers to guess what a neural network is calculating, CW-Net translates the self-driving system’s internal logic into human-friendly concepts. For example, as an autonomous driving system perceives the roadway around it, CW-Net could help by displaying a dashboard with various concept occurrences that are driving its decisions in real time, such as “Approaching Stopped Vehicle” or “Close to Cyclist.”

Crucially, these aren't just educated guesses generated after the fact. The explanations are directly determining driving behavior, meaning the vehicle’s final decision-making system only takes action based on these human-interpretable concepts. If the car brakes, it’s because a given concept triggered it, and it can tell you what that concept was. In that sense, the explanations are causally faithful, in contrast to alternative approaches that provide explanations in natural language which are understandable but may not be accurate. 

From the Lab to the Las Vegas Strip

Researchers have tried to make the AI in self-driving cars interpretable for many years; however, most research on so-called explainable AI has historically been confined to computer simulations in lab settings. The Motional and MIT team broke that mold, deploying CW-Net on an autonomous vehicle (AV), with an experienced safety operator in the driver’s seat, to collect data while operating on a private test track and on public roads around Las Vegas. We used an earlier experimental version of our deep-learning-based planning system that shows competitive performance but has notable gaps, which CW-Net could help explain.

These tests yielded insights into how CW-Net can help people better interpret the driving behaviors of autonomous systems in the real world. In one instance, the AV repeatedly stopped near a traffic cone. The vehicle operator assumed the cone was the problem. When researchers removed the cone, the car still stopped. CW-Net’s display revealed the truth: the experimental planning system was actually “hallucinating” a stopped vehicle ahead based on its past training data. The explanation allowed the researchers to more easily understand, predict, and ultimately resolve the car’s quirk.

In another test involving a cyclist, CW-Net revealed that, even though the AV was properly detecting and stopping for the cyclist, the experimental planning system was not, in fact, basing its driving decisions on it. Upon noticing this, the safety driver began exercising a higher level of caution around the cyclist. Follow-up analyses revealed that this level of caution was warranted: the AV was braking due to a safety backup system, not the experimental deep-learning-based planner.

Safer Without Compromise

Naïvely introducing layers of explainability into AI can degrade its speed and performance. But when researchers benchmarked CW-Net against top autonomous driving algorithms, the difference in driving capability was less than 1%. This highlights a key strength of this approach: it provides explainability without sacrificing performance.

As AVs continue to scale globally and regulators request more transparency about AI systems, technology like CW-Net could transition from research to a requirement.

Looking ahead

Our work shows that explainable deep learning can help people better understand and anticipate the behavior of self-driving systems in the real world. But the challenge extends far beyond AVs: as deep learning increasingly powers safety-critical systems, from autonomous drones to robotic surgery, operators and developers will need ways to understand their capabilities, limitations, and unexpected behavior. We hope CW-Net provides a step toward autonomous systems that are not only more capable, but also more transparent, predictable, and ultimately safer to integrate into our lives.

Citation

If you find this work useful in your research, please consider citing:

@article{kenny2026explainable, title={Explainable deep learning improves human mental models of self-driving cars}, author={Kenny, Eoin M. and Dharmavaram, Akshay and Lee, Sang Uk and Phan-Minh, Tung and Rajesh, Shreyas and Hu, Yunqing and Major, Laura and Tomov, Momchil S. and Shah, Julie A.}, journal={Nature}, year={2026}}

Link: Project Website