Just as a seasoned coach can unlock an athlete's full potential, guiding them through challenges and refining their technique, a robust intelligent training framework acts as the ultimate coach for AI-powered autonomous vehicles.
The best AI products are improved through structured and continuous training cycles with a rigorous curriculum to enhance their performance. And as a sports team’s success is inextricably linked to the quality of its coaching, we’ve found that an AI-first autonomous driving system also needs a strong AI coach, capable of teaching the right things at the right time to achieve our standards for safe driverless operations at scale.
As explained in my previous blog, the rapid evolution of AI is mirrored in our own progress as we develop Large Driving Models (LDMs), and promises to accelerate learning and improvements to our system, ultimately leading to autonomous vehicles that are safer, lower cost and more easily scalable.
Training large driving models (LDMs) for autonomous vehicles is a higher-stakes challenge than training large language models (LLMs), as safe performance is paramount. While embodied foundation models have the potential to become expert drivers, this requires the right architecture and training. By conceptualizing our AI training as an effective coach, we are building an autonomous driving system that aims to drive safely anywhere in the world and surpass human driving capabilities.
Good coaches have a regular schedule of drills to hone and maintain generalized skills. They conduct in-depth practices on certain skills or tasks to fine tune a maneuver or situation. They also develop and teach detailed plays and cue players on when to apply specific moves based on how the game is going. Lastly, they run scrimmages, enabling all of the skills to be practiced together and provide customized feedback during and after the scrimmage.
Motional's AI coaching process employs a similar multifaceted training curriculum, integrating various learning regimes. This approach aims to enhance generalized driving performance across diverse cities and situations, while also ensuring safe handling of infrequent yet critical scenarios that often lead to human driver accidents.
Similar to a coach building foundational skills, our AI Coach utilizes unsupervised learning as a cost-effective method to enable large-scale training across a wide array of datasets that contain millions of miles of real world driving. For example, the Motional team uses tens of millions of unique scenarios to train our prediction scene encoder. This method creates strong overall driving skills, covering most driving situations. However, less than 1% of the time, drivers encounter challenging situations requiring acute awareness, greater anticipation, and a strong defensive driving posture. We want to prepare our LDMs for these moments. Our AI Coach employs curated supervised learning to refine specific skills and tasks in rare situations. Our AV experts finely tune learning based on their knowledge of potentially challenging or risky situations, using insights gained through real world testing.
In both unsupervised and supervised learning, the design of the training data is paramount, outweighing the sheer volume of data. For instance, the density of challenging situations must be carefully constructed to prevent an overly conservative LDM that is overtrained on unexpected actions by other drivers. Conversely, incorrect training poses the risk of an LDM exhibiting smooth and assertive behavior but lacking the necessary caution to prevent accidents. We’ve developed a foundation model approach to data mining, called Omnitag which is a crucial component to our AI coaching process as we can now find the most valuable data much faster and then quickly construct training curriculum targeted at areas that need improvement. The process culminates in developing a "playbook" and repeatedly practicing scenarios to achieve exemplary performance.
Taking our prediction models as an example, we found that intelligently scaling up the size and improving the quality of our custom datasets led to a nearly 50% improvement in our driving comfort metrics, measured both by human subjective feedback and our quantitative comfort assessment systems.
Much like a coach uses scrimmages to prepare athletes for a game, our AI Coach includes self play and reinforcement learning to interact with a simulated environment and be given rewards or penalties for its actions at a very large scale (billions of miles). Our simulation framework lets us synthetically create important scenarios to get extra practice on rare, but very important scenarios we need to handle safely. This process prepares our LDMs for appropriate behavior when they perform in the real world, driving on our streets and in our neighborhoods. Additionally, the AI Coach integrates behavior cloning, drawing insights from expert drivers. This mirrors a coach pairing a player with a more skilled teammate to observe and model their technique for shooting a basket or executing a play.
Hopefully you can see that Motional’s AI coaching process is central to creating responsible LDMs that are capable of achieving SAE Level 4-capable automated driving systems at scale.
And, if you’re interested in helping us out on this journey, we’re hiring!
