Smart Choices: How Robotaxis Partner with Remote Vehicle Operators to Work Through Tricky Spots Safely

The more we drive, the more we learn from all that time behind the wheel. Then, when we run into an unusual situation – the road is blocked by a downed tree, another driver opens their door and is blocking the thru lane – we can pull from that internal database of personal driving experience to figure out how to handle the situation safely. 

Robotaxis are already highly advanced drivers, capable of competently handling most of what they encounter while operating on public roadways. However, like any driver, robotaxis will sometimes encounter unique scenarios they haven’t experienced before. 

Whenever uncertain about how to proceed through a situation safely and legally, Motional robotaxis can connect to trained human agents capable of working with the AV to maneuver through a situation safely, legally, and quickly. 

“Coming out of the gate, there are going to be certain cases the AV doesn’t have the training to handle, so having a human on the backend to help is important,” said Obinna Asinugo, a senior product manager leading the development of Motional’s Remote Vehicle Assistance feature, also called RVA. 

But RVA agents aren’t remotely driving or controlling the AV. Rather, they’re working hand-in-glove with the robotaxi’s onboard autonomous driving software to review and verify the best way to proceed. This system of digital checks and balances ensures the vehicle learns how to figure out tricky situations, is able to proceed carefully and safely, while storing the information away to learn from later. 

MANAGING UNCERTAINTIES

Human drivers aren’t born knowing the rules of the road, or with the confidence needed to handle any driving situation they come across. That expertise is built up over years of experience behind the wheel. 

But between unpredictable human behavior, sudden severe weather, miles of underground and roadside infrastructure, and other macro-factors, sometimes things happen that flummox even the most experienced human drivers. Pedestrians step out between cars. Trees fall across roads. Broken water lines flood streets. Emergency crews block lanes. Any of these occurrences can give an experienced driver pause. 

Robotaxis are carefully programmed to handle 99% of the routine driving situations they come across. That other 1% – construction debris blowing across a roadway, a vehicle illegally parked across traveling lanes – is tricky and comes with a learning curve. 

“Safety is our top priority. That means any moves made by our robotaxis need to be safe for those inside and outside the vehicle,” said Motional’s Chief Operating Officer Abe Ghabra. “RVA helps Motional’s robotaxis navigate through complex scenarios efficiently, follow the rules of the roads, and maintain a high level of safety.”

WORKING TOGETHER

When a Motional robotaxi encounters a situation it isn’t sure about, it doesn’t press forward. It prioritizes safety and comes to a safe stop.

The RVA team back in Motional’s dispatch center kicks into action when the vehicle stops and doesn’t resume forward.

There can be many reasons why a vehicle isn’t sure how to proceed, Asinugo said. A sensor could be blocked, for example, or the only available path forward would break a traffic rule, like crossing over a lane divider – something the vehicle is programmed not to do. 

“Most of the time when it’s stopped, it’s just confused by the environment and trying to process which option to take,” he said.

The RVA agent, who is a highly trained Motional employee, doesn’t take control of the vehicle remotely and maneuver it to safety. Instead, the robotaxi is looking to the RVA agent for guidance. It’s already generated multiple solutions – it just wants to double-check its math.

“The robotaxi is essentially saying ‘I have options but I’m confused which one to take,’” said Asinugo. “It’s thinking ‘I would prefer to have human input into whether this is the best action, and if it is safe, I will go with that.’”

The robotaxi knows not to accept any suggested maneuver that breaks driving rules. It won’t drive up and over a curb, for example, or cut across a parking lot.

Before it takes off, the robotaxi also checks to make sure the maneuver is safe. It only takes seconds for an RVA agent to draw a digital path forward, but the driving environment could have changed in that time – a pedestrian stepped off a curb, a vehicle showed up in the periphery – and the planned movement is now unsafe. If that happens, the vehicle and RVA agent simply come up with new solutions. 

WORKING SMARTER

Like human drivers, AVs improve with experience. Motional’s robotaxis use machine learning principles to get smarter each mile they drive. Each unusual scenario the vehicles encounter provides data that can be used to train and fine-tune the technology. And every time a robotaxi engages RVA, it builds up a database it can learn from. This information can be shared across the fleet, allowing robotaxis to learn from each other. The AVs also learn by running through simulated scenarios with the RVA agents. 

Ultimately, Asinugo said, the goal is that AVs will accumulate enough learned experience that they can require less reliance on a support feature like RVA.

BUILDING CONFIDENCE

Passengers may be understanding that robotaxis need occasional assistance navigating tricky situations – but not if it means interrupting their trip for several minutes, Asinugo said.  

“The goal is to have RVA agents assist vehicles in a matter of seconds, and for the whole experience to feel seamless, to the point where passengers may not even realize that the vehicle connected with RVA,” he said.  

Motional robotaxis are outfitted with 4G cellular connections that are regularly monitored for signal strength, and the RVA center is staffed whenever vehicles are on the road. A particularly tricky scenario may require the vehicle to stop for a minute. 

While passengers get comfortable with the idea of riding in a car with no driver, Asinugo said RVA provides a sense of confidence, even if the human RVA agents are essentially just checking the vehicle’s decision-making. 

“Passengers may think ‘I’m in a self-driving car. There’s nobody else in here. If something was to happen, I’m glad to know someone has my back,’” he said.