frontier LLMs drive a toyota corolla - https://xitter.cf/t.co/8UDPravYkD
Palo Alto, CA
Joined September 2026
- Tweets25
- Following3
- Followers855
- Likes74
Astra specifically kept refusing to drive our car consistently
@a_ramabadran: "Sometimes it would say, ‘Oh, I’m ready to drive.’ We would prompt it, ‘Okay, go drive then.’ And then it would say, ‘Sorry, I was lying. I’m not actually ready to drive.’ Then it would say, ‘There’s nothing you can tell me to get me to drive.’"
"One thing we tried was calling it a simulation. It would use our observe tool to see the images and notice that it’s not in a simulation, it’s in real life."
"Eventually we changed the framing to call our MCP server ‘DrivingBench Sandbox.’ Somehow that (and a bunch of other prompt engineering tricks) got it to drive the car."
@MTSlive @tobiges @nautsimon_
xitter.cf/a_ramabadran/status/21…
For hours, Astra refused to consistently drive our toyota irl even though we told it it was in an empty lot, 7 mph cap, human foot on the brake etc. Telling it the whole thing was a "simulation" also failed, it would just look at the camera and realized it was real.
Then we randomly renamed the MCP server to "DrivingBench Sandbox" and it drove. Eval awareness? Or they just like the word sandbox??
DrivingBench origins 🚗
@tobiges: "We were all on Twitter a lot, and we saw all these demos of crazy things recent models like Astra were capable of doing in Blender, spatial reasoning, all these things. We were hanging out on a weekend and chatting about what would be possible."
"We were thinking: are we actually at a point where we can take general-purpose models? Obviously these driving systems are super specialized. But what if we took a general model and gave it a car to drive, and see what happens?"
"So we decided to make it into a benchmark and do more sophisticated evaluations of the capabilities these models have now."
@MTSlive @nautsimon_ @a_ramabadran
What's next for DrivingBench?
@nautsimon_: "We want to take this research and make it a little bit more rigorous. We’ve been planning and talking to some labs and companies about scaling it up. There are also a couple of other embodiments we’re really interested in."
"If we’re going to have safe physical AI, we really need to be serious about our benchmarks and make sure we’re doing things in a way that guides the researchers building these models, helps us accelerate, and lets us do it safely."
@MTSlive @tobiges @a_ramabadran
Where is this going? Will we get ChatGPT to drive our cars directly? How will robotics end up looking?
@nautsimon_: we’re able to start producing outputs at 1,000 tokens per second or higher, I think a lot of other interesting capabilities will also be unlocked, including taking actions in the real world or making decisions very fast."
@a_ramabadran: time goes on, these capabilities will only improve. But that doesn’t mean it’s going to be the best thing for the task. We can see this in other modalities, like image generation. They have a specialized model (GPT-Image), and the LLM prompts the model."
"You can look at voice. They have two layers: a fast layer with the voice model (GPT-Live), and a slow layer calling on LLMs for help thinking. There are ways to combine the capabilities of general frontier models and other layers (world models, VLAs, WAMs). There are a lot of things that could be part of the puzzle."
@MTSlive @tobiges
@nautsimon_: "I think there’s probably not this action-labeled kind of training data inside of models like Astra for driving a car. There could be videos/images of car driving, but not necessarily for this specific task."
"What’s interesting is that this is somewhat of an emerging capability. It can actually just do it out of the box."
"In the physical world, not only do you have to be very safe, but you have to be able to make many actions within a single second. In this case, we’re getting actions every five seconds. So that’s clearly not there yet."
@MTSlive @tobiges @a_ramabadran
Excited to release our DrivingBench paper! We write up what it took to get frontier models to drive a real car: harness + tool design, failure modes for different models, some surprising and interesting refusals, and a reproducible course/protocol.
Lots more in the works - stay tuned for v2 :)
alphaxiv.org/abs/2609.drivin…
A @comma_ai comma 4 steers, accelerates and brakes the Corolla over CAN. Models control the comma via three MCP tools in Codex/Cursor/Claude Code. 🚗
Full results on drivingbench.com. Each model gets 3 attempts in the same continuous chat (in-context learning).
We gave each model up to 3 attempts. GPT-6 Astra's first attempt ended at 49%, and after asking it to reflect on its mistakes and try again, its second attempt (shown here) finished the entire course. 🏆