Turning Reality into Action.
San Francisco, CA
Joined August 2025
- Tweets16
- Following88
- Followers12
- Likes6
Frontier VLMs have a reality gap: Of 30 models, they now match a human rater at judging physical work, but read a tape measure right only 36% of the time.
We need a benchmark for reality. Introducing RealityBench: 3D worlds you can bend 馃У
Does a simulated world predict the real one? On real photos of tape measures and damaged cars, model rankings track the simulator's.
The worlds are simulators, not image sets: built to extend to new tasks. Early access is on Hugging Face and relantic.com/realitybench Check out our video below.
Relantic retweeted
The pace of improvement has been shocking. I half-expected that the focus on coding and RSI would mean that progress on vision had slowed. Instead, it seems likely that VLMs will be clearly superhuman in performance and cost in the next year or so.
Relantic retweeted
Request early access to environments and images here: relantic.com/realitybench. Paper: arxiv.org/abs/2610.14397. Joint work with @arnabdotorg, @anmoltakiar and Dominik Winecki at @relanticai.
Relantic retweeted
We created simulated environments in Blender for benchmarking VLMs on real world physical work perception and reasoning tasks. Think construction and maintenance of houses and cars.