Cofounder @theworldlabs. Building Spatial Intelligence.
Joined January 2014
- Tweets442
- Following642
- Followers22.9K
- Likes3.9K
Pinned Tweet
Today we sharing Atlas, our new multimodal world model.
One reason this model is so special to me is that it combines two core visual intelligence tasks I have worked on for over a decade: generation and reconstruction
World Labs is joining @AMD!
Over the past few years our incredible team has pushed the boundaries of spatial intelligence through models like Marble, RTFM, and now Atlas.
By partnering with AMD we will train bigger and better models than ever before, and build out an ecosystem of open models. I can’t wait to get started!
We are excited to announce that World Labs is joining @AMD.
The research and technical breakthroughs we have achieved since our founding in 2024 have given us a clear vision for AI’s potential to solve problems in the spatial and physical world.
Accelerating the future of spatial and physical intelligence requires scaling our efforts, scaling our reach, and getting closer to the hardware.
RT @HaoZhang623: World Tracing is a NeurIPS 2026 Spotlight! 🎉 It’s also our first NeurIPS paper at World Labs. Grateful to everyone who wor…
Justin Johnson retweeted
From 32 input images to real-time flight through @nvidia's Voyager headquarters.
Trained on NVIDIA Blackwell GPUs, Atlas uses these images as 3D spatial context to generate new views, letting you explore with pixel-perfect camera control.
Take a look around.
Wow, I'm honored and humbled that our 2016 perceptual losses paper with @AlexAlahi and @drfeifei was recognized with a Test of Time Award by @eccvconf!
It blows my mind that this is still a core ingredient in many modern pipelines, including the VAEs used in diffusion models
Our new Atlas model now runs real-time after some intense inference optimization! Generating and navigating worlds is so much more visceral when it's interactive.
Inference runs on both B200 and MI355X with similar performance. Both devices are incredible workhorses!
Justin Johnson retweeted
This is sooo cool! -- I visited an airplane museum on the weekend and took some photos walking around with my phone. Now I can make a "drone shot" afterwards using Atlas!
This Atlas generation surprised me -- it's way outside the training distribution, but still does a very reasonable job! This is what emergent capabilities look like for a world model
ok this one was challenging! i guess SEM has an orthographic camera model, ended up getting posed as a very narrow FOV perspective cam and having some alignment errors as a result.
if there was calibration data available for the ray bundles it could potentially help (though the model was definitely never trained on data like this!)
One question that came up on stream was whether Atlas can handle black holes ... turns out it can!
This probably isn't physically accurate, but I'm surprised it works this well
Since Atlas can handle both generation and reconstruction in a unified framework, it can combine real-world spaces with fully generated ones, all with full 3D control. The creative possibilities are endless!
one of my favorite things about Atlas is its ability to blend reconstructing the real-world with generating imaginative worlds. the possibilities are endless.
so proud of our team, can't wait to have everyone try it out in the next generation of @theworldlabs 's marble :)
I had a blast talking about world models on @twimlai - I'd love to come back and tell you all about our latest one!
In this episode, @jcjohnss, co-founder of @theworldlabs, joins us to discuss world models and the emerging field of spatial AI. We explore why many researchers see capabilities beyond language as an important frontier for AI, and what it means to build models that can understand, generate, and simulate the environments around them.
Justin explains the different approaches to world modeling, including explicit 3D representations and generative models, and why there is still no established recipe for building these systems. We also discuss World Labs’ Marble system, which can generate navigable 3D worlds from images and other inputs, the challenges of evaluating world models, and the role of simulation, planning, and action. Finally, Justin shares his vision for models that bring these capabilities together, supporting everything from interactive virtual environments to agents and robots that can operate in the physical world.
🗒️ Full show notes: twimlai.com/go/775.
📖 CHAPTERS
===============================
00:00 - Introduction
03:19 - Defining World Models
08:10 - World Models as Theory Builders
11:37 - POMDPs and Agent–World Interaction
15:40 - Training Agents with Behavior Cloning
18:40 - Ground-Truth State and Learned State
23:34 - Explicit 3D vs. Implicit 3D
28:14 - Reconstruction vs. Generative World Modeling
30:38 - Gaussian Splat Anatomy and File Formats
33:52 - Why Gaussian Splats Work with Neural Networks
37:00 - Consistency by Construction and at Scale
40:31 - How Marble Generates 3D Worlds
44:25 - Training Data and Output Representations
47:48 - World Models as Renderers, Planners, and Simulators
51:30 - When Rendering and Simulation Overlap
56:28 - The Path to Unified World Models
59:30 - Architectures, Loss Functions, and Long Contexts
01:02:47 - Where to Learn More
Justin Johnson retweeted
One cool use for Atlas is creating "bullet time"-style effects using only a few cameras.
In this case, I was able to use 3 iPhones + my finely honed juggling skillz to synthesize a smooth camera trajectory between each pair of cameras.
Also I layered in an AI-generated beats because that's just the world we live in now. Deal with it.
Justin Johnson retweeted
I used to play myst a lot when I was younger. something with atlas gives me a similar feeling of moving through these worlds 🌎
Atlas is such a versatile model; it handles so many different modalities, tasks, and visual aesthetics that can be re-combined in different ways for so many creative effects
I’m so excited to finally talk about our new model, Atlas, and share some generations from it..so excited, in fact, that I made a fan trailer for it 😆
It’s an incredibly versatile, multi-task model with a wide range of capabilities. There are many powerful video models out there right now, but after working with Atlas and experimenting with it for so long, I’ve come to really appreciate its elegance and malleability...all rooted in the framework’s simple autoregressive design.
As you can see in this video, Atlas is, at a high level, a camera-conditioned visual generation model. But its formulation allows it to learn and perform a variety of tasks remarkably efficiently!