Chandler (@stewchan2) profile picture

Building an AI harness that generates game worlds in Unreal Engine from a single prompt. Daily updates. UIUC Alum ex-AWS

Joined July 2024
Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)Photo by Chandler (@stewchan2)
An entire city in 7 minutes. #buildinpublic day 13 unreal engine harness hot-reload Brackenford: a whole market town, 4,862 pieces, planned by an AI agent from a short story and built in 7 min 33 s. Here the build replays around the runner, then the streets fill with 143 townsfolk. Unreal 5 on an RTX 4060.
20
19
4
219
13,741
A bot sailing a boat by the wind past a harbour town. #buildinpublic day 18 The world, the town and its piers were all built by our agent from a written story, and rendered in Unreal Engine in a woodblock print style. Tons of updates to my harness recently, small little video to stay consistent X posts
5
10
365
I gave my world builder two spots on the map. It rebuilt both towns as game worlds in Unreal. #buildinpublic day 17 Hallstatt, Austria and Estes Park, Colorado: real elevation, real streets and building footprints from OpenStreetMap, real forests from satellite land cover. The pipeline does the rest. No hand edits.
11
23
1,092
One short story in, a playable Unreal village out. #buildinpublic day 16 The agent built the world and the game rules (find the planks, rebuild the cottage) in 3 minutes, 33 seconds. No hand-placed assets.
6
26
1,155
I typed a story and got back a full pixel-art platformer in 2 minutes. #buildinpublic day 15 of building an AI game engine harness Claude planned 5 levels and a hub, then picked one free art pack to cover the hero, tiles, backdrop, enemies and bosses. The build took 1 second. A bot then played every level through to the end. The clock is real: 2:09 from story to a played-through game. The build is slowed down in the video so you can watch it. Art: Warped Caves Pixel Art Pack by ansimuz (CC0)
4
6
307
My last post (nitter.gq/stewchan2/status/21159…) got way more attention than I expected, and a lot of you asked for access. Its not ready, and here is why. #buildinpublic day 14 Tech showcases only show the good takes, so here’s what the same town looks like when you actually walk around it and look for the issues. You can walk on the river. The river just ends. Trees have invisible walls. Also, Claude made this QC video and somehow put two mannequins in the same spot. Even the bug report has bugs.
An entire city in 7 minutes. #buildinpublic day 13 unreal engine harness hot-reload Brackenford: a whole market town, 4,862 pieces, planned by an AI agent from a short story and built in 7 min 33 s. Here the build replays around the runner, then the streets fill with 143 townsfolk. Unreal 5 on an RTX 4060.
2
6
423
This is getting awesome. #buildinpublic day 12 Building a world editor where you just ask. Sky and weather edits hit my Unreal window in 2-5 s. Land edits show a rough shape in seconds, then the full eroded terrain settles in once the rebuild lands. One RTX 4060, no editor clicks.
10
14
760
One story prompt → a whole medieval market town in 6 min 19 s. Agentic unreal harness. 54 buildings, each built inside and out: rooms, furniture, doors that open. 4,793 pieces checked in place, none off. The clock is real build time from one fresh run, no hand fixes. Rendered in Unreal Engine 5. #buildinpublic day 11
1
7
549
"Make the waterfall taller." One sentence becomes one change to the world's plan. The land rebuilds in 5.5 s, the walkable patch around the falls in 5 min 27 s on 4 CPU cores. The fall goes from 156 m to 216 m; the valley, river and peaks stay where they were. Same camera, both rendered in Unreal Engine 5.8. Nothing edited by hand. #buildinpublic day 10
4
399
This harness is getting crazy! #buildinpublic day 9 The land rises in under 2 s. The walkable patch around the waterfall (forest, river, a 134 m fall, a gorge of scanned rock) takes 5 min 23 s on 4 CPU cores. The clock on screen is the real measured time. Rendered in Unreal Engine 5.8 on one RTX 4060. Nothing placed by hand. Rendering time not represented.
8
1
30
2,663
One generated city. Five moods. #buildinpublic day 8 Same downtown, same camera. Each mood is a single line in the world plan ("sky": "dusk"), rendered in Unreal Engine 5 on one RTX 4060. Our city planner laid out every block from Epic's free City Sample buildings. Nothing placed by hand.
1
1
268
Yesterday I said this was the new 3D asset meta. So I tested it. My own agent rebuilt their cathedral, then a building it had never seen, with every round scored against the picture part by part. One picture in, a real 3D model out:
Using a single Nano Banana image and Blender MCP, Gemini 4 Argon built a 3D Gothic cathedral in @Antigravity. It iterates on the geometry, textures, and lighting to create the finished scene.
2
1
1
5
490
What's not there yet: less carving than DeepMind's cathedral, simpler stone, and the back sides are guesses because the picture can't show them. The segmenter is good on doors and chimneys, weak on thin timber.
1
30
Why I care: I'm building a pipeline that turns a story into a playable world. Buildings are the gap image-to-3D can't fill. One picture in, a matching model out, with a score you can trust. Full method in the article: nitter.gq/stewchan2/status/21140…
27