Youwang Kim

For 3D learning, the “Bitter Lesson” doesn’t properly respect Kolmogorov complexity. The colloquial version of the bitter lesson is that, in a nutshell, just scaling up data and compute always beats problem-specific modeling (note that this is not actually what Sutton said; more on that later). But look at the current situation with LLMs and agents generating 3D content: the really impressive examples depend on configuring an API, language, or software package (Three.js, OpenSCAD, Blender, etc.). Each representation impacts the description size for a given object—and for the classes of object that can be faithfully represented via one of these procedural descriptions, the encoding is vastly more efficient than an explicit description (points, splats, polygons, etc.) or even an implicit description (SDF, NeRF, etc.). Moreover, the smaller you can make the procedural description, the better the LLM works: it has to predict fewer tokens to describe the same object, because each token does a better job of compressing geometric information. E.g., the words “unit sphere” are far more compact than a list of points or a collection of voxels. So, in the procedural setting, building models for predicting shape becomes an exercise in bounding Kolmogorov complexity: what’s the smallest “program” I can use to describe a given object? (Or more precisely: which language minimizes description length *and* the entropy of the description, for a model trained on that language.) That’s why current tools lean on carefully crafted abstractions (again, Blender, OpenSCAD, Three.js, etc.), for which many examples are available. Astra doesn’t run off and build you a giant neural network, hoping that the “right representation” will somehow be learned deep within the network weights. None of this, by the way, contradicts what Rich Sutton actually said: the input/output encoding matters, even if the rest is just scaling up search and learning. And the reason we see examples in Three.js and not a more esoteric language is that there is a large corpus of examples to learn from. But the idea that the network will just “learn the best representation” by adjusting its weights is a misguided fiction. Instead, it really seems like we are entering an era of “code as geometry” counterposed to WorldLabs’ “geometry as code”, where design of domain-specific languages is a key task. (And of course, search and learning will inevitably play a role in this design…) Computer Graphics also plays a central role, because it’s the mechanism for decompressing the procedural description into an explicit representation (building a cylinder, tessellating a curve, etc.). That’s the tool calling part of the agentic process. And the better these graphics algorithms get, the more effective the language becomes. People are worried about the death of Computer Vision—but it seems Graphics is coming back from the grave. 🧟‍♂️ (And, might I add, that differential geometry is a very expressive language that few are taking advantage of in this setting. 😉)

Here's a nice "proof without words": The sum of the squares of several positive values can never be bigger than the square of their sum. This picture helps make sense of how ℓ₁ and ℓ₂ norms regularize and sparsify solutions (resp.). [1/n]