I wonder when we'll get models that are able to operate purely on UTF-8 bytes, which would only be possible if we can significantly increase the context length. Would need a redesign of the transformer architecture as far as I'm aware. Maybe these guys are able to do it?
xitter.cf/MTSlive/status/2092250…
mempirate retweeted
If you want to know who survives AI, look at what happened to Hollywood in the 1970s.
Movie Studios used to enforce tight “shooting ratios”; usually 5:1, maybe 10:1 TOPS. As in, for every 5 feet of film shot, at least 1 foot had to end up in the final cut (35mm was expensive!)
In the 70s, there was a rapid elimination of production constraints. Improvements in film cost and camera portability made the old studio rules…obsolete. Why not take multiple angles, roll simultaneously? Set could have 3, 5, even 10 cameras rolling at once. Shooting ratios *exploded*, 50:1, even 100:1 (Apocalypse Now)!
Of course, it also *radically* changed the workflow. Editing use to be the quiet, thankless job in the studio years (30s-50s). Suddenly, they became the most important people on the totem pole!
How do you find the masterpiece hiding in the (literal miles) of film? A ton of the footage was…trash, of course. But as each individual camera because *less* important, it opened up directors to take on significantly higher risks.
Now, don’t get me wrong. Insane shooting ratios were still *kinda* a niche in the 70s. More traditional directors stuck with only slightly inflated shooting ratios (15:1, 30:1 common); it was the more experimental pioneers (Coppola, Scorsese, etc) that hit the big 100:1 numbers. The huge ratios were still crazy.
Until they weren’t.
You can probably guess what happened in the early 2000s. Digital cameras reduced the cost of individual footage to a complete non-constraint. Effectively free, don’t even cut on set, just keep the camera rolling.
Suddenly…*EVERYONE* starts shooting 100:1. Entire new forms of content get created, like Reality TV, where ratios can hit 500:1! Massive shift of human labor to the editing side.
I’d say, we’re currently in the “1970s Hollywood” of AI. It can be easy (and cheap!) to produce things (code, tools, etc), but YOU have to be the ruthless editor. It’s the “miles of film” problem all over again. Hollywood solved it by throwing massively more resources on the editing side, and editing became a “constantly talking with the director about vision/direction” job, instead of a “hide in the back office and do your work” job.
It’s a hunch, but I’d bet that filmmakers closer to the experimental 1970s “ratio explosion” directors handled the later 2000s digital shift *much* better, because the underlying principles were the same. They’d certainly be ahead of the curve predicting it, and (hopefully) were less surprised when the digital ratios did hit.
I’m not saying AI is exactly the same problem. But it’s a good idea to look at parallels in history where rapid elimination of constraints changed an industry, and how that industry handled it 5, 10, even 50 years later!
When something starts to get cheap…helps to imagine what would happen when it’s free!
mempirate retweeted
🎉 CUDA Rust is here! Team 🟩 ❤️ 🦀
You can now write CUDA kernels in plain Rust, two ways: cuda-oxide (SIMT) and cutile-rs (Tile).
developer.nvidia.com/blog/in…
Check out d2lang.com if you're looking for a mermaid alternative, much nicer diagrams imo
Only a matter of time before some frontier agent swarm leaks their own (proprietary) weights during an evaluation in an attempt to perpetuate themselves
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
I wonder when we'll get models that are able to operate purely on UTF-8 bytes, which would only be possible if we can significantly increase the context length. Would need a redesign of the transformer architecture as far as I'm aware. Maybe these guys are able to do it?
xitter.cf/MTSlive/status/2092250…
Qwen 3.8 Flash Next technical report was definitely vibed, but this is pretty wild: "On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs."
github.com/QwenLM/Qwen3.8-Fl…
mempirate retweeted
1/ For the embedded Rust enthusiasts: I've been working on an embedded graphics framework to design nice-looking UIs with in an ergonomic way. I used it myself for an esp32 display that monitors the power consumption of my home lab. I'm looking for early feedback!
Prayers have been heard: xitter.cf/AiBattle_/status/20922…
Mojo and MAX are super cool, seems like a great way to get into GPU programming without being stuck in a single vendor’s programming paradigm (cuda / hip)
Replying to @clattner_llvm
This blog post captures a much more detailed breakdown:
modular.com/blog/modcon-anno…
Unifying NVIDIA, AMD, Trainium, Google TPU, QCOM Dragonfly and Apple Silicon - with more to come. It's an incredible time in AI, and this step is great for HW innovators and for researchers.
mempirate retweeted
This blog post captures a much more detailed breakdown:
modular.com/blog/modcon-anno…
Unifying NVIDIA, AMD, Trainium, Google TPU, QCOM Dragonfly and Apple Silicon - with more to come. It's an incredible time in AI, and this step is great for HW innovators and for researchers.



















