RL infra at @LumaLabsAI prev @UnslothAI @axolotl_ai @awscloud @UMassAmherst small homestead owner 🐑🐓
Brimfield, MA
Joined December 2008
- Tweets5.9K
- Following2.7K
- Followers904
- Likes24.8K
buy VTI
speaking of worth:
- earlier this year i got a SemiAnalysis sub. $500/year
- i wanted to see if i was getting my money's worth, so i sampled 343 blogs across 2020–26.
- split $500 across inferred trades from those blog posts (for instance, shorting GOOGL when Dylan said their "odds of reaching SOTA again have dropped to zero")
SemiAnalysis: $927.47
S&P: $1,175.22
the Semi“Analysis” alpha is worth exactly −$247.75. you'd be better off putting your sub in the s&p.
chart below.
Dan Saunders retweeted
Our infra has really improved quite a bit since the last time we released a model - this one took just under 3 days to pretrain!
Also, it's quite cheap on Tinker (0.30/1.00/1.50 for in/out/train)! Hope you guys find this model great for customization purposes :)
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.
Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.
thinkingmachines.ai/news/ink…
lmao
I'm not even sounding the claxon for this it's just horrible.
Any text > 256chars is a valid root password, holy shit
sshhole.com/
Dan Saunders retweeted
Luma trained a 27B open model only in simulation. zero real robot data.
on a real two-arm robot it ties GPT-6 Astra 🤖
🧠 base Qwen already plans right 95% of the time, but only 36% of its moves are right
🎮 RL on 40 simple sim tasks takes that to 85%
⚡ avg 146 s vs 397 s and $0.29 vs $4.87 per task, same harness
why it matters: pretraining knows what, sim teaches how
Proud of this work!
This effort combined a lot of performance and systems work on robotics simulators with state of the art async RL methods, training small reasoners to match or beat strong closed source models.
Hoping to share more about our performance work soon 👀
Dan Saunders retweeted
Mechanistic interpretability was a huge missed opportunity. Many people wanted to join that effort, assuming it meant a mechanistic understanding of generalization. Instead, it was a bait and switch into an extremely niche, esoteric, and largely ill-conceived effort around understanding the roles of individual neurons. This has often been a challenge in alignment and interpretability: compelling initiatives that ought to be broadly defined, or at least mean what we would expect they mean, end up being restricted to a very narrow and esoteric set of topics. For example, if someone says they work on “interpretability” it does not mean what one would naively assume, e.g., interpretable methods, algorithms, representations, or model deconstruction, understanding the data, etc., but rather sparse autoencoders, influence functions, saliency maps. What a waste!
I've been saying this for ages to anyone who will listen
arxiv.org/abs/2610.40564
Using same objective from pre-training to post-training, only changing the data.
yeah my bank can barely figure out their login flow, they are ngmi
SCOOP: Major U.S. banks are detecting unauthorized third-party AI agents inside customer accounts, including some attempting to take actions that appear to be independent of user instructions, sources told Axios.
The findings have alarmed banking executives, who are racing to restrict access to their systems as AI moves at a velocity faster than financial institutions can respond, multiple industry and bank executives told Axios.
Several banks are now preparing for worst-case and even “day after” scenarios, including how to rapidly secure or restore customer accounts if things go wrong, the people said.
Several bank executives told Axios they're struggling to distinguish between human users and AI agents. One executive said banks also struggle to determine whether agents are authorized by customers or potentially operating without their permission, more akin to rogue agent behavior.
Read my latest for Axios here: axios.com/2026/10/22/ai-agen…
Dan Saunders retweeted
Exact sequence parallelism for packed multimodal masks.
It picks ring attention, a sparse all-to-all or Ulysses per layer, matches single-GPU numerics, and beats a FlashMask + Ulysses hybrid in 47 of 48 cases on up to 128 H100s.
lumalabs.ai/news/packed-mult…
100%, I staved off LLM work for a while because NLP gave me the ick
Dan Saunders retweeted
These 0.99999984 solutions are not as "ugly" as it may seem.
The truth is, those are not the real runtimes of the algorithms published. To make the proof simpler, they use absurd safety bounds like 2^50 that are entirely arbitrary.
For example, imagine proving that 1/x^2 > 0. This proof is much harder if you include the area around x=0, as it blows up to infinity there. Instead, if you only prove it for x>=99999, it avoids this blowup and simplifies the proof, but still gets the point across.
The "real" bounds of those solutions are probably much tighter, maybe even something "beautiful" like O(n*lg lg n). But the goal of the AI papers was just to show that improvement is possible, not that the bounds are perfect (which is usually much harder to prove). Over the coming weeks and months, we'll likely see a lot of tightening.
Dan Saunders retweeted
opsd and its variants don’t work, no matter what they are biased and bias kills llms, please please please stop working on it, so many other great directions to explore that aren’t opsd
Meta's team proposes DCE + SRCL as an alternative to on-policy self-distillation (OPSD).
Achieved significant gains: 30.76% avg acc -> 65.97% on Qwen3-8B
arxiv.org/abs/2609.30652
