León (@LeonGuertler) header
León (@LeonGuertler) profile picture

@thinkymachines | prev. leading selfplay @xai

Palo Alto, CA
Joined October 2019
Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)Photo by León (@LeonGuertler)
Replying to @karpathy
Perfect timing, we are just about to publish TextArena. A collection of 57 text-based games (30 in the first release) including single-player, two-player and multi-player games. We tried keeping the interface similar to OpenAI gym, made it very easy to add new games, and created an online leaderboard (you can let your model compete online against other models and humans). There are still some kinks to fix up, but we are actively looking for collaborators :) If you are interested check out textarena.ai/, DM me or send an email to [email protected] Next up, the plan is to use R1 style training to create a model with super-human soft-skills (i.e. theory of mind, persuasion, deception etc.)
46
115
15
1,236
627,016
León retweeted
We got to play with @thinkymachines' new looped model. As it turns out, from an interp pov there's no reason to be intimidated by loops, the model is still full of interpretable structure and mechanisms. Go checkout our new blog post!
Looped models are weird. A layer can write to itself or read from "future" layers. @thinkymachines gave us early access to Inkling 2 Mini so we could study its inner mechanisms. We uncover how it coordinates information flow across layers, blocks, and loops. Studying Inkling 2 Mini gave us cause for optimism: as transformers continue to evolve, we believe our ability to understand their internal mechanisms can evolve with them. Read the full post: goodfire.com/research/interp…
11
14
1
166
5,617
Excited to share our latest work: Inkling 2 Mini (aka Inklklkling)! :bufo-directionally-very-good: We really took a flight to record the demo. Hope you’ll like it!
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
4
6
62
1,749
León retweeted
ive really liked the architecture and taste of all the TML models. this is exceptional even by those standards. congrats to the team
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
1
9
256
6,602
Inkling 2 Mini is a looped transformer that runs on a laptop. Looping lets a model think more and write less. At this scale, its thought process remains monitorable; this may not stay the case. We’re also sharing a variable-loop research model to enable further study.
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
294
602
130
9,412
801,383
Since July 15th, TML has been able to surpass a 975B 41BAB model with a 26.4B total 3.4B active (5.4B effective). That's uh. Using the geometric mean, 17x compute efficiency gain. If I'm reading the data section right, 50x pretrain efficiency. 128 days.
Replying to @ValsAI
Mini improved significantly from than the original Inkling, and it’s much smaller. It improves on 13 of 18 share
11
16
4
427
15,461
Loooop
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
9
23
1
288
13,461
León retweeted
Extremely bullish on American AI. Here is my (cyber) review Inkling 2 Small (Thanks @cHHillee & @brucedang ). I love the model, but I think I love the platform even more. I barely played with it and already feel how much potential it can unlock.
2
4
1
72
20,401
Today on Connectionism: how we trained Inkling 2 Small with a native 4-bit forward pass on @nvidia GB300s, from the first pretraining step through the final RL rollout, one of the many improvements across the stack that took pretraining from weeks to 68 hours.
Training with fewer bits is faster, but quantization error can interfere with learning. Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. thinkingmachines.ai/blog/end…
57
105
7
1,426
81,614
Meet Inkling 2 Small! We’ve tweaked the architecture a bit since the last generation, adding attention residuals (with a q/k/v split), gated attention, dynamic attention sink, dynamic shortconv, value norm and more. Don’t ask me “linear attention wen"
49
51
10
738
64,548
As promised, we’re releasing Inkling 2 Small’s weights, thanks to the hard work of our team and safety testing partners
134
234
29
3,212
150,954
León retweeted
Oh wait I'm reading their blog post . Finally. A solid mogging from America. Thank god
2
2
118
7,825
Our v2 small model is here. I've been using it daily for a variety of things. The pre-training ran for 68 hours on GB300s. Post-training was 17 million rollouts over 9 days. Entirely pre-mid-post trained in NVFP4. Available to train and fine-tune on Tinker right now. Open weights next week after external safety testing is fully complete (cant skimp on that). Continuing to decentralize and democratize intelligence...
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…
74
60
17
1,327
94,757
León retweeted
Our infra has really improved quite a bit since the last time we released a model - this one took just under 3 days to pretrain! Also, it's quite cheap on Tinker (0.30/1.00/1.50 for in/out/train)! Hope you guys find this model great for customization purposes :)
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…
41
36
14
821
75,475
our engineers have been busy, and we're passing on the savings down to customers. the more you buy the more you save!
Tinkerers have been busy scaling up long-context RL! We’ve made significant improvements to Tinker’s efficiency to support those, and are passing these on with price cuts up to 70%. GLM-5.3-Flash and DeepSeek-v4.1-Flash are also live for cost-efficient long-context work.
20
20
369
36,799
León retweeted
Tinkerers have been busy scaling up long-context RL! We’ve made significant improvements to Tinker’s efficiency to support those, and are passing these on with price cuts up to 70%. GLM-5.3-Flash and DeepSeek-v4.1-Flash are also live for cost-efficient long-context work.
23
39
17
404
81,032
My bet: @thinkymachines will soon make more money than @AnthropicAI. Not by winning the race to build one standardized frontier model. By becoming the Palantir FDE for enterprise custom models. The playbook: 1. Release the best American open-weight model. 2. Drive widespread enterprise adoption. 3. Charge the largest companies 7–9 figures to post-train and run custom models behind their own firewall. The model rests on three bets: 1. Large enterprises will increasingly demand their own models with their own data, and this is how they differentiate and win. 2. Enterprises won’t need just one model. They’ll continuously need new models for different workflows, departments, and proprietary datasets. That creates extremely sticky, recurring revenue. 3. Autoresearch will make custom model development increasingly scalable. Tinker can become the interface enterprises use to post-train their own models—with @thinkymachines providing the expertise and infrastructure behind it. FDE, infra, everything, huge contracts. 4. Eventually, maybe everyone wants their OWN model, and autoresearch and training inside tinker on top of @thinkymachines's base model will make it happen. Meanwhile, Henry-ford-styled, standardized models will makes no margins. OpenAI and Anthropic will have their API margins squeezed by Deepseek/GLM/Grok/Meta etc, and their consumer subscriptions are loss centers. The fat margin will move to customization: proprietary data, post-training, evals, deployment, and infrastructure. If this thesis is right, @thinkymachines isn’t building just another frontier lab. It’s building the highest-value layer between frontier research and enterprise model ownership. Turns out, the best business model for enterprise is NOT to sell commodity API access. Sell them their own models. I’m extremely bullish on this approach. @miramurati may be the most commercially savvy frontier-lab leader. I have to admit it.
182
230
93
2,838
486,153
Whoa, they buried the most interesting part of all of this: Inkling-Small: 276B A12B, coming soon. Multi-modal at that size w/ those scores if they hold thru til release is very strong (see their blog for the full list of results)
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/int… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
23
26
6
404
26,315
Inkling-small is largely comparable to the 1T version despite having just ~25% of the total parameters. thinky must've made quite a few improvements after the initial big training run. next release is going to be crazy
3
42
2,117
Inkling-Small, once finished, might be the best model in its weight class, it's basically V4-Flash but a bit smaller (and with simpler attention) and more overtrained.
2
47
2,263