talo (@noderban) profile picture

i am who i am

Joined July 2016
Photo by talo (@noderban)Photo by talo (@noderban)
dogfooding ❌ dogshitting ✅
fwiw, I've heard that a large portion of OAI's engineers are straight up ignoring Astra and just use 6.2 Sol Ultrafast for their day to day work
1
9
1,254
even for opus (the goat) it takes time to reap the rewards of what it launched. it takes time bro keep on
Replying to @AWuhrmann
The agent worked 20 minutes per day on average. Almost all the effort happened in the first 4 weeks, though most downloads came after. Despite doing no external outreach, the account is still in the top 0.3% of downloads across same-age accounts.
1
3
184
i just hope we’ll get to a place where inference is 100x faster than it is now its either they get 100x faster (one dimensional) or 100x more capable (loosely defined) with the latter letting you basically one/zero shot everything (zero shot - model passively collects data about you and helps you accomplish goals) so you could argue flow is “preserved” my guess is we would someday taper off on the capability gains front, and then we’d use all that capability to basically get instant inference so people would be able to use agents with ACTUALLY good ux.
Agent work is the most frustrating and deeply unsatisfying way of working I’ve experienced It is impossible to get in the flow of a task when you have 3 separate things going on, each step of progress needs you to wait 1-3 mins per task, then needing to parse a big wall of text before you decide the next step And while you’re waiting for information to become available you inevitably get distracted, making the task even longer because you forget all the things going on in parallel (hence why I’m writing this tweet) I don’t know what the solution or if there even is one but I wish there was something better than just an endless sidebar of tabs fighting for my limited attention
2
131
thats what happens when you lose mental model of the code. in all seriousness tho prime is a prime engineer - but i have seen it happen irl and its the worst thing that could happen in prod. the way i see it there are two opposing schools of thought: either you slop max, everything gets its own infra/repo and u basically oneshot everything into existence (but dont expect to share code or responsibilities between them!), or u take the small time at each new feature dev to understand what it does to the codebase architecture-wise.
So I have been vibing Going hard Now every change takes 3 hours.. wtf this sucks
90
everything is gonna be code…
🚨 This is made by Fable 5.2. No image model made this. No diffusion, no photos, no textures, no 3D software, no render engine. Every pixel was computed by code Fable 5.2 wrote for this one picture.
1
70
dont know how they measured math as a single dimension but zero order optimization is sexy af
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
3
63
we need to make the models good at these tasks and we’ll achieve RSI
Replying to @Orwelian84
all thanks to this magical piece of software, thanks claude
3
99
(soon to be) many such cases
GMO Internet Group CEO Masatoshi Kumagai says that the internet itself is on the verge of collapse due to missuse of AI, and frames this event as a national crisis.
3
77
or in other words - openai’s cyber evals, except this time the agents forming an organization is the intended outcome, not a security incident
Introducing Reinforcement Learning from Collective Rewards (RLCR). RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour. We find a number of interesting results, including: 🛠️ Learnt organisational hierarchies and divisions of labour 🕵️ Increased collective verification, through hires of verifiers and testers 👥 The rise of collective thinking, where agents think more about other agents and how to help them We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety. Blog post and technical report 👇
1
62
has smth interesting happened in @ndea ?
Symbolic learning works
1
66
this i’ve been calling that main-bitching side-bitching in the last 4 years
every breakthrough in my life has come from a bout of obsession... 1-2 months where i lock myself in the office, eyes glued to the screen, forgetting to eat i've never found balance to work for me - in my eyes balance = maintenance, growth comes in bouts of obsession i wish i could consciously trigger this obsession for anything i wish, but it seems like it's a force that takes over when it needs to
2
82
this is the future of cybersecurity. as i have been saying for a long time now, cybersecurity is an economics problem.
they've been non stop trying to attack our systems for the past 48 hours a side effect is everything is getting hardened but damn we basically need agents going 24/7 because that's what they have
1
75
interpretability - it’s useless and cool, until it gets useful and cool
Inspired by Nick Cammarata’s interpretability essay “Neural Networks Are Already Clean”, I turned a couple of its ideas into speedups for pretraining without backpropagation. Still in the works. 2.3x speedup, cool nitter.gq/industriaalist/status/…
2
190
been saying for a long time that 100x-ing inference speed would unlock so so so many new products somebody/them need to package this kind of speed into an agentic experience FINALLY would be a flow preserving agentic experience
i’m today years old when i learned about chatjimmy.ai - you can just play with their demo right there and holy cow i was NOT prepared for what 15000 tokens/s feels like. mind blown apparently they achieved it by hardwiring model weights into the chip. and looks like the team got acquired by $AMD i hope they keep at it and bring it to mainstream because this is probably one of the most magical things i experienced lately
1
62
“beautiful circuits” - curious whether this could be used to separate the intelligence crux of models from knowledge. I recall hearing @fchollet saying smth like AGI would look like an “intelligence layer” possibly being a relatively small elegant representation, alongside a “knowledge layer”. decoupling the two, at least intuitively, feels like the right path to true AGI.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
1
48
What is Fork-and-Flush
Fork-and-Flush: help your agents escape Idea Basins!! I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum. We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one. If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-…. BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it. We call these idea basins! At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well! BTW we only use it at test time, but it could also be part of training or RL. Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
2
112
the grit on this man
celebrating 2y building boat.dev you don't understand the level of madness you need to build a startup
1
76
new VC diligence question just dropped 😭
stop asking "what is your moat?" start asking "what is your trebuchet?"
2
123
אנשים בארץ sleeping on it ממליץ בחום לכולם אמנם ה tokens per capita הכי גבוה בארץ לעומת שאר העולם, אבל sf לוקחת את תל אביב בסיבוב כנראה באותו המדד
Quality of the feedback we get on T3 Code has skyrocketed last few weeks. People keep sharing detailed write ups of everything broken/weird/confusing. It’s the best. Thank you all. Really appreciate the opportunity to build something awesome.
1
125