Writing about and working on AI, DSPy, geo, and data. Trying to prevent prompt debt and help others build maintainable intelligent applications.
Bay Area
Joined March 2008
- Tweets16.5K
- Following1.2K
- Followers10.2K
- Likes10.2K
Updating slides for AGNTCon this week on context fails and fixes.
Each gen, there's a target model makers aim for that alleviates some problems but introduce new ones. Context poisoning, rot, confusion came because longer contexts let us fit in more info, some bad. So we embraced harnesses to manage info.
But then context inertia and fighting the weights became issues labs built models that used harnesses better. Now harnesses are adjusting to know when to 'forget' the context and manage tool loadouts.
What do you think the next gen will bring? What problems will be created by the behaviors labs are currently targeting?
Can someone who isn't selling a solution for this problem run an experiment supporting or undermining the claim that Chinese models write insecure code for specific groups?
Excited to see this land – had a great time with Jeff discussing agents in production, DSPy, RLMs, memory, and the future of context engineering.
Models perform worse as their context grows. In Context Engineering, built in partnership with @trychroma you'll learn to give agents the right information, in the right format, at the right time. Taught by @jeffreyhuber with guest speaker @dbreunig.
Enroll for free: bit.ly/4BxLVA3
Drew Breunig retweeted
Foursquare was acquired last month. The outcome was disappointing. It's taken me a while to write this, but I think I'm finally done grieving it. Here are some thoughts on our final years, an effort to honor the team who helped build & maintain FSQ's legacy, and my attempt to finally get some closure. teendrama.substack.com/p/som…
“discern unstated intentions”
This will be good for the small detains, but if it leads the model to make significant, out of scope assumptions I’m curious how this will shake out.
Source: Anthropic is preparing a model with superior intuition and reasoning that can discern unstated intentions and is better at thinking outside of the box (@steph_palazzolo / The Information)
(Visit Techmeme dot com for the link and full context!)
As the models get better and better, the thing that matters is how you capture and express intent.
The rest is hacks along the way.
This is probably why GLM 5.2 is a much better writer.
GLM 5.3’s biggest leap came from post-training alone.
“GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.”
“Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.”
“There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”
The UX vision of Anthropic and OpenAI can be summed up by what each mobile app does immediately after returning a long response.
ChatGPT scrolls to the top of the response so you can read it.
Claude scrolls to the bottom of the response so you can start replying.
You cannot hide the keyboard in Claude.app, you’re forced to read through a tiny window of 2-3 lines after you’ve entered a paragraph or two of feedback.
Credit where credit’s due though: the new doc artifact in Claude, that you can actually edit is amazing. Catches up to Codex, which for a bit was my daily driver partly because it let me edit the files and changes it proposed in app.
That Claude launched editable docs is huge for me: for awhile it felt like a quasi religious conviction over there that the only way you should work with Claude is by talking to Claude.
New context failure mode: Context Inertia 🚂
During long-horizon tasks, persistent models can hyper-focus. The solve? Throw out your context. dbreunig.com/2026/10/14/cont…
And another one! xitter.cf/Mao_Yuzhen/status/2111…
Introducing RSI-top: a framework that helps autonomous research agents explore new directions when progress stalls 🧭
On Anthropic’s kernel optimization task, RSI-top outperforms OpenEvolve and ShinkaEvolve by up to 15%. On MLE-bench Lite, it achieves a 77.3% medal rate, outperforming the strongest baseline by 11.4 percentage points.
RSI-top tackles this problem from two angles: context management and diversity control 🔍
A compact metric dashboard helps agents selectively retrieve useful past trajectories, while three diversity control mechanisms encourage fresh perspectives, new ideas, and different starting points when progress stalls.
More details in the thread below 🧵
Undeniably cool, but a show to support the main product (the rest of the product without rockets that are street legal)
The Tesla Roadster demo was a fantastic example of a vibe-coded demo in the real world.
It's the ThreeJS "videogame" someone made with only a sentence. A thin cleanroom build of iconic software. An amazing app built for three people, where maintenance has never been mentioned.
Essential viewing if you like design, geo, or just deep dive videos.
How Real are the Maps in GTA?
I went full on data science mode:
- Pulled all the maps from game files
- Counted & classified buildings
- Estimated population
- Looked at the path of the Sun and weather patterns
All to answer that simple question
Watch: youtu.be/z0i-gtwCm98
Incredible work here. These plots are wild: as the context grows the agents made smaller edits and reasoned less.
Why do agents plateau? We analyzed the agent trajectories and found that agents become more conservative as their sessions grow. On AHC014, the team's edits shrink from about 150 lines to 30 per proposal before the first forget, and its thinking tokens fall from about 28k to 7k per turn. After each forget, both jump back up.
Personal agents are going to learn what you like and find it on the internet. Your own personal algo, and this is going to change so much because it makes attention economics so wonky.
The closest historical comparison I can think of is the rise of gunpowder, when it suddenly became possible to project force beyond your reach. Suddenly strength matter dramatically less. Better equipment, more supplies, everything changed.
Agents, going out hunting for what you like, allow us to *project our attention*. It sounds less significant, but attention is the currency of our entire media, political, and consumer tech economies. So yeah, it’s big.
I’m not sure how it’s going to work out but this is why open models, open harnesses, efficient inference…all matter so much.
