hope and optimism | ex-Ando, Columbia CS

Joined January 2021
Replying to @Google
cool now i can watch myself miss the exit in cinematic detail
2
14
2
915
36,523
rex multiplexer is so fucking fast 💨💨💨
60
Peter Choi retweeted
I’m thrilled to release this course on Context Engineering!
Models perform worse as their context grows. In Context Engineering, built in partnership with @trychroma you'll learn to give agents the right information, in the right format, at the right time. Taught by @jeffreyhuber with guest speaker @dbreunig. Enroll for free: bit.ly/4BxLVA3
3
5
3
86
8,305
Peter Choi retweeted
iiiiiiIIIT'S TIIIIME we proudly present: solidSF v1.0 hardware engineers, this one is for you a platform of platforms, amplifying or supporting all of the tools you love and hate folded into a clean mesh of cohesive hardware engineering excellence so what can it do?
19
15
4
209
232,547
Peter Choi retweeted
Introducing GPT-Live, a new generation of voice models for natural human-AI interaction. Rolling out in ChatGPT starting today. You’ll want to turn the sound on for this one.
1,248
2,201
1,856
24,308
8,490,905
Hex + ClickHouse hackathon 🪐
8
430
Cursor cafe ❤️
9
220
Peter Choi retweeted
Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available.
4,914
14,105
9,948
103,397
57,696,975
Peter Choi retweeted
pay by magic
580
320
441
5,565
2,017,203
Peter Choi retweeted
Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors. Available today at the same price.
3,588
8,314
5,300
66,154
15,514,077
Peter Choi retweeted
Introducing Claude Opus 4.7, our most capable Opus model yet. It handles long-running tasks with more rigor, follows instructions more precisely, and verifies its own outputs before reporting back. You can hand off your hardest work with less supervision.
4,598
9,872
6,729
79,675
14,471,555
Peter Choi retweeted
The cost of not tinkering and exploring has never been higher.
39
126
9
807
30,031
Where does knowledge transfer go when AI reviews AI? If agents write code and agents review code, the human is no longer in every loop. They're managing the loops. But code review was never just about bug-catching. It was how context and standards passed across a team. Code review was always imperfect, but the knowledge transfer happened anyway, almost by accident. That accident is going away, so what matters now is the quality of the original spec, and how deliberate teams are about teaching and documenting what review used to teach by accident.
Introducing Code Review, a new feature for Claude Code. When a PR opens, Claude dispatches a team of agents to hunt for bugs.
2
383
MiniMax just shipped M2.5 that benchmarks near Claude Opus 4.6 at a fraction of the cost. A couple years ago, the frontier was a small club. A handful of labs, massive compute requirements, and a meaningful capability gap between the best models and everyone else. That gap is closing fast. This changes the math for founders building on top of these models. "Which model is best?" still matters. But when the answer keeps changing every few months, betting your product on that answer is a losing strategy. The teams building durable products right now are winning on how tightly they've scoped their use case, how well they understand their user's workflow, and how fast they can iterate on feedback. Those are hard to copy regardless of which lab ships next. The best model you can build on today may not be in 6 months. Build like that's true.
1
1
225
In the AI era, strong revenue is no longer proof that your thesis is right. Cursor called an all-hands titled "War Time." Developers could now ship features without reviewing every line of code. Cursor's whole premise as a collaborative editor was suddenly in question. Meanwhile, annualized revenue hit $2 billion. Revenue is still growing. They're in a strange position: growing fast on a thesis they know needs to change. The new mandate is to build their own coding model. That's a company trying to stay in search mode even while scaling. The question for every AI product company right now is whether they can read the phase shift fast enough to build the next one.
1
2
382
Bank accounts, credit cards, payment APIs, crypto wallets. All built around the same assumption: a human is on the other end. Coinbase just launched infrastructure that breaks that assumption. Agentic Wallets give AI agents their own financial identity. They can hold crypto, pay for API access, execute trades, and manage funds autonomously without human approval. Giving agents a wallet is easy, but knowing how much autonomy to grant in financial decisions is a design challenge the industry hasn't solved yet. Coinbase is betting that agent-to-agent transactions eventually dwarf human-to-human ones. If that's right, the entire payments stack gets rebuilt around a new set of assumptions. The biggest crypto user of the next few years may not be human.
1
145
Build the model that writes the code. Sell the model that reviews the code. Repeat...
Introducing Code Review, a new feature for Claude Code. When a PR opens, Claude dispatches a team of agents to hunt for bugs.
1
164
AI agents are now managing billions in enterprise procurement spend. Not assisting the process. Executing it. Lio just raised $30M from a16z to do exactly this. Their agents read supplier docs, compare quotes, negotiate terms, and execute purchases. Humans step in for policy exceptions and approvals, but the default workflow runs autonomously. One global manufacturer automated 75% of its previously outsourced procurement work within six months. The pattern extends beyond procurement. Find a workflow that's high-value, rules-based, and buried in docs. Insurance, compliance, etc. where the work is mostly reading, cross-referencing, and following procedures. That's exactly what agents are good at. Enterprises spend over $180B a year on procurement talent vs. roughly $10B on procurement software. That ratio tells you how much work still happens manually around existing tools. Lio is betting it inverts as agents take over execution and humans shift to strategy and oversight.
1
147
You can't iterate on what you can't see. Teams adjust prompts, swap models, tweak retrieval pipelines, and have no way to measure whether any of it improved their product. So they ship on vibes. The fix starts with tracing. Every user interaction should produce a structured record: what the user asked for, what the system decided, what external calls it made, and what it returned. Without traces, debugging is guesswork. Once you have traces, convert real problems into test cases. Pull examples of actual user frustrations from production, categorize them, and turn them into eval sets with simple pass/fail scoring. Vercel's v0 team shows what this looks like at the eval layer. They feed user feedback and production failures back into eval sets, iterate on prompts almost daily, and gate every PR that touches the output pipeline on eval results. When models introduced subtle bugs or formatting issues, they tracked those patterns and built correction pipelines around them. No vibes, just data. Automate your evals. Gate deployments on whether they pass. Pair automated scores with periodic human review for subjective outputs. The teams improving their AI products predictably built this feedback infrastructure early. Everyone else is still guessing. Eval-driven development: Build better AI faster : vercel.com/blog/eval-driven-…
1
143
If your AI agent doesn't have an eval suite, you're debugging in production. You push a prompt change, something breaks downstream, and you can't tell if the regression is real or just noise. That's the cycle Anthropic's engineering team described after working with dozens of companies building agents: teams without evals get stuck in reactive loops where every fix risks creating a new problem. The playbook to breakout of this cycle is to test small pieces in isolation. Test everything separately: prompt behavior, retrieval quality, tool-call accuracy. When something breaks, you know exactly where to look. Anthropic's engineering team published a detailed breakdown of how companies like Descript and Bolt built their eval systems. Descript's video editing agent structured evals around three dimensions: don't break things, do what I asked, and do it well. They started with manual grading and evolved to LLM-based graders calibrated with periodic human review/judgement. Bolt took a different path. They already had a widely used agent before they built evals. In 3 months, they shipped a system that grades agent outputs using static analysis, browser-based testing, and LLM judges for behaviors like instruction following. Both teams started scrappy and got more rigorous over time, and that progression matters more than getting it right on day one. When you can rerun failures quickly and pinpoint exactly what changed, small targeted improvements add up fast without surprise outages. Demystifying evals for AI agents: anthropic.com/engineering/de…
2
3
223
A year ago, you could filter candidates by how fast they wrote clean code. That skill still matters, but it's harder to use as a signal when any engineer with a good prompt can produce clean code in minutes. What you're hiring for has shifted. The signal that still works: give candidates AI-generated code and ask what's wrong with it. Can they spot the subtle bug? The wrong abstraction? Why it might fail at scale? That's the actual job now.
133