Maximilian Henrich

Been thinking about this a lot: there's still a gap between humans and agents in development. Not capability; agents will keep getting smarter. It's three things that look like weaknesses but aren't: Curiosity, pushback, and rest. Curiosity: we wander off & try things nobody asked for. We get bored/obsessed, want things to exist (like me with state machine tooling), and a lot of really awesome things come from that. Agents can explore, but only inside a "prompt" someone else wrote. They can't come up with ideas on its own (prompting "come up with ideas" doesn't count!) They can build our ideas and riff on them, but they don't have ideas of their own that they obsess over on their own. Pushback: "is this actually needed?" "will this move the needle?" We ask questions like that because we have experience/scar tissue. We've been burned by recognizable patterns before, and push back because of the emotional toll of being the one that has to maintain it. Agents push back too sometimes but there are no stakes; it's just them pretending to pushback without real emotional motive. Of course you can tell them to push back, but ironically that's just another instruction they're obeying. Rest: We know when to stop, more or less. We get tired, call it a day, and that boundary (not laziness!) is a feature. Every change widens the blast radius. Stopping is where you can check the last batch of work and verify everything before the state space explodes again. Agents working in their fancy overnight loops nonstop looks more productive, but it's just accumulating tech/cognitive debt faster than can be paid off. And sure there are tests, but tests catch what you thought to test for; they don't shrink the state space of things that can go wrong. Exhaustion is a feature: it tells us our judgment is slipping. Agents degrade too (context length, drift, etc.) but they don't "feel" it like we do. They just push on with full confidence and worse judgment. Sure you can approximate all three with fancy skills or prompts or harnesses or whatever. They help, but they're all simulations, because those three things come from the one thing agents don't have: a "self" that persists and needs to pay for its own mistakes. They don't need curiosity; we need to scratch an itch, they don't They don't need to give pushback; nothing's at stake for them They obviously don't need rest; although sometimes I wish they'd chill out they were close to hitting their token limits 😅 And no, this isn't something that's "fixed in the next model update" or "this harness/tool/skill/etc solves it". All these things are really the same thing: - Curiosity = what's worth doing? - Pushback = is it actually worth doing? - Rest = when is more not worth it? Agents are incredible at the *doing* part, but they don't have much to say about *worth*, because worth only matters to whoever has to live with the result. That's the human part. IMO it's still extremely important and will continue to be, and it's not something we should automate away. It's what gives agents their purpose.