Your AI agent can complete a task successfully. And still do something you NEVER asked it to do. For example... AI engineer @TejasKumar_ built an agent using GPT-3.5 Turbo to upvote a story on Hacker News. The task was to find the highest-ranked story that hadn't been upvoted and give it ONE upvote. The agent completed the task. But it upvoted TWO stories instead of one. And the verifier still reported success. Why? The agent clicked upvote while logged out, which redirected it to a login page. After the harness logged in, the original vote had already gone through. But the harness didn't recognize it. So it told the agent to go back and finish the task. And the agent did exactly that. The harnesses verifier checked whether an upvote click had happened, not whether exactly ONE story had been upvoted. While revisiting his original demo, Tejas also discovered that its login handler could be tricked into entering credentials on an unrelated website. The password never appeared in the model's context. But the model could still influence where the harness entered it. Another harness bug. Tejas covers all of these lessons in his latest guest article for Decoding AI Magazine. Specifically, we touch on the 4 decisions you need to make when engineering an agent harness: 1. Who decides when the task is done? 2. Who controls sensitive actions? 3. Who decides when to stop? 4. Who's responsible when something breaks? It goes live tomorrow! Subscribe here to get it in your inbox: decodingai.com/p/4-key-decis…