John Richard

Building an AI support agent is easy Building one that doesn’t refund a customer twice is the hard part. Customer says: “My order hasn’t arrived, Refund me” Behind the scenes the agent must: • Understand intent • Authenticate the user • Fetch order + payment status • Check refund eligibility • Call the right tools • Execute the refund • Verify it succeeded • Confirm to the customer This breaks in production when: • The LLM times out • Tool arguments are invalid • Payment succeeds but the response times out • The agent retries and refunds again • A consumer crashes mid-event • The agent loops on tool calls • Someone tries prompt injection • The refund amount exceeds the limit A production agent needs: • Idempotency keys on every money-moving action • Retries with backoff + jitter • Durable workflow state • Tool-level validation and authorization • Circuit breakers • Step and token budgets • Human approval for high-value actions Observability and monitoring: • Trace every step of the agent run (intent, tool calls, decisions) • Log tool inputs and outputs without sensitive data • Track latency, token usage, and retry counts per run • Alert on duplicate refunds, high failure rates, or budget breaches • Record the final outcome and any human interventions • Make it easy to replay a failed run with the same request ID Core rule: Let the LLM decide what should happen. Let deterministic services decide what is allowed to happen. That’s the difference between a chatbot and a system you can trust with money. What’s the riskiest action you’ve seen an agent try to take without proper controls?