2026-07-16 · Confirm.dev

The $10,000 refund an agent almost sent

Here is the scenario we hear from almost every team shipping a support agent. The details vary; the shape never does.

A ticket comes in: "I was charged wrong, please refund me." The agent does everything right. It reads the ticket, pulls the customer's order history, checks the refund policy. The newest order is $10.00. Then it drafts the refund: $10,000.00.

No step failed. The model just filled in an amount with confidence, the way models do. The ticket never said a number, so it inferred one, badly. If the refund tool executes what the agent drafted, that money is gone before anyone reads the ticket.

Why this failure mode is the common one

Agents fail on the parameters of an action far more often than on the choice of action. Refunding this customer was correct. Emailing that prospect was correct. Scaling that service was correct. The amount, the recipient list, the instance count: that's where the hallucination lives, and it's exactly the part a quick glance from a human catches.

Which is why the standard fixes miss. Better prompts lower the rate, not the stakes. A confirmation step inside the model ("are you sure?") is the model grading its own homework. And a Slack ping to a human helps until you ask: who approved it, can they fix the amount instead of just blocking it, what happens when nobody answers, and what do you show the auditor? A ✅ reaction answers none of those.

Where the human belongs

The fix is structural: pause at the action layer, not the model layer. The tool that moves money does not run until a human has seen the exact payload, with the power to edit it.

import { guard } from "@confirm/sdk";

const tools = guard(agentTools, {
  policies: [
    { when: (t) => t.name === "refund" && t.args.amount > 100, notify: "finance@acme.com" },
  ],
  default: "require", // fail-closed: a tool nobody wrote a rule for is still gated
});

Now the $10,000 draft becomes an email to a human with the agent's reasoning and its recent steps above the payload. The reviewer sees the order history says $10.00, edits the amount, and approves. The agent resumes and executes effectivePayload: the corrected action, not the hallucinated one. The whole exchange lands in an audit log with a name, a timestamp, and the edit.

The agent stays fast for everything safe. Reads never wait. Small refunds under the policy threshold execute instantly. The human is only in the loop where an error is irreversible.

Watch it get caught

We turned this exact scenario into our live demo: it emails you a real approval request for the $10,000 refund, you fix the amount to $10.00, and you approve the corrected version. No account needed.

Try the live demo

Building an agent that acts? npm i @confirm/sdk and the quickstart take about five minutes.