Guide · step 3 of 4
Configuring an AI agent: the mistakes that cost
Seven failures we have actually measured in production — confident absences, invented figures, actions claimed but never performed — and what to change so they stop.
None of these mistakes produces an error message. That is what makes them expensive: the conversation looks like it went fine, and the customer is gone.
1. Letting it assert an absence without checking
The costliest failure, and the quietest. A visitor asks for something you do sell; the agent answers "we do not have that" without running the search. No error appears anywhere — the conversation looks fine. Measured here: an agent wired to a live catalogue announced zero results where there were twenty-two. The rule: an absence is announced after verification, never before.
2. Treating an empty result as proof of absence
A zero can come from the query, not the stock. Same measurement: "maison" returns nothing on a catalogue indexed in English, "house" returns twenty-two. Before concluding, the agent must retry with other words — and in the language of the corpus, which is not necessarily the visitor’s.
3. Letting it announce an action it never performed
"That is sent", "I am passing your request on", "it is recorded" — with no tool ever called. The text is plausible, and the customer waits for a call back that will not come. It only shows up by cross-checking what the bot SAID against what it actually CALLED.
4. Letting it state a figure it never read
The most contagious class of error: a volume, a price, a lead time that appears in no tool result. It then gets requoted as a fact about your site. Common trap: a search total is often capped by the engine — presenting it as a count of your catalogue is a fabrication, even when the number is literally in the data.
5. Putting the doctrine in a doc on the side
The rules that matter belong in the tool DESCRIPTION, not in a document nobody loads. That is where the agent reads them at the moment of choosing. A rule in the right place changes behaviour; the same rule elsewhere changes nothing.
6. Confusing an outage with a gap
An API returning 500 already has its own path: monitoring, alerts, logs. Ask the agent to report outages too and the channel fills with noise that already has an owner, and nobody reads it any more. Report what is missing, not what is broken.
7. Believing an instruction guarantees a behaviour
Our most useful measurement this year: an agent can QUOTE a rule verbatim when asked, and not apply it. Two rewrites changed nothing. When a procedural rule does not land in two attempts, the problem is no longer the wording — you need a deterministic anchor in code, not a third version of the text.
Frequently asked questions
How do I know if my agent is making things up?
By cross-checking what it wrote against the tools it actually called, turn by turn. That is mechanical and depends on no judgement: either the call exists or it does not. At Animam it runs weekly, by a second model, separate from the one holding the conversation.
Should I lock everything down to be safe?
No, and that is the symmetric mistake. An agent that can do nothing does nothing wrong and is worth nothing. What must be locked down are the irreversible moves — money, personal data, commitments. The rest is better open: that is where the value is.
My agent answers well. Is that enough?
That is half the question. The other half is what it could not serve, and that shows up in no successful conversation. It is the subject of the next step.
Next in the series