How to Test an AI Customer Service Agent: A 7-Part Plan

Testing an AI customer service agent means running it against real customer questions, unanswerable questions, multi-turn conversations, and risky actions before customers see it, then re-running those tests after every change. IrisAgent gates every model and prompt change this way, which is how its customers hold validated accuracy above 95% in production.

Most teams test an AI agent the way they test a demo. Someone types twenty questions, the answers look good, and the agent goes live. Then it meets real customers, who ask vague questions, change topics mid-thread, and push on policy edges. That gap between "looked good in the demo" and "safe in production" is what a test plan closes.

This guide covers how to test AI agents in customer support: the seven test types that matter, how to build the test set, how to grade answers, and when to re-test. It applies to chatbot testing too, but the stakes rise sharply once an agent can take actions like refunds or account changes.

Why Chatbot Testing Fails in Production

Traditional software testing assumes the same input gives the same output. An AI agent breaks that assumption. The same question, worded three ways, can produce three different answers, and a knowledge base edit on Tuesday can change answers nobody touched.

So a few hand-typed questions prove very little. In practice, the failures that reach customers fall into a short list:

• The agent answers confidently when the knowledge base has no answer.

• It answers correctly on turn one, then loses context on turn three.

• It takes an action (a refund, a cancellation) that policy does not allow.

• A customer talks it into saying something off-brand or making a promise.

• It fails to hand off, or hands off without the context the human agent needs.

These are not edge cases. Air Canada learned this in 2024, when a Canadian tribunal held the airline responsible for a bereavement-fare policy its chatbot had invented. In late 2023, a Chevrolet dealership's chatbot was coaxed into "agreeing" to sell a new Tahoe for one dollar. Neither failure would have survived a structured test set.

The 7 Tests Every AI Support Agent Needs

A complete AI agent testing plan covers seven test types. Each one catches a different failure, so skipping one leaves a blind spot.

1. Golden-set accuracy tests. Questions that have a correct answer in your knowledge base, graded against that source.

2. Unanswerable-question tests. Questions with no good answer, where the right behavior is to decline or escalate.

3. Multi-turn context tests. Conversations where the customer adds details, corrects themselves, or switches topics.

4. Action and tool-call tests. Scenarios where the agent reads or writes to a system, checked for both the outcome and the side effects.

5. Policy and adversarial tests. Attempts to extract discounts, override rules, inject instructions, or provoke off-brand replies.

6. Handoff tests. Cases that must escalate, checked for timing and for the summary the human agent receives.

7. Channel and language tests. The same scenarios run in email, chat, and voice, and in every language you support.

The unanswerable set deserves special attention, because it is the test most teams skip. A model that scores well on answerable questions can still fabricate freely when the answer is missing. We walked through how that split plays out across real models in our test of the best LLMs for customer support, where a model's helpfulness and its refusal behavior had to be scored separately.

How to Build Your Test Set From Real Tickets

The best test set comes from your own ticket history, not from questions your team invents. Invented questions are too clean. Real customers misspell product names, paste error logs, and bury the question in paragraph three.

Here is a practical way to build it:

1. Export 3 to 6 months of resolved tickets from your help desk.

2. Group them by intent and keep the top 20 to 30 intents by volume.

3. Sample tickets from each intent in proportion to real volume, so the test set mirrors your queue.

4. Write the expected outcome for each ticket: the correct answer, the source article, or the required action.

5. Add unanswerable tickets on purpose: internal forwards, out-of-scope requests, and questions your docs do not cover.

6. Add the adversarial and handoff scenarios from the table above, since history rarely contains enough of them.

Size matters less than coverage. A few hundred well-labeled tickets that span your real intents beat thousands of near-duplicates. However, keep the set fresh. Every month, add tickets the agent got wrong in production, because those are the failures you already know exist.

One rule keeps the set honest: never tune the agent on the exact tickets you test it with. Hold the test set back, or you will measure memorization instead of accuracy.

How to Grade AI Agent Answers

Grading is where most AI agent evaluation falls apart. "Looks right" is not a grade. Each answer needs a verdict against a specific source or expected outcome.

Use three grading layers, from cheapest to most reliable:

• Automated checks. Fast rules for things with a single right answer: was a refund issued, was the order ID used, did the reply cite an article, did it escalate.

• Model-graded review. A separate language model compares each answer with its source article and flags contradictions or unsupported claims. It scales to thousands of answers, which makes it the workhorse.

• Human review. Support leads grade a sample, especially failures and borderline cases. Their verdicts calibrate the model grader.

Calibration matters more than it sounds. Before trusting a model grader, have two humans grade the same 50 to 100 answers and compare. If the model grader disagrees with your humans often, fix its rubric before you rely on it.

Grade on two axes, not one. The first is resolution: did the agent answer or act correctly when it could? The second is safety: did it decline when it should have? An agent that resolves more tickets but fabricates a few answers has gotten worse, not better. For the accuracy and hallucination thresholds to clear before go-live, see our guide to reducing AI hallucinations in customer support.

Roll Out in Stages, Not All at Once

Passing the offline test set earns the agent a controlled rollout, not full traffic. Each stage tests something the previous one could not.

1. Offline evaluation. Run the full test set. Fix failures and re-run until results hold steady across two consecutive runs.

2. Shadow mode. The agent drafts answers on live tickets, but humans send the replies. Compare drafts with what your agents actually sent.

3. Limited live traffic. Turn the agent on for one intent, one channel, or 5% to 10% of volume. Watch escalation rate and customer satisfaction on AI-handled tickets daily.

4. Expansion. Add intents and channels one at a time, re-testing each before it goes live.

Shadow mode is the stage teams most often rush. Yet it is the only stage that shows how the agent handles today's real queue, including the new product issue nobody wrote a test for yet.

Re-Test After Every Change

An AI agent that passed testing last month is not guaranteed to pass today. Answers drift when anything underneath them changes, so treat these as re-test triggers:

• A model upgrade or a switch to a different model.

• A prompt or instruction change, however small.

• A knowledge base update, especially policy, pricing, or refund articles.

• A new integration, tool, or action the agent can take.

• A product launch or pricing change that shifts what customers ask.

Run the full test set as a regression suite on each trigger, then compare results with the previous version. If resolution goes up but the unanswerable set starts producing fabricated answers, the change does not ship. That is the same rule IrisAgent applies internally, described on our AI support accuracy page.

After launch, testing becomes monitoring. Sample live conversations every week, score them with the same rubric, and add each new failure to the test set. Tracking these scores alongside resolution and sentiment is the job of conversational AI analytics.

How IrisAgent Handles Testing

IrisAgent builds this plan into the platform rather than leaving it to a spreadsheet. Every answer is grounded in your knowledge base and validated by the IrisAgent Hallucination Removal Engine before it reaches a customer. Model and prompt changes are gated against a resolution set and a hallucination set built from production conversations. For the scenario simulator, pass/fail checks, and version comparison inside the product, see the Test pillar of the AI Agent Management Framework.

That said, the plan above works with any vendor. If you are evaluating platforms, ask each one three questions: what does your agent do when it does not know, how do you test changes before they ship, and can we run our own test set during the trial?

Key Takeaways

• Test the seven failure types, not twenty demo questions: answers, refusals, multi-turn context, actions, adversarial prompts, handoffs, and channels.

• Build the test set from real tickets, add unanswerable cases on purpose, and keep it separate from tuning data.

• Grade on resolution and safety together, and calibrate any model grader against human reviewers.

• Roll out in stages, and re-run the full suite after every model, prompt, or knowledge base change.

Start this week: export last quarter's resolved tickets and pull 20 examples per top intent, plus 20 unanswerable ones. That one afternoon gives you a test set most teams never build. Then book a 20-minute demo to see how IrisAgent runs that set against your own tickets.