What Are AI Evals? A Plain-English Guide With Examples

AI evals are repeatable tests of an AI system's output. Each eval has three parts: a set of inputs, a way to grade every output, and a score you track over time. They tell you whether a prompt change, model upgrade, or new workflow made your product better or worse.

Here is why one score is never enough. On IrisAgent's Support Model Arena, Gemini 3 Flash answered 98.8% of answerable support tickets correctly. On tickets it should have declined, it held back only 70% of the time. A single accuracy eval would have ranked it near the top. A second eval caught the fabrications.

If you lead a product or engineering team, you keep hearing the word "evals" in planning meetings. This guide explains what they are, how they differ from the tests your team already writes, and what they look like for real AI workflows.

Key Takeaways

• An eval is a test with a grader. Inputs, a way to grade each output, and a score you compare across versions.

• AI outputs vary, so exact matching rarely works. Most evals need a rubric, a judge model, or both.

• There are three kinds of grader: code-based checks, human review, and LLM-as-a-judge. Strong eval suites use all three.

• Run evals offline before a change and on production traffic after it. Each catches failures the other misses.

• Score each failure mode separately. A model can top one eval and fail the next, as the Arena data shows.

What Is an AI Eval?

An AI eval is a fixed procedure for measuring how well an AI system does a specific job. You run the same inputs through the system, grade each output the same way, and record the result. Then you change something and run it again. If the score drops, you caught a regression before your customers did.

People use "evals" for both the test and the practice. When someone says "we need better evals," they usually mean they cannot tell whether last week's prompt edit helped. LLM evals are the same idea applied to large language model apps, which is where most teams meet the term today.

Every eval has the same three parts:

The inputs usually live in a golden dataset, the reference set you score every version against. We cover how to build that set in our guide to building a golden dataset from production logs, so this post focuses on the other two parts.

How Evals Differ From Unit Tests

Evals borrow the idea of automated testing from software, but the grading works differently. A unit test checks that a function returns an exact value. Run it a thousand times and you get the same answer. An LLM can phrase a correct answer a hundred ways, and it can phrase a wrong answer persuasively.

So a single example rarely tells you much. Evals work on volume: hundreds of inputs, each graded, rolled into a rate. Hamel Husain, an ML engineer who has written widely on the topic, makes the same point in his practical guide to evals: start with cheap assertion-style tests and move to model and human review as the product matures.

Some checks still look like unit tests. "Is the output valid JSON?" has one right answer. Most evals that matter, though, ask questions like "is this answer correct and grounded?", which need a rubric.

The Three Types of AI Evals

The type of an eval mostly comes down to who, or what, does the grading.

OpenAI's evaluation best practices describe the same three groups. They also warn that judge models can favor longer answers or the answer shown first, and they recommend pass or fail verdicts and side-by-side comparisons over open-ended scoring. Anthropic's guide to building evals adds that many automatically graded items beat a few hand-graded ones, and that a judge should be a different model from the one it grades.

In practice, you combine them. Code checks catch the cheap failures, a judge model grades meaning at scale, and humans grade a sample to keep the judge honest. For a support-specific version of that layering, see our guide on how to test AI customer service agents.

Want to see evals on your own calls? Get a free AutoEval sample report by sending 10 to 20 example calls. We grade them against default rules and yours, then replay them on leading models, with price.

Offline Evals vs. Evals on Production Traffic

Evals run in two places, and they answer different questions.

Offline evals are your release gate. Because every candidate runs on identical inputs, the comparison is fair. Evals on production traffic are your smoke alarm. They grade real outputs against rules, so you learn about a broken workflow the day it breaks.

You need both. An offline set can't show you a question type that started last Tuesday. Production grading can't compare two models fairly, because each live call only ran on one of them. Feed failures from production back into your offline set, and the two loops improve each other.

AI Eval Examples Across Four Workflows

The three-part structure stays the same across use cases. What changes is the grader. The examples below are illustrative, not drawn from a specific customer.

Example 1: Grounded Answers

A support assistant answers questions from a help center.

The twin matters. Without must-decline inputs, a model that guesses confidently scores the same as one that knows when to stop.

Example 2: Classification and Routing

A model tags incoming tickets with one of 40 categories.

This is the closest an eval gets to a unit test. No judge needed.

Example 3: Agents and Tool Use

An agent looks up an order and issues a refund when policy allows it.

Grade the actions, not only the words. A polite reply means nothing if the agent refunded the wrong order.

Example 4: Voice and Multilingual Conversations

A voice agent serves customers who switch between Hindi and English mid-sentence.

Public benchmarks rarely test code-switching like this, which is exactly why teams with multilingual users need their own evals.

What the Support Model Arena Shows About Evals

IrisAgent runs a public eval of its own: the Support Model Arena, which scores leading LLMs on real support work. It splits the job into five evals: answer accuracy, hallucination resistance, cited sources, intent routing, and ticket tagging. The board below reflects the run dated October 15, 2026.

Two lessons for any eval program come out of this table.

First, one eval hides failure modes. Gemini 3 Flash nearly aced the answerable tickets but made up an answer on almost a third of the tickets it should have declined. A team that only measured accuracy would ship it.

Second, evals change cost decisions. GLM 5.3 Flash tied Claude Fable 5 on the overall score at one hundredth of the output price. Only an eval on your own work can tell you whether a cheaper model clears your bar.

Scores move as the suites are re-run, so read the shape of the board rather than any single percentage.

How to Write Your First AI Eval This Week

You do not need a platform to start. You need one workflow, a spreadsheet, and a few hours with the person who knows what a good answer looks like.

1. Pick one workflow and one decision. For example: "Can we switch the ticket tagger to a cheaper model?" A clear decision tells you what to measure.

2. Pull 50 to 100 real inputs. Take them from recent logs, redact personal data, and include a few known failures. Our golden dataset guide covers sampling and sizing in depth.

3. Write down what "good" means. Two to five plain-English rules per workflow, such as "cites the help article it used" or "never promises a refund amount."

4. Choose the grader for each rule. Use a code check wherever a rule has one right answer, and a judge model for the rest.

5. Grade 30 outputs by hand first. Then run the judge on the same 30. Where it disagrees with you, rewrite the rubric until it agrees.

6. Record a baseline. Score your current production prompt and model, and save the result with the date and dataset version.

7. Run the eval on every change. Every prompt edit and model candidate runs on the same set, and nothing ships below the baseline without a reason.

8. Add every new production failure. When a customer reports a bad answer, it becomes a test case so the bug can't return silently.

Use binary pass or fail verdicts to start. They are faster to label, easier to agree on, and easier to read than a 1 to 10 scale.

Common Mistakes With AI Evals

Most eval programs stall for the same few reasons:

• Writing the set once. A set written for the pilot stops matching production within months. Plan to add items every week.

• Trusting an uncalibrated judge. If no human has checked the judge's verdicts, its score is a guess with a decimal point.

• Reporting one blended number. An average hides the slice that broke. Report by workflow, language, and failure mode.

• Testing only the happy path. Include must-decline inputs, adversarial prompts, and the long, messy tickets real customers send.

• Choosing models from public leaderboards. Benchmarks are useful for a shortlist, but they don't know your policies, formats, or languages.

Where AutoEval Fits

Disclosure: IrisAgent makes AutoEval, so weigh this section accordingly.

There are several good ways to run evals. Open-source frameworks such as DeepEval, Ragas, and promptfoo suit engineers who want code-first tests in CI. Observability platforms such as Braintrust, LangSmith, Langfuse, and Arize trace production calls and give your team tools to build datasets and scorers. If your team wants to own that process, those are often the better fit.

AutoEval is for teams that want evals on their own traffic without building and maintaining the eval set. It generates a golden eval set from your LLM call logs and keeps it current as traffic changes. A judge model grades every output for correctness on a 0 to 5 scale, with an explanation, plus pass or fail on custom rules you write in plain English. You can then replay the set on any model, including your own fine-tuned ones, with cost and latency next to every score. It runs on a copy of your logs, never on your request path, and it recommends whether to ship or hold a model without switching anything on its own.

Next Steps

AI evals turn "this feels better" into a number your team can defend. To recap:

• An eval is inputs, a grader, and a score you track across versions.

• Use code checks where there is one right answer, a calibrated judge for the rest, and humans to keep the judge honest.

• Run evals offline as a release gate and on production traffic as an alarm.

• Score each failure mode on its own, because one blended number hides what broke.

This week, pick your highest-volume AI workflow, pull 50 real inputs, and write three rules with the person who knows what a good answer looks like. You will learn more from grading those 50 outputs by hand than from any leaderboard.

Rather not build the eval set yourself? Send 20 example calls for a free AutoEval sample report. We grade them against default rules and yours, then replay them on leading models, with price.