Know which model to run, what it costs, and keep it up
Evaluating AI gateways? Iris Control covers the gateway and the governance around it.
By the IrisAgent team · Last updated
Try to break an LLM call
Take a model down, cross a data boundary, or slip in a made-up answer, and watch what Iris Control does. Every call runs on live AI.
Your LLM stack grew faster than your controls
Multi-model products ship quickly. Then the platform team inherits a split stack: one vendor for routing, another for logs, a notebook for evals, a guardrail library per app. Nobody owns the policy in between.
Today: a split stack
With Iris Control: one layer
Outages become incidents
A provider outage or a model swap takes a live feature down when there is no routing policy and no failover.
Every new model is a guess
Public leaderboards don't tell you which model wins on your workflows, or what switching will cost you.
Nobody knows who is spending
The LLM bill arrives as one number per provider. Nobody can say which model, workflow, user, or customer drove it, or stop a runaway agent mid-month.
Secrets leak through prompts
Users and agents paste customer PII, API keys, and tokens into prompts. Every router and log on the path keeps a copy.
Data crosses segments
Multi-product and multi-tenant orgs must decide which models may touch which data. A budget cap does not answer that.
No single view of I/O
Prompts and completions are scattered across apps, so nobody can answer what a model was asked and what it said.
Tool sprawl
Agents keep picking up tools. Someone has to decide which models and workflows may call which ones.
Ungrounded answers ship
Without checks on the production path, hallucinated answers reach users before an offline eval ever sees them.
Eight controls. One layer.
Everything a platform team needs to run LLMs in production, sharing one policy model and one audit trail.
Model routing and failover
Keep production LLM paths up when a provider or model degrades. Iris Control picks the right model for each use case and falls back automatically when a call fails.
- Route each use case to the model that fits it
- Fail over when a model or provider call fails
- One routing policy shared across every product
Evals on your data, with AutoEval
Pick models by how they perform on your production data, not a public leaderboard. AutoEval grades every output for correctness and against your own rules, then replays the same prompts on any model you consider.
- Every output graded for correctness and your rules
- Leaderboard of every model on your golden set
- New model releases tested before they reach production
Observability and cost
Every LLM call lands on one path with its model, workflow, user, customer, tokens, and cost, so you can see what a model was asked, what it returned, and what it cost.
- Spend and tokens by model, workflow, user, and customer
- Prompts and completions logged on one path
- Trace any cost spike back to the calls behind it
Granular spend limits
Set budgets and rate limits on any model, workflow, user, or customer. Iris Control enforces them on the live path, before a runaway agent or a heavy tenant runs up the bill.
- Daily and monthly budgets at every level
- Alert, switch to a cheaper model you approved, or block
- Per-customer limits for multi-tenant products
PII and secret redaction
Prompts carry whatever users and agents paste in. Iris Control strips personal data and credentials from every call before it reaches a model provider or a stored log.
- Emails, phone numbers, card numbers, SSNs, and addresses
- Passwords, login strings, and long API keys or tokens
- Redacted before the provider call and before the record is stored
Data-segment governance
Multi-product and multi-tenant stacks need more than a spend cap. Iris Control keeps retrieval inside the product or segment a request is allowed to search.
- Filter knowledge by product or segment
- Keep each corpus with the calls that should see it
- One place for security to reason about data access
Tool-use controls
Agents stay useful and safe when someone decides which tools they may call. Iris Control keeps tool use inside the set each workflow is allowed to invoke.
- Invoke tools as part of the model workflow
- Limit a workflow to the tools it may call
- Add Prompt Shield for argument checks in your own apps
Output guards
Check the answer before it reaches a user or a tool. Grounding and judge-style review keep outputs tied to approved sources.
- URL grounding on the production path
- Judge-style review of every answer
- Prompt-leak and citation checks via Prompt Shield
Running a support AI agent on this path? See which production signals to monitor for support AI agents, from guard activity to escalation reasons.
Anatomy of a governed LLM call
Every request takes the same path through Iris Control. Your apps call one endpoint. The control plane does the rest.
- 01One path for I/O
Request in
An app sends a prompt to one endpoint, tagged with its product, tenant, and use case.
- 02Redaction + Prompt Shield
Input check
PII and secrets are redacted, and Prompt Shield inspects the input and isolates untrusted content before a model sees it.
- 03Governance
Segment policy
Retrieval, tools, and spend limits are checked for the data segment, workflow, and customer this call belongs to.
- 04Routing
Route and fail over
The router picks the model for the use case and falls back if the provider call fails.
- 05Quality gate
Output guard
Grounding and judge-style review check the answer against approved sources.
- 06Observe + evals
Logged and scored
Inputs, outputs, and cost land in one record, and AutoEval grades every output for correctness and against your rules.
An eval set that builds itself from production
Which model should you upgrade to, and what will it cost you? Most teams answer with a public leaderboard and a few hand-written test prompts. AutoEval grades every output in your own LLM logs for correctness and against rules you write, then replays the same prompts on every model you consider, with its price.
- 1
Capture every call
The system prompt, the user prompt, and the model's output, from your logs or the Iris Control endpoint.
- 2
Grade correctness
A judge model checks whether each output correctly and completely answers the prompt it was given, with a score and an explanation.
- 3
Check your rules
Rules you write in plain English, such as "always include a link" or "reply in the user's language", pass or fail on every output.
- 4
Build the eval set
Pick the inputs that matter from your own calls: a workflow, a tenant, last month, or only the failures. Duplicates drop out.
- 5
Replay on every model
The same inputs run on each model you consider, including your own, and every output is graded the same way, with cost and latency.
- 6
Decide the upgrade
Switch only when a model beats what you run today on your own prompts and your own rules. Nothing changes without your approval.
Eval set · support-answers
Upgrade check
Current primary
in production
New release
candidate
Hold: grounded rate below current
Current fallback
in production
More traffic, better evals
Every week in production adds more of your own prompts to grade and replay, so each model decision rests on more of your own data. A new model release is never a guess: it either beats what you run today on your prompts and rules, or it waits.
Not just support answers
Each workflow gets a rubric that fits its job, including your own in-house models.
Grounded answers
Support answers, internal search, RAG
Answer accuracy, hallucination resistance, cited sources, grounding score
Routing and tagging
Intent routing, ticket tagging, classification
Accuracy per label and which labels get confused with each other
Agents and tool use
Billing agents, triage agents, multi-step workflows
Right tool, right arguments, task completed, steps taken
Generated content
Scripts, summaries, drafts
Correctness against the prompt, plus your rules for format, tone, length, and required content
Voice and multilingual
Voice agents, conversations that switch languages mid-sentence
Language fidelity, code-mixed comprehension, turn accuracy, and response time per turn
Always next to price
Every workflow is scored on cost per call and response time too, so quality is never read without what it costs.
Start with 20 examples
Send 10 to 20 example calls as a JSONL file: the system prompt, the user prompt, your model's output, and any rules the output must follow. We return a sample eval report, and the same prompts replayed on leading models, with price. Nothing changes in your call path.
See who is spending, and cap it
Every call carries its model, workflow, user, and customer, so the bill stops being one number per provider. See what drives your LLM spend, then set limits at exactly the level that needs one.
LLM spend · this month
Where the bill comes from
Find the spend you didn't plan for
A load test left running, an agent stuck in a tool loop, one customer sending ten times the usual traffic: each shows up as its own line, traced back to the calls behind it.
Observe every dimension
Spend, tokens, and call volume by model, workflow, user, and customer, with the prompts and answers behind every number.
Set limits at any level
Daily or monthly budgets and rate limits per model, workflow, user, or customer, so one tenant or agent can't consume everyone's budget.
Choose what happens at the limit
Send an alert, move the call to a cheaper model you approved for that workflow, queue it, or block it, per limit.
Pair cost with quality
AutoEval shows whether a cheaper model holds up on your golden set before you move traffic to it.
Spend limits
82% used · Alert at 80%, then route to a cheaper model
64% used · Alert at 90%, block at 100%
100% used · Blocked until tomorrow
37% used · Queue extra calls
More than an AI gateway
Gateways stop at the proxy: one endpoint, many models, failover. Iris Control does that job and adds the policy a CTO org needs around it.
| Capability | Proxy-only AI gateway | Iris Control |
|---|---|---|
| One endpoint across model providers | Yes | ✓ Yes |
| Failover when a provider degrades | Yes | ✓ Yes |
| PII and secret redaction before the model call | Add-on | ✓ Built in |
| Cost by model, workflow, user, and customer | By key or team | ✓ Built in |
| Spend limits that fall back to a model you approved | Rarely | ✓ Built in |
| Data-segment governance for retrieval | Rarely | ✓ Built in |
| Tool-use policy per workflow | Rarely | ✓ Built in |
| Evals on your production data | Separate tool | ✓ Built in |
| Every output graded against your own rules | No | ✓ AutoEval |
| New models tested on your data before rollout | No | ✓ AutoEval |
| Output grounding on the live path | Separate tool | ✓ Built in |
| Prompt injection protection | Add-on | ✓ Prompt Shield |
Prompt Shield, built into the control plane
Prompt injection protection sits on the same path as routing, governance, and output guards. Prompt Shield inspects inputs, isolates untrusted content, and checks outputs. The same protections already run inside IrisAgent apps in production.
Built for the platform org
The CTO and the platform or AI engineering team own Iris Control. Security joins for data access, audit, and enforcement. Every line-of-business app consumes the same layer.
Economic buyer
CTO
One control layer for production risk, cost, and quality, in place of four vendors that do not share policy.
Technical buyer
Platform and AI engineering
Owns model choice, failover, call visibility, spend limits, tool limits, and evals on your data. Ships faster when the guardrails are already on the path.
Co-buyer
Security
Gets PII and secret redaction on every call, data-segment access rules, an audit trail of model use, and guards on inputs and outputs. Prompt Shield is the module to review with them.
Any questions?
We got you.
Put every LLM call behind one control plane
We will walk through routing, governance, failover, and quality gates with your platform and security leads, on your use cases.