The control plane for every LLM call in production
Evaluating AI gateways? Iris Control covers the gateway and the governance around it.
By the IrisAgent team · Last updated
Try to break an LLM call
Take a model down, cross a data boundary, or slip in a made-up answer, and watch what Iris Control does. Every call runs on live AI.
Your LLM stack grew faster than your controls
Multi-model products ship quickly. Then the platform team inherits a split stack: one vendor for routing, another for logs, a notebook for evals, a guardrail library per app. Nobody owns the policy in between.
Today: a split stack
With Iris Control: one layer
Outages become incidents
A provider outage or a model swap takes a live feature down when there is no routing policy and no failover.
Data crosses segments
Multi-product and multi-tenant orgs must decide which models may touch which data. A budget cap does not answer that.
No single view of I/O
Prompts and completions are scattered across apps, so nobody can answer what a model was asked and what it said.
Benchmarks, not your data
Generic leaderboards do not tell you which model wins on your corpora, your traces, and your users.
Tool sprawl
Agents keep picking up tools. Someone has to decide which models and workflows may call which ones.
Ungrounded answers ship
Without checks on the production path, hallucinated answers reach users before an offline eval ever sees them.
Six controls. One layer.
Everything a platform team needs to run LLMs in production, sharing one policy model and one audit trail.
Model routing and failover
Keep production LLM paths up when a provider or model degrades. Iris Control picks the right model for each use case and falls back automatically when a call fails.
- Route each use case to the model that fits it
- Fail over when a model or provider call fails
- One routing policy shared across every product
Data-segment governance
Multi-product and multi-tenant stacks need more than a spend cap. Iris Control keeps retrieval inside the product or segment a request is allowed to search.
- Filter knowledge by product or segment
- Keep each corpus with the calls that should see it
- One place for security to reason about data access
Observe every call
Inputs and outputs for every LLM call land on one path, so platform and security teams can audit what a model was asked and what it returned.
- Log prompts and completions as calls happen
- Inspect traffic across models from one place
- A durable record of model I/O for review
Tool-use controls
Agents stay useful and safe when someone decides which tools they may call. Iris Control keeps tool use inside the set each workflow is allowed to invoke.
- Invoke tools as part of the model workflow
- Limit a workflow to the tools it may call
- Add Prompt Shield for argument checks in your own apps
Evals on your data, with AutoEval
Pick models by how they perform on your production data, not a public leaderboard. AutoEval turns real customer feedback into a golden eval set that grows with your traffic.
- Eval set built from customer votes and agent signals
- Leaderboard of every model on your golden set
- New model releases tested before they reach production
Output guards
Check the answer before it reaches a user or a tool. Grounding and judge-style review keep outputs tied to approved sources.
- URL grounding on the production path
- Judge-style review of every answer
- Prompt-leak and citation checks via Prompt Shield
Anatomy of a governed LLM call
Every request takes the same path through Iris Control. Your apps call one endpoint. The control plane does the rest.
- 01One path for I/O
Request in
An app sends a prompt to one endpoint, tagged with its product, tenant, and use case.
- 02Prompt Shield
Input check
Prompt Shield inspects the input and isolates untrusted content before a model sees it.
- 03Governance
Segment policy
Retrieval and tools are scoped to the data segment and tool set this call is allowed to use.
- 04Routing
Route and fail over
The router picks the model for the use case and falls back if the provider call fails.
- 05Quality gate
Output guard
Grounding and judge-style review check the answer against approved sources.
- 06Observe + evals
Logged and scored
Inputs, outputs, and feedback land in one record, and AutoEval turns the best cases into your golden eval set.
An eval set that builds itself from production
Most teams choose models from a public leaderboard and a handful of hand-written test questions. AutoEval turns real customer feedback into a golden eval set for each use case, keeps it clean, and runs every model you consider against it.
- 1
Capture signals
Customer votes, agent edits, escalations, and resolved tickets flag the answers worth learning from, good and bad.
- 2
Qualify with a decision model
A decision model checks each candidate: is the question clear, is the right answer known and sourced, and does it test something the set doesn't cover yet?
- 3
Add to the golden set
Cases that pass join the use case's eval set with the expected answer and the sources that support it.
- 4
Keep it golden
Near-duplicates merge, stale cases retire when your knowledge changes, and the set compresses to the cases that separate strong models from weak ones.
- 5
Run the leaderboard
Every model you run or consider is scored on the golden set for accuracy, grounding, latency, and cost.
- 6
Gate every upgrade
A new model release runs against your golden set before it sees production traffic, and ships only if it beats what you run today.
Golden set · support-answers
Upgrade check
Current primary
in production
New release
candidate
Hold: grounded rate below current
Current fallback
in production
More traffic, better evals
Every week in production adds high-quality cases to your golden set, so each model decision rests on more of your own data. A new model release is never a guess: it either beats what you run today on your cases, or it waits.
More than an AI gateway
Gateways stop at the proxy: one endpoint, many models, failover. Iris Control does that job and adds the policy a CTO org needs around it.
| Capability | Proxy-only AI gateway | Iris Control |
|---|---|---|
| One endpoint across model providers | Yes | ✓Yes |
| Failover when a provider degrades | Yes | ✓Yes |
| Data-segment governance for retrieval | Rarely | ✓Built in |
| Tool-use policy per workflow | Rarely | ✓Built in |
| Evals on your production data | Separate tool | ✓Built in |
| Eval set that grows from production feedback | No | ✓AutoEval |
| New models tested on your data before rollout | No | ✓AutoEval |
| Output grounding on the live path | Separate tool | ✓Built in |
| Prompt injection protection | Add-on | ✓Prompt Shield |
Prompt Shield, built into the control plane
Prompt injection protection sits on the same path as routing, governance, and output guards. Prompt Shield inspects inputs, isolates untrusted content, and checks outputs. The same protections already run inside IrisAgent apps in production.
Built for the platform org
The CTO and the platform or AI engineering team own Iris Control. Security joins for data access, audit, and enforcement. Every line-of-business app consumes the same layer.
Economic buyer
CTO
One control layer for production risk, cost, and quality, in place of four vendors that do not share policy.
Technical buyer
Platform and AI engineering
Owns model choice, failover, call visibility, tool limits, and evals on your data. Ships faster when the guardrails are already on the path.
Co-buyer
Security
Gets data-segment access rules, an audit trail of model use, and guards on inputs and outputs. Prompt Shield is the module to review with them.
Any questions?
We got you.
Put every LLM call behind one control plane
We will walk through routing, governance, failover, and quality gates with your platform and security leads, on your use cases.