Schema validation
Confirms the profile matches the AOP format. Missing fields, bad types and conflicting settings are caught here.
Agent Studio
Testing and Evals
Find the problem in the Studio, not on a live call. Dry run catches wiring errors, test drive lets you talk to the agent and evals score it on real cases before anyone else sees it.
Three test modes
Validate, converse, score.
Each mode catches a different class of problem. Together they cover structural errors, conversation quality and outcome accuracy.
Dry run
Validates the profile, its tools and its guardrails without executing anything. Catches wiring errors before the agent runs.
Test drive
Talk to the draft agent yourself and see every guardrail check, tool call and decision it makes in real time.
Evals
Score the agent on a set of cases with expected outcomes. An agent that falls below the eval bar cannot run on its own.
Dry run
Catch wiring errors in seconds.
Dry run validates the profile without executing anything. It checks schema, tools, guardrails and the contract in one pass.
Tool resolution
Every tool in the profile is resolved against the connector hub. An unregistered tool fails the dry run.
Guardrail attachment
Checks that every listed guardrail exists and that universal guardrails are present. Reports coverage gaps.
Test drive
Talk to the agent before your customers do.
Open a live conversation with the draft agent. Every guardrail check, tool call and approval decision appears in real time so you see exactly what happens under the hood.
Multi-channel
Test drive works across chat, voice and email channels. The same profile, the same checks, whichever channel the job uses.
Full trace
Every step of the test drive is recorded on the ledger. You can replay the conversation and inspect any decision.
Evals
Score the agent on real cases.
Write cases with inputs and expected outcomes. The eval runner scores each case and an agent that falls below the bar is clamped to assisted mode.
Case library
Build a library of test cases for each agent. Reuse them across versions to catch regressions.
Eval gate
The platform enforces the eval bar. An agent below the threshold cannot run at full autonomy regardless of who asks.
Score history
Track scores across versions. See which changes improved or degraded the agent and by how much.
Comparison
Studio testing vs manual QA.
| Capability | Manual QA | Studio testing |
|---|---|---|
| Profile validation | No | Yes |
| Tool resolution check | No | Yes |
| Guardrail coverage | Partly | Yes |
| Conversational test | Yes | Yes |
| Scored eval suite | No | Yes |
| Audit trail of test | No | Yes |
| Blocks launch on failure | No | Yes |
FAQ
Common questions about testing.
Schema validation, tool resolution against the connector hub, guardrail attachment, contract negotiation (whether the profile meets the runtime contract) and tool-kind classification (whether read and write tools are labelled correctly).
Yes. Test drive works across channels. For voice agents it opens a call session so you can speak with the draft agent and see every check it makes.
A configurable threshold (set per tenant or per agent). If the agent scores below it on the eval suite, the platform clamps the agent to assisted autonomy. It cannot run unsupervised until it passes.
Yes. You write cases with inputs and expected outcomes. The eval runner scores the agent on each case and reports pass, fail and edge cases.
See the test modes live.
We will dry run, test drive and eval an agent together in one session so you see how the three modes work.