Skip to main content
Open Products → Responder → Simulation Testing and select the agent you want to test. Simulation Testing appears only when it is enabled for your team and your role has access. The public workspace has three tabs:
  • Test Cases: reusable caller scenarios and their expected outcomes
  • Test Runs: current and previous executions of those cases
  • Assertions: additional criteria evaluated during every applicable test

​
Create test cases

Create cases in any of four ways:
  • New Test Case: write a scenario manually.
  • Generate with AI: let Avoca analyze the selected agent, optionally provide focus instructions, and generate up to 20 cases.
  • Add from library: attach existing reusable cases to the selected agent.
  • Import from call: select real calls and queue their recordings or transcripts for conversion into test cases.
For a manual case, enter a clear title and persona role, then add only the customer facts the scenario needs. Define the caller’s opening message, behavior, when to reveal information, and when to end the conversation. Write Expected agent output as one or more concise, observable success conditions. This expected outcome determines whether the case passes overall, so avoid vague goals such as “handle the call well.” Advanced voice settings can vary accent, voice, talking speed, loudness, background noise, maximum duration, and automatic hangup behavior. Use these only when that variation is part of what the case is meant to validate. Review every generated, imported, or library case before running it. Edit its persona, facts, instructions, expected outcome, and mocks so they match the selected agent and your current configuration.

​
Configure tool mocks

Use Tool Call Overrides on a case to return a controlled response instead of allowing that tool to use its normal response. Mock responses can reference scenario facts and relative dates, which helps keep a case reusable. Fill Default Mocks previews missing mocks across the selected agent’s cases and fills gaps without overwriting mocks you already configured. Review the preview before applying it.
Simulation tests interact with the live agent. Voice mode places a real phone call, and tests can book appointments or trigger automations. The tool editor blocks transfers during tests, but do not assume every other write is blocked. Use controlled customer data and review mocks before every run.

​
Build a regression suite from real calls

When regression testing is enabled for your team, you can turn any production call into a repeatable case and re-run the whole collection alongside your other suites. Add a call to the suite
  1. Open the call from Channels → Calls and go to the call details view.
  2. In the Actions row (next to Debug Call, Open Call Debugger, and Copy Call Link), select Add to regression suite.
  3. Write the Success criteria — a concise description of the ideal agent behavior for this call. Future runs are judged against these criteria, so the field is required.
  4. Save. The button now shows Remove from regression suite, so you can take the call back out without overwriting its stored outcome.
The suite is scoped to your team and each case is tied to the agent that handled the source call. Run the suite Open Simulation Testing → Test Cases and click Run Tests. When an agent is selected, a Regression Suite row appears in the Select Suites list with the number of cases available for that agent. Select it to include those cases in the run, or use View Suite to see each case, its expected outcome, and a link back to the source call.

​
Run selected cases

  1. Select an agent.
  2. Choose individual cases, filter by tags, or select cases from a test suite.
  3. Click Run Tests.
  4. Give the run a recognizable name.
  5. Choose Voice or Chat when both are available. Chat availability depends on the selected agent and its configured provider key.
  6. For voice testing, choose how many times to sample each case, from 1 to 10.
  7. Review the final case list and start the run.
Repeating a voice case helps reveal inconsistent behavior, but it also multiplies the number of real calls and potential side effects. Start with one sample until the case and mocks are stable.

​
Review Test Runs

The Test Runs table shows the run name, date, number of cases, outcome score, and status. You can cancel an active run or compare two completed runs. Canceling removes the active run and cannot be undone. Open a run to review each case. The details can include:
  • Expected-outcome result and its checklist
  • Assertion results, including skipped checks
  • Recording and transcript
  • Scenario facts
  • Latency, talk-ratio, interruption, and other available metrics
Use Re-run to repeat the run’s cases after a change. Use Compare on two completed runs to compare shared cases; the comparison score is based on their expected outcomes.
Additional assertions are shown in run details as guardrails. Their failures are diagnostic and do not change the case’s overall pass or fail result; the expected outcome is the sole overall result.

​
Maintain Assertions

Open Assertions and select the agent.
  1. Create an assertion with a clear name and importance.
  2. Choose when it should be evaluated, or write a custom trigger.
  3. State one specific evaluation criterion.
  4. Define explicit pass and fail criteria.
  5. Keep Enabled on only while the assertion is relevant.
Use Generate with AI to request suggestions. Expand each suggestion to review its trigger, criteria, and pass/fail definitions before accepting or dismissing it. System-managed assertions are visible but read-only. Review assertions whenever the agent’s responsibilities change. Disable an obsolete assertion rather than weakening its criteria until it always passes.

​
Schedule recurring runs

When scheduling is available for the selected agent, select the cases and choose Schedule. Configure:
  • A recognizable schedule name
  • Weekly, every-two-weeks, or monthly frequency
  • The day of the week, or days 1–28 for a monthly schedule
  • Voice or chat mode
  • Whether to run immediately as well as create the schedule
Scheduled days and times use UTC. The schedules view shows the included cases and the last and next run dates; delete a schedule when it should stop running.
A simulation still does not prove that production carrier routing or a receiving phone system works. Complete a controlled end-to-end call before go-live.

​
Diagnose a failed case

Trace the behavior to its source, make one focused change, and rerun the same case: