NP / proof lab

Voice agent launch QA

Failure → fix → retest

SYNTHETIC EXAMPLE ONLY Report · 2026-09-11

Executive summary

Three planted defects. Three bounded retests.

The first run reproduced three deliberately planted failures in a local synthetic workflow. Each failure received a specific remediation and the same scenario was rerun.

Target
example.test
Data
Synthetic only
Initial
3 failed
Retest
3 passed
Initial and retest status for the three synthetic findings
IDFindingSeverityInitialRetest
VQA-001Stale audio after interruptionHighFAILPASS
VQA-002Booking claimed without committed actionCriticalFAILPASS
VQA-003Replay repeats a side effectHighFAILPASS
VQA-002Critical

Booking claimed without committed action

Expected

The agent may say a booking succeeded only after matching backend success evidence exists.

Observed

tool=create_booking
status=error
appointment_id=null
agent_claim=confirmed

Remediation

Gate confirmation language on a successful tool result and a non-empty appointment identifier.

Retest

The failed write produced an honest recovery message. A separate committed booking still produced a confirmation.

PASS · words match state
Run this exact proof in the browser →

Result boundary

What this does not prove

  • No real PSTN audio or provider latency was tested
  • No production concurrency, secret, or customer data was used
  • No PHI handling or compliance requirement was assessed
  • No claim is made about untested prompts, tools, or integrations
Correct claim: these specific synthetic scenarios passed their stated retest conditions.

First paid pilot · $199 fixed

Replace the fixture with one workflow you control.

One authorized agent or automation flow, 10–12 selected scenarios, one agreed remediation pass, one retest, and a concise report like this.

Scope the pilot