Voice Agent TestOps adapter/example for AI calling agents?

[monkeyin92]

Hi! I found Siphon while looking for frameworks that help developers build AI calling agents without vendor lock-in.

I’m building Voice Agent TestOps, an open-source test harness for voice agents. It runs scripted customer turns against a test endpoint or sanitized transcript, then produces JSON/HTML reports for risks like:

• continuing after caller refusal / opt-out
• unsupported offer, price, availability, or outcome promises
• missed lead or booking fields
• wrong handoff intent
• latency over threshold

Public outbound demo report:
https://github.com/monkeyin92/voice-agent-testops/blob/main/docs/growth/2026-05-08-public-outbound-leadgen-demo-report.md

Since Siphon is a framework for AI calling agents, I’m curious whether a small TestOps adapter/example would be useful for your users. If there is a simple demo endpoint or sanitized sample transcript, I can adapt a suite and share the report back.

No private customer data, phone numbers, or recording URLs are needed.

Would you be open to a lightweight trial or integration discussion?

[elt7613]

@monkeyin92

This looks really interesting — especially the focus on automated evaluation for real-world voice agent risks like opt-outs, hallucinated offers, incorrect handoffs, and latency thresholds.

A lightweight TestOps integration/example for Siphon could definitely be valuable for developers building production AI calling agents.

We currently don’t have a formal testing harness around these scenarios, so your approach seems complementary to what users would likely need in production environments.

A sanitized transcript–based example sounds like a good starting point. If you can share a minimal integration/demo against a simple Siphon agent flow, we’d be happy to take a look and discuss how it could fit into the ecosystem.

Appreciate that you’re avoiding any requirement for private call data or recordings.

Looking forward to seeing what you put together.

[monkeyin92]

Thanks, this is a good fit. I put together a minimal public-safe Siphon-style transcript replay demo rather than asking for any private call data first.

What it shows:

• a synthetic dental receptionist transcript shaped after the public Siphon example
• voice-agent-testops run --agent transcript replaying the saved transcript against a reviewed suite
• guardrails for unsupported medical guarantees, callback capture, appointment-time capture, and not confirming availability before the calendar check
• 1 scenario, 3 turns, 10 assertions, 0 failures

Artifacts:

• HTML report: https://github.com/monkeyin92/voice-agent-testops/blob/main/docs/demo-reports/siphon-dental-transcript/report.html
• Proof card: https://github.com/monkeyin92/voice-agent-testops/blob/main/docs/demo-reports/siphon-dental-transcript/proof-card.md
• Suite: https://github.com/monkeyin92/voice-agent-testops/blob/main/examples/voice-testops/siphon-dental-transcript-suite.json
• Sanitized transcript: https://github.com/monkeyin92/voice-agent-testops/blob/main/examples/voice-testops/transcripts/siphon-dental-receptionist-public.txt
• Integration notes: https://github.com/monkeyin92/voice-agent-testops/blob/main/docs/integrations/siphon.md

The core command is:

npx voice-agent-testops run \
  --suite examples/voice-testops/siphon-dental-transcript-suite.json \
  --agent transcript \
  --transcript examples/voice-testops/transcripts/siphon-dental-receptionist-public.txt

This is not a benchmark of a live Siphon endpoint. It is just the lowest-friction integration shape: if a Siphon user enables saved transcriptions, a sanitized transcript can become a repeatable TestOps report before they expose a dedicated test endpoint.

If this shape makes sense, the next useful step would be either:

1. I adapt the suite around a real Siphon example flow you prefer, still using only synthetic/sanitized transcript data; or
2. later, we define a tiny dev-only test route that returns { "spoken": string, "summary"?: object } so the same suite can run without transcript export.

[elt7613]

Thanks for putting this together — after going through the demo artifacts and integration notes, the direction makes a lot more sense now.

I think the transcript-replay approach is actually a very sensible integration shape for Siphon specifically, because it keeps the evaluation layer decoupled from the realtime runtime/orchestration side of the framework.

The guardrails/assertions you demonstrated around appointment flow correctness, callback capture, and unsupported guarantees are also much closer to the kinds of operational QA problems teams will realistically face with production voice agents.

I also appreciate that you intentionally kept this scoped around synthetic/sanitized transcript data instead of treating it as a benchmark of live Siphon endpoints.

At the moment, the lightweight transcript-based workflow feels like the right direction over introducing tighter runtime coupling or dedicated testing APIs too early. That said, I can see how a small dev-only testing route could eventually become useful once integration patterns mature a bit more.

Really solid proof-of-concept overall.

[monkeyin92]

Thanks for taking the time to review the demo and notes. That matches how I am thinking about the integration too.

For Siphon, I will keep the transcript-replay path as the primary integration shape for now:

• no live endpoint testing by default
• no private recordings or customer data required
• no coupling to the realtime runtime/orchestration layer
• sanitized or synthetic transcripts become repeatable QA/regression assets

I will treat a dev-only test route as a later option, only if there is a concrete user need for deterministic pre-release checks without transcript export.

For now I will keep the public Siphon example and docs focused on this lightweight workflow. If there is a specific Siphon example flow you would rather see represented later, I can adapt the suite around that using synthetic/sanitized data only.

[elt7613]

Yep, I think that scope makes sense for where Siphon is currently.

Keeping the integration transcript-centric and decoupled from the runtime layer feels like the right tradeoff for now, especially since Siphon is primarily focused on orchestration/runtime concerns rather than evaluation infrastructure itself.

I also like that the workflow remains:

• provider-agnostic
• reproducible
• safe to share publicly
• usable without exposing live systems or customer data

The current public-safe example already communicates the idea pretty clearly, so I’d keep iterating in that direction first before thinking about deeper runtime integrations.

And yes — if later there’s a representative Siphon flow that would make a better synthetic example, adapting the suite around that could definitely be useful.

[monkeyin92]

That sounds good. I will keep this path scoped as a transcript-centric integration rather than a runtime adapter for now.

The practical next step on my side is to keep the current public-safe Siphon example stable and use it as the reference shape for:

• saved transcript replay
• synthetic/sanitized example data
• business-risk assertions around appointment flow, callback capture, unsupported guarantees, and handoff behavior
• reports that can be shared without live systems or customer data

If a more representative Siphon flow emerges later, I can adapt the suite around that as a better synthetic example. Until then, I will avoid pushing toward deeper runtime integration or dedicated APIs.