DATASETS2SCOVAL BENCHMARK ROTATIONACTIVE

Ultra Bank caller speech-to-speech benchmark dataset

Ultra Bank caller is a speech-to-speech test set in Coval's daily rotation — every measured speech-to-speech model runs the same fixed inputs from it.

Models measured
10
last 30 days
Current leader · V2V#1 / 10
840ms
Dialog Rsn 1via PolyAI

How models rank on Ultra Bank caller

Full S2S dashboard
Voice-to-Voice Latency on Ultra Bank callerMilliseconds · lower is better · 30-day average on this datasetEvery S2S model measured on the Ultra Bank caller dataset, ranked on Voice-to-Voice Latency.
median of all models · 1350 ms
S2S models on the Ultra Bank caller dataset over the last 30 days, ranked on Voice-to-Voice Latency.
#ModelHostV2VInstruction adherenceSamples
1Dialog Rsn 1PolyAI840 ms95%160
2Gemini 3.8 LiveGoogle845 ms86%200
3PrismRime893 ms85%36
4Gemini 3.1 Flash Live (Preview)Google1010 ms94%193
5Phonic 1Phonic1327 ms94%90
6Qwen3.8 Omni Flash RealtimeAlibaba1373 ms93%160
7Eleven Realtime v1 MiniElevenLabs1414 ms92%130
8Eleven Realtime v1 MaxElevenLabs1419 ms96%130
9GPT-Live-1OpenAI1767 ms95%110
10Grok Voice 3.0SpaceXAI1785 ms91%160
Under-sampled models are excluded; tied models share a place.

Metrics reported for this dataset

Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.

Evaluate your own voice agent

Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.

Book a Demo