DATASETS2SCOVAL BENCHMARK ROTATIONACTIVE
Ultra Bank caller speech-to-speech benchmark dataset
Ultra Bank caller is a speech-to-speech test set in Coval's daily rotation — every measured speech-to-speech model runs the same fixed inputs from it.
- Models measured
- 10
- last 30 days
How models rank on Ultra Bank caller
Full S2S dashboardmedian of all models · 1350 ms
median of all models · 93%
| # | Model | Host | V2V | Instruction adherence | Samples |
|---|---|---|---|---|---|
| 1 | Dialog Rsn 1 | PolyAI | 840 ms | 95% | 160 |
| 2 | Gemini 3.8 Live | 845 ms | 86% | 200 | |
| 3 | Prism | Rime | 893 ms | 85% | 36 |
| 4 | Gemini 3.1 Flash Live (Preview) | 1010 ms | 94% | 193 | |
| 5 | Phonic 1 | Phonic | 1327 ms | 94% | 90 |
| 6 | Qwen3.8 Omni Flash Realtime | Alibaba | 1373 ms | 93% | 160 |
| 7 | Eleven Realtime v1 Mini | ElevenLabs | 1414 ms | 92% | 130 |
| 8 | Eleven Realtime v1 Max | ElevenLabs | 1419 ms | 96% | 130 |
| 9 | GPT-Live-1 | OpenAI | 1767 ms | 95% | 110 |
| 10 | Grok Voice 3.0 | SpaceXAI | 1785 ms | 91% | 160 |
Metrics reported for this dataset
Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.
Evaluate your own voice agent
Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.