DATASETS2SCOVAL BENCHMARK ROTATIONACTIVE
Ultra Bank caller speech-to-speech benchmark dataset
Ultra Bank caller is a speech-to-speech test set in Coval's daily rotation — every measured speech-to-speech model runs the same fixed inputs from it.
- Models measured
- 2
- last 30 days
How models rank on Ultra Bank caller
Full S2S dashboardmedian of all models · 1368 ms
median of all models · 90%
| # | Model | Host | V2V | Instruction adherence | Samples |
|---|---|---|---|---|---|
| 1 | Gemini 3.1 Flash Live (Preview) | 1227 ms | 94% | 250 | |
| 2 | GPT Realtime 2 | OpenAI | 1509 ms | 86% | 252 |
Metrics reported for this dataset
Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.
Evaluate your own voice agent
Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.