Voice AI BenchmarksBy

DATASETSTTWILDASR / FLEURSACTIVE

manifest

WildASR clipping speech-to-text benchmark dataset

284 clips with peak distortion: the same utterances as the clean set, clipped.

Items
284
fixed public inputs
Models measured
31
last 30 days
Current leader · WER#1 / 31
3%
Universal 3.6 Provia AssemblyAI

How models rank on WildASR clipping

Full STT dashboard
Word Error Rate on WildASR clippingPercent · lower is better · 30-day average on this datasetEvery STT model measured on the WildASR clipping dataset, ranked on Word Error Rate.
  1. #1Universal 3.6 Pro3.0%
  2. #2Universal 3.5 Pro3.2%
  3. #3Qwen3 ASR Fast5.2%
  4. #4resonant-15.9%
  5. #5Qwen3 ASR 1.7b6.0%
  6. #6GPT-4o Transcribe6.9%
  7. #7Linden 17.4%
  8. #8Whisper Large v3via Baseten8.6%
  9. #9STT 18.9%
  10. #10GPT-4o mini Transcribe9.4%
  11. #11Chirp 29.5%
  12. #12Ink 210.3%
Show all 31 models
  1. #13Chirp 310.4%
  2. #14Pulse10.5%
  3. #15Gemini 3.5 Transcribe Live10.7%
  4. #16Scribe v2 Realtime11.8%
  5. #17STT RT v512.8%
  6. #18Solaria 113.7%
  7. #19Grok STT14.4%
  8. #20Nova 314.6%
  9. #21Whisper Large v3via Together AI14.7%
  10. #22Daytona STT15.1%
  11. #23GPT Realtime Whisper15.1%
  12. #24Parakeet TDT 0.6B v315.6%
  13. #25Voxtral Mini Transcribe Realtime 260216.7%
  14. #26Universal Streaming17.1%
  15. #27Flux18.2%
  16. #28Nova 222.9%
  17. #29Flux Multilingual24.0%
  18. #30Default24.6%
  19. #31Nemotron 3.5 ASR Streaming38.4%
median of all models · 11.8%
STT models on the WildASR clipping dataset over the last 30 days, ranked on Word Error Rate.
#ModelHostTTFSWERTTFTSamples
1Universal 3.6 ProAssemblyAI139 ms1026 ms1,283
2Universal 3.5 ProAssemblyAI178 ms924 ms1,405
3Qwen3 ASR FastNari Labs46 ms1751 ms1,405
4resonant-1Reson8250 ms—1,405
5Qwen3 ASR 1.7bBaseten105 ms904 ms720
6GPT-4o TranscribeOpenAI796 ms—1,402
7Linden 1Speechmatics282 ms1574 ms1,399
8Whisper Large v3Baseten137 ms835 ms720
9STT 1Inworld AI70 ms1127 ms1,304
10GPT-4o mini TranscribeOpenAI731 ms—1,403
11Chirp 2Google888 ms6500 ms1,404
12Ink 2Cartesia150 ms1923 ms1,405
13Chirp 3Google744 ms6343 ms1,404
14PulseSmallest212 ms2095 ms1,405
15Gemini 3.5 Transcribe LiveGemini308 ms1766 ms1,368
16Scribe v2 RealtimeElevenLabs130 ms2133 ms1,404
17STT RT v5Soniox43 ms1555 ms1,241
18Solaria 1Gladia666 ms1760 ms1,379
19Grok STTSpaceXAI235 ms—1,399
20Nova 3Deepgram91 ms1272 ms1,371
21Whisper Large v3Together AI209 ms1162 ms1,390
22Daytona STTGuava384 ms5956 ms288
23GPT Realtime WhisperOpenAI638 ms1855 ms1,402
24Parakeet TDT 0.6B v3Together AI91 ms1107 ms1,394
25Voxtral Mini Transcribe Realtime 2602Mistral429 ms1820 ms1,344
26Universal StreamingAssemblyAI—1655 ms1,388
27FluxDeepgram112 ms825 ms1,371
28Nova 2Deepgram92 ms1362 ms1,370
29Flux MultilingualDeepgram111 ms1205 ms1,371
30DefaultGradium262 ms1903 ms1,404
31Nemotron 3.5 ASR StreamingTogether AI—1955 ms1,388
Under-sampled models are excluded; tied models share a place. Dotted WER values split into substitutions, deletions and insertions on hover or tap.

Inside the dataset

284 fixed public inputs — every model is tested on exactly these.

  • CLIP 1 · 12.4s

    “"he [wales] basically lied to us from the start. first by acting as if this was for legal reasons. second by pretending he was listening to us right up to his art deletion.”

  • CLIP 2 · 10.0s

    “a car bomb detonated at police headquarters in gaziantep turkey yesterday morning killed two police officers and injured more than twenty other people”

  • CLIP 3 · 8.9s

    “a civilization is a singular culture shared by a significant large group of people who live and work co-operatively a society”

3 of 284 clips, streamed from the public benchmark storage bucket.

Source: bosonai/WildASR environment_degradation__en__fleurs_clipping_en · License: Apache-2.0 (WildASR; audio derived from FLEURS, CC-BY-4.0)

WER change from the clean baseline

Same provider and model on this condition vs the WildASR clean dataset; positive = more recognition error.

WildASR clipping vs WildASR cleanPercentage points of WER · diverging from no changePer-model WER change between WildASR clean and WildASR clipping.
zero = no change from the clean baseline · right = more error under this condition
Per-model WER change between WildASR clean and WildASR clipping.
ModelHostClean WERCondition WERChange
Universal 3.6 ProAssemblyAI2.5%3.0%+0.5%
Universal 3.5 ProAssemblyAI2.8%3.2%+0.4%
Qwen3 ASR FastNari Labs3.5%5.2%+1.7%
resonant-1Reson83.2%5.9%+2.7%
Qwen3 ASR 1.7bBaseten3.7%6.0%+2.3%
GPT-4o TranscribeOpenAI2.8%6.9%+4.1%
Linden 1Speechmatics4.7%7.4%+2.7%
Whisper Large v3Baseten4.3%8.6%+4.4%
STT 1Inworld AI8.1%8.9%+0.8%
GPT-4o mini TranscribeOpenAI3.2%9.4%+6.1%
Chirp 2Google4.6%9.5%+5.0%
Ink 2Cartesia4.5%10.3%+5.8%
Chirp 3Google3.4%10.4%+7.1%
PulseSmallest5.1%10.5%+5.4%
Gemini 3.5 Transcribe LiveGemini3.1%10.7%+7.6%
Scribe v2 RealtimeElevenLabs4.0%11.8%+7.7%
STT RT v5Soniox6.1%12.8%+6.7%
Solaria 1Gladia6.0%13.7%+7.7%
Grok STTSpaceXAI4.1%14.4%+10.4%
Nova 3Deepgram6.1%14.6%+8.5%
Whisper Large v3Together AI6.7%14.7%+8.0%
Daytona STTGuava5.2%15.1%+10.0%
GPT Realtime WhisperOpenAI4.1%15.1%+11.1%
Parakeet TDT 0.6B v3Together AI10.7%15.6%+4.9%
Voxtral Mini Transcribe Realtime 2602Mistral4.6%16.7%+12.0%
Universal StreamingAssemblyAI6.1%17.1%+11.0%
FluxDeepgram5.4%18.2%+12.8%
Nova 2Deepgram6.9%22.9%+16.0%
Flux MultilingualDeepgram7.2%24.0%+16.8%
DefaultGradium7.8%24.6%+16.8%
Nemotron 3.5 ASR StreamingTogether AI10.5%38.4%+27.9%

What it tests

This condition applies peak clipping to the clean recordings, simulating an overloaded microphone or audio path. The clips remain paired one-to-one with the clean baseline and use the same transcripts and durations.

This condition is derived from the WildASR environment family and FLEURS read speech. Each paired condition uses the same utterance and transcript as the WildASR clean baseline, so the WER change isolates the effect of the test condition.

Metrics reported for this dataset

Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.

Evaluate your own voice agent

Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.

Book a Demo