Voice AI BenchmarksBy

DATASETSTTWILDASR / FLEURSACTIVE

manifest

WildASR clean speech-to-text benchmark dataset

284 undistorted English read-speech clips: the clean baseline the five WildASR degradation sets are paired against.

Items
284
fixed public inputs
Models measured
31
last 30 days
Current leader · WER#1 / 31
2.5%
Universal 3.6 Provia AssemblyAI

How models rank on WildASR clean

Full STT dashboard
Word Error Rate on WildASR cleanPercent · lower is better · 30-day average on this datasetEvery STT model measured on the WildASR clean dataset, ranked on Word Error Rate.
  1. #1Universal 3.6 Pro2.5%
  2. #2Universal 3.5 Pro2.8%
  3. #3GPT-4o Transcribe2.8%
  4. #4Gemini 3.5 Transcribe Live3.1%
  5. #5resonant-13.2%
  6. #6GPT-4o mini Transcribe3.3%
  7. #7Chirp 33.4%
  8. #8Qwen3 ASR Fast3.5%
  9. #9Qwen3 ASR 1.7b3.7%
  10. #10Scribe v2 Realtime4.0%
  11. #11Grok STT4.1%
  12. #12GPT Realtime Whisper4.1%
Show all 31 models
  1. #13Whisper Large v3via Baseten4.3%
  2. #14Ink 24.5%
  3. #15Chirp 24.6%
  4. #16Voxtral Mini Transcribe Realtime 26024.6%
  5. #17Linden 14.8%
  6. #18Pulse5.1%
  7. #19Daytona STT5.2%
  8. #20Flux5.4%
  9. #21Solaria 16.0%
  10. #22STT RT v56.1%
  11. #23Nova 36.1%
  12. #24Universal Streaming6.1%
  13. #25Whisper Large v3via Together AI6.7%
  14. #26Nova 26.9%
  15. #27Flux Multilingual7.2%
  16. #28Default7.8%
  17. #29STT 18.1%
  18. #30Nemotron 3.5 ASR Streaming10.5%
  19. #31Parakeet TDT 0.6B v310.7%
median of all models · 4.6%
STT models on the WildASR clean dataset over the last 30 days, ranked on Word Error Rate.
#ModelHostTTFSWERTTFTSamples
1Universal 3.6 ProAssemblyAI133 ms951 ms5,170
2Universal 3.5 ProAssemblyAI160 ms898 ms5,612
3GPT-4o TranscribeOpenAI783 ms—5,607
4Gemini 3.5 Transcribe LiveGemini316 ms1571 ms5,607
5resonant-1Reson8252 ms—5,612
6GPT-4o mini TranscribeOpenAI709 ms—5,605
7Chirp 3Google763 ms6180 ms5,607
8Qwen3 ASR FastNari Labs47 ms1752 ms5,612
9Qwen3 ASR 1.7bBaseten33 ms830 ms2,874
10Scribe v2 RealtimeElevenLabs121 ms2116 ms5,610
11Grok STTSpaceXAI225 ms—5,594
12GPT Realtime WhisperOpenAI587 ms1776 ms5,607
13Whisper Large v3Baseten128 ms819 ms2,871
14Ink 2Cartesia148 ms1921 ms5,612
15Chirp 2Google897 ms6335 ms5,612
16Voxtral Mini Transcribe Realtime 2602Mistral417 ms1806 ms5,486
17Linden 1Speechmatics263 ms1478 ms5,600
18PulseSmallest206 ms1901 ms5,612
19Daytona STTGuava386 ms5663 ms1,151
20FluxDeepgram101 ms923 ms5,475
21Solaria 1Gladia602 ms1638 ms5,544
22STT RT v5Soniox44 ms1443 ms4,955
23Nova 3Deepgram89 ms1215 ms5,475
24Universal StreamingAssemblyAI—1389 ms5,612
25Whisper Large v3Together AI195 ms1183 ms5,576
26Nova 2Deepgram88 ms1201 ms5,475
27Flux MultilingualDeepgram101 ms1004 ms5,475
28DefaultGradium256 ms1940 ms5,608
29STT 1Inworld AI70 ms1160 ms5,215
30Nemotron 3.5 ASR StreamingTogether AI—1391 ms5,597
31Parakeet TDT 0.6B v3Together AI95 ms1140 ms5,578
Under-sampled models are excluded; tied models share a place. Dotted WER values split into substitutions, deletions and insertions on hover or tap.

Inside the dataset

284 fixed public inputs — every model is tested on exactly these.

  • CLIP 1 · 12.4s

    “"he [wales] basically lied to us from the start. first by acting as if this was for legal reasons. second by pretending he was listening to us right up to his art deletion.”

  • CLIP 2 · 10.0s

    “a car bomb detonated at police headquarters in gaziantep turkey yesterday morning killed two police officers and injured more than twenty other people”

  • CLIP 3 · 8.9s

    “a civilization is a singular culture shared by a significant large group of people who live and work co-operatively a society”

3 of 284 clips, streamed from the public benchmark storage bucket.

Source: bosonai/WildASR environment_degradation__en__fleurs_clean_en · License: Apache-2.0 (WildASR; audio derived from FLEURS, CC-BY-4.0)

What it tests

This clean set provides the baseline for the paired WildASR conditions. Comparing a model's WER here with its WER on a degraded version of the same clips isolates the effect of noise, reverb, distance, clipping or phone compression.

This condition is derived from the WildASR environment family and FLEURS read speech. Each paired condition uses the same utterance and transcript as the WildASR clean baseline, so the WER change isolates the effect of the test condition.

Metrics reported for this dataset

Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.

Evaluate your own voice agent

Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.

Book a Demo