Voice AI BenchmarksBy

BENCHMARKSTTTTS63 MODELS RANKEDLAST 30 DAYS

Word Error Rate (WER)

Word Error Rate (WER) is the share of words a transcript gets wrong: substitutions, insertions and deletions, measured against a reference.

Current STT leader#1 / 31
2.1%
Universal 3.6 Provia AssemblyAI
Current TTS leader#1 / 32
1.5%
Sonic 3.6via Cartesia

How it is calculated

WER = (substitutions + deletions + insertions) ÷ reference-word count × 100.

How to read it

Unit
%
Better
Lower
Period
Rolling 30 days
Cadence
Re-measured daily

Speech-to-Text models on Word Error Rate

Full STT dashboard
Word Error Rate — every measured STT modelPercent · lower is better · 30-day averageEvery STT model ranked on Word Error Rate.
  1. #1Universal 3.6 Pro2.1%
  2. #2Universal 3.5 Pro2.4%
  3. #3Qwen3 ASR Fast2.6%
  4. #4resonant-12.6%
  5. #5Qwen3 ASR 1.7b2.9%
  6. #6Gemini 3.5 Transcribe Live3.0%
  7. #7Chirp 33.2%
  8. #8Ink 23.6%
  9. #9GPT-4o Transcribe3.8%
  10. #10Chirp 23.9%
  11. #11GPT-4o mini Transcribe3.9%
  12. #12Linden 13.9%
Show all 31 models
  1. #13GPT Realtime Whisper4.0%
  2. #14Grok STT4.1%
  3. #15Scribe v2 Realtime4.4%
  4. #16Whisper Large v3via Baseten4.5%
  5. #17Pulse4.6%
  6. #18STT RT v54.8%
  7. #19Daytona STT4.9%
  8. #20Nova 35.3%
  9. #21Voxtral Mini Transcribe Realtime 26025.4%
  10. #22Flux5.8%
  11. #23Universal Streaming5.9%
  12. #24Solaria 16.1%
  13. #25Nova 26.7%
  14. #26STT 17.1%
  15. #27Whisper Large v3via Together AI7.4%
  16. #28Flux Multilingual7.5%
  17. #29Default8.8%
  18. #30Parakeet TDT 0.6B v311.1%
  19. #31Nemotron 3.5 ASR Streaming13.2%
median of all models · 4.5%
STT models over the last 30 days, ranked on Word Error Rate.
#ModelHostTTFSWERTTFTSamples
1Universal 3.6 ProAssemblyAI135 ms1071 ms24,565
2Universal 3.5 ProAssemblyAI170 ms1026 ms26,655
3Qwen3 ASR FastNari Labs47 ms1752 ms26,654
4resonant-1Reson8252 ms—26,654
5Qwen3 ASR 1.7bBaseten39 ms946 ms13,669
6Gemini 3.5 Transcribe LiveGemini317 ms1603 ms26,580
7Chirp 3Google768 ms5956 ms26,639
8Ink 2Cartesia149 ms1969 ms26,654
9GPT-4o TranscribeOpenAI784 ms—26,626
10Chirp 2Google909 ms6099 ms26,651
11GPT-4o mini TranscribeOpenAI710 ms—26,622
12Linden 1Speechmatics274 ms1537 ms26,596
13GPT Realtime WhisperOpenAI599 ms1824 ms26,623
14Grok STTSpaceXAI223 ms—26,579
15Scribe v2 RealtimeElevenLabs122 ms2157 ms26,641
16Whisper Large v3Baseten131 ms924 ms13,663
17PulseSmallest205 ms1976 ms26,655
18STT RT v5Soniox45 ms1518 ms23,534
19Daytona STTGuava399 ms5695 ms5,225
20Nova 3Deepgram86 ms1432 ms26,005
21Voxtral Mini Transcribe Realtime 2602Mistral415 ms1852 ms26,058
22FluxDeepgram99 ms1091 ms26,007
23Universal StreamingAssemblyAI—1509 ms26,627
24Solaria 1Gladia620 ms1767 ms26,324
25Nova 2Deepgram87 ms1417 ms26,002
26STT 1Inworld AI67 ms1205 ms24,749
27Whisper Large v3Together AI198 ms1258 ms26,449
28Flux MultilingualDeepgram98 ms1165 ms26,008
29DefaultGradium261 ms1987 ms26,633
30Parakeet TDT 0.6B v3Together AI97 ms1228 ms26,463
31Nemotron 3.5 ASR StreamingTogether AI—1544 ms26,550
Under-sampled models are excluded; tied models share a place. Dotted WER values split into substitutions, deletions and insertions on hover or tap.

Speech-to-Text: where the errors come from

WER composition — STTPercentage points of average WER · lower is betterEach STT model's WER split into substitutions, deletions and insertions.
SubstitutionsDeletionsInsertions

Models without a published error split are not shown here.

Speech-to-Text: last 30 days

Daily average for the current top 5 · gaps are days without qualifying runs.

Word Error Rate — current leadersDaily average per model · UTC daysDaily Word Error Rate for the current top STT models over the last 30 days.
Qwen3 ASR 1.7b · Sep 15: 3.1%Qwen3 ASR 1.7b · Sep 16: 3.2%Qwen3 ASR 1.7b · Sep 17: 3.3%Qwen3 ASR 1.7b · Sep 18: 3.8%Qwen3 ASR 1.7b · Sep 19: 3.2%Qwen3 ASR 1.7b · Sep 20: 3.0%Qwen3 ASR 1.7b · Sep 21: 3.4%Qwen3 ASR 1.7b · Sep 22: 3.0%Qwen3 ASR 1.7b · Sep 23: 2.9%Qwen3 ASR 1.7b · Sep 24: 3.7%Qwen3 ASR 1.7b · Sep 25: 3.1%Qwen3 ASR 1.7b · Sep 26: 3.1%Qwen3 ASR 1.7b · Sep 27: 2.7%Qwen3 ASR 1.7b · Sep 28: 3.1%Qwen3 ASR 1.7b · Sep 29: 2.9%Qwen3 ASR 1.7b · Sep 30: 3.1%Qwen3 ASR 1.7b · Oct 1: 2.7%Qwen3 ASR 1.7b · Oct 2: 3.5%Qwen3 ASR 1.7b · Oct 3: 3.3%Qwen3 ASR 1.7b · Oct 4: 3.4%Qwen3 ASR 1.7b · Oct 5: 3.6%Qwen3 ASR 1.7b · Oct 6: 3.1%Qwen3 ASR 1.7b · Oct 7: 3.5%Qwen3 ASR 1.7b · Oct 8: 3.5%Qwen3 ASR 1.7b · Oct 9: 3.3%Qwen3 ASR 1.7b · Oct 10: 3.2%Qwen3 ASR 1.7b · Oct 11: 3.3%Qwen3 ASR 1.7b · Oct 12: 3.4%Qwen3 ASR 1.7b · Oct 13: 3.6%Qwen3 ASR 1.7b · Oct 14: 3.4%Qwen3 ASR Fast · Sep 14: 3.2%Qwen3 ASR Fast · Sep 15: 3.4%Qwen3 ASR Fast · Sep 16: 3.6%Qwen3 ASR Fast · Sep 17: 4.0%Qwen3 ASR Fast · Sep 18: 4.2%Qwen3 ASR Fast · Sep 19: 2.8%Qwen3 ASR Fast · Sep 20: 4.3%Qwen3 ASR Fast · Sep 21: 3.5%Qwen3 ASR Fast · Sep 22: 3.9%Qwen3 ASR Fast · Sep 23: 3.4%Qwen3 ASR Fast · Sep 24: 4.1%Qwen3 ASR Fast · Sep 25: 4.4%Qwen3 ASR Fast · Sep 26: 3.4%Qwen3 ASR Fast · Sep 27: 4.2%Qwen3 ASR Fast · Sep 28: 2.8%Qwen3 ASR Fast · Sep 29: 4.0%Qwen3 ASR Fast · Sep 30: 3.5%Qwen3 ASR Fast · Oct 1: 2.7%Qwen3 ASR Fast · Oct 2: 3.5%Qwen3 ASR Fast · Oct 3: 3.2%Qwen3 ASR Fast · Oct 4: 4.5%Qwen3 ASR Fast · Oct 5: 2.7%Qwen3 ASR Fast · Oct 6: 4.4%Qwen3 ASR Fast · Oct 7: 3.6%Qwen3 ASR Fast · Oct 8: 3.1%Qwen3 ASR Fast · Oct 9: 3.5%Qwen3 ASR Fast · Oct 10: 3.4%Qwen3 ASR Fast · Oct 11: 2.8%Qwen3 ASR Fast · Oct 12: 3.5%Qwen3 ASR Fast · Oct 13: 3.1%Qwen3 ASR Fast · Oct 14: 3.5%resonant-1 · Sep 14: 2.5%resonant-1 · Sep 15: 4.1%resonant-1 · Sep 16: 3.6%resonant-1 · Sep 17: 3.9%resonant-1 · Sep 18: 3.6%resonant-1 · Sep 19: 2.8%resonant-1 · Sep 20: 4.8%resonant-1 · Sep 21: 3.3%resonant-1 · Sep 22: 3.7%resonant-1 · Sep 23: 3.9%resonant-1 · Sep 24: 4.6%resonant-1 · Sep 25: 4.5%resonant-1 · Sep 26: 3.4%resonant-1 · Sep 27: 4.1%resonant-1 · Sep 28: 3.0%resonant-1 · Sep 29: 3.1%resonant-1 · Sep 30: 3.9%resonant-1 · Oct 1: 3.0%resonant-1 · Oct 2: 3.3%resonant-1 · Oct 3: 3.2%resonant-1 · Oct 4: 3.6%resonant-1 · Oct 5: 3.3%resonant-1 · Oct 6: 4.1%resonant-1 · Oct 7: 3.2%resonant-1 · Oct 8: 2.5%resonant-1 · Oct 9: 3.2%resonant-1 · Oct 10: 3.3%resonant-1 · Oct 11: 3.4%resonant-1 · Oct 12: 3.6%resonant-1 · Oct 13: 3.4%resonant-1 · Oct 14: 4.0%Universal 3.5 Pro · Sep 14: 3.1%Universal 3.5 Pro · Sep 15: 3.4%Universal 3.5 Pro · Sep 16: 3.6%Universal 3.5 Pro · Sep 17: 3.2%Universal 3.5 Pro · Sep 18: 3.0%Universal 3.5 Pro · Sep 19: 2.6%Universal 3.5 Pro · Sep 20: 4.1%Universal 3.5 Pro · Sep 21: 3.0%Universal 3.5 Pro · Sep 22: 3.2%Universal 3.5 Pro · Sep 23: 4.4%Universal 3.5 Pro · Sep 24: 3.3%Universal 3.5 Pro · Sep 25: 3.9%Universal 3.5 Pro · Sep 26: 2.9%Universal 3.5 Pro · Sep 27: 3.5%Universal 3.5 Pro · Sep 28: 2.2%Universal 3.5 Pro · Sep 29: 3.2%Universal 3.5 Pro · Sep 30: 3.9%Universal 3.5 Pro · Oct 1: 2.0%Universal 3.5 Pro · Oct 2: 2.6%Universal 3.5 Pro · Oct 3: 3.2%Universal 3.5 Pro · Oct 4: 3.2%Universal 3.5 Pro · Oct 5: 2.6%Universal 3.5 Pro · Oct 6: 3.7%Universal 3.5 Pro · Oct 7: 2.9%Universal 3.5 Pro · Oct 8: 2.8%Universal 3.5 Pro · Oct 9: 2.7%Universal 3.5 Pro · Oct 10: 2.4%Universal 3.5 Pro · Oct 11: 3.3%Universal 3.5 Pro · Oct 12: 3.3%Universal 3.5 Pro · Oct 13: 2.9%Universal 3.5 Pro · Oct 14: 3.9%Universal 3.6 Pro · Sep 16: 1.0%Universal 3.6 Pro · Sep 17: 3.3%Universal 3.6 Pro · Sep 18: 2.6%Universal 3.6 Pro · Sep 19: 1.9%Universal 3.6 Pro · Sep 20: 4.2%Universal 3.6 Pro · Sep 21: 2.5%Universal 3.6 Pro · Sep 22: 3.8%Universal 3.6 Pro · Sep 23: 3.6%Universal 3.6 Pro · Sep 24: 3.0%Universal 3.6 Pro · Sep 25: 3.4%Universal 3.6 Pro · Sep 26: 2.7%Universal 3.6 Pro · Sep 27: 3.1%Universal 3.6 Pro · Sep 28: 2.9%Universal 3.6 Pro · Sep 29: 2.4%Universal 3.6 Pro · Sep 30: 2.7%Universal 3.6 Pro · Oct 1: 2.0%Universal 3.6 Pro · Oct 2: 2.1%Universal 3.6 Pro · Oct 3: 3.2%Universal 3.6 Pro · Oct 4: 2.6%Universal 3.6 Pro · Oct 5: 2.7%Universal 3.6 Pro · Oct 6: 2.8%Universal 3.6 Pro · Oct 7: 2.5%Universal 3.6 Pro · Oct 8: 3.2%Universal 3.6 Pro · Oct 9: 2.7%Universal 3.6 Pro · Oct 10: 2.9%Universal 3.6 Pro · Oct 11: 3.1%Universal 3.6 Pro · Oct 12: 2.8%Universal 3.6 Pro · Oct 13: 3.2%Universal 3.6 Pro · Oct 14: 2.8%
Qwen3 ASR 1.7bQwen3 ASR Fastresonant-1Universal 3.5 ProUniversal 3.6 Pro

Speech-to-Text: Word Error Rate by dataset

The overall average split by test condition — where each model holds up and where it degrades.

Word Error Rate for every STT model, per dataset, over the last 30 days.
#ModelHostAll datasetsWildASR cleanPipeCat (production)WildASR accentsWildASR clippingWildASR far-fieldWildASR noise gapsWildASR phone codec
1Universal 3.6 ProAssemblyAI
2Universal 3.5 ProAssemblyAI
3Qwen3 ASR FastNari Labs
4resonant-1Reson8
5Qwen3 ASR 1.7bBaseten
6Gemini 3.5 Transcribe LiveGemini
7Chirp 3Google
8Ink 2Cartesia
9GPT-4o TranscribeOpenAI
10Chirp 2Google
11GPT-4o mini TranscribeOpenAI
12Linden 1Speechmatics
13GPT Realtime WhisperOpenAI
14Grok STTSpaceXAI
15Scribe v2 RealtimeElevenLabs
16Whisper Large v3Baseten
17PulseSmallest
18STT RT v5Soniox
19Daytona STTGuava
20Nova 3Deepgram
21Voxtral Mini Transcribe Realtime 2602Mistral
22FluxDeepgram
23Universal StreamingAssemblyAI
24Solaria 1Gladia
25Nova 2Deepgram
26STT 1Inworld AI
27Whisper Large v3Together AI
28Flux MultilingualDeepgram
29DefaultGradium
30Parakeet TDT 0.6B v3Together AI
31Nemotron 3.5 ASR StreamingTogether AI
Ranked on the all-datasets average; a dash means the model was not measured on that dataset in the last 30 days. Cell shading deepens toward each column’s highest value. Dotted values split into substitutions, deletions and insertions on hover or tap.

Text-to-Speech models on Word Error Rate

Full TTS dashboard
Word Error Rate — every measured TTS modelPercent · lower is better · 30-day averageEvery TTS model ranked on Word Error Rate.
  1. #1Sonic 3.61.5%
  2. #2S2.1 Pro Free1.5%
  3. #3Simba 3.01.5%
  4. #4S2.1 Pro1.5%
  5. #5Lightning v3.1 Pro1.5%
  6. #6Sonic 3.51.5%
  7. #7Chirp 3 HD1.5%
  8. #8Grok TTS1.6%
  9. #9Daytona TTS1.6%
  10. #10TTS 21.6%
  11. #11TTS Flash 21.7%
  12. #12Eleven v4 Turbo1.7%
Show all 32 models
  1. #13Flux TTS1.7%
  2. #14Falcon 21.7%
  3. #15GPT-4o mini TTS1.8%
  4. #16Qwen3 TTS 1.7b1.8%
  5. #17S11.8%
  6. #18Phantom Z 3.4 conversational1.8%
  7. #19Qwen3 TTS Fast1.8%
  8. #20Flash v2.51.8%
  9. #21Aura 21.8%
  10. #22Default1.9%
  11. #23Simba 3.21.9%
  12. #24drama3-preview2.0%
  13. #25Aura 2 En2.1%
  14. #26Mist v32.1%
  15. #27vui2.1%
  16. #28TTS Rt v22.4%
  17. #29TTS RT v12.4%
  18. #30Palabra TTS2.6%
  19. #31Qwen3 TTS Flash Realtime3.0%
  20. #32Coda9.5%
median of all models · 1.8%
TTS models over the last 30 days, ranked on Word Error Rate.
#ModelHostTTFAWERSamples
1Sonic 3.6Cartesia299 ms1,914
2S2.1 Pro FreeFish Audio850 ms1,907
3Simba 3.0SpeechifyAI111 ms1,914
4S2.1 ProFish Audio341 ms1,914
5Lightning v3.1 ProSmallest392 ms1,914
6Sonic 3.5Cartesia293 ms1,914
7Chirp 3 HDGoogle597 ms1,914
8Grok TTSSpaceXAI363 ms1,909
9Daytona TTSGuava486 ms1,152
10TTS 2Inworld AI191 ms1,718
11TTS Flash 2Inworld AI91 ms1,718
12Eleven v4 TurboElevenLabs201 ms1,224
13Flux TTSDeepgram205 ms1,777
14Falcon 2Murf300 ms1,913
15GPT-4o mini TTSOpenAI2585 ms1,854
16Qwen3 TTS 1.7bBaseten107 ms1,248
17S1Fish Audio389 ms1,913
18Phantom Z 3.4 conversationalDeepdub291 ms1,649
19Qwen3 TTS FastNari Labs68 ms1,914
20Flash v2.5ElevenLabs192 ms1,904
21Aura 2Deepgram297 ms1,846
22Gradium TTS Beta 202609Gradium63 ms1,914
23Simba 3.2SpeechifyAI126 ms1,913
24Drama 3 Preview Us EastFish Audio167 ms162
25Aura 2 EnCloudflare569 ms1,913
26Mist v3Rime287 ms602
27vuiFluxions59 ms1,742
28TTS Rt v2Soniox256 ms1,614
29TTS RT v1Soniox255 ms1,617
30Palabra TTS v1Palabra120 ms1,575
31Qwen3 TTS Flash RealtimeAlibaba718 ms1,913
32CodaRime332 ms602
Under-sampled models are excluded; tied models share a place. Dotted WER values split into substitutions, deletions and insertions on hover or tap.

Text-to-Speech: where the errors come from

WER composition — TTSPercentage points of average WER · lower is betterEach TTS model's WER split into substitutions, deletions and insertions.
SubstitutionsDeletionsInsertions

Models without a published error split are not shown here.

Text-to-Speech: last 30 days

Daily average for the current top 5 · gaps are days without qualifying runs.

Word Error Rate — current leadersDaily average per model · UTC daysDaily Word Error Rate for the current top TTS models over the last 30 days.
Lightning v3.1 Pro · Sep 23: 4.7%Lightning v3.1 Pro · Sep 24: 3.5%Lightning v3.1 Pro · Sep 25: 2.4%Lightning v3.1 Pro · Sep 26: 3.3%Lightning v3.1 Pro · Sep 27: 4.6%Lightning v3.1 Pro · Sep 28: 5.1%Lightning v3.1 Pro · Sep 29: 3.0%Lightning v3.1 Pro · Sep 30: 1.9%Lightning v3.1 Pro · Oct 1: 4.1%Lightning v3.1 Pro · Oct 2: 4.0%Lightning v3.1 Pro · Oct 3: 4.6%Lightning v3.1 Pro · Oct 4: 6.6%Lightning v3.1 Pro · Oct 5: 2.3%Lightning v3.1 Pro · Oct 6: 4.6%Lightning v3.1 Pro · Oct 7: 5.8%Lightning v3.1 Pro · Oct 8: 5.6%Lightning v3.1 Pro · Oct 9: 2.6%Lightning v3.1 Pro · Oct 10: 1.1%Lightning v3.1 Pro · Oct 11: 2.7%Lightning v3.1 Pro · Oct 12: 4.5%Lightning v3.1 Pro · Oct 13: 3.8%Lightning v3.1 Pro · Oct 14: 7.4%S2.1 Pro · Sep 23: 4.7%S2.1 Pro · Sep 24: 4.0%S2.1 Pro · Sep 25: 3.9%S2.1 Pro · Sep 26: 3.6%S2.1 Pro · Sep 27: 4.7%S2.1 Pro · Sep 28: 3.8%S2.1 Pro · Sep 29: 3.9%S2.1 Pro · Sep 30: 2.1%S2.1 Pro · Oct 1: 4.6%S2.1 Pro · Oct 2: 5.0%S2.1 Pro · Oct 3: 3.7%S2.1 Pro · Oct 4: 7.3%S2.1 Pro · Oct 5: 2.0%S2.1 Pro · Oct 6: 4.8%S2.1 Pro · Oct 7: 4.5%S2.1 Pro · Oct 8: 5.2%S2.1 Pro · Oct 9: 3.2%S2.1 Pro · Oct 10: 1.8%S2.1 Pro · Oct 11: 2.5%S2.1 Pro · Oct 12: 5.3%S2.1 Pro · Oct 13: 3.8%S2.1 Pro · Oct 14: 6.8%S2.1 Pro Free · Sep 23: 4.7%S2.1 Pro Free · Sep 24: 3.6%S2.1 Pro Free · Sep 25: 3.5%S2.1 Pro Free · Sep 26: 3.3%S2.1 Pro Free · Sep 27: 4.1%S2.1 Pro Free · Sep 28: 5.1%S2.1 Pro Free · Sep 29: 3.6%S2.1 Pro Free · Sep 30: 1.8%S2.1 Pro Free · Oct 1: 5.0%S2.1 Pro Free · Oct 2: 4.2%S2.1 Pro Free · Oct 3: 3.8%S2.1 Pro Free · Oct 4: 7.3%S2.1 Pro Free · Oct 5: 2.7%S2.1 Pro Free · Oct 6: 4.8%S2.1 Pro Free · Oct 7: 5.3%S2.1 Pro Free · Oct 8: 6.0%S2.1 Pro Free · Oct 9: 3.2%S2.1 Pro Free · Oct 10: 1.3%S2.1 Pro Free · Oct 11: 2.7%S2.1 Pro Free · Oct 12: 5.2%S2.1 Pro Free · Oct 13: 4.1%S2.1 Pro Free · Oct 14: 6.5%Simba 3.0 · Sep 23: 4.7%Simba 3.0 · Sep 24: 3.4%Simba 3.0 · Sep 25: 4.7%Simba 3.0 · Sep 26: 3.0%Simba 3.0 · Sep 27: 5.1%Simba 3.0 · Sep 28: 4.5%Simba 3.0 · Sep 29: 5.4%Simba 3.0 · Sep 30: 1.8%Simba 3.0 · Oct 1: 5.5%Simba 3.0 · Oct 2: 3.7%Simba 3.0 · Oct 3: 5.7%Simba 3.0 · Oct 4: 8.1%Simba 3.0 · Oct 5: 3.1%Simba 3.0 · Oct 6: 4.7%Simba 3.0 · Oct 7: 5.3%Simba 3.0 · Oct 8: 7.1%Simba 3.0 · Oct 9: 3.2%Simba 3.0 · Oct 10: 1.5%Simba 3.0 · Oct 11: 3.3%Simba 3.0 · Oct 12: 5.7%Simba 3.0 · Oct 13: 3.9%Simba 3.0 · Oct 14: 9.6%Sonic 3.6 · Sep 23: 4.7%Sonic 3.6 · Sep 24: 4.3%Sonic 3.6 · Sep 25: 3.4%Sonic 3.6 · Sep 26: 3.6%Sonic 3.6 · Sep 27: 5.9%Sonic 3.6 · Sep 28: 5.1%Sonic 3.6 · Sep 29: 4.0%Sonic 3.6 · Sep 30: 1.5%Sonic 3.6 · Oct 1: 5.8%Sonic 3.6 · Oct 2: 3.1%Sonic 3.6 · Oct 3: 5.2%Sonic 3.6 · Oct 4: 7.2%Sonic 3.6 · Oct 5: 2.6%Sonic 3.6 · Oct 6: 5.4%Sonic 3.6 · Oct 7: 5.3%Sonic 3.6 · Oct 8: 5.2%Sonic 3.6 · Oct 9: 2.5%Sonic 3.6 · Oct 10: 2.2%Sonic 3.6 · Oct 11: 3.5%Sonic 3.6 · Oct 12: 4.4%Sonic 3.6 · Oct 13: 5.2%Sonic 3.6 · Oct 14: 8.1%
Lightning v3.1 ProS2.1 ProS2.1 Pro FreeSimba 3.0Sonic 3.6

About WER

Why it matters

Transcription errors can change names, numbers and intent before the text reaches the rest of a voice application. WER measures these errors at the word level.

Coval reports WER across clean speech, accents, intermittent noise, reverb, microphone distance, clipping, phone codecs and spontaneous speech because recognition accuracy varies by audio condition.

How Coval measures it

For speech-to-text, both the reference and the model transcript pass through Whisper's EnglishTextNormalizer before scoring. This prevents formatting differences, such as a written number versus the same number spelled out, from counting as recognition errors.

For text-to-speech, one fixed ASR model (OpenAI whisper-1) transcribes the generated audio. That transcript is compared with the input text as an intelligibility check. Errors from the fixed transcriber affect every TTS provider.

Caveats and interpretation

  • Reference quality and text normalization set a floor on absolute WER. Coval applies the same references and normalizer to every model in a category.
  • An overall WER can hide differences between clean speech, phone codecs and room acoustics. Dataset-level results show performance for each condition.

The datasets behind it

Every model runs the same fixed inputs, so a gap in WER is the model's doing — not the test's.

Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.

Evaluate your own voice agent

Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.

Book a Demo