Voice AI BenchmarksBy

OUT OF SETTEXT-TO-SPEECH3,656 SAMPLES / 30 DAYSLAST RUN OCT 19, 2026, 21:00 UTC

Airy TTS text-to-speech benchmarks

Airy TTS, hosted by Out of Set, measures mean 270 ms time to first audio (18th of 32) and 1.6% word error rate (8th of 32) among TTS systems. Results cover the last 30 days. Coval benchmarks hosted endpoints daily using a consistent evaluation methodology. Last measured .

Coval has active text-to-speech measurements for Airy TTS, created by Out of Set and served on Out of Set.

Time to First Audio#18 / 32
270ms
TTFA Network Roundtrip#25 / 32
265ms
TTFA Leading Silence#2 / 32
5ms
Word Error Rate#8 / 32
1.6%

Overview

Airy TTS is tested every day on a fixed set of text prompts, measuring how quickly audible speech starts and how intelligible the result is.

Technical specifications

Made by
Out of Set
Hosted by
Out of Set
Source
Official API
Licensing
Proprietary
Deployment
Cloud
Region
US

How Airy TTS ranks

Full TTS dashboard
Time to First Audio — every measured TTS modelMilliseconds · lower is better · 30-day averageEvery TTS model ranked on Time to First Audio, with Airy TTS highlighted.
  1. #1vui56 ms
  2. #2Default63 ms
  3. #3Qwen3 TTS Fast68 ms
  4. #4TTS Flash 285 ms
  5. #5Qwen3 TTS 1.7b106 ms
  6. #6Simba 3.0110 ms
  7. #7Palabra TTS120 ms
  8. #8Simba 3.2129 ms
  9. #9drama3-preview175 ms
  10. #10TTS 2182 ms
  11. #11Flash v2.5194 ms
  12. #12Eleven v4 Turbo197 ms
  13. #13Flux TTS204 ms
  14. #14Phantom Z 3.4 conversational241 ms
  15. #15TTS RT v1256 ms
  16. #16TTS Rt v2257 ms
  17. #17Falcon 2264 ms
  18. #18Airy TTS270 ms
Show all 32 models
  1. #19Mist v3287 ms
  2. #20Sonic 3.5290 ms
  3. #21Sonic 3.6295 ms
  4. #22Aura 2298 ms
  5. #23Coda332 ms
  6. #24S2.1 Pro347 ms
  7. #25Grok TTS356 ms
  8. #26Lightning v3.1 Pro394 ms
  9. #27Daytona TTS485 ms
  10. #28Aura 2 En559 ms
  11. #29Chirp 3 HD597 ms
  12. #30Qwen3 TTS Flash Realtime720 ms
  13. #31S2.1 Pro Free844 ms
  14. #32GPT-4o mini TTS2549 ms
median of all models · 261 ms
TTS models over the last 30 days, ranked on Time to First Audio.
#ModelHostTTFAWERSamples
1vuiFluxions56 ms2,166
2Gradium TTS Beta 202609Gradium63 ms2,418
3Qwen3 TTS FastNari Labs68 ms2,418
4TTS Flash 2Inworld AI85 ms2,222
5Qwen3 TTS 1.7bBaseten106 ms1,728
6Simba 3.0SpeechifyAI110 ms2,415
7Palabra TTS v1Palabra120 ms1,997
8Simba 3.2SpeechifyAI129 ms2,412
9Drama 3 Preview Us EastFish Audio175 ms664
10TTS 2Inworld AI182 ms2,222
11Flash v2.5ElevenLabs194 ms2,408
12Eleven v4 TurboElevenLabs197 ms1,728
13Flux TTSDeepgram204 ms2,259
14Phantom Z 3.4 conversationalDeepdub241 ms2,153
15TTS RT v1Soniox256 ms2,044
16TTS Rt v2Soniox257 ms2,042
17Falcon 2Murf264 ms2,417
18Airy TTSOut of Set270 ms912
19Mist v3Rime287 ms602
20Sonic 3.5Cartesia290 ms2,418
21Sonic 3.6Cartesia295 ms2,418
22Aura 2Deepgram298 ms2,350
23CodaRime332 ms602
24S2.1 ProFish Audio347 ms2,418
25Grok TTSSpaceXAI356 ms2,413
26Lightning v3.1 ProSmallest394 ms2,411
27Daytona TTSGuava485 ms1,632
28Aura 2 EnCloudflare559 ms2,416
29Chirp 3 HDGoogle597 ms2,418
30Qwen3 TTS Flash RealtimeAlibaba720 ms2,417
31S2.1 Pro FreeFish Audio844 ms2,409
32GPT-4o mini TTSOpenAI2549 ms2,273
Under-sampled models are excluded; tied models share a place. Dotted WER values split into substitutions, deletions and insertions on hover or tap.

Highest relative placement: 8th of 32 on Word Error Rate.

Latency vs accuracy

TTFA vs WEREach point is one measured model · Airy TTS highlighted · 30-day averagesTime to First Audio against Word Error Rate for every measured TTS model, with Airy TTS highlighted.
vui — 56 ms, 2.1%Default — 63 ms, 1.9%Qwen3 TTS Fast — 68 ms, 1.9%TTS Flash 2 — 85 ms, 1.7%Qwen3 TTS 1.7b — 106 ms, 1.9%Simba 3.0 — 110 ms, 1.6%Palabra TTS — 120 ms, 2.7%Simba 3.2 — 129 ms, 2.0%drama3-preview — 175 ms, 1.8%TTS 2 — 182 ms, 1.7%Flash v2.5 — 194 ms, 1.9%Eleven v4 Turbo — 197 ms, 1.7%Flux TTS — 204 ms, 1.7%Phantom Z 3.4 conversational — 241 ms, 2.2%TTS RT v1 — 256 ms, 2.5%TTS Rt v2 — 257 ms, 2.4%Falcon 2 — 264 ms, 1.7%Mist v3 — 287 ms, 2.1%Sonic 3.5 — 290 ms, 1.6%Sonic 3.6 — 295 ms, 1.5%Aura 2 — 298 ms, 1.9%Coda — 332 ms, 9.5%S2.1 Pro — 347 ms, 1.6%Grok TTS — 356 ms, 1.7%Lightning v3.1 Pro — 394 ms, 1.6%Daytona TTS — 485 ms, 1.7%Aura 2 En — 559 ms, 2.1%Chirp 3 HD — 597 ms, 1.6%Qwen3 TTS Flash Realtime — 720 ms, 3.1%S2.1 Pro Free — 844 ms, 1.6%GPT-4o mini TTS — 2549 ms, 1.8%Airy TTS — 270 ms, 1.6%

Where the errors come from

WER compositionAiry TTS's Word Error Rate split by error type · 30-day averageAiry TTS's WER split into substitutions, deletions and insertions.
  • Airy TTS1.6%
SubstitutionsDeletionsInsertions

Averages and tail latency

Averages hide slow outliers — these are the distributions behind each figure.

Airy TTS latency distributionMilliseconds · shared axis across metrics · last 30 daysAiry TTS's latency percentiles per metric: p25–p75 band, p50 tick, whisker to max.
  • Time to First Audiop50 212 ms · Max 2040 ms
  • TTFA Network Roundtripp50 207 ms · Max 2035 ms
  • TTFA Leading Silencep50 5 ms · Max 11 ms
band p25–p75 · tick p50 · whisker to max with p90 and p95 stops
Average and percentile values per metric, with the number of samples behind each row.
MetricAveragep25p50p75p90p95MaxSamples
Time to First Audio270 ms191 ms212 ms285 ms378 ms489 ms2040 ms912
TTFA Network Roundtrip265 ms187 ms207 ms282 ms373 ms487 ms2035 ms912
TTFA Leading Silence5 ms2 ms5 ms7 ms9 ms9 ms11 ms912
Word Error Rate1.6%0.0%0.0%2.3%6.5%10.7%43.8%920

Last 30 days

Daily medians from the same measurement runs · gaps are days without qualifying runs.

Time to First Audio — daily p50Line p50 · band p25–p75 · UTC daysAiry TTS's daily median Time to First Audio over the last 30 days.
Airy TTS · Oct 10: 298 msAiry TTS · Oct 11: 242 msAiry TTS · Oct 12: 207 msAiry TTS · Oct 13: 229 msAiry TTS · Oct 14: 223 msAiry TTS · Oct 15: 219 msAiry TTS · Oct 16: 220 msAiry TTS · Oct 17: 206 msAiry TTS · Oct 18: 221 msAiry TTS · Oct 19: 220 ms
Word Error Rate — daily averageDaily average · UTC daysAiry TTS's daily Word Error Rate over the last 30 days.
Airy TTS · Oct 10: 1.0%Airy TTS · Oct 11: 2.0%Airy TTS · Oct 12: 3.3%Airy TTS · Oct 13: 3.1%Airy TTS · Oct 14: 4.0%Airy TTS · Oct 15: 4.2%Airy TTS · Oct 16: 3.7%Airy TTS · Oct 17: 3.2%Airy TTS · Oct 18: 2.3%Airy TTS · Oct 19: 3.5%

How fast is Airy TTS?

On Out of Set, Airy TTS measures mean 270 ms time to first audio (18th of 32). Last measured 2026-10-19.

How accurate is Airy TTS?

On Out of Set, Airy TTS measures 1.6% word error rate (8th of 32). Last measured 2026-10-19.

Who hosts Airy TTS?

Airy TTS is created by Out of Set and served by Out of Set. Coval measures each hosted endpoint separately.

Results are re-measured daily using fixed datasets and reported over a rolling 30-day window. Same datasets, prompts and metric definitions for every model, measured by Coval’s open-source runner. Full methodology on the overview.

Evaluate your own voice agent

Use Coval to test your production configuration, prompts and calls—not only the public benchmark endpoints.

Book a Demo