Artificial Analysis launches the Speech Agent Arena for speech-to-speech voice agents
On 2026-08-24 Artificial Analysis launched the Speech Agent Arena, where people hold live conversations with two hidden speech-to-speech models across 15 agentic (tool-calling) and 20 non-agentic scenarios, then vote. It reports a preference Elo plus a task-success rate. At launch Gemini 3.1 Flash Live Preview led on preference, and Grok Voice Think Fast 2.0 led on task success (94.7%). It joined AA's 2026 voice leaderboards, which also include the Controlled Voice TTS arena (July 2026) and multilingual TTS arenas (Sept 2026).
Key facts
- Method: pairwise human votes after separate live conversations → Preference Elo; agentic task success = share of eligible conversations completed with the correct final tool call(s)
- Launch preference Elo: Gemini 3.1 Flash Live Preview (Minimal) 1,046; Gemini 3.1 Flash Live Preview (High) 1,014; OpenAI GPT-Realtime-1.5 1,000
- Launch task success: Grok Voice Think Fast 2.0 (High) 94.7%; OpenAI GPT-Realtime-2.1 (High) 91.5%
- Controlled Voice Arena (announced 2026-07-08): TTS models compared on the same 8 cloned voices (US/UK, male/female). Initial leader Cartesia Sonic 3.5 (1,122), then Eleven v3 (1,088), Inworld Realtime TTS-2 (1,070)
- Multilingual TTS arenas for 9 languages beyond English announced 2026-09-22
What happened
Artificial Analysis, the independent benchmarking firm, added an arena for end-to-end voice agents. Earlier speech arenas (TTS preference, STT WER) scored single components. The Speech Agent Arena scores whole conversations with speech-to-speech models, including whether the agent actually did the requested action through tool calls.
Why it matters
Voice agents are being sold into customer service, where finishing the task matters more than sounding natural. The two launch leaderboards disagree: the preferred-sounding model is not the most reliable one. That split is now measured in public. Launch rankings were taken from AA's article and will change as models are added.
Changelog
- 2026-09-29: created (also covers the Controlled Voice and multilingual TTS arenas)
Related events
- xAI releases Grok Voice Think Fast 2.0 speech-to-speech model for voice agents ★★★
- ElevenLabs launches Eleven v4 and Eleven v4 Turbo, #1 on Artificial Analysis TTS arena ★★★★
- Cartesia Sonic-3.6 goes GA and tops the Artificial Analysis Speech Arena ★★★
Sources (4)
- officialArtificial Analysis: Announcing the Speech Agent Arena
- officialArtificial Analysis on X: Controlled Voice Arena announcement
- officialArtificial Analysis Controlled Voice leaderboard
- officialArtificial Analysis on X: Multilingual TTS Arena leaderboards
id: 2026-08-24-artificial-analysis-speech-agent-arena · updated 2026-09-29 · open in the interactive timeline