Speaker tracking is the biggest complaint we hear from our users. We are not even close to fixing it.
That is the problem behind this model comparison. Digital reporters and legal transcriptionists working from audio need to know who said the words. A readable sentence with the wrong name attached still leaves a problem for the person preparing the transcript.
The published Record Test Bench leaderboard lets us look beyond a Gemini-only comparison. GPT-6.1 Sol and Claude Sonnet 5.5 belong in the discussion too. Here are their published Speaker Correct name accuracy scores alongside the three Gemini entries.
Gemini, Sol and Sonnet side by side
| Model | Speaker Correct name accuracy |
|---|---|
| gemini-3.8-flash (low) | 63.9% |
| gemini-4-argon (low) | 54.8% |
| gemini-3.1-flash-lite (minimal) | 43.1% |
| gpt-6.1-sol | 52.5% |
| claude-sonnet-5-5 | 48.1% |
Sol and Sonnet worked from transcript text; Gemini from audio.
From audio, Gemini 3.8 Flash scores highest at 63.9%; from transcript text, Sol leads at 52.5%. Even the best result leaves roughly a third of speaker turns not correctly named. Adding Sol and Sonnet does not change the central message: no model here is close to solving speaker tracking.
The Gemini results are spread out
Gemini 3.8 Flash at low reasoning scores 63.9%. Gemini 4 Argon at low reasoning scores 54.8%, while Gemini 3.1 Flash Lite at minimal reasoning scores 43.1%.
The gap between the two Flash entries is substantial. The shared family name does not make their speaker-name results interchangeable. If we are discussing the published audio chains, Gemini 3.8 Flash is the stronger result in this table.
But the user complaint is still there. We should not treat the leading score as a declaration that speaker tracking is fixed. Our users are still raising the problem, and the highest audio-track score still shows how much work is left.
Sol and Sonnet have the same unfinished job
GPT-6.1 Sol scores 52.5% on speaker-name accuracy. Claude Sonnet 5.5 scores 48.1%. Within that pair, Sol has the higher published result.
Neither result makes the speaker-tracking problem disappear. The question for the person editing the transcript is not whether a model belongs to a well-known family. It is whether the name attached to a speaker turn is right.
Including these entries matters because the problem is not confined to one model family. We can see different scores across Gemini, Sol and Sonnet, but none of these results justifies telling users that the problem they keep reporting has been solved.
Roughly a third is still a major gap
The best audio-track score on display is 63.9%. Put plainly, the remaining gap is roughly a third of speaker turns. That is why a leading result can be useful without being good enough to celebrate as a solution.
An ordinary sentence under the wrong name can send an editor back to the recording. That is exactly the kind of work our users want help with. A score improvement should serve that job, not become a substitute for it.
We need to keep comparing models, but we also need to stay honest about where the work stands. A stronger speaker-name result and a solved speaker-tracking problem are different milestones. We have the former in this comparison, not the latter.
The work left is ours
Speaker tracking is our users’ biggest complaint, and we are not even close to fixing it. That is the point we should lead with when we show these numbers.
For the broader results, read the first Record Test Bench comparison or open the published leaderboard. The result worth celebrating will be one that makes speaker tracking stop being our users’ biggest complaint.