FLEURS

Pulse word error rate on FLEURS, streaming and pre-recorded, per language.

FLEURS Streaming - English

WER on the English subset of FLEURS across providers in streaming mode. Lower is better.

Provider	Smallest Pulse	Assembly Universal 3 Pro	AWS Transcribe	Azure	Deepgram Nova 3	Grok	Sarvam Saras V3	ElevenLabs Scribe V2
WER	5.55%	3.13%	6.54%	13.79%	11.59%	60.00%	6.34%	3.88%

A note on audio amplitude normalization

Audio amplitude normalization materially changes WER on FLEURS. Most competitors benchmark on raw FLEURS - which has variable, often low amplitude - without normalizing peak audio to −10 dBFS. This makes some models look much better than they actually are. Pulse is stable across all amplitude regimes.

Model	Raw FLEURS	−10 dBFS	−20 dBFS	Stable across regimes?
Smallest Pulse	5.55%	6.06%	5.81%	Yes
Deepgram Nova 3	11.59%	6.57%	6.51%	Partial - 1.8× degradation on raw
Grok	60.00%	7.58%	8.59%	Collapses on raw

Pre-recorded - FLEURS

Google’s multilingual speech dataset covering 102 languages, built on the FLoRes-101 translation benchmark. Contains ~12 hours of read speech per language and is the standard benchmark for evaluating multilingual ASR, including low-resource languages.

Evaluated on the FLEURS dataset (non-streaming / batch mode).

Language	Smallest Pulse	Deepgram Nova 2	Deepgram Nova 3
Italian	3.0%	10.7%	6.2%
English	4.5%	7.9%	6.7%
Spanish	3.2%	8.6%	4.1%
Portuguese	5.0%	9.9%	7.5%
German	6.4%	8.2%	8.5%
French	7.1%	13.3%	10.7%
Russian	9.6%	7.9%	11.8%
Ukrainian	7.5%	12.4%	NA
Polish	10.3%	12.2%	NA
Hindi	6.3%	23.5%	23.6%
Czech	12.4%	22.9%	19.2%
Slovak	13.5%	31.2%	NA
Dutch	15.0%	16.3%	12.5%
Swedish	18.7%	17.7%	14.3%
Finnish	18.3%	14.1%	13.2%
Latvian	16.5%	48.7%	NA
Romanian	17.8%	36.0%	NA
Estonian	17.8%	49.0%	NA
Bulgarian	24.1%	32.7%	NA
Danish	19.8%	21.1%	16.1%
Hungarian	22.5%	31.8%	28.6%
Maltese	25.5%	NA	NA
Lithuanian	25.1%	44.9%	NA

Sources: Deepgram internal benchmarks; Smallest AI internal evaluation.

Streaming - FLEURS

Evaluated on the FLEURS dataset (streaming mode). For Indian languages, see the FLEURS - Indic languages section below.

Language	Smallest Pulse	Deepgram Nova 2	Deepgram Nova 3
Italian	4.41	11.05	6.99
English	5.55	15.59	11.59
Spanish	5.99	10.67	7.52
Portuguese	8.32	14.15	11.46
German	9.5	11.1	10.15
French	10.71	14.3	12.07
Russian	14.35	NA	NA
Dutch	11.90	NA	NA

Sources: Deepgram internal benchmarks; Smallest AI internal evaluation.

FLEURS - Indic languages

FLEURS WER on nine Indian languages. Lower is better. - means the provider does not support that language, so it was not evaluated. Per-dataset results are on the Indic languages page. Results are from the Pulse streaming (WebSocket) API.

Language	Pulse	Deepgram	Scribe	Soniox	Sarvam code-switch	Sarvam	Cartesia
Hindi	6.17%	14.08%	10.67%	5.54%	8.62%	4.18%	9.06%
Marathi	13.6%	29.48%	32.80%	19.75%	17.13%	12.56%	-
Gujarati	10.8%	22.12%	110.70%	30.18%	16.34%	11.83%	-
Bengali	12.4%	23.18%	78.05%	15.13%	16.68%	11.34%	-
Odia	16.6%	-	77.65%	-	19.46%	14.36%	-
Tamil	16.14%	25.67%	49.76%	23.89%	15.60%	14.60%	-
Telugu	18.90%	26.44%	66.51%	23.56%	25.48%	16.46%	-
Kannada	12.79%	26.51%	64.59%	19.30%	20.92%	12.00%	-
Malayalam	13.32%	-	63.84%	25.74%	29.81%	14.21%	-