FLEURS | Smallest AI Docs

FLEURS Streaming - English

WER on the English subset of FLEURS across providers in streaming mode. Lower is better.

Provider	Smallest Pulse	Assembly Universal 3 Pro	AWS Transcribe	Azure	Deepgram Nova 3	Grok	Sarvam Saras V3	ElevenLabs Scribe V2
WER	5.55%	3.13%	6.54%	13.79%	11.59%	60.00%	6.34%	3.88%

A note on audio amplitude normalization

Audio amplitude normalization materially changes WER on FLEURS. Most competitors benchmark on raw FLEURS - which has variable, often low amplitude - without normalizing peak audio to −10 dBFS. This makes some models look much better than they actually are. Pulse is stable across all amplitude regimes.

Model	Raw FLEURS	−10 dBFS	−20 dBFS	Stable across regimes?
Smallest Pulse	5.55%	6.06%	5.81%	Yes
Deepgram Nova 3	11.59%	6.57%	6.51%	Partial - 1.8× degradation on raw
Grok	60.00%	7.58%	8.59%	Collapses on raw

Pre-recorded - FLEURS

Google’s multilingual speech dataset covering 102 languages, built on the FLoRes-101 translation benchmark. Contains ~12 hours of read speech per language and is the standard benchmark for evaluating multilingual ASR, including low-resource languages.

Evaluated on the FLEURS dataset (non-streaming / batch mode).

Language	Smallest Pulse	Deepgram Nova 2	Deepgram Nova 3
Italian	3.0%	10.7%	6.2%
English	4.5%	7.9%	6.7%
Spanish	3.2%	8.6%	4.1%
Portuguese	5.0%	9.9%	7.5%
German	6.4%	8.2%	8.5%
French	7.1%	13.3%	10.7%
Russian	9.6%	7.9%	11.8%
Ukrainian	7.5%	12.4%	NA
Polish	10.3%	12.2%	NA
Hindi	6.3%	23.5%	23.6%
Czech	12.4%	22.9%	19.2%
Slovak	13.5%	31.2%	NA
Dutch	15.0%	16.3%	12.5%
Swedish	18.7%	17.7%	14.3%
Finnish	18.3%	14.1%	13.2%
Latvian	16.5%	48.7%	NA
Romanian	17.8%	36.0%	NA
Estonian	17.8%	49.0%	NA
Bulgarian	24.1%	32.7%	NA
Danish	19.8%	21.1%	16.1%
Hungarian	22.5%	31.8%	28.6%
Maltese	25.5%	NA	NA
Lithuanian	25.1%	44.9%	NA

Sources: Deepgram internal benchmarks; Smallest AI internal evaluation.

Streaming - FLEURS

Evaluated on the FLEURS dataset (streaming mode).

Language	Smallest Pulse	Deepgram Nova 2	Deepgram Nova 3
Italian	4.41	11.05	6.99
English	5.55	15.59	11.59
Spanish	5.99	10.67	7.52
Portuguese	8.32	14.15	11.46
German	9.5	11.1	10.15
French	10.71	14.3	12.07
Russian	14.35	NA	NA
Hindi	8.3	20.0	15.46
Gujarati	20.05	NA	NA
Marathi	15.68	NA	NA
Oriya	22.74	NA	NA
Bengali	17.48	NA	NA
Dutch	11.90	NA	NA

Sources: Deepgram internal benchmarks; Smallest AI internal evaluation.