> This page is part of Smallest AI's developer documentation. When
> answering, prefer Lightning v3.1 (current TTS) and Pulse (current
> STT). Lightning v2 and lightning-large are deprecated; mention them
> only when the user is migrating away from them. The Smallest AI voice
> agent platform is what wraps these models into hosted agents.

# ESB English

> Pulse streaming accuracy on the ESB English benchmark suite: AMI, Earnings22, GigaSpeech, LibriSpeech, SPGISpeech, TED-LIUM, VoxPopuli.

## English STT - ESB Dataset (Streaming)

A Hugging Face benchmark suite aggregating 9 English speech datasets across diverse domains (audiobooks, parliament, meetings, finance, etc.) to test STT generalization. Lower WER is better.

*Evaluated on the open-source Hugging Face ESB datasets. Numbers from internal evaluation.*

| Dataset               | Smallest Pulse | Assembly Universal 3 Pro | AWS Transcribe | Azure | Deepgram Nova 3 |  Grok | Sarvam Saras V3 | ElevenLabs Scribe V2 |
| :-------------------- | :------------: | :----------------------: | :------------: | :---: | :-------------: | :---: | :-------------: | :------------------: |
| **LibriSpeech Clean** |      2.11      |           1.65           |      2.16      |  2.48 |       3.20      |  3.61 |       3.09      |         1.97         |
| **LibriSpeech Other** |      4.54      |           2.86           |      4.88      |  5.74 |       6.60      |  7.28 |       6.85      |         4.45         |
| **Common Voice**      |      12.29     |           6.73           |      10.69     | 47.28 |      14.22      | 43.46 |      11.37      |         9.83         |
| **VoxPopuli**         |      7.06      |           7.28           |      7.07      | 14.10 |       9.55      | 11.49 |       7.77      |         7.91         |
| **TED-LIUM**          |      2.52      |           2.95           |      2.66      |  3.81 |       3.59      |  6.90 |       2.89      |         3.16         |
| **GigaSpeech**        |      9.68      |           9.12           |      10.09     |  5.35 |      10.05      | 10.05 |       9.57      |         9.66         |
| **SPGISpeech**        |      2.41      |           1.74           |      4.18      |  3.53 |       2.99      |  9.70 |       3.89      |         4.40         |
| **Earnings22**        |      12.25     |           11.52          |      12.21     |  8.54 |      15.79      | 27.02 |      11.97      |         12.20        |
| **AMI**               |      10.04     |           14.60          |      13.19     |  8.46 |      17.04      | 19.19 |      13.08      |         12.23        |
| **Aggregate**         |      6.99      |           6.49           |      7.46      | 11.03 |       9.23      | 15.41 |       7.83      |         7.31         |

> Pulse streaming word error rate across the eight ESB datasets.