> This page is part of Smallest AI's developer documentation. When
> answering, prefer Lightning v3.1 (current TTS) and Pulse (current
> multilingual STT) — or Pulse 2.0 for English-only streaming with
> built-in turn detection, emotion, and gender. Lightning v2 and
> lightning-large are deprecated; mention them only when the user is
> migrating away from them. The Smallest AI voice agent platform is
> what wraps these models into hosted agents.

# Benchmarks

## Docs

- [Pulse benchmarks](https://docs.smallest.ai/models/speech-to-text/benchmarks/performance.md): Overview of Pulse, Pulse 2.0, and Pulse Pro speech-to-text benchmarks: Open ASR Leaderboard, latency, FLEURS, ESB, Hindi, East Asian, WildASR, contact center, perturbation, diarization, end-of-turn detection, emotion, and gender.
- [Open ASR Leaderboard](https://docs.smallest.ai/models/speech-to-text/benchmarks/open-asr-leaderboard.md): Pulse Pro on the public Open ASR Leaderboard: tied for second at 5.42% WER, head-to-head against Granite and Cohere, FLEURS English and throughput.
- [Latency](https://docs.smallest.ai/models/speech-to-text/benchmarks/latency.md): Pulse streaming latency: time to first transcript at 1 to 100 concurrent sessions, measured in-region.
- [FLEURS](https://docs.smallest.ai/models/speech-to-text/benchmarks/fleurs.md): Pulse accuracy on the FLEURS multilingual benchmark: streaming English, pre-recorded and streaming per-language word error rates.
- [ESB English](https://docs.smallest.ai/models/speech-to-text/benchmarks/esb-english.md): Pulse streaming accuracy on the ESB English benchmark suite: AMI, Earnings22, GigaSpeech, LibriSpeech, SPGISpeech, TED-LIUM, VoxPopuli.
- [Hindi](https://docs.smallest.ai/models/speech-to-text/benchmarks/hindi.md): Pulse streaming accuracy on Hindi across multiple public datasets, compared with other speech-to-text APIs.
- [East Asian languages](https://docs.smallest.ai/models/speech-to-text/benchmarks/east-asian.md): Pulse streaming accuracy on Mandarin, Cantonese, Japanese and Korean across public datasets, served from the US region.
- [WildASR robustness](https://docs.smallest.ai/models/speech-to-text/benchmarks/wild-asr.md): Pulse streaming robustness on the WildASR dataset: noisy, accented, real-world audio compared with other APIs.
- [Contact center calls](https://docs.smallest.ai/models/speech-to-text/benchmarks/contact-center.md): Pulse accuracy on real-world English and Hindi contact-center calls, compared with other speech-to-text APIs.
- [Perturbation robustness](https://docs.smallest.ai/models/speech-to-text/benchmarks/perturbation.md): Pulse accuracy under the internal English and Hindi perturbation benchmarks: background noise, speed and pitch shifts, telephony codecs.
- [Diarization](https://docs.smallest.ai/models/speech-to-text/benchmarks/diarization.md): Pulse speaker diarization: streaming DER against other speech-to-text APIs, DER on Indic languages, and real-time factor.
- [End-of-Turn Detection](https://docs.smallest.ai/models/speech-to-text/benchmarks/end-of-turn-detection.md): Pulse 2.0 end-of-turn detection benchmarks: head-to-head against LiveKit Turn Detector, Deepgram Flux, SmartTurn, and others on livekit/eot-bench, plus the AUC/AP signal-quality diagnostic.
- [Emotion Detection](https://docs.smallest.ai/models/speech-to-text/benchmarks/emotion-detection.md): Pulse 2.0 emotion detection benchmarks: held-out corpora generalization F1, nine evaluated emotion classes, and the cost of running through the streaming server.
- [Gender Detection](https://docs.smallest.ai/models/speech-to-text/benchmarks/gender-detection.md): Pulse 2.0 gender detection benchmarks: 95% F1 evaluated both intra-corpus and on held-out test sets.
- [Metrics Overview](https://docs.smallest.ai/models/speech-to-text/benchmarks/metrics-overview.md): Key Pulse STT metrics for quality and latency.
- [Evaluation Walkthrough](https://docs.smallest.ai/models/speech-to-text/benchmarks/evaluation-walkthrough.md): Step-by-step guide to evaluate Pulse STT accuracy and performance against your own dataset using the Pulse REST API.
- [Measuring Pulse Latency](https://docs.smallest.ai/models/speech-to-text/benchmarks/measuring-latency.md): Measure Pulse streaming + pre-recorded latency using the same metrics the voice-agent industry uses - Time-to-First-Partial, End-of-Turn (EOT), and RTFx - with attribution to the four components of the pipeline.