Pulse 2.0

English-only streaming speech to text with built-in end-of-turn detection, per-utterance emotion, and per-utterance gender - all from one WebSocket. Start with the realtime quickstart.

Who it’s for

• Voice agents and live call analytics that need to know when the caller finished speaking and how they sounded, on every final.
• English-only streaming audio where built-in turn detection replaces a hand-tuned silence timeout.
• Not the pick for multilingual or batch audio. Use Pulse or Pulse Pro.

Model Overview

Key Capabilities

Performance & Benchmarks

Best for: voice agents and live call analytics, where you need to know when the caller finished speaking and how they sounded.

Transcription accuracy (WER)

Unchanged from Pulse. The emotion and gender heads are a LoRA adapter plus classification heads added on top of the shared encoder, not a retrain of it, so recognition accuracy is identical to shipped Pulse - see the Pulse model card for WER benchmarks.

End-of-turn detection

Evaluated on livekit/eot-bench, English validation split (400 turns, 705 hold spans, 400 EOT spans). Every model streamed through its own live server and was scored against the same reference spans.

Sorted by false cutoffs at 300 ms, matching LiveKit’s own comparison table. Lower is better on every column; – means the model can’t reach that operating point.

Pulse EoT v1 ranks 3rd of 11 on false cutoffs at 600 ms and 3rd/4th on latency at the 5%/10% budgets - second only to LiveKit v1 among self-hosted models. Against Deepgram Flux specifically: it wins on false cutoffs at 600 ms (8.8% vs 9.9%) and latency at 5% (784 ms vs 1151 ms), and loses on false cutoffs at 300 ms and latency at 10%.

Signal-quality diagnostic (AUC / AP over the full span, independent of any operating-point choice):

2nd of 10 on raw signal quality. The gap between 2nd on discrimination and mid-pack on false cutoffs at the 300 ms budget is a timing effect, not a signal-quality one - the model knows the turn has ended, it just says so a little late.

Emotion detection

Evaluated on two corpora held out from training:

UA = unweighted accuracy, WA = weighted accuracy. Evaluated across nine emotion classes (anger, happiness, sadness, neutral, excitement, frustration, fear, surprise, disgust); the streaming API surfaces six classes per final - see Response Format.

Streaming introduces essentially no accuracy cost versus the direct (offline) model - per-corpus deltas average within about ±0.3 points, consistent with run-to-run noise rather than degradation.

Gender detection

95% F1, evaluated both intra-corpus and on held-out test sets.

Supported Languages

Pulse 2.0 is English-only.

Requesting any other language value returns a LANGUAGE_NOT_SUPPORTED_BY_MODEL error and closes the socket (see Error Frames). For multilingual or pre-recorded transcription, use standard Pulse (21 streaming + 12 pre-recorded languages).

Authentication

Include your API key in the Authorization header when opening the WebSocket:

Authorization: Bearer SMALLEST_API_KEY

Browsers can’t set headers on a WebSocket - mint a short-lived access token on your server and pass it as the api_key query parameter instead.

Request Parameters

Passed as query parameters on the WebSocket URL, for example:

wss://api.smallest.ai/waves/v1/stt/live?model=pulse-2&language=en&sample_rate=16000&encoding=linear16&emotion_detection=false

An unknown model is rejected at the handshake with HTTP 400 INVALID_MODEL.

Control Messages

Sent from the client as JSON text frames on the same WebSocket:

Response Format

Every server message is a JSON frame. Interim (is_final: false) messages carry a running transcript string only; finals carry the full set of fields below, subject to which params you set.

Real final from prod (words trimmed to two):

{
  "type": "transcription",
  "status": "success",
  "session_id": "e119d986-9ebc-449b-826d-94650fc09a5f",
  "transcript": " Good afternoon.",
  "is_final": true,
  "is_last": false,
  "eot_finalized": true,
  "language": "en",
  "languages": ["en"],
  "words": [
    {"word": "Good", "start": 1.14, "end": 1.46, "confidence": 0.9517, "speaker": 0, "speaker_confidence": 1},
    {"word": "afternoon.", "start": 1.46, "end": 2.1, "confidence": 0.9717, "speaker": 0, "speaker_confidence": 0.875}
  ],
  "emotion": {
    "label": "happy", "confidence": 0.5343,
    "scores": {"neutral": 0.4515, "happy": 0.5343, "angry": 0.0118, "sad": 0.0023, "fear": 0, "disgust": 0.0001}
  },
  "gender": {
    "label": "male", "confidence": 1,
    "scores": {"male": 1, "female": 0}
  }
}

Three things to keep in mind when integrating:

1. Treat emotion and gender as optional. They are left out (not null) when an utterance is too short or classification times out.
2. Expect fewer, longer finals. On a 25 s speech clip, Pulse 2.0 returned 5 finals, 4 of them eot_finalized, each ending on a sentence or clause.
3. The last final has no eot_finalized. It comes from closing the stream, marked by is_last: true instead.

Error Frames

{
  "type": "error",
  "error_code": "LANGUAGE_NOT_SUPPORTED_BY_MODEL",
  "message": "model=pulse-2 is English-only (language=en); got language='hi'.",
  "model": "pulse-2",
  "language": "hi"
}

Error frames always carry type: "error", error_code, and a human-readable message.

API Reference

Endpoint

See Transcribe (Realtime / WebSocket) for the full generated request/response schema, including this page’s params and fields.

There is no batch / pre-recorded Pulse 2.0. For high-volume English batch transcription, use Pulse Pro.

Throughput, Latency & Pricing

Customer pricing: $0.005 per minute of audio (Standard plan, streaming). Standard plan rate-limit default: 100 concurrent requests. Enterprise tier is unlimited and configurable per-customer.

Use Cases

Limitations

• English only. Use Pulse for every other language.
• Emotion and gender are not guaranteed on every final. They’re estimated from the voice of each utterance and can be left out on a very short utterance (under ~0.4 s) or a slow classification pass.
• Gender is a binary acoustic estimate of the voice, not an identity attribute. It describes how the speaker’s voice sounds to the model, not how the speaker identifies.

Safety & Compliance

Pulse 2.0 must not be used for:

• Recording or transcribing individuals without their explicit consent
• Surveillance, stalking, or any form of unauthorized monitoring
• Any illegal or unethical purposes

Additionally:

• Usage is monitored for policy compliance
• Gender output is an acoustic estimate, not an identity signal - do not use it to infer or record a speaker’s gender identity
• For compliance documentation (GDPR, SOC2, HIPAA), contact support@smallest.ai

FAQ

Support