Speaker diarization

Diarization labels each word and utterance with an integer speaker id in the same response you already get for transcription. One API call, no separate diarization vendor, no reconciliation of two timelines.

Available on Pulse in both batch (POST /waves/v1/stt/) and streaming (WSS /waves/v1/stt/live).

Language-agnostic: works across every language Pulse supports.

Enable

Add diarize=true. Pair with word_timestamps=true if you want per-word speaker labels (per-utterance labels come regardless).

Batch (Pre-Recorded)

$
curl -sL -o audio.wav "https://github.com/smallest-inc/cookbook/raw/main/speech-to-text/getting-started/samples/audio.wav"
$
$
curl --request POST \
>
  --url "https://api.smallest.ai/waves/v1/stt/?model=pulse&language=en&diarize=true&word_timestamps=true" \
>
  --header "Authorization: Bearer $SMALLEST_API_KEY" \
>
  --header "Content-Type: audio/wav" \
>
  --data-binary "@audio.wav"

Streaming (Real-Time WebSocket)

1
const url = new URL("wss://api.smallest.ai/waves/v1/stt/live?model=pulse");
2
url.searchParams.append("language", "en");
3
url.searchParams.append("encoding", "linear16");
4
url.searchParams.append("sample_rate", "16000");
5
url.searchParams.append("diarize", "true");
6
url.searchParams.append("word_timestamps", "true");
7
8
const ws = new WebSocket(url.toString(), {
9
  headers: { Authorization: `Bearer ${API_KEY}` },
10
});

Response shape

The response carries two arrays that both gain speaker labels when diarize=true:

• words[]. Per-word entries. Present when word_timestamps=true (batch) or on final events (streaming). Each entry gains speaker (integer) and speaker_confidence (0.0 to 1.0) when diarization is on.
• utterances[]. Sentence-level entries. Present by default on batch, and on streaming with sentence_timestamps=true. Each entry gains speaker (integer) when diarization is on.

Speaker labels are integers, zero-indexed (0, 1, 2, …) on both batch and streaming. Labels are stable within a single response or session, not across separate requests. Speaker 0 in one call is not necessarily the same physical speaker as speaker 0 in the next.

Batch response

1
{
2
  "status": "success",
3
  "transcription": "Agent: Hello, how can I help. Customer: I need to reset my password.",
4
  "words": [
5
    { "start": 0.24, "end": 0.40, "word": "Hello,",   "confidence": 0.998, "speaker": 0, "speaker_confidence": 1.00 },
6
    { "start": 0.40, "end": 0.72, "word": "how",      "confidence": 0.996, "speaker": 0, "speaker_confidence": 1.00 },
7
    { "start": 0.72, "end": 1.10, "word": "can",      "confidence": 0.999, "speaker": 0, "speaker_confidence": 1.00 },
8
    { "start": 1.20, "end": 1.60, "word": "I",        "confidence": 0.999, "speaker": 1, "speaker_confidence": 0.92 },
9
    { "start": 1.60, "end": 1.92, "word": "need",     "confidence": 0.997, "speaker": 1, "speaker_confidence": 0.94 }
10
  ],
11
  "utterances": [
12
    { "start": 0.24, "end": 1.10, "speaker": 0, "text": "Hello, how can I help." },
13
    { "start": 1.20, "end": 3.44, "speaker": 1, "text": "I need to reset my password." }
14
  ]
15
}

Streaming response

Streaming emits transcription events. Only events with is_final: true carry the words[] array; interim (partial) events send a running transcript string only.

1
// interim event (no words, no speaker attribution yet)
2
{ "type": "transcription", "transcript": "Hello, how can I", "is_final": false, "is_last": false }
3
4
// final event (words[] with speaker + speaker_confidence)
5
{
6
  "type": "transcription",
7
  "transcript": "Hello, how can I help.",
8
  "is_final": true,
9
  "is_last": false,
10
  "words": [
11
    { "word": "Hello,", "start": 0.32, "end": 0.80, "confidence": 0.964, "speaker": 0, "speaker_confidence": 0.87 },
12
    { "word": "how",    "start": 0.80, "end": 0.88, "confidence": 0.931, "speaker": 0, "speaker_confidence": 1.00 },
13
    { "word": "can",    "start": 1.28, "end": 1.52, "confidence": 0.959, "speaker": 0, "speaker_confidence": 1.00 },
14
    { "word": "I",      "start": 1.52, "end": 2.00, "confidence": 0.980, "speaker": 0, "speaker_confidence": 1.00 },
15
    { "word": "help.",  "start": 2.40, "end": 2.96, "confidence": 0.943, "speaker": 0, "speaker_confidence": 1.00 }
16
  ]
17
}

Practical consequences of the interim/final split:

• Do not render speaker attribution off interim events. They are transcript-string only. Speaker labels are correct only from is_final: true events.
• If you are driving live captions and want a running speaker chip on-screen, hold the current speaker from the last is_final event and update on the next one.

Response fields

Every other field on words (word, start, end, confidence) is present regardless of diarize. Same for utterances (text, start, end).

Gotchas

• Pulse Pro does not diarize. POST /waves/v1/stt/?model=pulse-pro&diarize=true returns a transcript-only response with no speaker fields on words[] and no utterances[]. Route diarization traffic to ?model=pulse.
• Interim events on streaming carry no speaker. Only finals do. See the note above.
• Labels are not identity. speaker: 0 in call A and speaker: 0 in call B are unrelated. If you need to persist “this is Alice” across calls, do the mapping yourself (voice-print embedding, dashboard tagging, etc).
• word_timestamps=true is required for per-word speakers on batch. Without it, words[] is empty and only utterances[] carries speaker labels.