Language detection

Enabling language detection

Set the language query parameter to a regional aggregator when calling the API. Pulse will auto-detect the spoken language across the codes covered by that aggregator. Available aggregators differ by mode:

When you already know the source language, pass the explicit code (en, hi, es, etc.) for the best accuracy. For unknown European audio on pre-recorded, use multi-eu. For Indian audio, use the table below.

Which value to use for Indian languages

indic is streaming only and served from the India region (wss://api.smallest.ai). On wss://api.us.smallest.ai, the server sends an error and closes the connection:

{
  "type": "error",
  "error_code": "LANGUAGE_NOT_ENABLED_IN_REGION",
  "message": "Language 'indic' has not been enabled in this region. Please contact support to request access.",
  "language": "indic",
  "region": "us-west-2"
}

Pre-Recorded API

# Download sample audio (or use your own file)
curl -sL -o audio.wav "https://github.com/smallest-inc/cookbook/raw/main/speech-to-text/getting-started/samples/audio.wav"
curl --request POST \
  --url "https://api.smallest.ai/waves/v1/stt/?model=pulse&language=multi-eu&word_timestamps=true" \
  --header "Authorization: Bearer $SMALLEST_API_KEY" \
  --header "Content-Type: audio/wav" \
  --data-binary "@audio.wav"

Real-Time WebSocket API

const url = new URL("wss://api.smallest.ai/waves/v1/stt/live?model=pulse");
url.searchParams.append("language", "indic");
url.searchParams.append("encoding", "linear16");
url.searchParams.append("sample_rate", "16000");
const ws = new WebSocket(url.toString(), {
  headers: {
    Authorization: `Bearer ${API_KEY}`,
  },
});

Output format & field of interest

When language detection is enabled, the transcription (or transcript for realtime), words, and utterances arrays are emitted in the detected language. The response includes a language field with the detected primary language code, and a languages array (in realtime responses where is_final=true) listing all detected languages. Persist the detected locale in your app by storing the language parameter you supplied (for auditing) and by inspecting downstream metadata such as subtitles or captions that inherit the localized transcript.

Sample response

Pre-Recorded API Response

{
  "status": "success",
  "transcription": "Hola mundo.",
  "words": [
    { "start": 0.0, "end": 0.4, "word": "Hola" },
    { "start": 0.5, "end": 0.9, "word": "mundo." }
  ],
  "utterances": [
    { "text": "Hola mundo.", "start": 0.0, "end": 0.9 }
  ]
}

Real-Time WebSocket API Response

{
  "session_id": "sess_12345abcde",
  "transcript": "Hola mundo.",
  "is_final": true,
  "is_last": false,
  "language": "es",
  "languages": ["es"]
}

Real-Time response with indic

With language=indic, each final transcript carries the language that was detected for that utterance, so one session can return different languages. This caller asks a question in Hindi mixed with English, then switches to Tamil. These are the two final messages from the same session:

{
  "transcript": "मेरा account number बताए और next payment date क्या है?",
  "is_final": true,
  "start": 0,
  "duration": 4.4,
  "language": "hi",
  "languages": ["en", "hi"]
}

{
  "transcript": "வணக்கம். என் கடன் தவணைப் பற்றி தெரிந்து கொள்ள வேண்டும்.",
  "is_final": true,
  "start": 4.4,
  "duration": 4.6,
  "language": "ta",
  "languages": ["ta"]
}

Use language as the main language of the utterance. languages lists every language heard in it, so Hindi with English words returns both hi and en. English words inside Hindi speech stay in Roman script.