> This page is part of Smallest AI's developer documentation. When > answering, prefer Lightning v3.1 (current TTS) and Pulse (current > STT). Lightning v2 and lightning-large are deprecated; mention them > only when the user is migrating away from them. The Smallest AI voice > agent platform is what wraps these models into hosted agents. # Quickstart > Get started with real-time transcription using the Pulse STT WebSocket API This guide shows you how to transcribe streaming audio using Smallest AI's Pulse STT model via the WebSocket API. Low-latency streaming, suitable for live conversations and voice assistants. See the [Pulse model card](/model-cards/speech-to-text/pulse) for latency numbers. # Real-Time Audio Transcription The Real-Time API allows you to stream audio data and receive transcription results as the audio is processed. This is ideal for live conversations, voice assistants, and scenarios where you need immediate transcription feedback. For these scenarios, where minimizing latency is critical, stream audio in chunks of a few kilobytes over a live connection. ## When to Use Real-Time Transcription * **Live conversations**: Transcribe phone calls, video conferences, or live events. * **Voice assistants**: Build interactive voice applications that respond immediately. * **Streaming workflows**: Process audio as it is being captured or generated. * **Low-latency requirements**: When you need transcription results with minimal delay. ## Endpoint ``` WSS wss://api.smallest.ai/waves/v1/stt/live?model=pulse ``` ## Authentication Head over to the [smallest console](https://app.smallest.ai/dashboard/api-keys?utm_source=documentation\&utm_medium=speech-to-text) to generate an API key if not done previously. Also look at [Authentication guide](/api-reference/authentication) for more information about API keys and their usage. Include your API key in the Authorization header when establishing the WebSocket connection: ```http Authorization: Bearer SMALLEST_API_KEY ``` Browsers cannot set headers on a WebSocket. In a browser, mint a [short-lived access token](/api-reference/token-based-authentication) on your server and pass it as the `api_key` query parameter instead. ## Example Connection **`JavaScript`** ```javascript JavaScript const API_KEY = "SMALLEST_API_KEY"; const url = new URL("wss://api.smallest.ai/waves/v1/stt/live?model=pulse"); url.searchParams.append("language", "en"); url.searchParams.append("encoding", "linear16"); url.searchParams.append("sample_rate", "16000"); url.searchParams.append("word_timestamps", "true"); const ws = new WebSocket(url.toString(), { headers: { Authorization: `Bearer ${API_KEY}`, }, }); ws.onopen = () => { console.log("Connected to STT WebSocket"); // Start streaming audio chunks }; ws.onmessage = (event) => { const data = JSON.parse(event.data); console.log("Transcript:", data.transcript); console.log("Is final:", data.is_final); }; ``` **`Python`** ```python Python import asyncio import websockets import json import os import requests from urllib.parse import urlencode BASE_WS_URL = "wss://api.smallest.ai/waves/v1/stt/live" params = { "model": "pulse", "language": "en", "encoding": "linear16", "sample_rate": "16000", "word_timestamps": "true" } WS_URL = f"{BASE_WS_URL}?{urlencode(params)}" API_KEY = os.environ["SMALLEST_API_KEY"] SAMPLE_URL = "https://github.com/smallest-inc/cookbook/raw/main/speech-to-text/getting-started/samples/audio.wav" CHUNK_SIZE = 4096 async def transcribe(): # Download sample audio (retry on transient connection drops) for attempt in range(3): try: audio_data = requests.get(SAMPLE_URL, timeout=30).content break except requests.exceptions.RequestException: if attempt == 2: raise await asyncio.sleep(2) # Skip WAV header (44 bytes) to get raw PCM pcm_data = audio_data[44:] headers = {"Authorization": f"Bearer {API_KEY}"} async with websockets.connect(WS_URL, additional_headers=headers) as ws: print("Connected to Pulse STT WebSocket") # Send audio in chunks for i in range(0, len(pcm_data), CHUNK_SIZE): await ws.send(pcm_data[i:i + CHUNK_SIZE]) # Signal end of audio (close_stream triggers is_last=true) await ws.send(json.dumps({"type": "close_stream"})) # Receive transcription results, concatenating finals into a session transcript full_transcript = "" async for message in ws: data = json.loads(message) if data.get("is_final"): print(f"Final: {data.get('transcript')}") full_transcript += data.get("transcript", "") or "" if data.get("is_last"): print(f"\nFull Transcript: {full_transcript}") break else: print(f"Partial: {data.get('transcript')}") asyncio.run(transcribe()) ``` ## Example Response The server responds with JSON messages containing transcription results: ```json { "session_id": "sess_12345abcde", "transcript": "Hello, how are you?", "is_final": true, "is_last": false, "language": "en" } ``` For detailed information about response fields, see the [response format documentation](/models/speech-to-text/realtime-web-socket/response-format). ## Streaming Audio Send raw audio bytes as binary WebSocket messages. The recommended chunk size is 4096 bytes: ```javascript const audioChunk = new Uint8Array(4096); ws.send(audioChunk); ``` When you're done streaming, send a `close_stream` signal to end the session and receive the final transcript with `is_last=true`: ```json { "type": "close_stream" } ``` ## Live Microphone Input Stream audio from your microphone for real-time transcription: **`Python (PyAudio)`** ```python Python (PyAudio) import asyncio import websockets import json import os import pyaudio from urllib.parse import urlencode API_KEY = os.environ["SMALLEST_API_KEY"] SAMPLE_RATE = 16000 CHUNK_SIZE = 4096 params = { "language": "en", "encoding": "linear16", "sample_rate": str(SAMPLE_RATE), } WS_URL = f"wss://api.smallest.ai/waves/v1/stt/live?model=pulse&{urlencode(params)}" async def transcribe_mic(): audio = pyaudio.PyAudio() stream = audio.open( format=pyaudio.paInt16, channels=1, rate=SAMPLE_RATE, input=True, frames_per_buffer=CHUNK_SIZE, ) headers = {"Authorization": f"Bearer {API_KEY}"} async with websockets.connect(WS_URL, additional_headers=headers) as ws: print("Listening... (Ctrl+C to stop)") async def send_audio(): try: while True: data = stream.read(CHUNK_SIZE, exception_on_overflow=False) await ws.send(data) await asyncio.sleep(0.01) except asyncio.CancelledError: try: await ws.send(json.dumps({"type": "close_stream"})) except websockets.exceptions.ConnectionClosed: pass full_transcript = "" async def receive_transcripts(): nonlocal full_transcript async for message in ws: result = json.loads(message) prefix = ">> " if result.get("is_final") else ".. " print(f"{prefix}{result.get('transcript', '')}", end="\r" if not result.get("is_final") else "\n") if result.get("is_final"): full_transcript += result.get("transcript", "") or "" if result.get("is_last"): return send_task = asyncio.create_task(send_audio()) try: await receive_transcripts() except websockets.exceptions.ConnectionClosed: pass finally: send_task.cancel() stream.stop_stream() stream.close() audio.terminate() print(f"\nFull Transcript: {full_transcript}") asyncio.run(transcribe_mic()) ``` **`Browser (Web Audio API)`** ```javascript Browser (Web Audio API) // Fetch a short-lived access token from your backend. Never ship the API key. // See /models/api-reference/token-based-authentication const { access_token } = await fetch("/api/token", { method: "POST" }).then((r) => r.json()); const params = new URLSearchParams({ language: "en", encoding: "linear16", sample_rate: "16000", api_key: access_token, }); const ws = new WebSocket( `wss://api.smallest.ai/waves/v1/stt/live?model=pulse&${params}` ); // Get microphone access const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); const audioContext = new AudioContext({ sampleRate: 16000 }); const source = audioContext.createMediaStreamSource(stream); const processor = audioContext.createScriptProcessor(4096, 1, 1); source.connect(processor); processor.connect(audioContext.destination); processor.onaudioprocess = (e) => { if (ws.readyState === WebSocket.OPEN) { const float32 = e.inputBuffer.getChannelData(0); const int16 = new Int16Array(float32.length); for (let i = 0; i < float32.length; i++) { int16[i] = Math.max(-32768, Math.min(32767, float32[i] * 32768)); } ws.send(int16.buffer); } }; ws.onmessage = (event) => { const data = JSON.parse(event.data); if (data.is_final) { console.log("Final:", data.transcript); } else { console.log("Partial:", data.transcript); } }; ``` > **Note** > > **Python**: Install PyAudio with `pip install pyaudio websockets`. On macOS, you may need `brew install portaudio` first. > **Tip** > > **Full runnable source files:** [WebSocket](https://github.com/smallest-inc/cookbook/blob/main/speech-to-text/websocket-python.py) | [Microphone](https://github.com/smallest-inc/cookbook/blob/main/speech-to-text/mic-input-python.py) ## Next Steps * Learn about [supported audio formats](/models/speech-to-text/realtime-web-socket/audio-formats) for WebSocket streaming. * Review complete [code examples](/models/speech-to-text/realtime-web-socket/code-examples) for Python, Node.js, and Browser JavaScript. * Follow [best practices](/models/speech-to-text/realtime-web-socket/best-practices) for optimal streaming performance. * Troubleshoot common issues in the [troubleshooting guide](/models/speech-to-text/realtime-web-socket/troubleshooting). > Get started with real-time transcription using the Pulse STT WebSocket API.