Pipecat TTS, STT, and LLM: Smallest AI Services

This guide walks you through integrating Smallest AI TTS, STT, and LLM into a Pipecat voice pipeline. Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents using a frame-based architecture.

Code Example

The complete runnable example lives in the Pipecat repository, using Smallest AI for STT, LLM, and TTS end to end:

Pipecat Example - Smallest AI STT + LLM + TTS

Setup

1. Create a Virtual Environment

python3.11 -m venv .venv

Activate it:

On Linux/Mac:
```
source .venv/bin/activate
```
On Windows:
```
.venv\Scripts\activate
```

2. Install Pipecat with Smallest AI support

The smallest extra installs the TTS, STT, and LLM services for Smallest AI:

pip install "pipecat-ai[smallest]"

To run the full voice agent example, you also need:

daily - Daily transport, which the bot uses to manage audio rooms and connect participants
openai - Required by SmallestLLMService, which connects to Electron through Pipecat’s OpenAI-compatible LLM service
silero - Silero VAD for voice activity detection and interruption handling
runner - Pipecat development runner that creates Daily rooms automatically and serves the bot locally

pip install "pipecat-ai[smallest,daily,openai,silero,runner]"

3. Create a `.env` file

SMALLEST_API_KEY=...
DAILY_API_KEY=...

DAILY_API_KEY is required - the Pipecat runner creates a Daily room automatically at startup. If you want to reuse an existing room instead of creating a new one each run, set the optional DAILY_ROOM_URL variable. A single SMALLEST_API_KEY covers STT, LLM, and TTS - no separate LLM provider key is needed.

Services

`SmallestSTTService`

from pipecat.services.smallest.stt import SmallestSTTService
from pipecat.transcriptions.language import Language
stt = SmallestSTTService(
    api_key=os.getenv("SMALLEST_API_KEY"),
    settings=SmallestSTTService.Settings(
        language=Language.EN,
        word_timestamps=True,
    ),
)

Constructor parameters:

Parameter	Type	Default	Description
`api_key`	`str`	-	Your Smallest AI API key (required)

Settings (SmallestSTTService.Settings):

Parameter	Type	Default	Description
`model`	`SmallestSTTModel`	`SmallestSTTModel.PULSE`	STT model. `PULSE` - multilingual streaming, 38 languages. `PULSE_2` (`model="pulse-2"`) - English-only streaming with built-in end-of-turn detection and per-final emotion/gender (see below)
`language`	`Language`	`Language.EN`	Language for transcription. `PULSE_2` is English-only — regional strings like `en-US` are accepted and normalized to `en`; any other language raises `ValueError` at construction or on a settings update
`word_timestamps`	`bool`	`False`	Include word-level timestamps
`full_transcript`	`bool`	`False`	Include cumulative transcript
`sentence_timestamps`	`bool`	`False`	Include sentence-level timestamps
`redact_pii`	`bool`	`False`	Redact personally identifiable information
`redact_pci`	`bool`	`False`	Redact payment card information
`numerals`	`str`	`"auto"`	Convert spoken numerals to digits
`diarize`	`bool`	`False`	Enable speaker diarization
`endpointing`	`bool`	`True`	Finalize transcripts promptly on trailing silence. No effect on `PULSE_2` (see Pulse 2.0 notes below)
`eou_timeout_ms`	`int`	`None`	Trailing-silence duration, in milliseconds (100-10000), before Pulse finalizes a segment on its own. Defaults to 600 with `endpointing=True` and 800 with `endpointing=False`. Governs mid-utterance segmentation, not turn-ending latency — Pipecat’s VAD-triggered `finalize` already flushes the turn’s last segment ahead of this timer. No effect on `PULSE_2`, which uses its own built-in end-of-turn detection. See End-of-Utterance Timeout
`keywords`	`str`	`""`	Comma-separated `KEYWORD:INTENSIFIER` pairs to boost recognition of domain-specific words/phrases (e.g. `"NVIDIA:2"`)
`format`	`bool`	`True`	Apply punctuation and capitalization to transcripts
`emotion_detection`	`bool`	`True`	`PULSE_2` only. Adds a six-class `emotion` object to every final transcript’s raw `result` payload. No effect on `PULSE`
`gender_detection`	`bool`	`True`	`PULSE_2` only. Adds a `gender` object to every final transcript’s raw `result` payload. No effect on `PULSE`
`itn_normalize`	`bool`	`False`	Run inverse text normalization so spoken numbers come back as digits (e.g. “twenty five” -> “25”)
`no_format_delay`	`bool`	`False`	With `itn_normalize` on, a number still being dictated at the end of a final is held back and merged into the next final so it normalizes whole. Set to `True` to emit every final as soon as it is ready instead. No effect unless `itn_normalize` is `True`; text is never dropped either way

The STT service connects to the Waves v4 STT endpoint (wss://api.smallest.ai/waves/v1/stt/live?model=pulse) and streams audio frames from the pipeline, returning transcriptions with 64ms TTFT. Pipecat’s VAD triggers a finalize message on each end-of-utterance to flush the transcript while keeping the WebSocket session open for the next utterance.

Pulse 2.0:

from pipecat.services.smallest.stt import SmallestSTTService, SmallestSTTModel
from pipecat.transcriptions.language import Language
stt = SmallestSTTService(
    api_key=os.getenv("SMALLEST_API_KEY"),
    settings=SmallestSTTService.Settings(
        model=SmallestSTTModel.PULSE_2,
        language=Language.EN,
    ),
)

Pass model=SmallestSTTModel.PULSE_2 to connect to wss://api.smallest.ai/waves/v1/stt/live?model=pulse-2 instead. Pulse 2.0 has a built-in end-of-turn detector, but Pipecat’s VAD and user turn strategies still decide when the user’s turn ends — the VAD-triggered finalize flushes the final transcript as with Pulse, and endpointing has no effect on this model. The rest of SmallestSTTSettings (diarization, PII/PCI redaction, keyword boosting, etc.) still applies. See the Pulse 2.0 model card for benchmarks and the full response shape.

`SmallestTTSService`

from pipecat.services.smallest.tts import SmallestTTSService, SmallestTTSModel
tts = SmallestTTSService(
    api_key=os.getenv("SMALLEST_API_KEY"),
    output_format="pcm",
    settings=SmallestTTSService.Settings(
        model=SmallestTTSModel.LIGHTNING_V3_1_PRO,
        voice="meher",
    ),
)

Constructor parameters:

Parameter	Type	Default	Description
`api_key`	`str`	-	Your Smallest AI API key (required)
`output_format`	`str`	`pcm`	Audio output format: `pcm`, `mp3`, `wav`, `ulaw`, `alaw`
`word_timestamps`	`bool`	`True`	Emit per-word `TTSTextFrame`s aligned to audio playback. Enabled by default. Supported on base-queue English + Hindi voices (`meher`, `devansh`, `kartik`, `maithili`, `liam`, `avery`); other voices silently emit no word events, so leaving this on is safe. Pass `word_timestamps=False` to fall back to whole-text frames.
`max_buffer_delay_ms`	`int`	`None`	Upper bound, in milliseconds (0-5000), on how long the server buffers a continuation fragment before speaking it. If `None`, the server’s default (3000ms) applies.

Settings (SmallestTTSService.Settings):

Parameter	Type	Default	Description
`model`	`SmallestTTSModel`	`lightning_v3.1_pro`	TTS model. `lightning_v3.1_pro`, or `lightning_v3.1`, which will be retired on Nov 30, 2026
`voice`	`str`	model-dependent	Voice ID. Plugin defaults: `meher` for `lightning_v3.1_pro`, `sophia` for `lightning_v3.1`
`language`	`Language`	`Language.EN`	Language for synthesis
`speed`	`float`	`None`	Speech speed multiplier (0.5–2.0)
`math_notation`	`bool`	`None`	Read digit-flanked math operators as words (`2 + 2` as “two plus two”) instead of leaving them to the default number reader. If `None`, the API default (off) applies.
`content_filter`	`"reject" \| "flag" \| "redact"`	`None`	Check the text against Smallest’s profanity word list before synthesis, taking this action on a match: `"reject"` fails the request with a `CONTENT_FILTER_BLOCKED` error and synthesizes nothing; `"flag"` synthesizes normally and records the match; `"redact"` removes the matched term and speaks the rest. If `None`, the API default (off) applies.

The TTS service connects to wss://api.smallest.ai/waves/v1/tts/live and uses WebSocket streaming for low-latency, real-time audio delivery. The model field is sent per-message, so switching models takes effect on the next utterance without reconnecting. Text fragments within the same LLM turn share a continuation context: they’re sent with the turn’s context_id and continue: true so the server joins them into one continuous generation instead of resetting prosody on each fragment, and the context is closed with continue: false once the turn ends.

`SmallestLLMService`

Electron access is currently limited to enterprise customers. To request access, reach out to support@smallest.ai.

from pipecat.services.smallest.llm import SmallestLLMService
llm = SmallestLLMService(
    api_key=os.getenv("SMALLEST_API_KEY"),
    settings=SmallestLLMService.Settings(
        system_instruction="You are a helpful assistant in a voice conversation.",
    ),
)

SmallestLLMService connects to Electron, Smallest AI’s LLM, through the Waves API’s OpenAI-compatible chat completions endpoint. It extends Pipecat’s OpenAILLMService, so it’s a drop-in replacement wherever an OpenAI-compatible LLM service is expected - no provider-specific wiring beyond the API key and base URL.

Constructor parameters:

Parameter	Type	Default	Description
`api_key`	`str`	-	Your Smallest AI API key (required)
`base_url`	`str`	`https://api.smallest.ai/waves/v1`	Base URL for Smallest AI’s API
`settings`	`SmallestLLMService.Settings`	`Settings(model="electron")`	Runtime-updatable settings. Defaults to Electron, Smallest AI’s LLM
`default_headers`	`Mapping[str, str]`	`None`	Additional HTTP headers to include in requests

Settings (SmallestLLMService.Settings):

Inherits the common OpenAI-compatible settings - model, system_instruction, temperature, top_p, max_tokens/max_completion_tokens, frequency_penalty, presence_penalty, and seed - all of which can be updated at runtime via LLMUpdateSettingsFrame. model defaults to "electron" and generally doesn’t need to be set.

Because Electron speaks the OpenAI Chat Completions format, SmallestLLMService works with Pipecat’s existing LLMContext, tool calling, and streaming support unchanged. Electron also supports returning a short spoken filler in its response alongside a tool call (for example, “Let me check that”) when the system prompt asks for one, so the TTS can mask tool-call latency without any extra pipeline wiring - see function-calling-smallest.py for a complete example.

Running the Example

Clone the Pipecat repository and navigate to the examples directory:

git clone https://github.com/pipecat-ai/pipecat.git
cd pipecat/examples/voice

Create a .env file with the keys listed in the Setup section above, then run:

Daily transport - server mode (recommended):

python voice-smallest.py -t daily

Open http://localhost:7860 in your browser. The runner creates a Daily room automatically and redirects you to it.

Daily transport - direct mode (no web server, for quick testing):

python voice-smallest.py -d

The room URL is printed in the terminal. Open it in your browser to join.

The full source for voice-smallest.py is at examples/voice/voice-smallest.py. It sets up a complete interruptible voice bot using Smallest AI for STT, LLM (Electron), and TTS, Silero VAD for interruptions, and Daily as the transport - all wired together with the Pipecat runner, so the whole bot runs on a single SMALLEST_API_KEY.

Other examples

function-calling-smallest.py - tool calling with SmallestLLMService, including the filler-phrase pattern described above.
llm-smallest.py - updating SmallestLLMService settings (e.g. temperature) at runtime via LLMUpdateSettingsFrame.

Notes

The pipeline is interruptible: if a user speaks while the bot is talking, audio stops immediately and the pipeline re-engages - no custom logic needed.
For any issues or questions, open an issue in the Pipecat repository or contact us on Discord.

Code Example

Setup

1. Create a Virtual Environment

2. Install Pipecat with Smallest AI support

3. Create a .env file

Services

SmallestSTTService

SmallestTTSService

SmallestLLMService

Running the Example

Other examples

Notes

3. Create a `.env` file

`SmallestSTTService`

`SmallestTTSService`

`SmallestLLMService`