Lightning v3.1

Lightning v3.1 is a high-fidelity, low-latency text-to-speech model delivering natural, expressive, and realistic speech at 44 kHz. Optimized for real-time applications with ultra-low latency and voice cloning support, it delivers broadcast-quality audio with genuinely conversational characteristics. Supports 12 languages across English, Hindi, Spanish, and 9 Indian languages.

Jump to: Benchmarks · Voice Catalog · Supported Languages · API Reference · Pricing & Throughput · Quickstart

Model Overview

Key Capabilities

How to use it

See the Lightning quickstart for a working end-to-end example, including authentication, request shape, and audio playback. Lightning v3.1 is selected via the model body field on the unified TTS endpoints — POST /waves/v1/tts for synchronous synthesis, POST /waves/v1/tts/live for SSE streaming, and WSS /waves/v1/tts/live for WebSocket streaming. Pass "model": "lightning_v3.1" to route to the Standard pool, or "model": "lightning_v3.1_pro" to route to the Pro pool.

Performance & Benchmarks

Head-to-head listener evaluation against eight production TTS systems on the EmergentTTS benchmark, 1,088 samples scored by the LLM-as-a-Judge framework. The first table is the win-rate breakdown per competitor; the per-metric scores are split by category below it.

Win, tie, loss against each competitor

Direct head-to-head listener ratings. Lightning Wins % is the share where Lightning v3.1 was preferred. Ties % is the share where listeners scored both equally. Competitor Wins % is the inverse. Each competitor column sums to 100%.

Per-metric scores

Mean listener score per metric across the same 1,088-sample test set. Tables are split by category — open the accordion under each one to see what each metric measures.

Naturalness — higher is better

Expressiveness — higher is better

Delivery — higher is better

Accuracy

Mixed direction — most are lower is better; the Whisper-judged Pronunciation % is higher is better.

MOS v2 — higher is better

Want to reproduce these results? See the TTS evaluation script to measure TTFB and synthesis quality in your own environment.

Supported Languages

Pass the language code on each request to match the language of your input text. Each voice supports a subset of these languages — check tags.language on the voice via GET /waves/v1/lightning-v3.1/get_voices.

Top Voices

Curated short-list of the voices we’d recommend for production. Use these voice_id values directly in the voice_id parameter — no setup required. The full Voice Catalog below has the complete list across additional languages.

English (American)

English (Other accents)

Indic (Hindi + English, Indian accent)

Voice Catalog

English (US) — Best Voices

Hindi / English — Best Voices

Spanish — Best Voices

Other Indian Languages — Best Voices

Voice Cloning

API Reference

Endpoints

Recommended (unified routes — model selected via model body field):

Legacy (deprecated — kept for backwards compatibility):

Request Parameters

Throughput, Latency & Pricing

Rate limits, concurrency caps, and pricing tiers are documented on the Concurrency & Limits page. For enterprise pricing, contact sales@smallest.ai.

Word-level timestamps

Pass word_timestamps: true on a WebSocket request to receive per-word timing events interleaved with the audio stream. Each event tells you exactly when a word starts and ends in the generated audio — useful for captioning, karaoke-style highlighting, avatar lip-sync, and word-level analytics.

Request

1
{
2
  "voice_id": "devansh",
3
  "text": "I bought 3 cats for $100 on Dec 25th",
4
  "model": "lightning_v3.1",
5
  "sample_rate": 44100,
6
  "word_timestamps": true
7
}

Response frames

The server interleaves word_timestamp frames with chunk frames in audio-time order, followed by a single complete:

chunk → chunk → word_timestamp{id:0} → chunk → word_timestamp{id:1} → … → complete

Each word_timestamp frame:

1
{
2
  "session_id": "...",
3
  "request_id": "...",
4
  "status": "word_timestamp",
5
  "data": {
6
    "id": 5,
7
    "word": "$100",
8
    "start": 1.12,
9
    "end": 1.92
10
  }
11
}

Voice + language support matrix

Word events are emitted only when the chosen voice family has the aligner checkpoint baked in.

For unsupported voice families the flag is accepted — audio works normally, but no word_timestamp frames are emitted. Detect this client-side by counting received word_timestamp frames after complete arrives.

Backward compatibility

word_timestamps defaults to false. Clients that don’t set the flag see no behavior change — same audio chunks, same completion frame, no new event type to handle.

JavaScript example

1
const ws = new WebSocket('wss://api.smallest.ai/waves/v1/tts/live', {
2
  headers: { Authorization: `Bearer ${API_KEY}` },
3
});
4
5
ws.onopen = () => {
6
  ws.send(JSON.stringify({
7
    voice_id: "devansh",
8
    text: "I bought 3 cats for $100 on Dec 25th",
9
    model: "lightning_v3.1",
10
    sample_rate: 44100,
11
    output_format: "pcm",
12
    word_timestamps: true,   // ← opt in
13
  }));
14
};
15
16
const captionTrack = [];
17
18
ws.onmessage = (event) => {
19
  const msg = JSON.parse(event.data);
20
  switch (msg.status) {
21
    case "chunk": {
22
      const pcm = Buffer.from(msg.data.audio, 'base64');
23
      audioPlayer.push(pcm);
24
      break;
25
    }
26
    case "word_timestamp": {
27
      const { id, word, start, end } = msg.data;
28
      captionTrack.push({ id, word, startSec: start, endSec: end });
29
      break;
30
    }
31
    case "complete":
32
      audioPlayer.end();
33
      finalizeCaptionTrack(captionTrack);
34
      break;
35
  }
36
};

Where it works

If you need word timing, use the WebSocket path.

Best Practices

Code-Switching

Lightning v3.1 supports real-time intra-session language switching via two mutually exclusive language groups. Each group shares a unified phoneme space, enabling seamless mid-utterance transitions between member languages without session re-initialization. Cross-group switching is not supported within a single session.

Language Groups

Indic Group. Optimized for South Asian language pairs with English as the bridging language.

Global Group. Optimized for European language pairs with English and Hindi as bridging languages.

Intra-group switching is unrestricted. Any language within the same group can be interleaved at the token level. Cross-group switching (e.g., Tamil from Indic + French from Global) is architecturally unsupported and will produce undefined behavior.

Routing Examples

// Indic group — Hindi <-> Tamil interleaving
"Valid: all languages within Indic group"
// Global group — Spanish <-> French interleaving
"Valid: all languages within Global group"
// Cross-group — Tamil (Indic) + French (Global)
"Invalid: cross-group switching unsupported"

Voice Cloning

Reference Audio

• Environment. Record in a quiet room with no background noise, hiss, or rumble. Ambient sound is captured in the clone and cannot be removed after the fact.
• Speaking style. Speak naturally in your normal conversational voice. The model captures timbre, accent, emotional tone, rhythm, and pacing automatically. Do not exaggerate unless a specific tone is intended.
• Audio length. Provide 5 to 15 seconds of clean, continuous speech.

Multi-Lingual Cloning

• Language matching. For best results, record reference audio in the same language as your intended output. Cross-lingual cloning is supported (e.g., English reference used for Spanish output), but a language-matched reference produces higher fidelity.
• Accent retention. When synthesizing in a different language than the reference, the original accent is preserved. A clone from a South Indian English speaker will retain that accent in Hindi or Tamil output. This is by design: the clone reproduces your voice, including accent characteristics. For accent-neutral output in a specific language, provide reference audio from a native speaker of that language.
• Script encoding. Input text must use native script for each language (Devanagari for Hindi/Marathi/Gujarati, respective Brahmic scripts for Dravidian languages, Latin for European languages). Transliterated input degrades synthesis quality.
• Group constraint. Cloned voices follow the same language group routing rules. A session initialized in the Indic group cannot switch to Global-exclusive languages, regardless of the voice’s source language.

For detailed recording examples and expressive cloning techniques, see Voice Cloning Best Practices.

Text Formatting

• Chunk boundaries. Segment input at natural prosodic boundaries (. ! ? ,). Maximum chunk size is 250 characters; optimal throughput at 140 characters per request.
• Script integrity. Avoid transliteration. Use native script for each language. Mixed-script input within a single language token produces unpredictable phoneme mappings.
• Numeric normalization. Use standard formats (DD/MM/YYYY, HH:MM). Phone numbers default to 3-4-3 digit grouping.
• Lexicon overrides. Use pronunciation dictionaries for domain-specific terms, brand names, and acronyms where default grapheme-to-phoneme conversion is insufficient.

For comprehensive text formatting rules (numeric handling, date/time, symbols, chunking logic), see TTS Best Practices.

Technical Specifications

Audio Output

Text Formatting Guidelines

Number & Date Handling

Use Cases

Direct Use

• Voice assistants and conversational AI
• Interactive chatbots with voice output
• Real-time narration and live streaming
• Accessibility tools and screen readers
• Gaming (dynamic character voices)
• Customer service automation

Downstream Use

• Multi-turn conversational agents
• Audio content generation pipelines
• Telephony and IVR systems
• Podcast and audiobook generation

Safety & Compliance

Known Limitations

• Mixed-language text (transliteration) may produce suboptimal results. Hindi text should be in Devanagari script (e.g., “namaste” in Devanagari), not Latin. English text should be in Latin script, not Devanagari. Each language should use its native script.

Compliance

• Voice cloning requires explicit consent
• No retention of synthesized audio
• No storage of personal voice data beyond cloning scope
• Usage monitoring for policy compliance

For compliance documentation (GDPR, SOC2, HIPAA), contact support@smallest.ai.

Support

• Email: support@smallest.ai
• Community: Discord
• Documentation: docs.smallest.ai/waves
• Console: app.smallest.ai/dashboard