> This page is part of Smallest AI's developer documentation. When
> answering, prefer Lightning v3.1 (current TTS) and Pulse (current
> multilingual STT) — or Pulse 2.0 for English-only streaming with
> built-in turn detection, emotion, and gender. Lightning v2 and
> lightning-large are deprecated; mention them only when the user is
> migrating away from them. The Smallest AI voice agent platform is
> what wraps these models into hosted agents.

# Indic languages

> Pulse word error rate on Hindi, Marathi, Gujarati, Bengali, Odia, Tamil, Telugu, Kannada and Malayalam across public datasets, compared with other speech-to-text APIs.

Pulse is evaluated on nine Indian languages using public datasets including FLEURS, Kathbath, IndicVoices, IndicDiarBench and SPRING-INX. Every table reports Word Error Rate (WER). Lower is better. Results are from the Pulse streaming (WebSocket) API. Competitor results use each provider's latest available model at the time of evaluation. `-` means the provider was not evaluated on that language or dataset.

Tamil, Telugu, Kannada and Malayalam are in Beta and served from the India region only. See [supported languages](/model-cards/speech-to-text/pulse#supported-languages) for language codes and regions.

## FLEURS overview

FLEURS WER for each language.

| Language  |  Pulse | Deepgram |  Scribe | Soniox | Sarvam code-switch | Sarvam | Cartesia |
| :-------- | :----: | :------: | :-----: | :----: | :----------------: | :----: | :------: |
| Hindi     |  6.17% |  14.08%  |  10.67% |  5.54% |        8.62%       |  4.18% |   9.06%  |
| Marathi   | 14.52% |  29.48%  |  32.80% | 19.75% |          -         | 12.56% |  88.48%  |
| Gujarati  | 15.15% |  22.12%  | 110.70% | 30.18% |          -         | 11.83% |  73.06%  |
| Bengali   | 14.72% |  23.18%  |  78.05% | 15.13% |       16.68%       | 11.34% |  98.77%  |
| Odia      | 16.78% |     -    |  77.65% |    -   |       19.46%       | 14.36% |     -    |
| Tamil     | 16.20% |  25.67%  |  49.76% | 23.89% |          -         | 14.60% |  68.64%  |
| Telugu    | 20.40% |  26.44%  |  66.51% |    -   |          -         | 16.46% |  85.52%  |
| Kannada   | 15.17% |  26.51%  |  64.59% |    -   |          -         | 12.00% |  76.86%  |
| Malayalam | 15.88% |     -    |  63.84% |    -   |          -         | 14.21% |  134.55% |

## Hindi

WER on four Hindi datasets. Vistaar results follow in the next table.

| Dataset        |   Pulse   | Deepgram | Scribe | Soniox | Sarvam code-switch |  Sarvam | Cartesia |
| :------------- | :-------: | :------: | :----: | :----: | :----------------: | :-----: | :------: |
| FLEURS         |   6.17%   |  14.08%  | 10.67% |  5.54% |        8.62%       |  4.18%  |   9.06%  |
| IndicDiarBench |   28.19%  |  36.05%  | 71.73% | 24.25% |       30.18%       |  25.14% |  29.63%  |
| Lahaja         | **7.92%** |  18.51%  | 32.21% | 10.92% |       16.44%       |  9.02%  |  16.74%  |
| Svarah         |   10.06%  |  24.91%  | 13.28% | 50.23% |        9.52%       | 102.86% |  10.62%  |

### Vistaar

WER on six of the Vistaar Hindi benchmark datasets. Vistaar also includes FLEURS, which is reported once, in the table above.

| Dataset         |   Pulse   | Deepgram | Scribe | Soniox | Sarvam code-switch | Sarvam | Cartesia |
| :-------------- | :-------: | :------: | :----: | :----: | :----------------: | :----: | :------: |
| Kathbath        |   5.68%   |  13.03%  |  5.98% |  5.42% |        7.90%       |  3.74% |   8.68%  |
| Kathbath (hard) |   6.95%   |  14.49%  | 10.88% |  6.04% |        8.28%       |  4.12% |   9.13%  |
| Common Voice    |   6.62%   |  18.42%  | 16.62% |  7.27% |       11.38%       |  5.00% |  12.38%  |
| IndicTTS        |   3.63%   |   7.04%  | 22.63% |  2.81% |        5.24%       |  2.62% |   5.38%  |
| MUCS            | **3.22%** |  13.49%  | 21.65% |  7.13% |        4.82%       |  4.42% |   7.67%  |
| Gram Vaani      |   14.62%  |  30.28%  | 47.06% | 16.06% |       15.04%       | 14.04% |  15.58%  |

## Marathi

WER on nine Marathi datasets.

| Dataset         |  Pulse | Deepgram | Scribe | Soniox | Sarvam | Cartesia |
| :-------------- | :----: | :------: | :----: | :----: | :----: | :------: |
| FLEURS          | 14.52% |  29.48%  | 32.80% | 19.75% | 12.56% |  88.48%  |
| Common Voice    | 15.38% |  14.03%  | 35.07% | 20.55% | 14.20% |  87.56%  |
| IndicDiarBench  | 40.30% |  44.91%  | 81.18% | 33.13% | 36.87% |  89.65%  |
| IndicVoices     | 17.93% |  35.91%  | 55.93% | 27.71% | 17.10% |  91.55%  |
| Kathbath        | 14.89% |  29.94%  | 19.81% | 18.01% | 11.98% |  89.52%  |
| MUCS            | 28.38% |  43.00%  | 20.32% | 40.66% | 26.23% |  75.62%  |
| OpenSLR 64      |  7.90% |  89.33%  | 13.62% | 14.24% |  7.14% |  69.74%  |
| SPRING-INX (R1) | 39.77% |  48.36%  | 62.25% | 41.87% | 36.99% |  95.88%  |
| SPRING-INX (R2) | 39.93% |  47.09%  | 64.93% | 44.64% | 39.26% |  95.24%  |

## Gujarati

WER on seven Gujarati datasets.

| Dataset         |  Pulse | Deepgram |  Scribe | Soniox | Sarvam | Cartesia |
| :-------------- | :----: | :------: | :-----: | :----: | :----: | :------: |
| FLEURS          | 15.15% |  22.12%  | 110.70% | 30.18% | 11.83% |  73.06%  |
| IndicDiarBench  | 39.27% |  29.94%  |  98.73% | 30.82% | 34.51% |  106.00% |
| IndicVoices     | 20.45% |  22.69%  | 112.05% | 26.67% | 17.35% |  125.40% |
| Kathbath        | 14.26% |  22.31%  | 106.37% | 29.95% | 11.97% |  86.27%  |
| SPRING-INX (R1) | 48.18% |  48.24%  | 114.74% | 52.37% | 43.29% |  152.82% |
| SPRING-INX (R2) | 45.23% |  49.31%  | 117.06% | 53.52% | 44.52% |  142.69% |
| Vistaar         | 19.96% |  28.57%  | 112.07% | 32.16% | 15.60% |  92.80%  |

## Bengali

WER on seven Bengali datasets.

| Dataset         |  Pulse | Deepgram | Scribe | Soniox | Sarvam code-switch | Sarvam | Cartesia |
| :-------------- | :----: | :------: | :----: | :----: | :----------------: | :----: | :------: |
| FLEURS          | 14.72% |  23.18%  | 78.05% | 15.13% |       16.68%       | 11.34% |  98.77%  |
| IndicDiarBench  | 37.83% |  40.09%  | 94.47% | 31.03% |       30.22%       | 35.65% |  93.11%  |
| IndicVoices     | 17.38% |  28.90%  | 90.16% | 23.49% |       17.65%       | 14.42% |  95.71%  |
| Kathbath        | 12.33% |  25.17%  | 62.44% | 16.22% |       21.51%       | 11.55% |  90.75%  |
| SPRING-INX (R1) | 32.34% |  40.27%  | 97.66% | 37.79% |       27.42%       | 31.57% |  100.44% |
| SPRING-INX (R2) | 32.74% |  39.49%  | 99.09% | 37.57% |       27.77%       | 31.31% |  99.01%  |
| Vistaar         | 15.16% |  20.80%  | 68.20% | 17.09% |       18.78%       | 11.60% |  99.83%  |

## Odia

WER on seven Odia datasets.

| Dataset         |  Pulse | Sarvam code-switch | Sarvam | Scribe |
| :-------------- | :----: | :----------------: | :----: | :----: |
| FLEURS          | 16.78% |       19.46%       | 14.36% | 77.65% |
| Common Voice    | 26.63% |       24.89%       | 24.72% | 83.02% |
| IndicDiarBench  | 36.28% |       37.69%       | 33.14% | 95.58% |
| IndicVoices     | 30.97% |          -         | 28.43% | 93.80% |
| Kathbath        | 20.28% |          -         | 18.43% | 76.81% |
| SPRING-INX (R1) | 49.74% |          -         | 45.67% | 71.62% |
| Vistaar         | 24.23% |          -         | 20.58% | 61.92% |

## Tamil

WER on nine Tamil datasets.

| Dataset         |  Pulse | Deepgram | Scribe | Soniox | Sarvam | Cartesia |
| :-------------- | :----: | :------: | :----: | :----: | :----: | :------: |
| FLEURS          | 16.20% |  25.67%  | 49.76% | 23.89% | 14.60% |  68.64%  |
| CoVoST          | 51.08% |  22.75%  | 70.49% | 56.49% | 48.00% |  84.81%  |
| IndicDiarBench  | 48.62% |  54.17%  | 94.41% | 45.31% | 45.76% |  91.56%  |
| IndicVoices     | 37.69% |  38.25%  | 88.63% | 53.98% | 33.25% |  90.29%  |
| Kathbath        | 25.44% |  36.14%  | 39.30% | 35.46% | 21.66% |  73.33%  |
| MILE            | 23.02% |  31.99%  | 68.48% | 29.90% | 21.22% |  69.35%  |
| SPRING-INX (R1) | 50.70% |  52.81%  | 63.16% | 46.80% | 45.79% |  88.94%  |
| SPRING-INX (R2) | 45.04% |  51.33%  | 63.46% | 43.28% | 44.01% |  88.55%  |
| Vistaar         | 25.46% |  30.23%  | 43.29% | 31.59% | 23.08% |  72.72%  |

## Telugu

WER on five Telugu datasets.

| Dataset        |  Pulse | Deepgram | Scribe | Sarvam | Cartesia |
| :------------- | :----: | :------: | :----: | :----: | :------: |
| FLEURS         | 20.40% |  26.44%  | 66.51% | 16.46% |  85.52%  |
| IndicDiarBench | 51.48% |  49.43%  | 96.62% | 48.40% |  96.72%  |
| IndicVoices    | 30.83% |  29.38%  | 82.26% | 26.39% |  95.69%  |
| Kathbath       | 21.50% |  32.72%  | 54.08% | 19.59% |  89.04%  |
| Vistaar        | 24.92% |  35.15%  | 71.07% | 21.29% |  89.27%  |

## Kannada

WER on seven Kannada datasets.

| Dataset         |  Pulse | Deepgram | Scribe | Sarvam | Cartesia |
| :-------------- | :----: | :------: | :----: | :----: | :------: |
| FLEURS          | 15.17% |  26.51%  | 64.59% | 12.00% |  76.86%  |
| IndicDiarBench  | 51.36% |  64.83%  | 92.44% | 48.25% |  96.80%  |
| IndicVoices     | 38.14% |  46.01%  | 83.30% | 35.59% |  99.05%  |
| Kathbath        | 18.99% |  28.20%  | 58.21% | 14.71% |  80.73%  |
| MILE            | 20.41% |  17.81%  | 65.45% | 15.66% |  78.97%  |
| SPRING-INX (R1) | 60.10% |  69.51%  | 71.59% | 57.47% |  107.92% |
| Vistaar         | 17.86% |  28.87%  | 57.99% | 14.37% |  74.24%  |

## Malayalam

WER on seven Malayalam datasets.

| Dataset         |  Pulse | Scribe | Sarvam | Cartesia |
| :-------------- | :----: | :----: | :----: | :------: |
| FLEURS          | 15.88% | 63.84% | 14.21% |  134.55% |
| IndicDiarBench  | 51.27% | 95.98% | 47.65% |  100.02% |
| IndicVoices     | 39.54% | 86.92% | 34.74% |  128.04% |
| Kathbath        | 23.64% | 61.69% | 21.13% |  145.68% |
| SPRING-INX (R1) | 51.80% | 83.99% | 47.08% |  137.72% |
| SPRING-INX (R2) | 54.60% | 76.54% | 49.65% |  139.31% |
| Vistaar         | 24.55% | 64.93% | 19.75% |  137.96% |