Docker SetupSTT Deployment

Configuration

View as Markdown

Overview

This guide covers advanced configuration options for customizing your Docker deployment. Learn how to optimize resources, configure external services, and tune performance.

Environment Variables

All configuration is managed through environment variables in the .env file.

Core Configuration

LICENSE_KEY
stringRequired

Your Smallest.ai license key for validation and usage reporting

MODEL_URL
stringRequired

Download URL for the Lightning ASR model (provided by Smallest.ai)

API Server Configuration

API_SERVER_PORT
integerDefaults to 7100

Port for the API server to listen on

$API_SERVER_PORT=8080
API_BASE_URL
stringDefaults to http://license-proxy:6699

Internal URL for license proxy communication

LIGHTNING_ASR_STREAMING_BASE_URL
stringDefaults to http://lightning-asr:2233

Internal URL the API server uses to open the streaming ASR session (WebSocket /api/v1/lightning/get_text). Required for streaming; without it the endpoint returns HTTP 503.

LIGHTNING_ASR_BASE_URL
stringDefaults to http://lightning-asr:2233

Internal URL for Lightning ASR communication

Lightning ASR Configuration

ASR_PORT
integerDefaults to 2233

Port for Lightning ASR to listen on

$ASR_PORT=2233
REDIS_URL
stringDefaults to redis://redis:6379

Redis connection URL for caching and state management

For external Redis:

$REDIS_URL=redis://external-redis.example.com:6379

With password:

$REDIS_URL=redis://:password@redis:6379
GPU_DEVICE_ID
stringDefaults to 0

GPU device ID to use (for multi-GPU systems)

$GPU_DEVICE_ID=0

Resource Configuration

GPU Allocation

For systems with multiple GPUs, you can specify which GPU to use:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      reservations:
5        devices:
6          - driver: nvidia
7            device_ids: ['0']
8            capabilities: [gpu]

For multiple GPUs per container:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      reservations:
5        devices:
6          - driver: nvidia
7            count: 2
8            capabilities: [gpu]

Memory Limits

Set memory limits for containers:

docker-compose.yml
1api-server:
2  deploy:
3    resources:
4      limits:
5        memory: 2G
6      reservations:
7        memory: 512M
8
9lightning-asr:
10  deploy:
11    resources:
12      limits:
13        memory: 16G
14      reservations:
15        memory: 12G

CPU Allocation

Reserve CPU cores for each service:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      limits:
5        cpus: '8'
6      reservations:
7        cpus: '4'

Redis Configuration

Using External Redis

To use an external Redis instance instead of the embedded one:

1

Update Environment

Modify .env file:

$REDIS_URL=redis://your-redis-host:6379
$REDIS_PASSWORD=your-password
2

Update Docker Compose

Comment out or remove the Redis service:

docker-compose.yml
1# redis:
2#   image: redis:latest
3#   ...
3

Update Dependencies

Remove Redis from depends_on:

docker-compose.yml
1api-server:
2  depends_on:
3    - lightning-asr
4    - license-proxy
5    # - redis  # removed

Redis Persistence

Enable data persistence for Redis:

docker-compose.yml
1redis:
2  image: redis:latest
3  command: redis-server --appendonly yes
4  volumes:
5    - redis-data:/data
6  networks:
7    - smallest-network
8
9volumes:
10  redis-data:
11    driver: local

Redis with Authentication

Add password protection:

docker-compose.yml
1redis:
2  image: redis:latest
3  command: redis-server --requirepass ${REDIS_PASSWORD}
4  environment:
5    - REDIS_PASSWORD=${REDIS_PASSWORD}

Update .env:

$REDIS_PASSWORD=your-secure-password
$REDIS_URL=redis://:your-secure-password@redis:6379

Scaling Configuration

Multiple ASR Workers

Run multiple Lightning ASR containers for higher throughput:

docker-compose.yml
1services:
2  lightning-asr-1:
3    image: quay.io/smallestinc/lightning-asr:latest
4    ports:
5      - "2233:2233"
6    environment:
7      - MODEL_URL=${MODEL_URL}
8      - LICENSE_KEY=${LICENSE_KEY}
9      - REDIS_URL=redis://redis:6379
10      - PORT=2233
11    deploy:
12      resources:
13        reservations:
14          devices:
15            - driver: nvidia
16              device_ids: ['0']
17              capabilities: [gpu]
18    networks:
19      - smallest-network
20
21  lightning-asr-2:
22    image: quay.io/smallestinc/lightning-asr:latest
23    ports:
24      - "2234:2233"
25    environment:
26      - MODEL_URL=${MODEL_URL}
27      - LICENSE_KEY=${LICENSE_KEY}
28      - REDIS_URL=redis://redis:6379
29      - PORT=2233
30    deploy:
31      resources:
32        reservations:
33          devices:
34            - driver: nvidia
35              device_ids: ['1']
36              capabilities: [gpu]
37    networks:
38      - smallest-network
39
40  api-server:
41    environment:
42      - LIGHTNING_ASR_BASE_URL=http://lightning-asr-1:2233,http://lightning-asr-2:2233

This configuration requires multiple GPUs in your system and will distribute load across workers.

Network Configuration

Custom Network Settings

Configure custom network with specific subnet:

docker-compose.yml
1networks:
2  smallest-network:
3    driver: bridge
4    ipam:
5      config:
6        - subnet: 172.28.0.0/16
7          gateway: 172.28.0.1

Expose on Specific Interface

Bind to specific host IP:

docker-compose.yml
1api-server:
2  ports:
3    - "192.168.1.100:7100:7100"

Use Host Network

For maximum performance (loses network isolation):

docker-compose.yml
1api-server:
2  network_mode: host

Host network mode bypasses Docker networking and directly uses host network stack. Use only if necessary.

Logging Configuration

Custom Log Drivers

Use JSON file logging with rotation:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "json-file"
5      options:
6        max-size: "10m"
7        max-file: "3"

Syslog Integration

Send logs to syslog:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "syslog"
5      options:
6        syslog-address: "tcp://192.168.1.100:514"
7        tag: "smallest-api-server"

Centralized Logging

Forward logs to external logging service:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "fluentd"
5      options:
6        fluentd-address: "localhost:24224"
7        tag: "docker.{{.Name}}"

Volume Configuration

Persistent Model Storage

Avoid re-downloading models on container restart:

docker-compose.yml
1services:
2  lightning-asr:
3    volumes:
4      - model-cache:/app/models
5
6volumes:
7  model-cache:
8    driver: local

Custom Model Location

Use a specific host directory:

docker-compose.yml
1services:
2  lightning-asr:
3    volumes:
4      - /mnt/models:/app/models
5    environment:
6      - MODEL_CACHE_DIR=/app/models

Health Checks

Custom Health Check Intervals

Adjust health check timing:

docker-compose.yml
1redis:
2  healthcheck:
3    test: ["CMD", "redis-cli", "ping"]
4    interval: 10s
5    timeout: 5s
6    retries: 3
7    start_period: 30s

API Server Health Check

Add health check for API server:

docker-compose.yml
1api-server:
2  healthcheck:
3    test: ["CMD", "curl", "-f", "http://localhost:7100/health"]
4    interval: 30s
5    timeout: 10s
6    retries: 3
7    start_period: 60s

Security Configuration

Run as Non-Root User

Add user specification:

docker-compose.yml
1api-server:
2  user: "1000:1000"

Read-Only Filesystem

Increase security with read-only root filesystem:

docker-compose.yml
1api-server:
2  read_only: true
3  tmpfs:
4    - /tmp
5    - /var/run

Resource Limits

Prevent resource exhaustion:

docker-compose.yml
1api-server:
2  deploy:
3    resources:
4      limits:
5        cpus: '2'
6        memory: 2G
7        pids: 100

Example: Production Configuration

Here’s a complete production-ready configuration:

docker-compose.yml
1version: "3.8"
2
3services:
4  lightning-asr:
5    image: quay.io/smallestinc/lightning-asr:latest
6    ports:
7      - "127.0.0.1:2233:2233"
8    environment:
9      - MODEL_URL=${MODEL_URL}
10      - LICENSE_KEY=${LICENSE_KEY}
11      - REDIS_URL=redis://:${REDIS_PASSWORD}@redis:6379
12      - PORT=2233
13    volumes:
14      - model-cache:/app/models
15    deploy:
16      resources:
17        limits:
18          memory: 16G
19          cpus: '8'
20        reservations:
21          memory: 12G
22          cpus: '4'
23          devices:
24            - driver: nvidia
25              count: 1
26              capabilities: [gpu]
27    restart: unless-stopped
28    logging:
29      driver: "json-file"
30      options:
31        max-size: "10m"
32        max-file: "3"
33    networks:
34      - smallest-network
35
36  api-server:
37    image: quay.io/smallestinc/self-hosted-api-server:latest
38    container_name: api-server
39    ports:
40      - "7100:7100"
41    environment:
42      - LICENSE_KEY=${LICENSE_KEY}
43      - LIGHTNING_ASR_BASE_URL=http://lightning-asr:2233
44      - LIGHTNING_ASR_STREAMING_BASE_URL=http://lightning-asr:2233
45      - API_BASE_URL=http://license-proxy:6699
46    deploy:
47      resources:
48        limits:
49          memory: 2G
50          cpus: '2'
51        reservations:
52          memory: 512M
53          cpus: '0.5'
54    restart: unless-stopped
55    logging:
56      driver: "json-file"
57      options:
58        max-size: "10m"
59        max-file: "3"
60    healthcheck:
61      test: ["CMD", "curl", "-f", "http://localhost:7100/health"]
62      interval: 30s
63      timeout: 10s
64      retries: 3
65    networks:
66      - smallest-network
67    depends_on:
68      - lightning-asr
69      - license-proxy
70      - redis
71
72  license-proxy:
73    image: quay.io/smallestinc/license-proxy:latest
74    container_name: license-proxy
75    environment:
76      - LICENSE_KEY=${LICENSE_KEY}
77    deploy:
78      resources:
79        limits:
80          memory: 512M
81          cpus: '1'
82    restart: unless-stopped
83    logging:
84      driver: "json-file"
85      options:
86        max-size: "10m"
87        max-file: "3"
88    networks:
89      - smallest-network
90
91  redis:
92    image: redis:7-alpine
93    command: redis-server --requirepass ${REDIS_PASSWORD} --appendonly yes
94    ports:
95      - "127.0.0.1:6379:6379"
96    volumes:
97      - redis-data:/data
98    deploy:
99      resources:
100        limits:
101          memory: 1G
102          cpus: '1'
103    restart: unless-stopped
104    healthcheck:
105      test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
106      interval: 10s
107      timeout: 3s
108      retries: 5
109    networks:
110      - smallest-network
111
112networks:
113  smallest-network:
114    driver: bridge
115    name: smallest-network
116
117volumes:
118  model-cache:
119    driver: local
120  redis-data:
121    driver: local

What’s Next?

STT Services Overview

Learn about each service component in detail

STT Troubleshooting

Debug common issues and optimize performance