Docker SetupSTT Deployment

Configuration

View as MarkdownOpen in Claude

Overview

This guide covers advanced configuration options for customizing your Docker deployment. Learn how to optimize resources, configure external services, and tune performance.

Environment Variables

All configuration is managed through environment variables in the .env file.

Core Configuration

LICENSE_KEY
stringRequired

Your Smallest.ai license key for validation and usage reporting

MODEL_URL
stringRequired

Download URL for the Lightning ASR model (provided by Smallest.ai)

API Server Configuration

API_SERVER_PORT
integerDefaults to 7100

Port for the API server to listen on

$API_SERVER_PORT=8080
API_BASE_URL
stringDefaults to http://license-proxy:6699

Internal URL for license proxy communication

LIGHTNING_ASR_BASE_URL
stringDefaults to http://lightning-asr:2233

Internal URL for Lightning ASR communication

Lightning ASR Configuration

ASR_PORT
integerDefaults to 2233

Port for Lightning ASR to listen on

$ASR_PORT=2233
REDIS_URL
stringDefaults to redis://redis:6379

Redis connection URL for caching and state management

For external Redis:

$REDIS_URL=redis://external-redis.example.com:6379

With password:

$REDIS_URL=redis://:password@redis:6379
GPU_DEVICE_ID
stringDefaults to 0

GPU device ID to use (for multi-GPU systems)

$GPU_DEVICE_ID=0

Resource Configuration

GPU Allocation

For systems with multiple GPUs, you can specify which GPU to use:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      reservations:
5        devices:
6          - driver: nvidia
7            device_ids: ['0']
8            capabilities: [gpu]

For multiple GPUs per container:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      reservations:
5        devices:
6          - driver: nvidia
7            count: 2
8            capabilities: [gpu]

Memory Limits

Set memory limits for containers:

docker-compose.yml
1api-server:
2  deploy:
3    resources:
4      limits:
5        memory: 2G
6      reservations:
7        memory: 512M
8
9lightning-asr:
10  deploy:
11    resources:
12      limits:
13        memory: 16G
14      reservations:
15        memory: 12G

CPU Allocation

Reserve CPU cores for each service:

docker-compose.yml
1lightning-asr:
2  deploy:
3    resources:
4      limits:
5        cpus: '8'
6      reservations:
7        cpus: '4'

Redis Configuration

Using External Redis

To use an external Redis instance instead of the embedded one:

1

Update Environment

Modify .env file:

$REDIS_URL=redis://your-redis-host:6379
$REDIS_PASSWORD=your-password
2

Update Docker Compose

Comment out or remove the Redis service:

docker-compose.yml
1# redis:
2#   image: redis:latest
3#   ...
3

Update Dependencies

Remove Redis from depends_on:

docker-compose.yml
1api-server:
2  depends_on:
3    - lightning-asr
4    - license-proxy
5    # - redis  # removed

Redis Persistence

Enable data persistence for Redis:

docker-compose.yml
1redis:
2  image: redis:latest
3  command: redis-server --appendonly yes
4  volumes:
5    - redis-data:/data
6  networks:
7    - smallest-network
8
9volumes:
10  redis-data:
11    driver: local

Redis with Authentication

Add password protection:

docker-compose.yml
1redis:
2  image: redis:latest
3  command: redis-server --requirepass ${REDIS_PASSWORD}
4  environment:
5    - REDIS_PASSWORD=${REDIS_PASSWORD}

Update .env:

$REDIS_PASSWORD=your-secure-password
$REDIS_URL=redis://:your-secure-password@redis:6379

Scaling Configuration

Multiple ASR Workers

Run multiple Lightning ASR containers for higher throughput:

docker-compose.yml
1services:
2  lightning-asr-1:
3    image: quay.io/smallestinc/lightning-asr:latest
4    ports:
5      - "2233:2233"
6    environment:
7      - MODEL_URL=${MODEL_URL}
8      - LICENSE_KEY=${LICENSE_KEY}
9      - REDIS_URL=redis://redis:6379
10      - PORT=2233
11    deploy:
12      resources:
13        reservations:
14          devices:
15            - driver: nvidia
16              device_ids: ['0']
17              capabilities: [gpu]
18    networks:
19      - smallest-network
20
21  lightning-asr-2:
22    image: quay.io/smallestinc/lightning-asr:latest
23    ports:
24      - "2234:2233"
25    environment:
26      - MODEL_URL=${MODEL_URL}
27      - LICENSE_KEY=${LICENSE_KEY}
28      - REDIS_URL=redis://redis:6379
29      - PORT=2233
30    deploy:
31      resources:
32        reservations:
33          devices:
34            - driver: nvidia
35              device_ids: ['1']
36              capabilities: [gpu]
37    networks:
38      - smallest-network
39
40  api-server:
41    environment:
42      - LIGHTNING_ASR_BASE_URL=http://lightning-asr-1:2233,http://lightning-asr-2:2233

This configuration requires multiple GPUs in your system and will distribute load across workers.

Network Configuration

Custom Network Settings

Configure custom network with specific subnet:

docker-compose.yml
1networks:
2  smallest-network:
3    driver: bridge
4    ipam:
5      config:
6        - subnet: 172.28.0.0/16
7          gateway: 172.28.0.1

Expose on Specific Interface

Bind to specific host IP:

docker-compose.yml
1api-server:
2  ports:
3    - "192.168.1.100:7100:7100"

Use Host Network

For maximum performance (loses network isolation):

docker-compose.yml
1api-server:
2  network_mode: host

Host network mode bypasses Docker networking and directly uses host network stack. Use only if necessary.

Logging Configuration

Custom Log Drivers

Use JSON file logging with rotation:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "json-file"
5      options:
6        max-size: "10m"
7        max-file: "3"

Syslog Integration

Send logs to syslog:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "syslog"
5      options:
6        syslog-address: "tcp://192.168.1.100:514"
7        tag: "smallest-api-server"

Centralized Logging

Forward logs to external logging service:

docker-compose.yml
1services:
2  api-server:
3    logging:
4      driver: "fluentd"
5      options:
6        fluentd-address: "localhost:24224"
7        tag: "docker.{{.Name}}"

Volume Configuration

Persistent Model Storage

Avoid re-downloading models on container restart:

docker-compose.yml
1services:
2  lightning-asr:
3    volumes:
4      - model-cache:/app/models
5
6volumes:
7  model-cache:
8    driver: local

Custom Model Location

Use a specific host directory:

docker-compose.yml
1services:
2  lightning-asr:
3    volumes:
4      - /mnt/models:/app/models
5    environment:
6      - MODEL_CACHE_DIR=/app/models

Health Checks

Custom Health Check Intervals

Adjust health check timing:

docker-compose.yml
1redis:
2  healthcheck:
3    test: ["CMD", "redis-cli", "ping"]
4    interval: 10s
5    timeout: 5s
6    retries: 3
7    start_period: 30s

API Server Health Check

Add health check for API server:

docker-compose.yml
1api-server:
2  healthcheck:
3    test: ["CMD", "curl", "-f", "http://localhost:7100/health"]
4    interval: 30s
5    timeout: 10s
6    retries: 3
7    start_period: 60s

Security Configuration

Run as Non-Root User

Add user specification:

docker-compose.yml
1api-server:
2  user: "1000:1000"

Read-Only Filesystem

Increase security with read-only root filesystem:

docker-compose.yml
1api-server:
2  read_only: true
3  tmpfs:
4    - /tmp
5    - /var/run

Resource Limits

Prevent resource exhaustion:

docker-compose.yml
1api-server:
2  deploy:
3    resources:
4      limits:
5        cpus: '2'
6        memory: 2G
7        pids: 100

Example: Production Configuration

Here’s a complete production-ready configuration:

docker-compose.yml
1version: "3.8"
2
3services:
4  lightning-asr:
5    image: quay.io/smallestinc/lightning-asr:latest
6    ports:
7      - "127.0.0.1:2233:2233"
8    environment:
9      - MODEL_URL=${MODEL_URL}
10      - LICENSE_KEY=${LICENSE_KEY}
11      - REDIS_URL=redis://:${REDIS_PASSWORD}@redis:6379
12      - PORT=2233
13    volumes:
14      - model-cache:/app/models
15    deploy:
16      resources:
17        limits:
18          memory: 16G
19          cpus: '8'
20        reservations:
21          memory: 12G
22          cpus: '4'
23          devices:
24            - driver: nvidia
25              count: 1
26              capabilities: [gpu]
27    restart: unless-stopped
28    logging:
29      driver: "json-file"
30      options:
31        max-size: "10m"
32        max-file: "3"
33    networks:
34      - smallest-network
35
36  api-server:
37    image: quay.io/smallestinc/self-hosted-api-server:latest
38    container_name: api-server
39    ports:
40      - "7100:7100"
41    environment:
42      - LICENSE_KEY=${LICENSE_KEY}
43      - LIGHTNING_ASR_BASE_URL=http://lightning-asr:2233
44      - API_BASE_URL=http://license-proxy:6699
45    deploy:
46      resources:
47        limits:
48          memory: 2G
49          cpus: '2'
50        reservations:
51          memory: 512M
52          cpus: '0.5'
53    restart: unless-stopped
54    logging:
55      driver: "json-file"
56      options:
57        max-size: "10m"
58        max-file: "3"
59    healthcheck:
60      test: ["CMD", "curl", "-f", "http://localhost:7100/health"]
61      interval: 30s
62      timeout: 10s
63      retries: 3
64    networks:
65      - smallest-network
66    depends_on:
67      - lightning-asr
68      - license-proxy
69      - redis
70
71  license-proxy:
72    image: quay.io/smallestinc/license-proxy:latest
73    container_name: license-proxy
74    environment:
75      - LICENSE_KEY=${LICENSE_KEY}
76    deploy:
77      resources:
78        limits:
79          memory: 512M
80          cpus: '1'
81    restart: unless-stopped
82    logging:
83      driver: "json-file"
84      options:
85        max-size: "10m"
86        max-file: "3"
87    networks:
88      - smallest-network
89
90  redis:
91    image: redis:7-alpine
92    command: redis-server --requirepass ${REDIS_PASSWORD} --appendonly yes
93    ports:
94      - "127.0.0.1:6379:6379"
95    volumes:
96      - redis-data:/data
97    deploy:
98      resources:
99        limits:
100          memory: 1G
101          cpus: '1'
102    restart: unless-stopped
103    healthcheck:
104      test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
105      interval: 10s
106      timeout: 3s
107      retries: 5
108    networks:
109      - smallest-network
110
111networks:
112  smallest-network:
113    driver: bridge
114    name: smallest-network
115
116volumes:
117  model-cache:
118    driver: local
119  redis-data:
120    driver: local

What’s Next?

STT Services Overview

Learn about each service component in detail

STT Troubleshooting

Debug common issues and optimize performance