Services Overview

Architecture

The Docker deployment consists of four main services that work together:

API Server

The API Server is the main entry point for all client requests.

Purpose

• Routes incoming API requests to Lightning ASR workers
• Manages WebSocket connections for streaming
• Handles request queuing and load balancing
• Provides unified API interface

Container Details

Key Endpoints

Environment Variables

1
LICENSE_KEY: Your license key
2
LIGHTNING_ASR_BASE_URL: Internal URL to Lightning ASR
3
API_BASE_URL: Internal URL to License Proxy

Logs

Key log messages:

✓ Connected to Lightning ASR at http://lightning-asr:2233
✓ License validation successful
✓ API server listening on port 7100

Dependencies

• Requires Lightning ASR to be running
• Requires License Proxy for validation
• Optionally uses Redis for request coordination

Lightning ASR

The core speech recognition engine powered by GPU acceleration.

Purpose

• Performs audio-to-text transcription
• Processes both batch and streaming requests
• Manages GPU resources and model inference
• Handles audio preprocessing and postprocessing

Container Details

GPU Requirements

Lightning ASR requires NVIDIA GPU with CUDA support:

Environment Variables

1
MODEL_URL: Download URL for ASR model
2
LICENSE_KEY: Your license key
3
REDIS_URL: Redis connection string
4
PORT: Service port (default 2233)
5
GPU_DEVICE_ID: GPU to use (for multi-GPU)

Model Loading

On first startup, Lightning ASR:

1. Downloads model from MODEL_URL (~20 GB)
2. Validates model integrity
3. Loads model into GPU memory
4. Performs warmup inference

Logs

Key log messages:

✓ GPU detected: NVIDIA A10 (24GB)
✓ Downloading model from URL...
✓ Model loaded successfully (5.2GB)
✓ Warmup completed in 3.2s
✓ Server ready on port 2233

Performance

Typical performance metrics:

Dependencies

• Requires License Proxy for validation
• Requires Redis for request coordination
• Requires NVIDIA GPU

License Proxy

Validates license keys and reports usage to Smallest servers.

Purpose

• Validates license keys on startup
• Reports usage metadata to Smallest
• Provides grace period for offline operation
• Acts as licensing gateway for all services

Container Details

Environment Variables

1
LICENSE_KEY: Your license key

Network Requirements

Validation Process

1. On startup, validates license key with Smallest servers
2. Receives license terms and quotas
3. Caches validation (valid for grace period)
4. Periodically reports usage metadata

Usage Reporting

License Proxy reports only metadata:

Offline Mode

If connection to license server fails:

• Uses cached validation (24-hour grace period)
• Continues serving requests
• Logs warning messages
• Retries connection periodically

Logs

Key log messages:

✓ License validated successfully
✓ License valid until: 2024-12-31
✓ Server listening on port 6699
⚠ Connection to license server failed, using cached validation

Redis

Provides caching and state management for the system.

Purpose

• Request queuing and coordination
• Session state for streaming connections
• Caching of frequent requests
• Performance optimization

Container Details

Configuration Options

1
redis:
2
  image: redis:latest
3
  ports:
4
    - "6379:6379"

Data Stored

Redis stores:

• Request queue state
• WebSocket session data
• Temporary audio chunks (streaming)
• Worker status and health

Health Check

Built-in health check:

1
healthcheck:
2
  test: ["CMD", "redis-cli", "ping"]
3
  interval: 5s
4
  timeout: 3s
5
  retries: 5

Service Dependencies

Startup order and dependencies:

Recommended Startup Sequence

1. Redis - Starts immediately (5 seconds)
2. License Proxy - Validates license (10-15 seconds)
3. Lightning ASR - Downloads/loads model (30-600 seconds)
4. API Server - Connects to services (5-10 seconds)

Resource Planning

Minimum Configuration

For development/testing:

1
Total Resources:
2
  CPU: 6 cores
3
  Memory: 16 GB
4
  GPU: 1x T4 (16 GB VRAM)
5
  Storage: 100 GB

Production Configuration

For production workloads:

1
Total Resources:
2
  CPU: 12 cores
3
  Memory: 32 GB
4
  GPU: 1x A10 (24 GB VRAM)
5
  Storage: 200 GB

Multi-Worker Configuration

For high-volume production:

1
Total Resources:
2
  CPU: 24 cores
3
  Memory: 64 GB
4
  GPU: 2x A10 (24 GB VRAM each)
5
  Storage: 300 GB

Monitoring

Container Health

Check container status:

$
docker compose ps

Resource Usage

Monitor resource consumption:

$
docker stats

GPU Usage

Monitor GPU utilization:

$
watch -n 1 nvidia-smi

Logs

View service logs:

$
docker compose logs -f [service-name]

What’s Next?