Docker SetupSTT Deployment

Prerequisites

View as MarkdownOpen in Claude

Overview

Before deploying Smallest Self-Host with Docker, ensure your system meets the hardware and software requirements. This guide walks you through everything you need to prepare your environment.

Hardware Requirements

Minimum Specifications

CPU

4 cores minimum

8+ cores recommended for production

RAM

16 GB minimum

32+ GB recommended for production

GPU

NVIDIA GPU required

  • L4 or L40s (recommended)
  • A10, A100, H100, T4 (supported)
  • Minimum 16GB VRAM
Storage

100 GB minimum

  • 50 GB for models
  • 50 GB for logs and data

Software Requirements

Operating System

Smallest Self-Host supports the following operating systems:

$Ubuntu 20.04 LTS or later
$Ubuntu 22.04 LTS (recommended)
$Debian 11 or later

Required Software

1

Docker

Docker Engine 20.10 or later

$docker --version

Expected output: Docker version 20.10.0 or higher

2

Docker Compose

Docker Compose 2.0 or later

$docker compose version

Expected output: Docker Compose version v2.0.0 or higher

3

NVIDIA Driver

NVIDIA Driver 525+ for newer GPUs (A10, A100, L4)

NVIDIA Driver 470+ for older GPUs (T4, V100)

$nvidia-smi

Should display GPU information without errors

4

NVIDIA Container Toolkit

Required for GPU access in containers

$nvidia-container-cli --version

Installation Guides

Install Docker

$sudo apt-get update
$sudo apt-get install -y ca-certificates curl gnupg
$
$sudo install -m 0755 -d /etc/apt/keyrings
$curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
>  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$sudo chmod a+r /etc/apt/keyrings/docker.gpg
$
$echo \
>  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
>  https://download.docker.com/linux/ubuntu \
>  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
>  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$
$sudo apt-get update
$sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
>  docker-buildx-plugin docker-compose-plugin
$
$sudo usermod -aG docker $USER
$newgrp docker

Install NVIDIA Driver

$sudo apt-get update
$sudo apt-get install -y ubuntu-drivers-common
$
$sudo ubuntu-drivers autoinstall
$
$sudo reboot

After reboot, verify:

$nvidia-smi

Install NVIDIA Container Toolkit

$distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | \
>  sudo apt-key add -
$curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
>  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
$
$sudo apt-get update
$sudo apt-get install -y nvidia-container-toolkit
$
$sudo systemctl restart docker

Verify GPU Access

Test that Docker can access the GPU:

$docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi

You should see your GPU information displayed.

Credentials

Before installation, obtain the following from Smallest.ai:

Your unique license key for validation

Contact: support@smallest.ai

Credentials to pull Docker images:

  • Registry URL: quay.io
  • Username
  • Password
  • Email

Contact: support@smallest.ai

Download URLs for ASR models

Contact: support@smallest.ai

Network Requirements

Ensure the following ports are available:

PortServicePurpose
7100API ServerClient API requests
2233Lightning ASRInternal ASR processing
6699License ProxyInternal license validation
6379RedisInternal caching

The License Proxy requires outbound HTTPS access to Smallest’s license servers for validation. Ensure your firewall allows outbound connections to:

  • console-api.smallest.ai (port 443)

Verification Checklist

Before proceeding to installation, verify:

1

Docker Running

$docker ps

Should execute without errors

2

GPU Accessible

$docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi

Should display GPU information

3

Credentials Ready

  • License key obtained
  • Container registry username and password
  • Model download URLs
4

Ports Available

$sudo netstat -tuln | grep -E '(7100|2233|6699|6379)'

Should return no results (ports free)

What’s Next?

Once all prerequisites are met, proceed to the quick start guide:

STT Quick Start

Deploy Smallest Self-Host STT with Docker Compose