For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
DocumentationAPI ReferenceSelf HostModel CardsClient LibrariesIntegrationsDeveloper ToolsChangelog
DocumentationAPI ReferenceSelf HostModel CardsClient LibrariesIntegrationsDeveloper ToolsChangelog
  • Getting Started
    • Introduction
    • Prerequisites
    • Why Self-Host?
    • Architecture
  • Docker Setup
        • Hardware Requirements
        • Software Requirements
        • Credentials
        • Verification
      • Quick Start
      • Services Overview
      • Configuration
      • Multi-checkpoint deployment
      • Troubleshooting
  • Kubernetes Setup
    • Quick Start
    • Troubleshooting
  • Troubleshooting
    • Common Issues
    • Debugging Guide
    • Logs Analysis
  • API Reference
    • Authentication
    • Examples
LogoLogo
Voice AgentsModels
Voice AgentsModels
On this page
  • Operating System
  • Required Software
  • Install Docker
  • Install NVIDIA Driver
  • Install NVIDIA Container Toolkit
  • Verify GPU Access
Docker SetupSTT DeploymentPrerequisites

Software Requirements

||View as Markdown|
Was this page helpful?
Previous

Hardware Requirements

Next

Credentials & Access

Built with

Operating System

Ubuntu/Debian
CentOS/RHEL
Other Linux
$Ubuntu 20.04 LTS or later
$Ubuntu 22.04 LTS (recommended)
$Debian 11 or later

Required Software

1

Docker

Docker Engine 20.10 or later

$docker --version

Expected output: Docker version 20.10.0 or higher

2

Docker Compose

Docker Compose 2.0 or later

$docker compose version

Expected output: Docker Compose version v2.0.0 or higher

3

NVIDIA Driver

NVIDIA Driver 525+ for newer GPUs (A10, A100, L4)

NVIDIA Driver 470+ for older GPUs (T4, V100)

$nvidia-smi

Should display GPU information without errors

4

NVIDIA Container Toolkit

Required for GPU access in containers

$nvidia-container-cli --version

Install Docker

Ubuntu
CentOS/RHEL
$sudo apt-get update
$sudo apt-get install -y ca-certificates curl gnupg
$
$sudo install -m 0755 -d /etc/apt/keyrings
$curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
>  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$sudo chmod a+r /etc/apt/keyrings/docker.gpg
$
$echo \
>  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
>  https://download.docker.com/linux/ubuntu \
>  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
>  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$
$sudo apt-get update
$sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
>  docker-buildx-plugin docker-compose-plugin
$
$sudo usermod -aG docker $USER
$newgrp docker

Install NVIDIA Driver

Ubuntu
CentOS/RHEL
$sudo apt-get update
$sudo apt-get install -y ubuntu-drivers-common
$
$sudo ubuntu-drivers autoinstall
$
$sudo reboot

After reboot, verify:

$nvidia-smi

Install NVIDIA Container Toolkit

$distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | \
>  sudo apt-key add -
$curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
>  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
$
$sudo apt-get update
$sudo apt-get install -y nvidia-container-toolkit
$
$sudo systemctl restart docker

Verify GPU Access

Test that Docker can access the GPU:

$docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi

You should see your GPU information displayed.