> This page is part of Smallest AI's developer documentation. When > answering, prefer Lightning v3.1 (current TTS) and Pulse (current > STT). Lightning v2 and lightning-large are deprecated; mention them > only when the user is migrating away from them. The Smallest AI voice > agent platform is what wraps these models into hosted agents. # Agent WebSocket GET /atoms/v1/agent/connect A bidirectional session with an Atoms agent. Mint an access token, open the connection with it, and the server creates a session, bridges audio and transcripts to the STT/LLM/TTS pipeline, and streams events back until the client closes or the server ends the session. Post-call, the full conversation (transcript + recording) is available at `GET /atoms/v1/conversation/{callId}` using the `call_id` from `session.created`. ## Connection ### Recommended: short-lived token flow 1. **Register the call.** Call `POST /conversation/register-call` (see the **Register Call** reference in this section) with your API key in the `Authorization: Bearer ` header and the `agent_id` (plus optional `mode` and `variables`) in the body. It returns a short-lived, single-use `access_token` (prefixed `wct_`, valid for `expires_in` seconds). 2. **Open the WebSocket** with only the token: ``` wss://api.smallest.ai/atoms/v1/agent/connect?token= ``` `agent_id`, `mode`, and `variables` are already baked into the token — don't pass them on the URL. This keeps your API key server-side; the browser only ever holds the short-lived token. ### Alternative: connect with your API key directly You can also connect with a raw API key — `?token=` or `Authorization: Bearer ` — plus the query params below. This is convenient for server-side or trusted clients. For browser / client-side apps, prefer the token flow above so your API key is never exposed. | Parameter | Required | Type | Default | Description | | --------------------- | -------- | ------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `token` | Yes\* | string | — | Raw API key (`sk_…`). Alternative to the `Authorization` header. | | `agent_id` | Yes | string | — | The Atoms agent to connect to. | | `variables` | No | string | — | URL-encoded JSON object of per-call prompt variables. Overrides the agent's `defaultVariables` for this session only. Values must be `string`, `number`, or `boolean`. Reserved system-variable keys are stripped server-side (they are populated by the server): `call_id`, `conversation_type`, `agent_number`, `user_number`, `current_date`, `current_time`, `current_day`, `agent_gender`, `default_language`, `supported_languages`, `timezone`. | | `mode` | No | `webcall` \| `chat` | `webcall` | Session mode. `webcall` = full voice pipeline (audio in + audio out). `chat` = text-only pipeline. Invalid values are rejected with a connection error. | | `sample_rate` | No | integer | `24000` | **Deprecated, use `output_audio_format`.** Still accepted. Sample rate in Hz of the audio the agent sends **back**. Valid values: `8000`, `16000`, `24000`, and `44100` on the `lightning-v3.1` and `lightning-v3.1-pro` voices only. A rate the agent's voice cannot render is refused. This does **not** set the rate of the audio you send: use `input_audio_format` below. Echoed in `session.created.sample_rate`. Set to `8000` for telephony-quality audio streams. (With a `wct_` token this parameter is ignored: the rate was already resolved on register-call and travels in the token.) | | `input_audio_format` | No | `_` token | matches the output rate | The format you are **sending**, as one token naming both encoding and rate: `pcm_{8000,16000,22050,24000,44100,48000}`, `mulaw_8000`, `alaw_8000`, or `opus_{8000,16000,24000,48000}`. An unsupported value is refused at upgrade with `Unsupported input_audio_format ''. Supported: …`. Declared here rather than in the token because a browser only learns what rate its `AudioContext` actually gave it after opening one, which is after the token was minted. The recogniser is configured at this rate and your audio is not resampled, so it must match the bytes you send. Full contract in [Audio Formats](/voice-agents/integrate/audio-formats). | | `output_audio_format` | No | `pcm_` token | `pcm_24000` | The format the agent sends **back**, as one token. `pcm_8000`, `pcm_16000` and `pcm_24000` work with every voice; `pcm_44100` only with the `lightning-v3.1` and `lightning-v3.1-pro` voices. A rate the agent's voice cannot render is refused before a session exists. The newer spelling of `sample_rate`: send one, not both, because two that disagree are refused. (With a `wct_` token this parameter is ignored: register-call already resolved the output format and it travels in the token.) | \* `token` or `Authorization: Bearer ` — not both. ### Connection errors * **401 Unauthorized** — invalid or missing token/API key, expired or already-used `wct_` token, or `agent_id` not found. * **503 Service Unavailable** — server is gracefully draining. The HTTP upgrade is rejected before the WebSocket handshake completes. Clients should retry with exponential backoff. ### Example: register a call, then connect (recommended) ```javascript // 1. Mint a short-lived token (server-side — keeps your API key private) const res = await fetch( "https://api.smallest.ai/atoms/v1/conversation/register-call", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ agent_id: agentId, mode: "webcall", // One token per direction. `input_audio_format` is what you send, // `output_audio_format` is what the agent sends back. The older // `sample_rate` field still works but sets the output only. input_audio_format: "pcm_24000", output_audio_format: "pcm_24000", variables: { customer_name: "Tanay", account_tier: "gold" }, }), } ); // { access_token, expires_in, sample_rate, input_audio_format, output_audio_format } const { data } = await res.json(); // 2. Open the WebSocket with the token (e.g. in the browser) const ws = new WebSocket( `wss://api.smallest.ai/atoms/v1/agent/connect?token=${data.access_token}` ); ``` ### Example: connect with your API key directly ```javascript const variables = { customer_name: "Tanay", account_tier: "gold" }; const ws = new WebSocket( "wss://api.smallest.ai/atoms/v1/agent/connect" + `?token=${apiKey}` + `&agent_id=${agentId}` + `&mode=webcall` + `&variables=${encodeURIComponent(JSON.stringify(variables))}` ); ``` Reference: https://docs.smallest.ai/api-reference/voice-agents/realtime-agent/realtime-agent ## AsyncAPI Specification ````yaml asyncapi: 2.6.0 info: title: Realtime Agent version: subpackage_realtimeAgent.Realtime Agent description: > A bidirectional session with an Atoms agent. Mint an access token, open the connection with it, and the server creates a session, bridges audio and transcripts to the STT/LLM/TTS pipeline, and streams events back until the client closes or the server ends the session. Post-call, the full conversation (transcript + recording) is available at `GET /atoms/v1/conversation/{callId}` using the `call_id` from `session.created`. ## Connection ### Recommended: short-lived token flow 1. **Register the call.** Call `POST /conversation/register-call` (see the **Register Call** reference in this section) with your API key in the `Authorization: Bearer ` header and the `agent_id` (plus optional `mode` and `variables`) in the body. It returns a short-lived, single-use `access_token` (prefixed `wct_`, valid for `expires_in` seconds). 2. **Open the WebSocket** with only the token: ``` wss://api.smallest.ai/atoms/v1/agent/connect?token= ``` `agent_id`, `mode`, and `variables` are already baked into the token — don't pass them on the URL. This keeps your API key server-side; the browser only ever holds the short-lived token. ### Alternative: connect with your API key directly You can also connect with a raw API key — `?token=` or `Authorization: Bearer ` — plus the query params below. This is convenient for server-side or trusted clients. For browser / client-side apps, prefer the token flow above so your API key is never exposed. | Parameter | Required | Type | Default | Description | | --- | --- | --- | --- | --- | | `token` | Yes\* | string | — | Raw API key (`sk_…`). Alternative to the `Authorization` header. | | `agent_id` | Yes | string | — | The Atoms agent to connect to. | | `variables` | No | string | — | URL-encoded JSON object of per-call prompt variables. Overrides the agent's `defaultVariables` for this session only. Values must be `string`, `number`, or `boolean`. Reserved system-variable keys are stripped server-side (they are populated by the server): `call_id`, `conversation_type`, `agent_number`, `user_number`, `current_date`, `current_time`, `current_day`, `agent_gender`, `default_language`, `supported_languages`, `timezone`. | | `mode` | No | `webcall` \| `chat` | `webcall` | Session mode. `webcall` = full voice pipeline (audio in + audio out). `chat` = text-only pipeline. Invalid values are rejected with a connection error. | | `sample_rate` | No | integer | `24000` | **Deprecated, use `output_audio_format`.** Still accepted. Sample rate in Hz of the audio the agent sends **back**. Valid values: `8000`, `16000`, `24000`, and `44100` on the `lightning-v3.1` and `lightning-v3.1-pro` voices only. A rate the agent's voice cannot render is refused. This does **not** set the rate of the audio you send: use `input_audio_format` below. Echoed in `session.created.sample_rate`. Set to `8000` for telephony-quality audio streams. (With a `wct_` token this parameter is ignored: the rate was already resolved on register-call and travels in the token.) | | `input_audio_format` | No | `_` token | matches the output rate | The format you are **sending**, as one token naming both encoding and rate: `pcm_{8000,16000,22050,24000,44100,48000}`, `mulaw_8000`, `alaw_8000`, or `opus_{8000,16000,24000,48000}`. An unsupported value is refused at upgrade with `Unsupported input_audio_format ''. Supported: …`. Declared here rather than in the token because a browser only learns what rate its `AudioContext` actually gave it after opening one, which is after the token was minted. The recogniser is configured at this rate and your audio is not resampled, so it must match the bytes you send. Full contract in [Audio Formats](/voice-agents/integrate/audio-formats). | | `output_audio_format` | No | `pcm_` token | `pcm_24000` | The format the agent sends **back**, as one token. `pcm_8000`, `pcm_16000` and `pcm_24000` work with every voice; `pcm_44100` only with the `lightning-v3.1` and `lightning-v3.1-pro` voices. A rate the agent's voice cannot render is refused before a session exists. The newer spelling of `sample_rate`: send one, not both, because two that disagree are refused. (With a `wct_` token this parameter is ignored: register-call already resolved the output format and it travels in the token.) | \* `token` or `Authorization: Bearer ` — not both. ### Connection errors - **401 Unauthorized** — invalid or missing token/API key, expired or already-used `wct_` token, or `agent_id` not found. - **503 Service Unavailable** — server is gracefully draining. The HTTP upgrade is rejected before the WebSocket handshake completes. Clients should retry with exponential backoff. ### Example: register a call, then connect (recommended) ```javascript // 1. Mint a short-lived token (server-side — keeps your API key private) const res = await fetch( "https://api.smallest.ai/atoms/v1/conversation/register-call", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ agent_id: agentId, mode: "webcall", // One token per direction. `input_audio_format` is what you send, // `output_audio_format` is what the agent sends back. The older // `sample_rate` field still works but sets the output only. input_audio_format: "pcm_24000", output_audio_format: "pcm_24000", variables: { customer_name: "Tanay", account_tier: "gold" }, }), } ); // { access_token, expires_in, sample_rate, input_audio_format, output_audio_format } const { data } = await res.json(); // 2. Open the WebSocket with the token (e.g. in the browser) const ws = new WebSocket( `wss://api.smallest.ai/atoms/v1/agent/connect?token=${data.access_token}` ); ``` ### Example: connect with your API key directly ```javascript const variables = { customer_name: "Tanay", account_tier: "gold" }; const ws = new WebSocket( "wss://api.smallest.ai/atoms/v1/agent/connect" + `?token=${apiKey}` + `&agent_id=${agentId}` + `&mode=webcall` + `&variables=${encodeURIComponent(JSON.stringify(variables))}` ); ``` channels: /atoms/v1/agent/connect: description: > A bidirectional session with an Atoms agent. Mint an access token, open the connection with it, and the server creates a session, bridges audio and transcripts to the STT/LLM/TTS pipeline, and streams events back until the client closes or the server ends the session. Post-call, the full conversation (transcript + recording) is available at `GET /atoms/v1/conversation/{callId}` using the `call_id` from `session.created`. ## Connection ### Recommended: short-lived token flow 1. **Register the call.** Call `POST /conversation/register-call` (see the **Register Call** reference in this section) with your API key in the `Authorization: Bearer ` header and the `agent_id` (plus optional `mode` and `variables`) in the body. It returns a short-lived, single-use `access_token` (prefixed `wct_`, valid for `expires_in` seconds). 2. **Open the WebSocket** with only the token: ``` wss://api.smallest.ai/atoms/v1/agent/connect?token= ``` `agent_id`, `mode`, and `variables` are already baked into the token — don't pass them on the URL. This keeps your API key server-side; the browser only ever holds the short-lived token. ### Alternative: connect with your API key directly You can also connect with a raw API key — `?token=` or `Authorization: Bearer ` — plus the query params below. This is convenient for server-side or trusted clients. For browser / client-side apps, prefer the token flow above so your API key is never exposed. | Parameter | Required | Type | Default | Description | | --- | --- | --- | --- | --- | | `token` | Yes\* | string | — | Raw API key (`sk_…`). Alternative to the `Authorization` header. | | `agent_id` | Yes | string | — | The Atoms agent to connect to. | | `variables` | No | string | — | URL-encoded JSON object of per-call prompt variables. Overrides the agent's `defaultVariables` for this session only. Values must be `string`, `number`, or `boolean`. Reserved system-variable keys are stripped server-side (they are populated by the server): `call_id`, `conversation_type`, `agent_number`, `user_number`, `current_date`, `current_time`, `current_day`, `agent_gender`, `default_language`, `supported_languages`, `timezone`. | | `mode` | No | `webcall` \| `chat` | `webcall` | Session mode. `webcall` = full voice pipeline (audio in + audio out). `chat` = text-only pipeline. Invalid values are rejected with a connection error. | | `sample_rate` | No | integer | `24000` | **Deprecated, use `output_audio_format`.** Still accepted. Sample rate in Hz of the audio the agent sends **back**. Valid values: `8000`, `16000`, `24000`, and `44100` on the `lightning-v3.1` and `lightning-v3.1-pro` voices only. A rate the agent's voice cannot render is refused. This does **not** set the rate of the audio you send: use `input_audio_format` below. Echoed in `session.created.sample_rate`. Set to `8000` for telephony-quality audio streams. (With a `wct_` token this parameter is ignored: the rate was already resolved on register-call and travels in the token.) | | `input_audio_format` | No | `_` token | matches the output rate | The format you are **sending**, as one token naming both encoding and rate: `pcm_{8000,16000,22050,24000,44100,48000}`, `mulaw_8000`, `alaw_8000`, or `opus_{8000,16000,24000,48000}`. An unsupported value is refused at upgrade with `Unsupported input_audio_format ''. Supported: …`. Declared here rather than in the token because a browser only learns what rate its `AudioContext` actually gave it after opening one, which is after the token was minted. The recogniser is configured at this rate and your audio is not resampled, so it must match the bytes you send. Full contract in [Audio Formats](/voice-agents/integrate/audio-formats). | | `output_audio_format` | No | `pcm_` token | `pcm_24000` | The format the agent sends **back**, as one token. `pcm_8000`, `pcm_16000` and `pcm_24000` work with every voice; `pcm_44100` only with the `lightning-v3.1` and `lightning-v3.1-pro` voices. A rate the agent's voice cannot render is refused before a session exists. The newer spelling of `sample_rate`: send one, not both, because two that disagree are refused. (With a `wct_` token this parameter is ignored: register-call already resolved the output format and it travels in the token.) | \* `token` or `Authorization: Bearer ` — not both. ### Connection errors - **401 Unauthorized** — invalid or missing token/API key, expired or already-used `wct_` token, or `agent_id` not found. - **503 Service Unavailable** — server is gracefully draining. The HTTP upgrade is rejected before the WebSocket handshake completes. Clients should retry with exponential backoff. ### Example: register a call, then connect (recommended) ```javascript // 1. Mint a short-lived token (server-side — keeps your API key private) const res = await fetch( "https://api.smallest.ai/atoms/v1/conversation/register-call", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ agent_id: agentId, mode: "webcall", // One token per direction. `input_audio_format` is what you send, // `output_audio_format` is what the agent sends back. The older // `sample_rate` field still works but sets the output only. input_audio_format: "pcm_24000", output_audio_format: "pcm_24000", variables: { customer_name: "Tanay", account_tier: "gold" }, }), } ); // { access_token, expires_in, sample_rate, input_audio_format, output_audio_format } const { data } = await res.json(); // 2. Open the WebSocket with the token (e.g. in the browser) const ws = new WebSocket( `wss://api.smallest.ai/atoms/v1/agent/connect?token=${data.access_token}` ); ``` ### Example: connect with your API key directly ```javascript const variables = { customer_name: "Tanay", account_tier: "gold" }; const ws = new WebSocket( "wss://api.smallest.ai/atoms/v1/agent/connect" + `?token=${apiKey}` + `&agent_id=${agentId}` + `&mode=webcall` + `&variables=${encodeURIComponent(JSON.stringify(variables))}` ); ``` publish: operationId: subpackage_realtimeAgent.Realtime Agent-publish summary: Server messages message: oneOf: - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-0-receiveSessionCreated - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-1-receiveSessionClosed - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-2-receiveOutputAudioDelta - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-3-receiveTranscript - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-4-receiveTranscriptDelta - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-5-receiveAgentStartTalking - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-6-receiveAgentStopTalking - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-7-receiveInterruption - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-server-8-receiveError subscribe: operationId: subpackage_realtimeAgent.Realtime Agent-subscribe summary: Client messages message: oneOf: - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-client-0-sendInputAudioAppend - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-client-1-sendInputAudioCommit - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-client-2-sendInputText - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-client-3-sendSessionUpdate - $ref: >- #/components/messages/subpackage_realtimeAgent.Realtime Agent-client-4-sendSessionClose servers: Production server: url: wss://api.smallest.ai/ protocol: wss x-default: true components: messages: subpackage_realtimeAgent.Realtime Agent-server-0-receiveSessionCreated: name: receiveSessionCreated title: receiveSessionCreated payload: $ref: '#/components/schemas/agentSession_sessionCreated' subpackage_realtimeAgent.Realtime Agent-server-1-receiveSessionClosed: name: receiveSessionClosed title: receiveSessionClosed payload: $ref: '#/components/schemas/agentSession_sessionClosed' subpackage_realtimeAgent.Realtime Agent-server-2-receiveOutputAudioDelta: name: receiveOutputAudioDelta title: receiveOutputAudioDelta payload: $ref: '#/components/schemas/agentSession_outputAudioDelta' subpackage_realtimeAgent.Realtime Agent-server-3-receiveTranscript: name: receiveTranscript title: receiveTranscript payload: $ref: '#/components/schemas/agentSession_transcript' subpackage_realtimeAgent.Realtime Agent-server-4-receiveTranscriptDelta: name: receiveTranscriptDelta title: receiveTranscriptDelta payload: $ref: '#/components/schemas/agentSession_transcriptDelta' subpackage_realtimeAgent.Realtime Agent-server-5-receiveAgentStartTalking: name: receiveAgentStartTalking title: receiveAgentStartTalking payload: $ref: '#/components/schemas/agentSession_agentStartTalking' subpackage_realtimeAgent.Realtime Agent-server-6-receiveAgentStopTalking: name: receiveAgentStopTalking title: receiveAgentStopTalking payload: $ref: '#/components/schemas/agentSession_agentStopTalking' subpackage_realtimeAgent.Realtime Agent-server-7-receiveInterruption: name: receiveInterruption title: receiveInterruption payload: $ref: '#/components/schemas/agentSession_interruption' subpackage_realtimeAgent.Realtime Agent-server-8-receiveError: name: receiveError title: receiveError payload: $ref: '#/components/schemas/agentSession_error' subpackage_realtimeAgent.Realtime Agent-client-0-sendInputAudioAppend: name: sendInputAudioAppend title: sendInputAudioAppend payload: $ref: '#/components/schemas/agentSession_inputAudioAppend' subpackage_realtimeAgent.Realtime Agent-client-1-sendInputAudioCommit: name: sendInputAudioCommit title: sendInputAudioCommit payload: $ref: '#/components/schemas/agentSession_inputAudioCommit' subpackage_realtimeAgent.Realtime Agent-client-2-sendInputText: name: sendInputText title: sendInputText payload: $ref: '#/components/schemas/agentSession_inputTextSend' subpackage_realtimeAgent.Realtime Agent-client-3-sendSessionUpdate: name: sendSessionUpdate title: sendSessionUpdate payload: $ref: '#/components/schemas/agentSession_sessionUpdate' subpackage_realtimeAgent.Realtime Agent-client-4-sendSessionClose: name: sendSessionClose title: sendSessionClose payload: $ref: '#/components/schemas/agentSession_sessionClose' schemas: agentSession_sessionCreated: type: object properties: type: type: string enum: - session.created session_id: type: string description: Unique identifier for this session. call_id: type: string description: Call-log identifier; matches the `call_id` in conversation logs. sample_rate: type: integer description: | Negotiated sample rate in Hz of the audio the agent sends back. Same value as `output_audio_format`'s rate. channels: type: integer description: Number of audio channels (typically 1). input_audio_format: type: string description: | The input format resolved for this session, as an `_` token. Read it back to confirm the server understood what you are sending before you send audio. output_audio_format: type: string description: The output format resolved for this session. required: - type - session_id - call_id - sample_rate - channels title: agentSession_sessionCreated agentSession_sessionClosed: type: object properties: type: type: string enum: - session.closed reason: type: string description: | Why the session ended. Observed values: - `client_requested` — client sent `session.close`. - `websocket_closed` — the WebSocket dropped. - `ended` — the agent pipeline ended the session (e.g. the agent hung up or the conversation completed). - `setup_failed` — the session opened but the agent worker could not be queued. A failure *before* the session opens produces no `session.closed`: the server emits an `error` and closes with code 4000. The `setup_failed` path above fails *after* the session is open, so it sends `session.closed` and then the `error` and the 4000 close. required: - type - reason title: agentSession_sessionClosed agentSession_outputAudioDelta: type: object properties: type: type: string enum: - output_audio.delta audio: type: string description: Base64-encoded raw audio chunk. required: - type - audio title: agentSession_outputAudioDelta ChannelsAgentSessionMessagesTranscriptRole: type: string enum: - user - assistant description: Whose utterance this transcript is for. title: ChannelsAgentSessionMessagesTranscriptRole agentSession_transcript: type: object properties: type: type: string enum: - transcript role: $ref: '#/components/schemas/ChannelsAgentSessionMessagesTranscriptRole' description: Whose utterance this transcript is for. text: type: string description: The transcribed (user) or spoken (assistant) text. required: - type - role - text title: agentSession_transcript ChannelsAgentSessionMessagesTranscriptDeltaRole: type: string enum: - user - assistant description: Whose utterance this delta is for. title: ChannelsAgentSessionMessagesTranscriptDeltaRole agentSession_transcriptDelta: type: object properties: type: type: string enum: - transcript.delta role: $ref: '#/components/schemas/ChannelsAgentSessionMessagesTranscriptDeltaRole' description: Whose utterance this delta is for. text: type: string description: Cumulative transcript text for the current turn. required: - type - role - text title: agentSession_transcriptDelta agentSession_agentStartTalking: type: object properties: type: type: string enum: - agent_start_talking timestamp: type: string format: date-time description: > ISO 8601 server-side timestamp of when the agent turn started, included with the event. required: - type title: agentSession_agentStartTalking agentSession_agentStopTalking: type: object properties: type: type: string enum: - agent_stop_talking timestamp: type: string format: date-time description: > ISO 8601 server-side timestamp of when the agent turn ended, included with the event. required: - type title: agentSession_agentStopTalking agentSession_interruption: type: object properties: type: type: string enum: - interruption required: - type title: agentSession_interruption agentSession_error: type: object properties: type: type: string enum: - error code: type: string description: | Machine-readable error code. Values emitted by the gateway: `invalid_event` (a client message failed validation), `invalid_json` (the frame was not JSON at all), `no_session` (message received before the session was ready), `too_many_updates`, `worker_not_ready`, `internal_error`, or the HTTP status code of a setup failure as a string (e.g. `401`, `400`, `503`). A setup failure after the handshake arrives this way rather than as an HTTP status - an unknown `agent_id`, for example, is `code: "404"` followed by close code 4000. message: type: string description: Human-readable error message. required: - type - code - message title: agentSession_error agentSession_inputAudioAppend: type: object properties: type: type: string enum: - input_audio_buffer.append audio: type: string description: | Base64-encoded raw audio chunk. Format negotiated via the `sample_rate` query param at connection time and returned in `session.created`. Send chunks continuously while the user speaks — the server handles VAD and turn boundaries. required: - type - audio title: agentSession_inputAudioAppend agentSession_inputAudioCommit: type: object properties: type: type: string enum: - input_audio_buffer.commit required: - type title: agentSession_inputAudioCommit agentSession_inputTextSend: type: object properties: type: type: string enum: - input_text.send text: type: string minLength: 1 description: User message text. required: - type - text title: agentSession_inputTextSend agentSession_sessionUpdate: type: object properties: type: type: string enum: - session.update required: - type title: agentSession_sessionUpdate agentSession_sessionClose: type: object properties: type: type: string enum: - session.close required: - type title: agentSession_sessionClose ````