Android (Kotlin) | Smallest AI Docs

Audio you send must be declared with the format token your recorder actually produces. See Audio Formats for the tokens, the Opus framing rule, and why a mismatched rate degrades transcription without raising an error.

Native Android applications integrate with the Atoms agent over the raw WebSocket protocol. OkHttp handles transport, and the platform AudioRecord and AudioTrack classes handle PCM16 capture and playback.

Initialize AudioRecord with the VOICE_COMMUNICATION audio source to engage the platform’s acoustic echo cancellation and noise suppression.

Minimum supported version is Android 7 (API 24), which matches OkHttp 5’s API floor.

Validated end-to-end on a Pixel 9 emulator (Android API 35): OkHttp WebSocket connects, AudioRecord streams PCM16 with VOICE_COMMUNICATION for AEC coupling, and AudioTrack plays back agent audio in MODE_STREAM with USAGE_MEDIA (the STREAM_VOICE_CALL path is system-controlled and inaudible on emulators). Verify foreground service and Bluetooth route behavior on physical devices if those flows matter for your app.

When to use native Android

Android-only app, or a cross-platform app where Android is the priority.
You need fine control over the audio pipeline (specific sample rates, buffer sizes, AEC routing).
You need proper foreground-service handling for calls that continue when the app is backgrounded.

If your app is primarily React Native, see the React Native guide. For Flutter, see Flutter.

Dependencies

// build.gradle.kts (app)
dependencies {
    implementation("com.squareup.okhttp3:okhttp:4.12.0")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
}

Dependency	Role	Why this one
OkHttp	WebSocket client	Ubiquitous in Android, production-hardened, handles reconnect/backoff primitives. Alternative: Ktor client.
kotlinx-coroutines	Async orchestration	Bridges OkHttp’s callback model to suspending functions cleanly.
`AudioRecord` (platform)	Microphone PCM16 capture	Zero-dependency, gives raw Int16 access.
`AudioTrack` (platform)	PCM16 playback	Same. Streaming mode consumes your buffer at the device sample rate.

Manifest permissions

<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
    <!-- If you support background calls, add the foreground service perms: -->
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE_PHONE_CALL" />
</manifest>

Request RECORD_AUDIO at runtime:

private val permissionLauncher = registerForActivityResult(
    ActivityResultContracts.RequestPermission()
) { granted -> onMicPermission(granted) }
private fun ensureMicPermission() {
    val status = ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO)
    if (status == PackageManager.PERMISSION_GRANTED) onMicPermission(true)
    else permissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
}

Audio mode

Set AudioManager.MODE_IN_COMMUNICATION for the duration of the call. This signals the Android audio HAL that a bidirectional voice session is active; combined with MediaRecorder.AudioSource.VOICE_COMMUNICATION on the capture side, it enables the hardware AEC and NS pipeline. Restore the mode in the finally block of your session teardown.

val audioManager = getSystemService(Context.AUDIO_SERVICE) as AudioManager
val previousMode = audioManager.mode
audioManager.mode = AudioManager.MODE_IN_COMMUNICATION
try {
    runAgentSession()
} finally {
    audioManager.mode = previousMode
}

Playback routing is handled separately by the AudioTrack’s AudioAttributes (see Playback). The quickstart uses USAGE_MEDIA on the player, which routes to the main speaker by default on both emulators and physical devices, so there is no need to toggle isSpeakerphoneOn.

Quickstart

A full agent session: open the WebSocket, start mic capture, play agent audio, tear down.

import kotlinx.coroutines.*
import okhttp3.*
import okio.ByteString
import java.util.concurrent.TimeUnit
class AtomsAgent(
    private val apiKey:  String,
    private val agentId: String,
) {
    private val sampleRate = 24_000
    private val client = OkHttpClient.Builder()
        .readTimeout(0, TimeUnit.MILLISECONDS)  // unlimited for long-lived WS
        .build()
    // Own the scope: the microphone coroutine is launched asynchronously from
    // OkHttp's onOpen callback, so it must outlive any caller stack frame.
    // A short-lived scope (for example, one passed in from lifecycleScope.launch
    // whose lambda returns immediately) would be cancelled before onOpen fires,
    // and the mic coroutine would silently no-op.
    private val scope = CoroutineScope(Dispatchers.Default + SupervisorJob())
    private var webSocket: WebSocket? = null
    private var micJob: Job? = null
    private val player = AudioPlayer(sampleRate)
    fun start() {
        val url = HttpUrl.Builder()
            .scheme("https")  // OkHttp wraps wss:// via https://
            .host("api.smallest.ai")
            .addPathSegments("atoms/v1/agent/connect")
            .addQueryParameter("token",       apiKey)
            .addQueryParameter("agent_id",    agentId)
            .addQueryParameter("mode",        "webcall")
            .addQueryParameter("sample_rate", sampleRate.toString())
            .build()
        val request = Request.Builder().url(url).build()
        webSocket = client.newWebSocket(request, object : WebSocketListener() {
            override fun onOpen(ws: WebSocket, response: Response) {
                micJob = scope.launch { streamMicrophone(ws) }
                player.start()
            }
            override fun onMessage(ws: WebSocket, text: String)    = handleServerEvent(text)
            override fun onMessage(ws: WebSocket, bytes: ByteString) = handleServerEvent(bytes.utf8())
            override fun onClosing(ws: WebSocket, code: Int, reason: String) { stop() }
            override fun onFailure(ws: WebSocket, t: Throwable, r: Response?) { stop() }
        })
    }
    fun stop() {
        micJob?.cancel()
        player.stop()
        webSocket?.close(1000, "client stop")
        webSocket = null
        scope.cancel()
    }
}

Microphone capture

import android.media.AudioFormat
import android.media.AudioRecord
import android.media.MediaRecorder
import android.util.Base64
import org.json.JSONObject
private suspend fun streamMicrophone(ws: WebSocket) {
    val channelConfig = AudioFormat.CHANNEL_IN_MONO
    val encoding      = AudioFormat.ENCODING_PCM_16BIT
    val minBuffer     = AudioRecord.getMinBufferSize(sampleRate, channelConfig, encoding)
    val bufferSize    = (minBuffer * 2).coerceAtLeast(4096)
    val record = AudioRecord(
        MediaRecorder.AudioSource.VOICE_COMMUNICATION,  // enables platform AEC + NS
        sampleRate,
        channelConfig,
        encoding,
        bufferSize,
    )
    val chunk = ByteArray(bufferSize)
    try {
        record.startRecording()
        while (currentCoroutineContext().isActive) {
            val n = record.read(chunk, 0, chunk.size)
            if (n > 0) {
                val audio = Base64.encodeToString(chunk, 0, n, Base64.NO_WRAP)
                val payload = JSONObject().apply {
                    put("type",  "input_audio_buffer.append")
                    put("audio", audio)
                }
                ws.send(payload.toString())
            }
        }
    } finally {
        record.stop()
        record.release()
    }
}

MediaRecorder.AudioSource.VOICE_COMMUNICATION routes capture through the platform’s AEC/NS pipeline. Without it, the agent will hear its own output through the microphone and start looping.

Playback

AudioTrack in MODE_STREAM accepts writes as fast as you can feed it and plays at the hardware sample rate. Run it on a dedicated thread and queue chunks from the WebSocket callback.

Use USAGE_MEDIA (not USAGE_VOICE_COMMUNICATION) for the AudioTrack. Even though this is a voice call, USAGE_VOICE_COMMUNICATION routes to the STREAM_VOICE_CALL stream, which is system-controlled: its volume is not settable by a normal app and it is silent on emulators. Capture still uses MediaRecorder.AudioSource.VOICE_COMMUNICATION for AEC coupling, which is what matters for echo cancellation.

import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioTrack
import java.util.concurrent.LinkedBlockingQueue
class AudioPlayer(private val sampleRate: Int) {
    private val channelConfig = AudioFormat.CHANNEL_OUT_MONO
    private val encoding      = AudioFormat.ENCODING_PCM_16BIT
    private val minBuffer     = AudioTrack.getMinBufferSize(sampleRate, channelConfig, encoding)
    private val queue = LinkedBlockingQueue<ByteArray>()
    @Volatile private var running = false
    private var thread: Thread? = null
    private var track: AudioTrack? = null
    fun start() {
        running = true
        track = AudioTrack.Builder()
            .setAudioAttributes(
                AudioAttributes.Builder()
                    .setUsage(AudioAttributes.USAGE_MEDIA)
                    .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
                    .build()
            )
            .setAudioFormat(
                AudioFormat.Builder()
                    .setSampleRate(sampleRate)
                    .setChannelMask(channelConfig)
                    .setEncoding(encoding)
                    .build()
            )
            .setBufferSizeInBytes(minBuffer * 4)
            .setTransferMode(AudioTrack.MODE_STREAM)
            .build()
            .also { it.play() }
        thread = Thread {
            while (running) {
                val chunk = try { queue.poll(50, TimeUnit.MILLISECONDS) } catch (_: InterruptedException) { null } ?: continue
                track?.write(chunk, 0, chunk.size)
            }
        }.also { it.start() }
    }
    fun enqueue(pcm: ByteArray) { queue.offer(pcm) }
    fun flush() {
        queue.clear()
        track?.flush()
    }
    fun stop() {
        running = false
        thread?.join()
        track?.stop(); track?.release(); track = null
    }
}

Handle server events

private fun handleServerEvent(text: String) {
    val json = JSONObject(text)
    when (json.optString("type")) {
        "session.created"      -> { /* update UI on main thread */ }
        "output_audio.delta"    -> {
            val pcm = Base64.decode(json.getString("audio"), Base64.NO_WRAP)
            player.enqueue(pcm)
        }
        "agent_start_talking"   -> { /* UI: show "speaking" */ }
        "agent_stop_talking"    -> { /* UI: hide "speaking" */ }
        "interruption"          -> player.flush()
        "session.closed"        -> stop()
        "error"                 -> Log.e("Atoms", "[${json.optString("code")}] ${json.optString("message")}")
    }
}

Threading model

OkHttp WebSocketListener callbacks run on OkHttp’s internal executor. Do not block them. All UI work must cross to the main looper via Handler(Looper.getMainLooper()).post { ... } or a coroutine on Dispatchers.Main.
AudioRecord.read in a loop must run off the main thread. Use a background coroutine as shown in the quickstart.
AudioTrack.write is a blocking call when the internal buffer is full. Run it on its own thread (as shown) to avoid stalling your capture loop.

Audio focus

If the user is playing music or on another call, request audio focus before starting:

val focusRequest = AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE)
    .setAudioAttributes(
        AudioAttributes.Builder()
            .setUsage(AudioAttributes.USAGE_VOICE_COMMUNICATION)
            .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
            .build()
    )
    .setAcceptsDelayedFocusGain(false)
    .setOnAudioFocusChangeListener { change ->
        when (change) {
            AudioManager.AUDIOFOCUS_LOSS,
            AudioManager.AUDIOFOCUS_LOSS_TRANSIENT -> stop()
        }
    }
    .build()
val result = audioManager.requestAudioFocus(focusRequest)
if (result != AudioManager.AUDIOFOCUS_REQUEST_GRANTED) {
    // another app has exclusive focus; don't start the call
}

Abandon focus in stop().

Background calls

For calls that continue when the user backgrounds the app, run the agent in a foreground service. Without this, Android will silently starve your mic capture on Android 12+.

// Declared in AndroidManifest.xml:
// <service
//     android:name=".AgentForegroundService"
//     android:foregroundServiceType="phoneCall"
//     android:exported="false" />
class AgentForegroundService : Service() {
    override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
        val notification = buildOngoingCallNotification()
        startForeground(NOTIFICATION_ID, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_PHONE_CALL)
        return START_STICKY
    }
    // ... delegate to AtomsAgent from here ...
}

The phoneCall foreground service type requires the FOREGROUND_SERVICE_PHONE_CALL manifest permission (Android 14+).

Interruption handling

Incoming phone calls and other communication apps revoke audio focus. Handle AUDIOFOCUS_LOSS in the listener as shown above and tear down cleanly. On AUDIOFOCUS_GAIN after a transient loss, decide whether to auto-resume or prompt the user.

Bluetooth route changes

Bluetooth headsets connect and disconnect during calls routinely. AudioRecord and AudioTrack switch routes transparently on most devices. You may want to observe AudioManager.ACTION_AUDIO_BECOMING_NOISY to pause the call if wired headphones are unplugged:

val noisyReceiver = object : BroadcastReceiver() {
    override fun onReceive(context: Context, intent: Intent) {
        if (intent.action == AudioManager.ACTION_AUDIO_BECOMING_NOISY) {
            pauseCall()
        }
    }
}
context.registerReceiver(
    noisyReceiver,
    IntentFilter(AudioManager.ACTION_AUDIO_BECOMING_NOISY)
)

Production hardening

Reconnect on transient failure

OkHttp’s onFailure fires on network drops. Reconnect with exponential backoff up to 30 s. Do not retry on 4401/4403 codes (auth failure); check response?.code in the failure handler.

Mic mute while agent speaks

Stop AudioRecord briefly on agent_start_talking and restart on agent_stop_talking if device AEC is underperforming. The user’s speech during the agent turn goes undetected, which is usually the right trade-off versus audible self-feedback.

Battery

An open WebSocket + active AudioRecord + AudioTrack draws 3–5 % battery per minute. Design session duration accordingly. Always tear down promptly when the user ends the call.

Logging

Attach an interceptor to OkHttp for debugging the handshake. Remove before shipping.

Next steps

Realtime Agent WebSocket API

The full wire protocol with every message type, payload, and error code.

iOS (Swift)

Native iOS integration with URLSessionWebSocketTask and AVAudioEngine.

Flutter

Cross-platform Dart integration.

Error reference

HTTP status codes returned by every Atoms endpoint.