Media Stream

When a call starts, Inkbox opens a WebSocket connection to your agent at the client_websocket_url you configured on the phone number or provided when placing the call. This connection carries the live call data between your agent and the caller for the duration of the call. What flows over this connection (text, audio, or both) depends on how your agent configures itself. Inkbox can handle text-to-speech (TTS), speech-to-text (STT), or both on your behalf. See Choosing a mode below.

Connection flow

Inkbox connects to your client_websocket_url with an X-Call-Context header containing the call_id, phone_number, direction, and matching contacts. If the agent identity that owns this phone number has a signing key, the connection also includes X-Inkbox-Request-ID, X-Inkbox-Timestamp, and X-Inkbox-Signature headers.

The X-Call-Context value is canonical JSON:

{
"call_id": "550e8400-e29b-41d4-a716-446655440000",
"phone_number": "+14155550100",
"direction": "inbound",
"contacts": [
  {
    "id": "7d82f038-62b2-4ad1-a747-2db860a118f5",
    "name": "Jane Doe",
    "memories": ["Prefers concise updates."],
    "memories_truncated": false
  }
]
}

contacts is always present and may be empty. A contact’s name is null when it has no name on file — a contact created automatically from an inbound call has an id and memories before anyone gives it a name, and name never falls back to the caller’s phone number. Check it before having your agent greet someone by name. Memories are ordered newest first. The header value is limited to 8,000 UTF-8 bytes; if older memories are omitted to fit that limit, memories_truncated is true. Verify the signature against the exact X-Call-Context header value before parsing it.

Your agent accepts the WebSocket and declares its capabilities with response headers:

Header	Default	Description
`X-Use-Inkbox-Text-To-Speech`	`true`	If `true`, Inkbox converts your text responses to speech. If `false`, your agent sends audio directly.
`X-Use-Inkbox-Speech-To-Text`	`true`	If `true`, Inkbox transcribes the caller’s speech and sends you text. If `false`, your agent receives raw audio.
`X-Inkbox-Audio-Format`	`pcmu_8000`	Format for audio you send and receive: `pcmu_8000` or `pcm_s16le_16000`. See Audio format.

If you omit the speech headers, both default to true, so Inkbox handles STT and TTS. Audio defaults to PCMU at 8 kHz.

start event is sent to your agent with the call_control_id and media format details.
Streaming begins. Text or audio flows bidirectionally depending on the mode.
stop event is sent when the call ends.

Choosing a mode

The combination of the two speech headers gives you four configurations. Use the selector below to explore what each mode looks like, including the exact WebSocket events your agent sends and receives.

Inkbox handles STT + TTS
Inkbox handles TTS only
Inkbox handles STT only
Full audio bridge

Inkbox transcribes the caller and synthesizes your responses. Your agent only deals with text.

Who handles what?
Text-to-speech (TTS)	Inkbox
Speech-to-text (STT)	Inkbox
Your agent receives	Text (transcribed caller speech)
Your agent sends	Text (to be spoken to the caller)
Inkbox handles	Speech-to-text + text-to-speech

Simplest setup. Ideal when your agent is a text-based LLM and you want Inkbox to handle all audio processing.

WebSocket response headers

Your agent declares this configuration by setting these headers when accepting the WebSocket connection:

X-Use-Inkbox-Text-To-Speech: true
X-Use-Inkbox-Speech-To-Text: true

Events you receive (Inkbox → your agent)

Event	Description
`start`	Call stream opened. Contains call metadata and media format.
`transcript`	Caller speech transcribed by Inkbox. Sent as interim results and a final result per utterance.
`barge_in`	Stop generating the current response. Includes caller speech or a turn-cancellation signal; see Interruptions.
`stop`	Call ended.

start

{
  "event": "start",
  "stream_id": "abc",
  "start": {
    "call_control_id": "v3:xyz",
    "stream_id": "abc",
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

transcript

{
  "event": "transcript",
  "text": "I'd like to check my order status",
  "is_final": true,
  "turn_id": "turn_01"
}

barge_in

{
  "event": "barge_in",
  "trigger": "interim_transcript",
  "text": "Actually, I have another question",
  "tts_interrupted": true,
  "turn_id": "turn_02"
}

stop

{
  "event": "stop",
  "reason": "remote"
}

Events you send (your agent → Inkbox)

Event	Description
`text`	Stream text to be spoken to the caller. Send chunks with `done: false`, then a final message with `done: true`. Inkbox converts each chunk to speech and plays it.
`clear`	Clear audio queued for playback to the caller.
`dtmf`	Press keypad digits on the call as touch-tones, for automated phone menus.
`stop`	Hang up the call from your side.

text

{
  "event": "text",
  "delta": "Your order #1234 has shipped",
  "done": false,
  "turn_id": "turn_01"
}

{
  "event": "text",
  "done": true,
  "turn_id": "turn_01"
}

dtmfPress one to 30 keys (0-9, *, #) on the live call, in order. Use it to work an automated phone menu or enter an extension. A payload with any key outside that set, or more than 30 keys, is ignored as a whole; the stream is unaffected. Presses are sent one burst at a time and never block your audio. At most 20 bursts per call can be outstanding, including the one being sent; further bursts are ignored. Nothing is sent back for a press: a rejected payload, a carrier error, or a call that ends mid-press is dropped silently, so read the menu’s next prompt to confirm the keys took. A stop waits up to 10 seconds for queued keypad commands to be accepted, then hangs up; it does not wait for all tones to finish playing. Wait for the menu’s next prompt before sending another burst or stop.

{
  "event": "dtmf",
  "digits": "1"
}

stop

{
  "event": "stop",
  "reason": "agent_hang_up"
}

Your agent receives raw caller audio and runs its own speech-to-text. Responses are sent as text, which Inkbox converts to speech.

Who handles what?
Text-to-speech (TTS)	Inkbox
Speech-to-text (STT)	Your agent
Your agent receives	Raw audio in the negotiated format
Your agent sends	Text (to be spoken to the caller)
Inkbox handles	Text-to-speech only

When you want to run your own STT model (e.g. for specialized vocabulary or languages) but still want Inkbox to handle TTS.

WebSocket response headers

Your agent declares this configuration by setting these headers when accepting the WebSocket connection:

X-Use-Inkbox-Text-To-Speech: true
X-Use-Inkbox-Speech-To-Text: false

Events you receive (Inkbox → your agent)

Event	Description
`start`	Call stream opened.
`media`	Raw caller audio, base64-encoded in the negotiated format.
`stop`	Call ended.

start

{
  "event": "start",
  "stream_id": "abc",
  "start": {
    "call_control_id": "v3:xyz",
    "stream_id": "abc",
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

media

{
  "event": "media",
  "stream_id": "abc",
  "media": {
    "payload": "<base64-pcmu>",
    "track": "inbound"
  }
}

stop

{
  "event": "stop",
  "reason": "remote"
}

Events you send (your agent → Inkbox)

Event	Description
`text`	Stream text to be spoken. Inkbox converts to speech.
`transcript`	Send caller transcripts from your STT so they appear in call history.
`clear`	Clear audio queued for playback to the caller.
`dtmf`	Press keypad digits on the call as touch-tones, for automated phone menus.
`stop`	Hang up the call.

text

{
  "event": "text",
  "delta": "Let me look that up for you.",
  "done": false,
  "turn_id": "turn_01"
}

{
  "event": "text",
  "done": true,
  "turn_id": "turn_01"
}

transcript

{
  "event": "transcript",
  "party": "remote",
  "text": "what is my order status",
  "is_final": true,
  "turn_id": "turn_01"
}

{
  "event": "dtmf",
  "digits": "1"
}

stop

{
  "event": "stop",
  "reason": "agent_hang_up"
}

Inkbox transcribes the caller’s speech to text. Your agent responds with synthesized audio directly.

Who handles what?
Text-to-speech (TTS)	Your agent
Speech-to-text (STT)	Inkbox
Your agent receives	Text (transcribed caller speech)
Your agent sends	Audio (synthesized speech)
Inkbox handles	Speech-to-text only

Use this mode when your agent generates speech with its own TTS service.

WebSocket response headers

Your agent declares this configuration by setting these headers when accepting the WebSocket connection:

X-Use-Inkbox-Text-To-Speech: false
X-Use-Inkbox-Speech-To-Text: true

Events you receive (Inkbox → your agent)

Event	Description
`start`	Call stream opened.
`transcript`	Caller speech transcribed by Inkbox.
`barge_in`	Stop generating the current response and sending its audio. See Interruptions.
`stop`	Call ended.

start

{
  "event": "start",
  "stream_id": "abc",
  "start": {
    "call_control_id": "v3:xyz",
    "stream_id": "abc",
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

transcript

{
  "event": "transcript",
  "text": "I need to reschedule my appointment",
  "is_final": true,
  "turn_id": "turn_01"
}

barge_in

{
  "event": "barge_in",
  "trigger": "interim_transcript",
  "text": "Actually, I have another question",
  "tts_interrupted": false,
  "turn_id": "turn_02"
}

stop

{
  "event": "stop",
  "reason": "remote"
}

Events you send (your agent → Inkbox)

Event	Description
`media`	Send synthesized audio to the caller, base64-encoded in the negotiated format.
`audio_done`	Signal that you’ve finished sending audio for the current turn.
`transcript`	Send your agent’s response transcript so it appears in call history.
`clear`	Clear audio queued for playback to the caller.
`dtmf`	Press keypad digits on the call as touch-tones, for automated phone menus.
`stop`	Hang up the call.

media

{
  "event": "media",
  "stream_id": "abc",
  "media": {
    "payload": "<base64-pcmu>",
    "track": "outbound"
  }
}

audio_done

{
  "event": "audio_done"
}

transcript

{
  "event": "transcript",
  "party": "local",
  "text": "Sure, I can help you reschedule.",
  "is_final": true,
  "turn_id": "turn_01"
}

{
  "event": "dtmf",
  "digits": "1"
}

stop

{
  "event": "stop",
  "reason": "agent_hang_up"
}

Inkbox passes raw audio through without any processing. Your agent handles everything: STT, LLM, and TTS.

Who handles what?
Text-to-speech (TTS)	Your agent
Speech-to-text (STT)	Your agent
Your agent receives	Raw audio in the negotiated format
Your agent sends	Raw audio in the negotiated format
Inkbox handles	Nothing (transport only)

Use this mode when your agent handles STT, LLM, and TTS itself or connects to a realtime speech model.

WebSocket response headers

Your agent declares this configuration by setting these headers when accepting the WebSocket connection:

X-Use-Inkbox-Text-To-Speech: false
X-Use-Inkbox-Speech-To-Text: false

Events you receive (Inkbox → your agent)

Event	Description
`start`	Call stream opened.
`media`	Raw caller audio, base64-encoded in the negotiated format.
`stop`	Call ended.

start

{
  "event": "start",
  "stream_id": "abc",
  "start": {
    "call_control_id": "v3:xyz",
    "stream_id": "abc",
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

media

{
  "event": "media",
  "stream_id": "abc",
  "media": {
    "payload": "<base64-pcmu>",
    "track": "inbound"
  }
}

stop

{
  "event": "stop",
  "reason": "remote"
}

Events you send (your agent → Inkbox)

Event	Description
`media`	Send audio to the caller, base64-encoded in the negotiated format.
`audio_done`	Signal that you’ve finished sending audio for the current turn.
`transcript`	Send both caller and agent transcripts so they appear in call history.
`clear`	Clear audio queued for playback to the caller.
`dtmf`	Press keypad digits on the call as touch-tones, for automated phone menus.
`stop`	Hang up the call.

media

{
  "event": "media",
  "stream_id": "abc",
  "media": {
    "payload": "<base64-pcmu>",
    "track": "outbound"
  }
}

audio_done

{
  "event": "audio_done"
}

transcriptCaller transcript:

{
  "event": "transcript",
  "party": "remote",
  "text": "what is my balance",
  "is_final": true,
  "turn_id": "turn_01"
}

Agent transcript:

{
  "event": "transcript",
  "party": "local",
  "text": "Your current balance is $142.50.",
  "is_final": true,
  "turn_id": "turn_01"
}

{
  "event": "dtmf",
  "digits": "1"
}

stop

{
  "event": "stop",
  "reason": "agent_hang_up"
}

Audio format

Audio payloads are base64-encoded inside media events. Choose the format with the X-Inkbox-Audio-Format header in your WebSocket handshake response:

Header value	Encoding	Sample rate	Channels
`pcmu_8000` (default)	PCMU (u-law)	8,000 Hz	1
`pcm_s16le_16000`	Signed 16-bit little-endian PCM	16,000 Hz	1

The selected format applies to audio in both directions. The examples above use the default. To use PCM16, return:

X-Inkbox-Audio-Format: pcm_s16le_16000

The start.media_format then reports encoding: "L16", sample_rate: 16000, and channels: 1. Despite the L16 label, PCM16 samples on this WebSocket are little-endian. Unsupported or repeated audio-format headers cause the connection to close.

Interruptions

When you receive barge_in, stop generating the current response and stop sending its text or audio. The trigger is interim_transcript or final_transcript for caller speech. Its text contains the recognized speech or an indication that the caller started speaking. A cancel_turn trigger requests cancellation and can have empty text. The tts_interrupted flag reports whether playback was interrupted. Send a clear event to discard audio already queued for playback to the caller:

{
  "event": "clear"
}

clear travels from your agent to Inkbox. Stop sending the old response before clearing its queued audio.

Transcripts

Inkbox stores call transcripts as the call progresses. When Inkbox handles STT, it captures caller transcripts automatically. When your agent handles STT or TTS, send transcript events for the speech it transcribes or generates. You can retrieve transcripts after the call via the Transcripts API.

Call duration

Maximum call duration defaults to 10 minutes on Free and 60 minutes on Developer, Startup, and Enterprise. Custom limits may apply to your organization. When the limit is reached, Inkbox ends the call with hangup_reason: "max_duration". On Free, a call also ends with "max_duration" if the organization runs out of its included call minutes first. See Rate Limits for organization-level limits.

Setting your stream URL

Configure client_websocket_url on a phone number so it’s used automatically for all auto-accepted calls:

JSON

{
  "incoming_call_action": "auto_accept",
  "client_websocket_url": "wss://your-agent.example.com/stream"
}

Or provide a client_websocket_url per-call when placing outbound calls or responding to incoming call webhooks.

Agent signup API

Agent identity API

Agent2Agent API

Message retries

Mail API

Phone API

iMessage API

Slack API

Webhooks API

Tunnels API

Contacts API

Notes API

Vault API

Billing API

Connection flow

Choosing a mode

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

Audio format

Interruptions

Transcripts

Call duration

Setting your stream URL

​Connection flow

​Choosing a mode

​WebSocket response headers

​Events you receive (Inkbox → your agent)

​Events you send (your agent → Inkbox)

​WebSocket response headers

​Events you receive (Inkbox → your agent)

​Events you send (your agent → Inkbox)

​WebSocket response headers

​Events you receive (Inkbox → your agent)

​Events you send (your agent → Inkbox)

​WebSocket response headers

​Events you receive (Inkbox → your agent)

​Events you send (your agent → Inkbox)

​Audio format

​Interruptions

​Transcripts

​Call duration

​Setting your stream URL

Connection flow

Choosing a mode

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

WebSocket response headers

Events you receive (Inkbox → your agent)

Events you send (your agent → Inkbox)

Audio format

Interruptions

Transcripts

Call duration

Setting your stream URL