WSS
wss://api.modellix.ai/api/v1/live/openai/gpt-realtime-2.1-mini
For authentication, handshake errors, and close codes shared by all Live models, see the Live Overview.
Quick Start
SetMODELLIX_API_KEY, then run the script. It sends one text message, prints the spoken reply’s transcript, and saves the reply as raw 24 kHz PCM16 in reply.pcm.
Python
Sending Audio
Send base64-encoded audio in the format set byaudio.input.format. With voice activity detection (VAD) enabled, the model detects speech boundaries and responds automatically.
audio.input.turn_detection to null, then end each turn with these two messages:
Client Messages
Messages use the upstream event format. See the OpenAI client events reference for all fields.session.update
Sendsession.update after session.created. The following fields belong inside session.
Optional. The URL selects the model, and Modellix replaces this field with the upstream model identifier.
Allowed values:
["audio"], ["text"]. A single response cannot request both. Audio responses include a transcript.Allowed values:
{"type": "audio/pcm", "rate": 24000}, {"type": "audio/pcmu"}, {"type": "audio/pcma"}.Noise reduction with
type (near_field or far_field) and strength (0 to 1, default 1). Use null to turn it off.Attached input transcription is not available through Modellix. Only
null is accepted; any other value closes the connection with 1008. To transcribe speech, use GPT Live Transcribe or GPT Realtime Whisper on their own endpoints.Use
{"type": "semantic_vad"}, {"type": "server_vad"}, or null for manual turns. Inside the object, create_response and interrupt_response both default to true. Set them to false to keep VAD without automatic responses or interruptions.semantic_vad also accepts eagerness: low, medium, high, or auto (default). server_vad also accepts threshold (default 0.5), prefix_padding_ms (default 300), silence_duration_ms (default 500), and idle_timeout_ms (5000 to 30000, or null).Allowed values:
alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, cedar. OpenAI recommends marin or cedar. The voice cannot change after the model has replied with audio in the session.Function tool definitions. Your application runs each function and returns its result in the conversation. An empty array clears the tools.
Allowed strings:
none, auto, required. You can also pass an object that selects one function.Allowed values:
minimal, low, medium, high, xhigh. Higher effort can increase latency and output token usage.Other Client Events
Adds an item to the conversation. A user message can contain
input_text, input_audio, and input_image parts. An input_image part takes image_url as a base64 PNG or JPEG data URI, and detail as auto (default), low, or high.Cancels the response in progress. See Billing for how cancelled responses are charged.
Server Messages
Server events are forwarded unchanged. See the OpenAI server events reference for all fields.The final event of a response.
response.status is one of completed, cancelled, failed, incomplete, or in_progress. response.usage contains the token counts used for billing.Billing
Each response produces one request record, based on the finalresponse.usage for that response ID. A repeated final event is not charged twice.
- Token types. Text, audio, and image input, cached input, and text and audio output are priced separately. Cached tokens are part of the input count, and are charged at the cached rate rather than twice. Reasoning tokens are already part of output and are not added again. Unit prices and invoices follow the console.
- Cancelled and interrupted responses. Charges follow the usage the upstream reports, which can be zero or positive. Disconnecting does not guarantee a response is free: if you disconnect mid-response, Modellix cancels it and waits up to 2 seconds for the final usage.
- Missing usage. If the connection ends before final usage arrives, only usage already received is billed. Modellix does not estimate missing usage.
Sessions and Limits
- WebSocket only. WebRTC, browser client secrets, and other upstream HTTP endpoints are not available through Modellix.
- Attached input transcription is not available. Use GPT Live Transcribe or GPT Realtime Whisper on their own endpoints.
- Modellix closes any connection at its maximum connection duration with code
4408.