Realtime

Build low-latency, multi-modal experiences with the Realtime API.

The OPE.AI Realtime API enables low-latency, multimodal interactions including speech-to-speech conversational experiences and real-time transcription.

Connect with WebSockets

WebSockets are a broadly supported API for realtime data transfer, and a great choice for connecting to the OPE.AI Realtime API in server-to-server applications.

Overview

In a server-to-server integration with Realtime, your backend system will connect via WebSocket directly to the Realtime API. You can use a standard API key to authenticate this connection, since the token will only be available on your secure backend server.

Standard API tokens should only be used in secure server-side environments.

Connection details

Speech-to-SpeechTranscription

Connecting via WebSocket requires the following connection information:

URL	wss://api-platform.ope.ai/v1/realtime
Query Parameters	model Realtime model ID to connect to, like opeai/AudioLLM/Voice2
Headers	Authorization: Bearer $API_KEY OpenAI-Beta: realtime=v1 This header is required during the beta period.

Below are several examples of using these connection details to initialize a WebSocket connection to the Realtime API.

ws module(Node.js)websocket-client(Python)WebSocket(browsers)

import WebSocket from "ws";

const url = "wss://api-platform.ope.ai/v1/realtime?model=opeai/AudioLLM/Voice2";
const ws = new WebSocket(url, {
  headers: {
    "Authorization": "Bearer " + process.env.API_KEY,
    "OpenAI-Beta": "realtime=v1",
  },
});

ws.on("open", () => console.log("Connected"));
ws.on("message", (msg) => console.log(JSON.parse(msg.toString())));

import os
import json
import websocket

API_KEY = os.environ.get("API_KEY")

url = "wss://api-platform.ope.ai/v1/realtime?model=opeai/AudioLLM/Voice2"
headers = [
    "Authorization: Bearer " + API_KEY,
    "OpenAI-Beta: realtime=v1"
]

def on_open(ws):
    print("Connected to server.")

def on_message(ws, message):
    data = json.loads(message)
    print("Received event:", json.dumps(data, indent=2))

ws = websocket.WebSocketApp(
    url, 
    header=headers,
    on_open=on_open,
    on_message=on_message,
)

ws.run_forever()

const ws = new WebSocket(
  "wss://api-platform.ope.ai/v1/realtime?model=opeai/AudioLLM/Voice2",
  [
    "realtime",
    //Auth
    "openai-insecure-api-key." + API_KEY,
    //Beta protocol, required
    "openai-beta.realtime-v1"
  ]
);

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(message.data);
});  

Connecting via WebSocket requires the following connection information:

URL	wss://api-platform.ope.ai/v1/realtime
Query Parameters	intent The intent of the connection: transcription
Headers	Authorization: Bearer $API_KEY OpenAI-Beta: realtime=v1 This header is required during the beta period.

Below are several examples of using these connection details to initialize a WebSocket connection to the Realtime API.

ws module(Node.js)websocket-client(Python)WebSocket(browsers)

import WebSocket from "ws";

const url = "wss://api-platform.ope.ai/v1/realtime?intent=transcription";
const ws = new WebSocket(url, {
  headers: {
    "Authorization": "Bearer " + process.env.API_KEY,
    "OpenAI-Beta": "realtime=v1",
  },
});

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(JSON.parse(message.toString()));
});

import os
import json
import websocket

API_KEY = os.environ.get("API_KEY")

url = "wss://api-platform.ope.ai/v1/realtime?intent=transcription"
headers = [
    "Authorization: Bearer " + API_KEY,
    "OpenAI-Beta: realtime=v1"
]

def on_open(ws):
    print("Connected to server.")

def on_message(ws, message):
    data = json.loads(message)
    print("Received event:", json.dumps(data, indent=2))

ws = websocket.WebSocketApp(
    url, 
    header=headers,
    on_open=on_open,
    on_message=on_message,
)

ws.run_forever()

const ws = new WebSocket(
  "wss://api-platform.ope.ai/v1/realtime?intent=transcription",
  [
    "realtime",
    //Auth
    "openai-insecure-api-key." + API_KEY,
    //Beta protocol, required
    "openai-beta.realtime-v1"
  ]
);

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(message.data);
});