Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/seven-llamas-buy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@openai/agents-realtime': patch
---

feat: #439 add SIP support for realtime agent runner
56 changes: 56 additions & 0 deletions examples/realtime-twilio-sip/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Twilio SIP Realtime Example

This example shows how to handle OpenAI Realtime SIP calls with the Agents JS SDK. Incoming calls are accepted through the Realtime Calls API, a triage agent answers with a fixed greeting, and handoffs route the caller to specialist agents (FAQ lookup and record updates) similar to the realtime UI demo.

## Prerequisites

- Node.js 22+
- pnpm 10+
- An OpenAI API key with Realtime API access
- A configured webhook secret for your OpenAI project
- A Twilio account with a phone number and Elastic SIP Trunking enabled
- A public HTTPS endpoint for local development (for example, [ngrok](https://ngrok.com/))

## Configure OpenAI

1. In [platform settings](https://platform.openai.com/settings) select your project.
2. Create a webhook pointing to `https://<your-public-host>/openai/webhook` with the **realtime.call.incoming** event type and note the signing secret. The server verifies each webhook using `OPENAI_WEBHOOK_SECRET`.

## Configure Twilio Elastic SIP Trunking

1. Create (or edit) an Elastic SIP trunk.
2. On the **Origination** tab, add an origination SIP URI of `sip:proj_<your_project_id>@sip.api.openai.com;transport=tls` so Twilio sends inbound calls to OpenAI. (The Termination tab always ends with `.pstn.twilio.com`, so leave it unchanged.)
3. Attach at least one phone number to the trunk so inbound calls are forwarded to OpenAI.

## Setup

1. Install dependencies from the monorepo root (if you have not already):
```bash
pnpm install
```
2. Export the required environment variables:
```bash
export OPENAI_API_KEY="sk-..."
export OPENAI_WEBHOOK_SECRET="whsec_..."
export PORT=8000 # optional, defaults to 8000
```
3. (Optional) Adjust the multi-agent logic in `examples/realtime-twilio-sip/agents.ts` if you want to change the specialist agents or tools.
4. Start the Fastify server:
```bash
pnpm -F realtime-twilio-sip start
```
5. Expose the server publicly (example with ngrok):
```bash
ngrok http 8000
```

## Test a Call

1. Place a call to the Twilio number attached to the SIP trunk.
2. Twilio sends the call to `sip.api.openai.com`; OpenAI emits a `realtime.call.incoming` event, which this server accepts via the Realtime Calls API.
3. The triage agent greets the caller, then either keeps the conversation or hands off to:
- **FAQ Agent** – answers common questions via `faq_lookup_tool`.
- **Records Agent** – writes short notes using `update_customer_record`.
4. The background task attaches to the call and logs transcripts plus basic events in the console.

Tweak `server.ts` to customize instructions, add tools, or integrate with internal systems after the SIP session is active.
89 changes: 89 additions & 0 deletions examples/realtime-twilio-sip/agents.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
import { RECOMMENDED_PROMPT_PREFIX } from '@openai/agents-core/extensions';
import { RealtimeAgent, tool } from '@openai/agents/realtime';
import { z } from 'zod';

export const WELCOME_MESSAGE =
'Hello, this is ABC customer service. How can I help you today?';

const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

const faqLookupSchema = z.object({
question: z.string().describe('The caller question to search for.'),
});

const faqLookupTool = tool({
name: 'faq_lookup_tool',
description: 'Lookup frequently asked questions for the caller.',
parameters: faqLookupSchema,
execute: async ({ question }: z.infer<typeof faqLookupSchema>) => {
await wait(1000);

const normalized = question.toLowerCase();
if (normalized.includes('wi-fi') || normalized.includes('wifi')) {
return 'We provide complimentary Wi-Fi. Join the ABC-Customer network.';
}
if (normalized.includes('billing') || normalized.includes('invoice')) {
return 'Your latest invoice is available in the ABC portal under Billing > History.';
}
if (normalized.includes('hours') || normalized.includes('support')) {
return 'Human support agents are available 24/7; transfer to the specialist if needed.';
}
return "I'm not sure about that. Let me transfer you back to the triage agent.";
},
});

const updateCustomerRecordSchema = z.object({
customerId: z
.string()
.describe('Unique identifier for the customer you are updating.'),
note: z
.string()
.describe('Brief summary of the customer request to store in records.'),
});

const updateCustomerRecord = tool({
name: 'update_customer_record',
description: 'Record a short note about the caller.',
parameters: updateCustomerRecordSchema,
execute: async ({
customerId,
note,
}: z.infer<typeof updateCustomerRecordSchema>) => {
await wait(1000);
return `Recorded note for ${customerId}: ${note}`;
},
});

const faqAgent = new RealtimeAgent({
name: 'FAQ Agent',
handoffDescription:
'Handles frequently asked questions and general account inquiries.',
instructions: `${RECOMMENDED_PROMPT_PREFIX}
You are an FAQ specialist. Always rely on the faq_lookup_tool for answers and keep replies concise. If the caller needs hands-on help, transfer back to the triage agent.`,
tools: [faqLookupTool],
});

const recordsAgent = new RealtimeAgent({
name: 'Records Agent',
handoffDescription:
'Updates customer records with brief notes and confirmation numbers.',
instructions: `${RECOMMENDED_PROMPT_PREFIX}
You handle structured updates. Confirm the customer's ID, capture their request in a short note, and use the update_customer_record tool. For anything outside data updates, return to the triage agent.`,
tools: [updateCustomerRecord],
});

const triageAgent = new RealtimeAgent({
name: 'Triage Agent',
handoffDescription:
'Greets callers and routes them to the most appropriate specialist.',
instructions: `${RECOMMENDED_PROMPT_PREFIX}
Always begin the call by saying exactly '${WELCOME_MESSAGE}' before collecting details. Once the greeting is complete, gather context and hand off to the FAQ or Records agents when appropriate.`,
handoffs: [faqAgent, recordsAgent],
});

faqAgent.handoffs = [triageAgent, recordsAgent];
recordsAgent.handoffs = [triageAgent, faqAgent];

export function getStartingAgent(): RealtimeAgent {
return triageAgent;
}
17 changes: 17 additions & 0 deletions examples/realtime-twilio-sip/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"private": true,
"name": "realtime-twilio-sip",
"dependencies": {
"@openai/agents-core": "workspace:*",
"@openai/agents-realtime": "workspace:*",
"@openai/agents": "workspace:*",
"dotenv": "^16.5.0",
"fastify": "^5.3.3",
"fastify-raw-body": "^5.0.0",
"openai": "^6.7.0"
},
"scripts": {
"build-check": "tsc --noEmit",
"start": "tsx server.ts"
}
}
Loading