Skip to content

Commit dc5e851

Browse files
sahilsuman933sahilsuman933
authored
Update call-features.mdx
1 parent 499ded4 commit dc5e851

File tree

1 file changed

+68
-54
lines changed

1 file changed

+68
-54
lines changed

‎calls/call-features.mdx

Lines changed: 68 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -2,42 +2,93 @@
22
title: "Listen, Control, Language Detection"
33
sidebarTitle: "Live Call Features"
44
---
5+
Vapi offers two main features that provide enhanced control over live calls:
56

6-
In this documentation, we will showcase our three new features and how you can use them:
7+
1. **Call Control**: This feature allows you to inject conversation elements dynamically during an ongoing call.
8+
2. **Call Listen**: This feature enables real-time audio data streaming using WebSocket connections.
79

8-
1. **Call Control**: Enables dynamic injection of conversation elements during live calls.
9-
2. **Call Listen**: Provides real-time audio streaming and processing during the call.
10-
3. **Automatic Language Detection**: Detect the language in real-time conversation and talk in that particular language.
10+
To use these features, you first need to obtain the URLs specific to the live call. These URLs can be retrieved by triggering a `/call` endpoint, which returns the `listenUrl` and `controlUrl` within the `monitor` object.
1111

12-
## Call Control and Call Listen Feature
12+
## Obtaining URLs for Call Control and Listen
1313

14-
When you initiate a call with the `/call` endpoint, you will receive a call ID. You can listen to the call directly via the Call Listen feature, and if you want to inject some operations into it, you can use the Call Control functionality.
14+
To initiate a call and retrieve the `listenUrl` and `controlUrl`, send a POST request to the `/call` endpoint.
1515

16-
### Call Control
16+
### Sample Request
1717

18-
Call Control allows you to inject conversation elements dynamically during a live call via HTTP POST requests. Currently, we support injecting messages in real-time. More operations will be supported in the future.
18+
```bash
19+
curl 'https://api.vapi.ai/call/phone'
20+
-H 'authorization: Bearer YOUR_API_KEY'
21+
-H 'content-type: application/json'
22+
--data-raw '{
23+
"assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80",
24+
"customer": {
25+
"number": "+12345678913"
26+
},
27+
"phoneNumberId": "42b4b25d-031e-4786-857f-63b346c9580f"
28+
}'
29+
30+
```
1931

20-
To inject a message, send a POST request in this format:
32+
### Sample Response
33+
34+
```json
35+
{
36+
"id": "7420f27a-30fd-4f49-a995-5549ae7cc00d",
37+
"assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80",
38+
"phoneNumberId": "42b4b25d-031e-4786-857f-63b346c9580f",
39+
"type": "outboundPhoneCall",
40+
"createdAt": "2024-09-10T11:14:12.339Z",
41+
"updatedAt": "2024-09-10T11:14:12.339Z",
42+
"orgId": "eb166faa-7145-46ef-8044-589b47ae3b56",
43+
"cost": 0,
44+
"customer": {
45+
"number": "+12345678913"
46+
},
47+
"status": "queued",
48+
"phoneCallProvider": "twilio",
49+
"phoneCallProviderId": "CA4c6793d069ef42f4ccad69a0957451ec",
50+
"phoneCallTransport": "pstn",
51+
"monitor": {
52+
"listenUrl": "wss://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport",
53+
"controlUrl": "<https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control>"
54+
}
55+
}
56+
57+
```
58+
59+
## Call Control Feature
60+
61+
Once you have the `controlUrl`, you can inject a message into the live call using a POST request. This can be done by sending a JSON payload to the `controlUrl`.
62+
63+
### Example: Injecting a Message
2164

2265
```bash
23-
curl -X POST https://aws-us-west-2-production3-phone-call-websocket.vapi.ai/{call_id}/control \
24-
-H "Content-Type: application/json" \
25-
-d '{
66+
curl -X POST 'https://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/control'
67+
-H 'content-type: application/json'
68+
--data-raw '{
2669
"type": "say",
2770
"message": "Welcome to Vapi, this message was injected during the call."
2871
}'
72+
2973
```
3074

31-
### Call Listen
75+
The message will be spoken in real-time during the ongoing call.
76+
77+
## Call Listen Feature
78+
79+
The `listenUrl` allows you to connect to a WebSocket and stream the audio data in real-time. You can either process the audio directly or save the binary data to analyze or replay later.
3280

33-
Call Listen enables real-time streaming and processing of audio data using WebSocket connections. Here's an example implementation showcasing how you can receive audio packets and manipulate them based on your needs:
81+
### Example: Saving Audio Data from a Live Call
3482

35-
```javascript
83+
Here is a simple implementation for saving the audio buffer from a live call using Node.js:
84+
85+
```jsx
3686
const WebSocket = require('ws');
3787
const fs = require('fs');
3888

3989
let pcmBuffer = Buffer.alloc(0);
40-
const ws = new WebSocket(`${listenUrl}/listen`);
90+
91+
const ws = new WebSocket("wss://aws-us-west-2-production1-phone-call-websocket.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport");
4192

4293
ws.on('open', () => console.log('WebSocket connection established'));
4394

@@ -58,42 +109,5 @@ ws.on('close', () => {
58109
});
59110

60111
ws.on('error', (error) => console.error('WebSocket error:', error));
61-
```
62-
63-
## Automatic Language Detection
64-
65-
This feature allows you to automatically switch between languages during a call. It is currently supported only on Deepgram and supports the following languages:
66-
67-
<ul>
68-
<li>ar: Arabic</li>
69-
<li>bn: Bengali</li>
70-
<li>yue: Cantonese</li>
71-
<li>zh: Chinese</li>
72-
<li>en: English</li>
73-
<li>fr: French</li>
74-
<li>de: German</li>
75-
<li>hi: Hindi</li>
76-
<li>it: Italian</li>
77-
<li>ja: Japanese</li>
78-
<li>ko: Korean</li>
79-
<li>pt: Portuguese</li>
80-
<li>ru: Russian</li>
81-
<li>es: Spanish</li>
82-
<li>th: Thai</li>
83-
<li>vi: Vietnamese</li>
84-
</ul>
85112

86-
To enable automatic language detection for multilingual calls, set `transcriber.languageDetectionEnabled: true` through the `/assistant` API endpoint or use the assistantOverride.
87-
88-
### Requirements for Multilingual Support
89-
90-
To make multilingual support work, you need to choose the following models:
91-
92-
* **Transcriber**:
93-
* **Deepgram**: `nova-2` or `nova-2-general`
94-
95-
* **Voice Providers**:
96-
* **11labs**: Multilingual model or Turbo v2.5
97-
* **Cartesia**: `sonic-multilingual` model
98-
99-
By using these models and enabling automatic language detection, your application will be able to handle multilingual conversations seamlessly.
113+
```

0 commit comments

Comments
 (0)