Skip to content

Commit d880761

Browse files
opensearch-trigger-bot[bot]shengbo-ma
opensearch-trigger-bot[bot]
and
shengbo-ma
authored
Add connector blueprint for Azure OpenAI Embedding and Chat model (#1874) (#2062)
* add: connector blueprint for Azure OpenAI embedding * add: connector blueprint for azure openai chat model * minor: fix typo (cherry picked from commit 671457b) Co-authored-by: OysterMax <sma@esri.com>
1 parent 731aa72 commit d880761

File tree

2 files changed

+354
-0
lines changed

2 files changed

+354
-0
lines changed

‎docs/remote_inference_blueprints/azure_openai_connector_chat_blueprint.md

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Azure OpenAI connector Blueprint for Chat Completion:
2+
3+
## 1. Add Azure OpenAI Endpoint to Trusted URLs:
4+
5+
```json
6+
PUT /_cluster/settings
7+
{
8+
"persistent": {
9+
"plugins.ml_commons.trusted_connector_endpoints_regex": [
10+
"^https://.*\\.openai\\.azure\\.com/.*$"
11+
]
12+
}
13+
}
14+
```
15+
16+
## 2. Create Connector for Azure OpenAI Chat Model:
17+
18+
Refer to [Azure OpenAI Service REST API reference - Chat Completion](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions).
19+
20+
If you are using self-managed Opensearch, you should supply OpenAI API key:
21+
22+
23+
```json
24+
POST /_plugins/_ml/connectors/_create
25+
{
26+
"name": "<YOUR CONNECTOR NAME>",
27+
"description": "<YOUR CONNECTOR DESCRIPTION>",
28+
"version": "<YOUR CONNECTOR VERSION>",
29+
"protocol": "http",
30+
"parameters": {
31+
"endpoint": "<YOUR RESOURCE NAME>.openai.azure.com/",
32+
"deploy-name": "<YOUR DEPLOYMENT NAME>",
33+
"model": "gpt-4",
34+
"api-version": "<YOUR API VERSION>",
35+
"temperature": 0.7
36+
},
37+
"credential": {
38+
"openAI_key": "<YOUR API KEY>"
39+
},
40+
"actions": [
41+
{
42+
"action_type": "predict",
43+
"method": "POST",
44+
"url": "https://${parameters.endpoint}/openai/deployments/${parameters.deploy-name}/chat/completions?api-version=${parameters.api-version}",
45+
"headers": {
46+
"api-key": "${credential.openAI_key}"
47+
},
48+
"request_body": "{ \"messages\": ${parameters.messages}, \"temperature\": ${parameters.temperature} }"
49+
}
50+
]
51+
}
52+
```
53+
54+
### Sample response:
55+
```json
56+
{
57+
"connector_id": "EapnEY0BpYxvPx3Hxpmp"
58+
}
59+
```
60+
61+
## 3. Create model group:
62+
63+
```json
64+
POST /_plugins/_ml/model_groups/_register
65+
{
66+
"name": "remote_model_group",
67+
"description": "This is an example description"
68+
}
69+
```
70+
71+
### Sample response
72+
```json
73+
{
74+
"model_group_id": "YyvcbYsBjU568JRbdHqv",
75+
"status": "CREATED"
76+
}
77+
```
78+
79+
## 4. Register model to model group & deploy model:
80+
81+
```json
82+
POST /_plugins/_ml/models/_register
83+
{
84+
"name": "azure-openAI-gpt-4",
85+
"function_name": "remote",
86+
"model_group_id": "YyvcbYsBjU568JRbdHqv",
87+
"description": "Azure OpenAI GPT 4",
88+
"connector_id": "EapnEY0BpYxvPx3Hxpmp"
89+
}
90+
```
91+
92+
### Sample response
93+
```json
94+
{
95+
"task_id": "E6ppEY0BpYxvPx3HZZkL",
96+
"status": "CREATED",
97+
"model_id": "FKppEY0BpYxvPx3HZZk0"
98+
}
99+
```
100+
101+
Check if the task is completed
102+
```
103+
GET /_plugins/_ml/tasks/E6ppEY0BpYxvPx3HZZkL
104+
```
105+
106+
When model registration is completed, deploy it
107+
```
108+
POST /_plugins/_ml/models/FKppEY0BpYxvPx3HZZk0/_deploy
109+
```
110+
111+
## 5. Test model
112+
```json
113+
POST /_plugins/_ml/models/FKppEY0BpYxvPx3HZZk0/_predict
114+
{
115+
"parameters": {
116+
"messages": [
117+
{
118+
"role": "system",
119+
"content": "You are a helpful assistant."
120+
},
121+
{
122+
"role": "user",
123+
"content": "Hello!"
124+
}
125+
]
126+
}
127+
}
128+
```
129+
130+
### Sample response
131+
132+
```json
133+
{
134+
"inference_results": [
135+
{
136+
"output": [
137+
{
138+
"name": "response",
139+
"dataAsMap": {
140+
"id": "chatcmpl-8hZDlKJIaFLrC2sGTI2EYBiHQUhdI",
141+
"object": "chat.completion",
142+
"created": 1705394185,
143+
"model": "gpt-35-turbo",
144+
"prompt_filter_results": [
145+
{
146+
"prompt_index": 0,
147+
"content_filter_results": {
148+
"hate": {
149+
"filtered": false,
150+
"severity": "safe"
151+
},
152+
"self_harm": {
153+
"filtered": false,
154+
"severity": "safe"
155+
},
156+
"sexual": {
157+
"filtered": false,
158+
"severity": "safe"
159+
},
160+
"violence": {
161+
"filtered": false,
162+
"severity": "safe"
163+
}
164+
}
165+
}
166+
],
167+
"choices": [
168+
{
169+
"finish_reason": "stop",
170+
"index": 0,
171+
"message": {
172+
"role": "assistant",
173+
"content": "Hello! How can I assist you today?"
174+
},
175+
"content_filter_results": {
176+
"hate": {
177+
"filtered": false,
178+
"severity": "safe"
179+
},
180+
"self_harm": {
181+
"filtered": false,
182+
"severity": "safe"
183+
},
184+
"sexual": {
185+
"filtered": false,
186+
"severity": "safe"
187+
},
188+
"violence": {
189+
"filtered": false,
190+
"severity": "safe"
191+
}
192+
}
193+
}
194+
],
195+
"usage": {
196+
"prompt_tokens": 19,
197+
"completion_tokens": 9,
198+
"total_tokens": 28
199+
}
200+
}
201+
}
202+
],
203+
"status_code": 200
204+
}
205+
]
206+
}
207+
```
208+

‎docs/remote_inference_blueprints/azure_openai_connector_embedding_blueprint.md

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Azure OpenAI connector blueprint example for embedding model
2+
3+
## 1. Add Azure OpenAI endpoint to trusted URLs:
4+
5+
```json
6+
PUT /_cluster/settings
7+
{
8+
"persistent": {
9+
"plugins.ml_commons.trusted_connector_endpoints_regex": [
10+
"^https://.*\\.openai\\.azure\\.com/.*$"
11+
]
12+
}
13+
}
14+
```
15+
16+
## 2. Create connector for Azure OpenAI embedding model:
17+
18+
Refer to [Azure OpenAI Service REST API reference - Embedding](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#embeddings).
19+
20+
If you are using self-managed Opensearch, you should supply OpenAI API key:
21+
22+
23+
```json
24+
POST /_plugins/_ml/connectors/_create
25+
{
26+
"name": "<YOUR CONNECTOR NAME>",
27+
"description": "<YOUR CONNECTOR DESCRIPTION>",
28+
"version": "<YOUR CONNECTOR VERSION>",
29+
"protocol": "http",
30+
"parameters": {
31+
"endpoint": "<YOUR RESOURCE NAME>.openai.azure.com/",
32+
"deploy-name": "<YOUR DEPLOYMENT NAME>",
33+
"model": "text-embedding-ada-002",
34+
"api-version": "<YOUR API VERSION>"
35+
},
36+
"credential": {
37+
"openAI_key": "<YOUR API KEY>"
38+
},
39+
"actions": [
40+
{
41+
"action_type": "predict",
42+
"method": "POST",
43+
"url": "https://${parameters.endpoint}/openai/deployments/${parameters.deploy-name}/embeddings?api-version=${parameters.api-version}",
44+
"headers": {
45+
"api-key": "${credential.openAI_key}"
46+
},
47+
"request_body": "{ \"input\": ${parameters.input}}",
48+
"pre_process_function": "connector.pre_process.openai.embedding",
49+
"post_process_function": "connector.post_process.openai.embedding"
50+
}
51+
]
52+
}
53+
```
54+
55+
Sample response:
56+
```json
57+
{
58+
"connector_id": "OyB0josB2yd36FqHy3lO"
59+
}
60+
```
61+
62+
## 3. Create model group:
63+
64+
```json
65+
POST /_plugins/_ml/model_groups/_register
66+
{
67+
"name": "remote_model_group",
68+
"description": "This is an example description"
69+
}
70+
```
71+
72+
Sample response:
73+
```json
74+
{
75+
"model_group_id": "TWR0josByE8GuSOJ629m",
76+
"status": "CREATED"
77+
}
78+
```
79+
80+
## 4. Register model to model group & deploy model:
81+
82+
```json
83+
POST /_plugins/_ml/models/_register
84+
{
85+
"name": "OpenAI embedding model",
86+
"function_name": "remote",
87+
"model_group_id": "TWR0josByE8GuSOJ629m",
88+
"description": "test model",
89+
"connector_id": "OyB0josB2yd36FqHy3lO"
90+
}
91+
```
92+
93+
94+
Sample response:
95+
```json
96+
{
97+
"task_id": "PCB1josB2yd36FqHAXk9",
98+
"status": "CREATED"
99+
}
100+
```
101+
Get model id from task
102+
```json
103+
GET /_plugins/_ml/tasks/PCB1josB2yd36FqHAXk9
104+
```
105+
Deploy model, in this demo the model id is `PSB1josB2yd36FqHAnl1`
106+
```json
107+
POST /_plugins/_ml/models/PSB1josB2yd36FqHAnl1/_deploy
108+
```
109+
110+
## 5. Test model inference
111+
112+
```json
113+
POST /_plugins/_ml/models/PSB1josB2yd36FqHAnl1/_predict
114+
{
115+
"parameters": {
116+
"input": [ "What is the meaning of life?" ]
117+
}
118+
}
119+
```
120+
121+
Response:
122+
```json
123+
{
124+
"inference_results": [
125+
{
126+
"output": [
127+
{
128+
"name": "sentence_embedding",
129+
"data_type": "FLOAT32",
130+
"shape": [
131+
1536
132+
],
133+
"data": [
134+
-0.0043460787,
135+
-0.029653417,
136+
-0.008173223,
137+
...
138+
]
139+
}
140+
],
141+
"status_code": 200
142+
}
143+
]
144+
}
145+
```
146+

0 commit comments

Comments
 (0)