Skip to content

Commit a890e1c

Browse files
reidliu41yeqcharlotte
authored andcommitted
[Misc] refactor example - openai_transcription_client (vllm-project#19851)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
1 parent 0a28b1b commit a890e1c

File tree

1 file changed

+34
-5
lines changed

1 file changed

+34
-5
lines changed

examples/online_serving/openai_transcription_client.py

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,23 @@
11
# SPDX-License-Identifier: Apache-2.0
22
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3+
"""
4+
This script demonstrates how to use the vLLM API server to perform audio
5+
transcription with the `openai/whisper-large-v3` model.
6+
7+
Before running this script, you must start the vLLM server with the following command:
8+
9+
vllm serve openai/whisper-large-v3
10+
11+
Requirements:
12+
- vLLM with audio support
13+
- openai Python SDK
14+
- httpx for streaming support
15+
16+
The script performs:
17+
1. Synchronous transcription using OpenAI-compatible API.
18+
2. Streaming transcription using raw HTTP request to the vLLM server.
19+
"""
20+
321
import asyncio
422
import json
523

@@ -21,6 +39,9 @@
2139

2240

2341
def sync_openai():
42+
"""
43+
Perform synchronous transcription using OpenAI-compatible API.
44+
"""
2445
with open(str(mary_had_lamb), "rb") as f:
2546
transcription = client.audio.transcriptions.create(
2647
file=f,
@@ -37,11 +58,11 @@ def sync_openai():
3758
print("transcription result:", transcription.text)
3859

3960

40-
sync_openai()
41-
42-
4361
# OpenAI Transcription API client does not support streaming.
4462
async def stream_openai_response():
63+
"""
64+
Perform streaming transcription using vLLM's raw HTTP streaming API.
65+
"""
4566
data = {
4667
"language": "en",
4768
"stream": True,
@@ -68,7 +89,15 @@ async def stream_openai_response():
6889
# Extract and print the content
6990
content = chunk["choices"][0].get("delta", {}).get("content")
7091
print(content, end="")
92+
print() # Final newline after stream ends
93+
94+
95+
def main():
96+
sync_openai()
97+
98+
# Run the asynchronous function
99+
asyncio.run(stream_openai_response())
71100

72101

73-
# Run the asynchronous function
74-
asyncio.run(stream_openai_response())
102+
if __name__ == "__main__":
103+
main()

0 commit comments

Comments
 (0)