Skip to content

Commit a664ae0

Browse files
committed
Add streaming section to README
Signed-off-by: Mattt Zmuda <mattt@replicate.com>
1 parent cd5383d commit a664ae0

File tree

1 file changed

+28
-0
lines changed

1 file changed

+28
-0
lines changed

README.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,34 @@ Some models, like [methexis-inc/img2prompt](https://replicate.com/methexis-inc/i
7979
> print(results)
8080
> ```
8181
82+
## Run a model and stream its output
83+
84+
Replicate’s API supports server-sent event streams (SSEs) for language models.
85+
Use the `stream` method to consume tokens as they're produced by the model.
86+
87+
```python
88+
import replicate
89+
90+
# https://replicate.com/meta/llama-2-70b-chat
91+
model_version = "meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3"
92+
93+
tokens = []
94+
for event in replicate.stream(
95+
model_version,
96+
input={
97+
"prompt": "Please write a haiku about llamas.",
98+
},
99+
):
100+
print(event)
101+
tokens.append(str(event))
102+
103+
print("".join(tokens))
104+
```
105+
106+
For more information, see
107+
["Streaming output"](https://replicate.com/docs/streaming) in Replicate's docs.
108+
109+
82110
## Run a model in the background
83111
84112
You can start a model and run it in the background:

0 commit comments

Comments
 (0)