File tree Expand file tree Collapse file tree 1 file changed +28
-0
lines changed Expand file tree Collapse file tree 1 file changed +28
-0
lines changed Original file line number Diff line number Diff line change @@ -79,6 +79,34 @@ Some models, like [methexis-inc/img2prompt](https://replicate.com/methexis-inc/i
79
79
> print (results)
80
80
> ```
81
81
82
+ # # Run a model and stream its output
83
+
84
+ Replicate’s API supports server- sent event streams (SSEs) for language models.
85
+ Use the `stream` method to consume tokens as they' re produced by the model.
86
+
87
+ ```python
88
+ import replicate
89
+
90
+ # https://replicate.com/meta/llama-2-70b-chat
91
+ model_version = " meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3"
92
+
93
+ tokens = []
94
+ for event in replicate.stream(
95
+ model_version,
96
+ input = {
97
+ " prompt" : " Please write a haiku about llamas." ,
98
+ },
99
+ ):
100
+ print (event)
101
+ tokens.append(str (event))
102
+
103
+ print (" " .join(tokens))
104
+ ```
105
+
106
+ For more information, see
107
+ [" Streaming output" ](https:// replicate.com/ docs/ streaming) in Replicate' s docs.
108
+
109
+
82
110
# # Run a model in the background
83
111
84
112
You can start a model and run it in the background:
You can’t perform that action at this time.
0 commit comments