Commit 2f83350
Feat bedrock cohere support (#11230)
**Description:**
Added support for Cohere command model via Bedrock.
With this change it is now possible to use the `cohere.command-text-v14`
model via Bedrock API.
About Streaming: Cohere model outputs 2 additional chunks at the end of
the text being generated via streaming: a chunk containing the text
`<EOS_TOKEN>`, and a chunk indicating the end of the stream. In this
implementation I chose to ignore both chunks. An alternative solution
could be to replace `<EOS_TOKEN>` with `\n`
Tests: manually tested that the new model work with both
`llm.generate()` and `llm.stream()`.
Tested with `temperature`, `p` and `stop` parameters.
**Issue:** #11181
**Dependencies:** No new dependencies
**Tag maintainer:** @baskaryan
**Twitter handle:** mangelino
---------
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>1 parent 37f2f71 commit 2f83350
1 file changed
+15
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
68 | 69 | | |
69 | 70 | | |
70 | 71 | | |
| |||
74 | 75 | | |
75 | 76 | | |
76 | 77 | | |
77 | | - | |
| 78 | + | |
78 | 79 | | |
79 | 80 | | |
80 | 81 | | |
| |||
98 | 99 | | |
99 | 100 | | |
100 | 101 | | |
| 102 | + | |
| 103 | + | |
101 | 104 | | |
102 | 105 | | |
103 | 106 | | |
| |||
119 | 122 | | |
120 | 123 | | |
121 | 124 | | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
122 | 131 | | |
123 | 132 | | |
124 | 133 | | |
| |||
159 | 168 | | |
160 | 169 | | |
161 | 170 | | |
| 171 | + | |
162 | 172 | | |
163 | 173 | | |
164 | 174 | | |
| |||
259 | 269 | | |
260 | 270 | | |
261 | 271 | | |
262 | | - | |
263 | | - | |
264 | | - | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
265 | 276 | | |
266 | 277 | | |
267 | 278 | | |
| |||
0 commit comments