Skip to content

Commit 0b140fe

Browse files
feat(doc): add split_thinking docs (axolotl-ai-cloud#2613) [skip ci]
* feat(doc): add split_thinking docs * fix: link config.qmd to conversation.qmd for split_thinking example * update thinking => reasoning_content in messages format --------- Co-authored-by: Wing Lian <wing@axolotl.ai>
1 parent e4cfebe commit 0b140fe

File tree

2 files changed

+30
-2
lines changed

2 files changed

+30
-2
lines changed

docs/config.qmd

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -184,8 +184,8 @@ datasets:
184184
# adding a system turn with empty content.
185185
drop_system_message:
186186

187-
# Optional[bool]. Whether to split the assistant turn based on a reasoning trace inside delimited tags
188-
# defaults to False
187+
# Optional[bool]. (for Qwen3 template only) Whether to split the assistant content based on a reasoning trace inside delimited tags
188+
# See example at `docs/dataset-formats/conversation.qmd`
189189
split_thinking:
190190

191191
# IMPORTANT: The following fields determine which parts of the conversation to train on.

docs/dataset-formats/conversation.qmd

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,34 @@ datasets:
196196
It is not necessary to set both `message_field_training` and `message_field_training_detail` at once.
197197
:::
198198

199+
8. (For Qwen3 template only) Enable reasoning split, where the reasoning is split from the content and passed as a separate field into the template.
200+
201+
```yaml
202+
datasets:
203+
- path: ...
204+
type: chat_template
205+
chat_template: qwen3
206+
split_thinking: true
207+
```
208+
209+
For example, a content can look like:
210+
211+
```json
212+
{
213+
"content": "<think>Some thinking outputs</think>Output after thinking."
214+
}
215+
```
216+
217+
After split, it will look like:
218+
219+
```json
220+
{
221+
"reasoning_content": "Some thinking outputs",
222+
"content": "Output after thinking..."
223+
}
224+
```
225+
226+
199227
## sharegpt
200228

201229
::: {.callout-important}

0 commit comments

Comments
 (0)