From 0b140fef83462455a7b00ef18bdd62b26c61da95 Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Wed, 7 May 2025 07:05:32 +0700 Subject: [PATCH] feat(doc): add split_thinking docs (#2613) [skip ci] * feat(doc): add split_thinking docs * fix: link config.qmd to conversation.qmd for split_thinking example * update thinking => reasoning_content in messages format --------- Co-authored-by: Wing Lian --- docs/config.qmd | 4 ++-- docs/dataset-formats/conversation.qmd | 28 +++++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/config.qmd b/docs/config.qmd index 04d1320ca..18fc9dcf8 100644 --- a/docs/config.qmd +++ b/docs/config.qmd @@ -184,8 +184,8 @@ datasets: # adding a system turn with empty content. drop_system_message: - # Optional[bool]. Whether to split the assistant turn based on a reasoning trace inside delimited tags - # defaults to False + # Optional[bool]. (for Qwen3 template only) Whether to split the assistant content based on a reasoning trace inside delimited tags + # See example at `docs/dataset-formats/conversation.qmd` split_thinking: # IMPORTANT: The following fields determine which parts of the conversation to train on. diff --git a/docs/dataset-formats/conversation.qmd b/docs/dataset-formats/conversation.qmd index ee9f7391b..87c2941e6 100644 --- a/docs/dataset-formats/conversation.qmd +++ b/docs/dataset-formats/conversation.qmd @@ -196,6 +196,34 @@ datasets: It is not necessary to set both `message_field_training` and `message_field_training_detail` at once. ::: +8. (For Qwen3 template only) Enable reasoning split, where the reasoning is split from the content and passed as a separate field into the template. + +```yaml +datasets: + - path: ... + type: chat_template + chat_template: qwen3 + split_thinking: true +``` + +For example, a content can look like: + +```json +{ + "content": "Some thinking outputsOutput after thinking." +} +``` + +After split, it will look like: + +```json +{ + "reasoning_content": "Some thinking outputs", + "content": "Output after thinking..." +} +``` + + ## sharegpt ::: {.callout-important}