From a2ba36ec6399d18b1adf7f75541b9b6111e89a94 Mon Sep 17 00:00:00 2001 From: Quarto GHA Workflow Runner Date: Tue, 23 Sep 2025 14:28:06 +0000 Subject: [PATCH] Built site for gh-pages --- .nojekyll | 2 +- ...core.datasets.transforms.chat_builder.html | 18 +- docs/multimodal.html | 21 +- search.json | 6 +- sitemap.xml | 396 +++++++++--------- 5 files changed, 221 insertions(+), 222 deletions(-) diff --git a/.nojekyll b/.nojekyll index 0843099f8..d56ea6452 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -394adab2 \ No newline at end of file +cf30b694 \ No newline at end of file diff --git a/docs/api/core.datasets.transforms.chat_builder.html b/docs/api/core.datasets.transforms.chat_builder.html index cc5d0977a..b4d7b6f3a 100644 --- a/docs/api/core.datasets.transforms.chat_builder.html +++ b/docs/api/core.datasets.transforms.chat_builder.html @@ -544,7 +544,7 @@ dataset and converts it to a Chat.

chat_message_transform_builder

core.datasets.transforms.chat_builder.chat_message_transform_builder(
     train_on_inputs=False,
-    conversations_field='conversations',
+    conversations_field='messages',
     message_field_role=None,
     message_field_content=None,
     message_field_training=None,
@@ -555,9 +555,9 @@ dataset and converts it to a Chat.

---+++ @@ -577,25 +577,25 @@ dataset and converts it to a Chat.

- - + + - + - + - + diff --git a/docs/multimodal.html b/docs/multimodal.html index 67051594b..8ccc7b6f9 100644 --- a/docs/multimodal.html +++ b/docs/multimodal.html @@ -588,17 +588,16 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true}); - path: HuggingFaceH4/llava-instruct-mix-vsft type: chat_template split: train[:1%] - field_messages: messages - -# (optional) if doing lora, only finetune the Language model, -# leave the vision model and vision tower frozen -# load_in_8bit: true -adapter: lora -lora_target_modules: 'model.language_model.layers.[\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj' - -# (optional) if you want to resize images to a set size -image_size: 512 -image_resize_algorithm: bilinear + +# (optional) if doing lora, only finetune the Language model, +# leave the vision model and vision tower frozen +# load_in_8bit: true +adapter: lora +lora_target_modules: 'model.language_model.layers.[\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj' + +# (optional) if you want to resize images to a set size +image_size: 512 +image_resize_algorithm: bilinear

Please see examples folder for full configs.

diff --git a/search.json b/search.json index 470ccb577..c9f2174f6 100644 --- a/search.json +++ b/search.json @@ -494,7 +494,7 @@ "href": "docs/multimodal.html#usage", "title": "MultiModal / Vision Language Models (BETA)", "section": "Usage", - "text": "Usage\nMultimodal support is limited and doesn’t have full feature parity.\nHere are the hyperparams you’ll need to use to finetune a multimodal model.\nprocessor_type: AutoProcessor\n\nskip_prepare_dataset: true\nremove_unused_columns: false # leave columns in place as they are needed to handle image embeddings during training\nsample_packing: false # not yet supported with multimodal\n\nchat_template: # see in next section if specified\n\n# example dataset\ndatasets:\n - path: HuggingFaceH4/llava-instruct-mix-vsft\n type: chat_template\n split: train[:1%]\n field_messages: messages\n\n# (optional) if doing lora, only finetune the Language model,\n# leave the vision model and vision tower frozen\n# load_in_8bit: true\nadapter: lora\nlora_target_modules: 'model.language_model.layers.[\\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj'\n\n# (optional) if you want to resize images to a set size\nimage_size: 512\nimage_resize_algorithm: bilinear\nPlease see examples folder for full configs.\n\n\n\n\n\n\nWarning\n\n\n\nSome of our chat_templates have been extended to support broader dataset types. This should not break any existing configs.\n\n\n\nMllama\nbase_model: meta-llama/Llama-3.2-11B-Vision-Instruct\n\nchat_template: llama3_2_vision\n\n\nLlama4\nbase_model: meta-llama/Llama-4-Scout-17B-16E-Instruct\n\nchat_template: llama4\n\n\nPixtral\nbase_model: mistralai/Pixtral-12B-2409\n\nchat_template: pixtral\n\n\nLlava-1.5\nbase_model: llava-hf/llava-1.5-7b-hf\n\nchat_template: llava\n\n\nMistral-Small-3.1\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install vision lib via pip install 'mistral-common[opencv]==1.8.5'\n\n\nbase_model: mistralai/Mistral-Small-3.1-24B-Instruct-2503\n\n\nMagistral-Small-2509\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install vision lib via pip install 'mistral-common[opencv]==1.8.5'\n\n\nbase_model: mistralai/Magistral-Small-2509\n\n\nVoxtral\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install audio lib via pip3 install librosa==0.11.0 'mistral_common[audio]==1.8.3'\n\n\nbase_model: mistralai/Voxtral-Mini-3B-2507\n\n\nGemma-3\n\n\n\n\n\n\nTip\n\n\n\nThe Gemma3-1B model is a text-only model, so please train as regular text model.\n\n\nFor multi-modal 4B/12B/27B models, use the following config:\nbase_model: google/gemma-3-4b-it\n\nchat_template: gemma3\n\n\nGemma-3n\n\n\n\n\n\n\nWarning\n\n\n\nThe model’s initial loss and grad norm will be very high. We suspect this to be due to the Conv in the vision layers.\n\n\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install timm via pip3 install timm==1.0.17\n\n\nbase_model: google/gemma-3n-E2B-it\n\nchat_template: gemma3n\n\n\nQwen2-VL\nbase_model: Qwen/Qwen2-VL-7B-Instruct\n\nchat_template: qwen2_vl\n\n\nQwen2.5-VL\nbase_model: Qwen/Qwen2.5-VL-7B-Instruct\n\nchat_template: qwen2_vl # same as qwen2-vl\n\n\nSmolVLM2\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install num2words via pip3 install num2words==0.5.14\n\n\nbase_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct\n\n\nLFM2-VL\n\n\n\n\n\n\nWarning\n\n\n\nPlease uninstall causal-conv1d via pip3 uninstall -y causal-conv1d\n\n\nbase_model: LiquidAI/LFM2-VL-450M", + "text": "Usage\nMultimodal support is limited and doesn’t have full feature parity.\nHere are the hyperparams you’ll need to use to finetune a multimodal model.\nprocessor_type: AutoProcessor\n\nskip_prepare_dataset: true\nremove_unused_columns: false # leave columns in place as they are needed to handle image embeddings during training\nsample_packing: false # not yet supported with multimodal\n\nchat_template: # see in next section if specified\n\n# example dataset\ndatasets:\n - path: HuggingFaceH4/llava-instruct-mix-vsft\n type: chat_template\n split: train[:1%]\n\n# (optional) if doing lora, only finetune the Language model,\n# leave the vision model and vision tower frozen\n# load_in_8bit: true\nadapter: lora\nlora_target_modules: 'model.language_model.layers.[\\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj'\n\n# (optional) if you want to resize images to a set size\nimage_size: 512\nimage_resize_algorithm: bilinear\nPlease see examples folder for full configs.\n\n\n\n\n\n\nWarning\n\n\n\nSome of our chat_templates have been extended to support broader dataset types. This should not break any existing configs.\n\n\n\nMllama\nbase_model: meta-llama/Llama-3.2-11B-Vision-Instruct\n\nchat_template: llama3_2_vision\n\n\nLlama4\nbase_model: meta-llama/Llama-4-Scout-17B-16E-Instruct\n\nchat_template: llama4\n\n\nPixtral\nbase_model: mistralai/Pixtral-12B-2409\n\nchat_template: pixtral\n\n\nLlava-1.5\nbase_model: llava-hf/llava-1.5-7b-hf\n\nchat_template: llava\n\n\nMistral-Small-3.1\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install vision lib via pip install 'mistral-common[opencv]==1.8.5'\n\n\nbase_model: mistralai/Mistral-Small-3.1-24B-Instruct-2503\n\n\nMagistral-Small-2509\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install vision lib via pip install 'mistral-common[opencv]==1.8.5'\n\n\nbase_model: mistralai/Magistral-Small-2509\n\n\nVoxtral\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install audio lib via pip3 install librosa==0.11.0 'mistral_common[audio]==1.8.3'\n\n\nbase_model: mistralai/Voxtral-Mini-3B-2507\n\n\nGemma-3\n\n\n\n\n\n\nTip\n\n\n\nThe Gemma3-1B model is a text-only model, so please train as regular text model.\n\n\nFor multi-modal 4B/12B/27B models, use the following config:\nbase_model: google/gemma-3-4b-it\n\nchat_template: gemma3\n\n\nGemma-3n\n\n\n\n\n\n\nWarning\n\n\n\nThe model’s initial loss and grad norm will be very high. We suspect this to be due to the Conv in the vision layers.\n\n\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install timm via pip3 install timm==1.0.17\n\n\nbase_model: google/gemma-3n-E2B-it\n\nchat_template: gemma3n\n\n\nQwen2-VL\nbase_model: Qwen/Qwen2-VL-7B-Instruct\n\nchat_template: qwen2_vl\n\n\nQwen2.5-VL\nbase_model: Qwen/Qwen2.5-VL-7B-Instruct\n\nchat_template: qwen2_vl # same as qwen2-vl\n\n\nSmolVLM2\n\n\n\n\n\n\nTip\n\n\n\nPlease make sure to install num2words via pip3 install num2words==0.5.14\n\n\nbase_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct\n\n\nLFM2-VL\n\n\n\n\n\n\nWarning\n\n\n\nPlease uninstall causal-conv1d via pip3 uninstall -y causal-conv1d\n\n\nbase_model: LiquidAI/LFM2-VL-450M", "crumbs": [ "How To Guides", "MultiModal / Vision Language Models (BETA)" @@ -3368,14 +3368,14 @@ "href": "docs/api/core.datasets.transforms.chat_builder.html", "title": "core.datasets.transforms.chat_builder", "section": "", - "text": "core.datasets.transforms.chat_builder\nThis module contains a function that builds a transform that takes a row from the\ndataset and converts it to a Chat.\n\n\n\n\n\nName\nDescription\n\n\n\n\nchat_message_transform_builder\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\ncore.datasets.transforms.chat_builder.chat_message_transform_builder(\n train_on_inputs=False,\n conversations_field='conversations',\n message_field_role=None,\n message_field_content=None,\n message_field_training=None,\n)\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntrain_on_inputs\nbool\nIf True, the transform will train on the inputs. If False, the transform will train on the targets. Defaults to False.\nFalse\n\n\nconversations_field\nstr\nThe field name of the conversations. Defaults to “conversations”.\n'conversations'\n\n\nmessage_field_role\nstr | list[str]\nThe field name of the role. Defaults to “role”.\nNone\n\n\nmessage_field_content\nstr | list[str]\nThe field name of the message content. Defaults to “content”.\nNone\n\n\nmessage_field_training\nstr | list[str]\nThe field name of the train/weight. Defaults to “weight”.\nNone\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\nCallable\n\nA function that takes a list of conversations and returns a list of messages." + "text": "core.datasets.transforms.chat_builder\nThis module contains a function that builds a transform that takes a row from the\ndataset and converts it to a Chat.\n\n\n\n\n\nName\nDescription\n\n\n\n\nchat_message_transform_builder\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\ncore.datasets.transforms.chat_builder.chat_message_transform_builder(\n train_on_inputs=False,\n conversations_field='messages',\n message_field_role=None,\n message_field_content=None,\n message_field_training=None,\n)\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntrain_on_inputs\nbool\nIf True, the transform will train on the inputs. If False, the transform will train on the targets. Defaults to False.\nFalse\n\n\nconversations_field\nstr\nThe field name of the conversations. Defaults to “messages”.\n'messages'\n\n\nmessage_field_role\nstr | list[str]\nThe field name of the role.\nNone\n\n\nmessage_field_content\nstr | list[str]\nThe field name of the message content.\nNone\n\n\nmessage_field_training\nstr | list[str]\nThe field name of the train/weight.\nNone\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\nCallable\n\nA function that takes a list of conversations and returns a list of messages." }, { "objectID": "docs/api/core.datasets.transforms.chat_builder.html#functions", "href": "docs/api/core.datasets.transforms.chat_builder.html#functions", "title": "core.datasets.transforms.chat_builder", "section": "", - "text": "Name\nDescription\n\n\n\n\nchat_message_transform_builder\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\ncore.datasets.transforms.chat_builder.chat_message_transform_builder(\n train_on_inputs=False,\n conversations_field='conversations',\n message_field_role=None,\n message_field_content=None,\n message_field_training=None,\n)\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntrain_on_inputs\nbool\nIf True, the transform will train on the inputs. If False, the transform will train on the targets. Defaults to False.\nFalse\n\n\nconversations_field\nstr\nThe field name of the conversations. Defaults to “conversations”.\n'conversations'\n\n\nmessage_field_role\nstr | list[str]\nThe field name of the role. Defaults to “role”.\nNone\n\n\nmessage_field_content\nstr | list[str]\nThe field name of the message content. Defaults to “content”.\nNone\n\n\nmessage_field_training\nstr | list[str]\nThe field name of the train/weight. Defaults to “weight”.\nNone\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\nCallable\n\nA function that takes a list of conversations and returns a list of messages." + "text": "Name\nDescription\n\n\n\n\nchat_message_transform_builder\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\ncore.datasets.transforms.chat_builder.chat_message_transform_builder(\n train_on_inputs=False,\n conversations_field='messages',\n message_field_role=None,\n message_field_content=None,\n message_field_training=None,\n)\nBuilds a transform that takes a row from the dataset and converts it to a Chat\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntrain_on_inputs\nbool\nIf True, the transform will train on the inputs. If False, the transform will train on the targets. Defaults to False.\nFalse\n\n\nconversations_field\nstr\nThe field name of the conversations. Defaults to “messages”.\n'messages'\n\n\nmessage_field_role\nstr | list[str]\nThe field name of the role.\nNone\n\n\nmessage_field_content\nstr | list[str]\nThe field name of the message content.\nNone\n\n\nmessage_field_training\nstr | list[str]\nThe field name of the train/weight.\nNone\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\nCallable\n\nA function that takes a list of conversations and returns a list of messages." }, { "objectID": "docs/api/utils.trainer.html", diff --git a/sitemap.xml b/sitemap.xml index b01ea6639..493bc22d4 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,794 +2,794 @@ https://docs.axolotl.ai/examples/colab-notebooks/colab-axolotl-example.html - 2025-09-23T04:31:24.847Z + 2025-09-23T14:22:24.588Z https://docs.axolotl.ai/src/axolotl/integrations/LICENSE.html - 2025-09-23T04:31:24.864Z + 2025-09-23T14:22:24.605Z https://docs.axolotl.ai/FAQS.html - 2025-09-23T04:31:24.837Z + 2025-09-23T14:22:24.578Z https://docs.axolotl.ai/docs/unsloth.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/reward_modelling.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/docker.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/batch_vs_grad.html - 2025-09-23T04:31:24.838Z + 2025-09-23T14:22:24.579Z https://docs.axolotl.ai/docs/streaming.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/nccl.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/quantize.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/nd_parallelism.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/custom_integrations.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.579Z https://docs.axolotl.ai/docs/debugging.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/gradient_checkpointing.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/multimodal.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/dataset_loading.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/api/integrations.spectrum.args.html - 2025-09-23T04:34:52.953Z + 2025-09-23T14:25:51.436Z https://docs.axolotl.ai/docs/api/loaders.adapter.html - 2025-09-23T04:34:52.101Z + 2025-09-23T14:25:50.584Z https://docs.axolotl.ai/docs/api/core.builders.base.html - 2025-09-23T04:34:51.738Z + 2025-09-23T14:25:50.217Z https://docs.axolotl.ai/docs/api/cli.merge_sharded_fsdp_weights.html - 2025-09-23T04:34:51.938Z + 2025-09-23T14:25:50.421Z https://docs.axolotl.ai/docs/api/cli.cloud.modal_.html - 2025-09-23T04:34:51.967Z + 2025-09-23T14:25:50.451Z https://docs.axolotl.ai/docs/api/monkeypatch.stablelm_attn_hijack_flash.html - 2025-09-23T04:34:52.518Z + 2025-09-23T14:25:51.002Z https://docs.axolotl.ai/docs/api/integrations.kd.trainer.html - 2025-09-23T04:34:52.943Z + 2025-09-23T14:25:51.426Z https://docs.axolotl.ai/docs/api/cli.delinearize_llama4.html - 2025-09-23T04:34:51.903Z + 2025-09-23T14:25:50.387Z https://docs.axolotl.ai/docs/api/cli.evaluate.html - 2025-09-23T04:34:51.851Z + 2025-09-23T14:25:50.335Z https://docs.axolotl.ai/docs/api/utils.schemas.model.html - 2025-09-23T04:34:52.725Z + 2025-09-23T14:25:51.209Z https://docs.axolotl.ai/docs/api/cli.art.html - 2025-09-23T04:34:51.874Z + 2025-09-23T14:25:50.358Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.llama3.html - 2025-09-23T04:34:52.284Z + 2025-09-23T14:25:50.767Z https://docs.axolotl.ai/docs/api/core.chat.format.llama3x.html - 2025-09-23T04:34:51.786Z + 2025-09-23T14:25:50.268Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.chatml.html - 2025-09-23T04:34:52.315Z + 2025-09-23T14:25:50.798Z https://docs.axolotl.ai/docs/api/cli.args.html - 2025-09-23T04:34:51.871Z + 2025-09-23T14:25:50.354Z https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_cpu.html - 2025-09-23T04:34:52.543Z + 2025-09-23T14:25:51.027Z https://docs.axolotl.ai/docs/api/evaluate.html - 2025-09-23T04:34:51.661Z + 2025-09-23T14:25:50.140Z https://docs.axolotl.ai/docs/api/prompt_strategies.input_output.html - 2025-09-23T04:34:52.246Z + 2025-09-23T14:25:50.730Z https://docs.axolotl.ai/docs/api/cli.vllm_serve.html - 2025-09-23T04:34:51.958Z + 2025-09-23T14:25:50.441Z https://docs.axolotl.ai/docs/api/cli.quantize.html - 2025-09-23T04:34:51.951Z + 2025-09-23T14:25:50.434Z https://docs.axolotl.ai/docs/api/cli.utils.sweeps.html - 2025-09-23T04:34:51.998Z + 2025-09-23T14:25:50.480Z https://docs.axolotl.ai/docs/api/core.builders.rl.html - 2025-09-23T04:34:51.748Z + 2025-09-23T14:25:50.228Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chatml.html - 2025-09-23T04:34:52.294Z + 2025-09-23T14:25:50.778Z https://docs.axolotl.ai/docs/api/prompt_strategies.metharme.html - 2025-09-23T04:34:52.257Z + 2025-09-23T14:25:50.741Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.passthrough.html - 2025-09-23T04:34:52.298Z + 2025-09-23T14:25:50.782Z https://docs.axolotl.ai/docs/api/core.datasets.chat.html - 2025-09-23T04:34:51.793Z + 2025-09-23T14:25:50.275Z https://docs.axolotl.ai/docs/api/utils.distributed.html - 2025-09-23T04:34:52.664Z + 2025-09-23T14:25:51.148Z https://docs.axolotl.ai/docs/api/integrations.base.html - 2025-09-23T04:34:52.931Z + 2025-09-23T14:25:51.414Z https://docs.axolotl.ai/docs/api/index.html - 2025-09-23T04:34:51.593Z + 2025-09-23T14:25:50.069Z https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_disk.html - 2025-09-23T04:34:52.569Z + 2025-09-23T14:25:51.053Z https://docs.axolotl.ai/docs/api/utils.quantization.html - 2025-09-23T04:34:52.704Z + 2025-09-23T14:25:51.188Z https://docs.axolotl.ai/docs/api/utils.dict.html - 2025-09-23T04:34:52.669Z + 2025-09-23T14:25:51.153Z https://docs.axolotl.ai/docs/api/core.trainers.dpo.trainer.html - 2025-09-23T04:34:52.051Z + 2025-09-23T14:25:50.534Z https://docs.axolotl.ai/docs/api/utils.schemas.config.html - 2025-09-23T04:34:52.718Z + 2025-09-23T14:25:51.202Z https://docs.axolotl.ai/docs/api/prompt_strategies.orcamini.html - 2025-09-23T04:34:52.261Z + 2025-09-23T14:25:50.745Z https://docs.axolotl.ai/docs/api/utils.collators.mamba.html - 2025-09-23T04:34:52.996Z + 2025-09-23T14:25:51.479Z https://docs.axolotl.ai/docs/api/datasets.html - 2025-09-23T04:34:51.668Z + 2025-09-23T14:25:50.146Z https://docs.axolotl.ai/docs/api/convert.html - 2025-09-23T04:34:51.681Z + 2025-09-23T14:25:50.159Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.llama3.html - 2025-09-23T04:34:52.307Z + 2025-09-23T14:25:50.790Z https://docs.axolotl.ai/docs/api/utils.tokenization.html - 2025-09-23T04:34:52.576Z + 2025-09-23T14:25:51.060Z https://docs.axolotl.ai/docs/api/core.trainers.base.html - 2025-09-23T04:34:52.024Z + 2025-09-23T14:25:50.507Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_expand_mask.html - 2025-09-23T04:34:52.473Z + 2025-09-23T14:25:50.957Z https://docs.axolotl.ai/docs/api/loaders.model.html - 2025-09-23T04:34:52.086Z + 2025-09-23T14:25:50.569Z https://docs.axolotl.ai/docs/api/logging_config.html - 2025-09-23T04:34:51.732Z + 2025-09-23T14:25:50.211Z https://docs.axolotl.ai/docs/api/core.chat.format.shared.html - 2025-09-23T04:34:51.788Z + 2025-09-23T14:25:50.270Z https://docs.axolotl.ai/docs/api/models.mamba.modeling_mamba.html - 2025-09-23T04:34:52.972Z + 2025-09-23T14:25:51.455Z https://docs.axolotl.ai/docs/api/utils.schemas.enums.html - 2025-09-23T04:34:52.788Z + 2025-09-23T14:25:51.271Z https://docs.axolotl.ai/docs/api/utils.callbacks.profiler.html - 2025-09-23T04:34:53.052Z + 2025-09-23T14:25:51.534Z https://docs.axolotl.ai/docs/api/utils.bench.html - 2025-09-23T04:34:52.592Z + 2025-09-23T14:25:51.075Z https://docs.axolotl.ai/docs/api/utils.samplers.multipack.html - 2025-09-23T04:34:53.042Z + 2025-09-23T14:25:51.524Z https://docs.axolotl.ai/docs/api/core.trainers.mamba.html - 2025-09-23T04:34:52.044Z + 2025-09-23T14:25:50.527Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_chat.html - 2025-09-23T04:34:52.200Z + 2025-09-23T14:25:50.684Z https://docs.axolotl.ai/docs/api/monkeypatch.btlm_attn_hijack_flash.html - 2025-09-23T04:34:52.511Z + 2025-09-23T14:25:50.995Z https://docs.axolotl.ai/docs/api/core.chat.messages.html - 2025-09-23T04:34:51.783Z + 2025-09-23T14:25:50.265Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_patch_multipack.html - 2025-09-23T04:34:52.513Z + 2025-09-23T14:25:50.996Z https://docs.axolotl.ai/docs/api/monkeypatch.mixtral.html - 2025-09-23T04:34:52.540Z + 2025-09-23T14:25:51.024Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.scheduler.html - 2025-09-23T04:34:52.128Z + 2025-09-23T14:25:50.611Z https://docs.axolotl.ai/docs/api/loaders.patch_manager.html - 2025-09-23T04:34:52.111Z + 2025-09-23T14:25:50.594Z https://docs.axolotl.ai/docs/api/utils.optimizers.adopt.html - 2025-09-23T04:34:52.677Z + 2025-09-23T14:25:51.161Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_xformers.html - 2025-09-23T04:34:52.465Z + 2025-09-23T14:25:50.949Z https://docs.axolotl.ai/docs/api/utils.callbacks.perplexity.html - 2025-09-23T04:34:53.048Z + 2025-09-23T14:25:51.531Z https://docs.axolotl.ai/docs/api/loaders.tokenizer.html - 2025-09-23T04:34:52.094Z + 2025-09-23T14:25:50.577Z https://docs.axolotl.ai/docs/api/prompt_strategies.pygmalion.html - 2025-09-23T04:34:52.268Z + 2025-09-23T14:25:50.751Z https://docs.axolotl.ai/docs/api/utils.lora.html - 2025-09-23T04:34:52.583Z + 2025-09-23T14:25:51.066Z https://docs.axolotl.ai/docs/api/prompt_strategies.user_defined.html - 2025-09-23T04:34:52.221Z + 2025-09-23T14:25:50.705Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.rng_state_loader.html - 2025-09-23T04:34:52.121Z + 2025-09-23T14:25:50.604Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.user_defined.html - 2025-09-23T04:34:52.297Z + 2025-09-23T14:25:50.781Z https://docs.axolotl.ai/docs/api/kernels.swiglu.html - 2025-09-23T04:34:52.449Z + 2025-09-23T14:25:50.933Z https://docs.axolotl.ai/docs/api/utils.schemas.multimodal.html - 2025-09-23T04:34:52.767Z + 2025-09-23T14:25:51.251Z https://docs.axolotl.ai/docs/api/cli.main.html - 2025-09-23T04:34:51.835Z + 2025-09-23T14:25:50.318Z https://docs.axolotl.ai/docs/api/utils.ctx_managers.sequence_parallel.html - 2025-09-23T04:34:52.152Z + 2025-09-23T14:25:50.635Z https://docs.axolotl.ai/docs/api/monkeypatch.multipack.html - 2025-09-23T04:34:52.468Z + 2025-09-23T14:25:50.952Z https://docs.axolotl.ai/docs/api/core.chat.format.chatml.html - 2025-09-23T04:34:51.785Z + 2025-09-23T14:25:50.267Z https://docs.axolotl.ai/docs/api/cli.cloud.base.html - 2025-09-23T04:34:51.961Z + 2025-09-23T14:25:50.444Z https://docs.axolotl.ai/docs/api/core.trainers.trl.html - 2025-09-23T04:34:52.039Z + 2025-09-23T14:25:50.522Z https://docs.axolotl.ai/docs/mixed_precision.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/installation.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/mac.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/sequence_parallelism.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/faq.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/tokenized.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/stepwise_supervised.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/pretraining.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/template_free.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/index.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/conversation.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/dataset-formats/inst_tune.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/lr_groups.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/inference.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/lora_optims.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/multipack.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/amd_hpc.html - 2025-09-23T04:31:24.838Z + 2025-09-23T14:22:24.579Z https://docs.axolotl.ai/docs/api/monkeypatch.trainer_fsdp_optim.html - 2025-09-23T04:34:52.521Z + 2025-09-23T14:25:51.005Z https://docs.axolotl.ai/docs/api/cli.inference.html - 2025-09-23T04:34:51.917Z + 2025-09-23T14:25:50.401Z https://docs.axolotl.ai/docs/api/utils.collators.mm_chat.html - 2025-09-23T04:34:53.001Z + 2025-09-23T14:25:51.483Z https://docs.axolotl.ai/docs/api/utils.model_shard_quant.html - 2025-09-23T04:34:52.588Z + 2025-09-23T14:25:51.072Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_flash.html - 2025-09-23T04:34:52.463Z + 2025-09-23T14:25:50.947Z https://docs.axolotl.ai/docs/api/integrations.grokfast.optimizer.html - 2025-09-23T04:34:52.935Z + 2025-09-23T14:25:51.418Z https://docs.axolotl.ai/docs/api/utils.schemas.training.html - 2025-09-23T04:34:52.732Z + 2025-09-23T14:25:51.216Z https://docs.axolotl.ai/docs/api/utils.schemas.integrations.html - 2025-09-23T04:34:52.779Z + 2025-09-23T14:25:51.263Z https://docs.axolotl.ai/docs/api/utils.schemas.utils.html - 2025-09-23T04:34:52.794Z + 2025-09-23T14:25:51.277Z https://docs.axolotl.ai/docs/api/cli.checks.html - 2025-09-23T04:34:51.880Z + 2025-09-23T14:25:50.364Z https://docs.axolotl.ai/docs/api/utils.callbacks.qat.html - 2025-09-23T04:34:53.068Z + 2025-09-23T14:25:51.550Z https://docs.axolotl.ai/docs/api/monkeypatch.mistral_attn_hijack_flash.html - 2025-09-23T04:34:52.466Z + 2025-09-23T14:25:50.950Z https://docs.axolotl.ai/docs/api/utils.data.sft.html - 2025-09-23T04:34:52.685Z + 2025-09-23T14:25:51.168Z https://docs.axolotl.ai/docs/api/utils.chat_templates.html - 2025-09-23T04:34:52.578Z + 2025-09-23T14:25:51.061Z https://docs.axolotl.ai/docs/api/monkeypatch.utils.html - 2025-09-23T04:34:52.510Z + 2025-09-23T14:25:50.993Z https://docs.axolotl.ai/docs/api/cli.utils.fetch.html - 2025-09-23T04:34:51.986Z + 2025-09-23T14:25:50.469Z https://docs.axolotl.ai/docs/api/core.builders.causal.html - 2025-09-23T04:34:51.743Z + 2025-09-23T14:25:50.223Z https://docs.axolotl.ai/docs/api/utils.schemas.trl.html - 2025-09-23T04:34:52.762Z + 2025-09-23T14:25:51.246Z https://docs.axolotl.ai/docs/api/utils.data.streaming.html - 2025-09-23T04:34:52.678Z + 2025-09-23T14:25:51.162Z https://docs.axolotl.ai/docs/api/monkeypatch.unsloth_.html - 2025-09-23T04:34:52.529Z + 2025-09-23T14:25:51.013Z https://docs.axolotl.ai/docs/api/prompt_strategies.base.html - 2025-09-23T04:34:52.153Z + 2025-09-23T14:25:50.637Z https://docs.axolotl.ai/docs/api/cli.merge_lora.html - 2025-09-23T04:34:51.926Z + 2025-09-23T14:25:50.409Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_instruct.html - 2025-09-23T04:34:52.201Z + 2025-09-23T14:25:50.685Z https://docs.axolotl.ai/docs/api/loaders.constants.html - 2025-09-23T04:34:52.112Z + 2025-09-23T14:25:50.595Z https://docs.axolotl.ai/docs/api/monkeypatch.transformers_fa_utils.html - 2025-09-23T04:34:52.528Z + 2025-09-23T14:25:51.012Z https://docs.axolotl.ai/docs/api/monkeypatch.relora.html - 2025-09-23T04:34:52.471Z + 2025-09-23T14:25:50.955Z https://docs.axolotl.ai/docs/api/utils.schedulers.html - 2025-09-23T04:34:52.644Z + 2025-09-23T14:25:51.128Z https://docs.axolotl.ai/docs/api/kernels.lora.html - 2025-09-23T04:34:52.428Z + 2025-09-23T14:25:50.912Z https://docs.axolotl.ai/docs/api/utils.collators.batching.html - 2025-09-23T04:34:52.992Z + 2025-09-23T14:25:51.475Z https://docs.axolotl.ai/docs/api/prompt_strategies.completion.html - 2025-09-23T04:34:52.240Z + 2025-09-23T14:25:50.724Z https://docs.axolotl.ai/docs/api/core.trainers.grpo.trainer.html - 2025-09-23T04:34:52.062Z + 2025-09-23T14:25:50.545Z https://docs.axolotl.ai/docs/api/common.architectures.html - 2025-09-23T04:34:52.954Z + 2025-09-23T14:25:51.437Z https://docs.axolotl.ai/docs/api/cli.utils.html - 2025-09-23T04:34:51.969Z + 2025-09-23T14:25:50.452Z https://docs.axolotl.ai/docs/api/prompt_strategies.orpo.chat_template.html - 2025-09-23T04:34:52.337Z + 2025-09-23T14:25:50.820Z https://docs.axolotl.ai/docs/api/core.training_args.html - 2025-09-23T04:34:51.760Z + 2025-09-23T14:25:50.241Z https://docs.axolotl.ai/docs/api/utils.schemas.datasets.html - 2025-09-23T04:34:52.750Z + 2025-09-23T14:25:51.234Z https://docs.axolotl.ai/docs/api/prompt_strategies.chat_template.html - 2025-09-23T04:34:52.186Z + 2025-09-23T14:25:50.670Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_w_system.html - 2025-09-23T04:34:52.213Z + 2025-09-23T14:25:50.697Z https://docs.axolotl.ai/docs/api/integrations.cut_cross_entropy.args.html - 2025-09-23T04:34:52.934Z + 2025-09-23T14:25:51.417Z https://docs.axolotl.ai/docs/api/common.const.html - 2025-09-23T04:34:52.956Z + 2025-09-23T14:25:51.439Z https://docs.axolotl.ai/docs/api/monkeypatch.lora_kernels.html - 2025-09-23T04:34:52.502Z + 2025-09-23T14:25:50.985Z https://docs.axolotl.ai/docs/api/kernels.quantize.html - 2025-09-23T04:34:52.456Z + 2025-09-23T14:25:50.940Z https://docs.axolotl.ai/docs/api/cli.config.html - 2025-09-23T04:34:51.898Z + 2025-09-23T14:25:50.382Z https://docs.axolotl.ai/docs/api/prompt_strategies.llama2_chat.html - 2025-09-23T04:34:52.234Z + 2025-09-23T14:25:50.718Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.zephyr.html - 2025-09-23T04:34:52.295Z + 2025-09-23T14:25:50.779Z https://docs.axolotl.ai/docs/api/kernels.geglu.html - 2025-09-23T04:34:52.439Z + 2025-09-23T14:25:50.922Z https://docs.axolotl.ai/docs/api/utils.freeze.html - 2025-09-23T04:34:52.599Z + 2025-09-23T14:25:51.083Z https://docs.axolotl.ai/docs/api/utils.callbacks.mlflow_.html - 2025-09-23T04:34:53.057Z + 2025-09-23T14:25:51.539Z https://docs.axolotl.ai/docs/api/core.trainers.grpo.sampler.html - 2025-09-23T04:34:52.074Z + 2025-09-23T14:25:50.557Z https://docs.axolotl.ai/docs/api/cli.utils.train.html - 2025-09-23T04:34:52.009Z + 2025-09-23T14:25:50.492Z https://docs.axolotl.ai/docs/api/cli.utils.load.html - 2025-09-23T04:34:51.991Z + 2025-09-23T14:25:50.475Z https://docs.axolotl.ai/docs/api/utils.collators.core.html - 2025-09-23T04:34:52.973Z + 2025-09-23T14:25:51.456Z https://docs.axolotl.ai/docs/api/integrations.liger.args.html - 2025-09-23T04:34:52.946Z + 2025-09-23T14:25:51.429Z https://docs.axolotl.ai/docs/api/cli.preprocess.html - 2025-09-23T04:34:51.946Z + 2025-09-23T14:25:50.429Z https://docs.axolotl.ai/docs/api/kernels.utils.html - 2025-09-23T04:34:52.458Z + 2025-09-23T14:25:50.942Z https://docs.axolotl.ai/docs/api/utils.callbacks.lisa.html - 2025-09-23T04:34:53.053Z + 2025-09-23T14:25:51.536Z https://docs.axolotl.ai/docs/api/monkeypatch.data.batch_dataset_fetcher.html - 2025-09-23T04:34:52.538Z + 2025-09-23T14:25:51.022Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.optimizer.html - 2025-09-23T04:34:52.118Z + 2025-09-23T14:25:50.601Z https://docs.axolotl.ai/docs/api/loaders.processor.html - 2025-09-23T04:34:52.095Z + 2025-09-23T14:25:50.578Z https://docs.axolotl.ai/docs/api/core.trainers.utils.html - 2025-09-23T04:34:52.076Z + 2025-09-23T14:25:50.559Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chat_template.html - 2025-09-23T04:34:52.273Z + 2025-09-23T14:25:50.757Z https://docs.axolotl.ai/docs/api/prompt_tokenizers.html - 2025-09-23T04:34:51.723Z + 2025-09-23T14:25:50.201Z https://docs.axolotl.ai/docs/api/utils.callbacks.comet_.html - 2025-09-23T04:34:53.061Z + 2025-09-23T14:25:51.543Z https://docs.axolotl.ai/docs/api/core.datasets.transforms.chat_builder.html - 2025-09-23T04:34:51.801Z + 2025-09-23T14:25:50.283Z https://docs.axolotl.ai/docs/api/utils.trainer.html - 2025-09-23T04:34:52.616Z + 2025-09-23T14:25:51.100Z https://docs.axolotl.ai/docs/api/cli.train.html - 2025-09-23T04:34:51.843Z + 2025-09-23T14:25:50.326Z https://docs.axolotl.ai/docs/api/integrations.lm_eval.args.html - 2025-09-23T04:34:52.950Z + 2025-09-23T14:25:51.433Z https://docs.axolotl.ai/docs/api/prompt_strategies.messages.chat.html - 2025-09-23T04:34:52.272Z + 2025-09-23T14:25:50.755Z https://docs.axolotl.ai/docs/api/utils.schemas.peft.html - 2025-09-23T04:34:52.759Z + 2025-09-23T14:25:51.242Z https://docs.axolotl.ai/docs/api/prompt_strategies.bradley_terry.llama3.html - 2025-09-23T04:34:52.341Z + 2025-09-23T14:25:50.824Z https://docs.axolotl.ai/docs/api/cli.utils.args.html - 2025-09-23T04:34:51.980Z + 2025-09-23T14:25:50.464Z https://docs.axolotl.ai/docs/api/prompt_strategies.stepwise_supervised.html - 2025-09-23T04:34:52.250Z + 2025-09-23T14:25:50.734Z https://docs.axolotl.ai/docs/api/common.datasets.html - 2025-09-23T04:34:52.971Z + 2025-09-23T14:25:51.454Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.user_defined.html - 2025-09-23T04:34:52.316Z + 2025-09-23T14:25:50.800Z https://docs.axolotl.ai/docs/api/train.html - 2025-09-23T04:34:51.651Z + 2025-09-23T14:25:50.129Z https://docs.axolotl.ai/docs/multi-node.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/input_output.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/ray-integration.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/getting-started.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/optimizers.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/multi-gpu.html - 2025-09-23T04:31:24.842Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/docs/dataset_preprocessing.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/torchao.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/config-reference.html - 2025-09-23T04:35:08.477Z + 2025-09-23T14:26:07.014Z https://docs.axolotl.ai/docs/rlhf.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.584Z https://docs.axolotl.ai/docs/cli.html - 2025-09-23T04:31:24.838Z + 2025-09-23T14:22:24.579Z https://docs.axolotl.ai/docs/fsdp_qlora.html - 2025-09-23T04:31:24.839Z + 2025-09-23T14:22:24.580Z https://docs.axolotl.ai/docs/qat.html - 2025-09-23T04:31:24.843Z + 2025-09-23T14:22:24.583Z https://docs.axolotl.ai/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html - 2025-09-23T04:31:24.864Z + 2025-09-23T14:22:24.605Z https://docs.axolotl.ai/index.html - 2025-09-23T04:31:24.859Z + 2025-09-23T14:22:24.601Z
conversations_field strThe field name of the conversations. Defaults to “conversations”.'conversations'The field name of the conversations. Defaults to “messages”.'messages'
message_field_role str | list[str]The field name of the role. Defaults to “role”.The field name of the role. None
message_field_content str | list[str]The field name of the message content. Defaults to “content”.The field name of the message content. None
message_field_training str | list[str]The field name of the train/weight. Defaults to “weight”.The field name of the train/weight. None