From dd36fe4391b9c1f6256d82c8c209da729a7dc0ab Mon Sep 17 00:00:00 2001 From: Quarto GHA Workflow Runner Date: Wed, 28 May 2025 20:22:40 +0000 Subject: [PATCH] Built site for gh-pages --- .nojekyll | 2 +- docs/faq.html | 13 ++ search.json | 2 +- sitemap.xml | 376 +++++++++++++++++++++++++------------------------- 4 files changed, 203 insertions(+), 190 deletions(-) diff --git a/.nojekyll b/.nojekyll index c250ef015..0737c307a 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -f25724bb \ No newline at end of file +d2901af9 \ No newline at end of file diff --git a/docs/faq.html b/docs/faq.html index fb0d081b3..f1bd54943 100644 --- a/docs/faq.html +++ b/docs/faq.html @@ -621,6 +621,19 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});

Internally, eot_tokens: tokenizer.eos_token and train_on_eot: train_on_eos (which defaults to turn). This transition helps clarify the naming and behavior of EOT/EOS tokens.

+

Q: Data processing error: CAS service error

+
+

A: Try disabling XET with export HF_HUB_DISABLE_XET=1

+
+

Q: torch._inductor.exc.LoweringException: NoValidChoicesError: No choices to select, please consider adding ATEN into max_autotune_gemm_backends config (defined in torch/_inductor/config.py) to allow at least one choice.

+
+

A: Depending on the version of torch, you may need to include this in your YAML:

+
+
+
flex_attn_compile_kwargs:
+  dynamic: false
+  mode: max-autotune-no-cudagraphs
+
diff --git a/search.json b/search.json index 92fbe8ac9..948ea715b 100644 --- a/search.json +++ b/search.json @@ -696,7 +696,7 @@ "href": "docs/faq.html", "title": "FAQ", "section": "", - "text": "General\nQ: The trainer stopped and hasn’t progressed in several minutes.\n\nA: Usually an issue with the GPUs communicating with each other. See the NCCL doc\n\nQ: Exitcode -9\n\nA: This usually happens when you run out of system RAM.\n\nQ: Exitcode -7 while using deepspeed\n\nA: Try upgrading deepspeed w: pip install -U deepspeed\n\nQ: AttributeError: ‘DummyOptim’ object has no attribute ‘step’\nQ: ModuleNotFoundError: No module named ‘mpi4py’ using single GPU with deepspeed\n\nA: You may be using deepspeed with single gpu. Please remove the deepspeed: section in the yaml file or --deepspeed CLI flag.\n\nQ: The codes is stuck on saving preprocessed datasets.\n\nA: This is usually an issue with the GPU. This can be resolved through setting the os environment variable CUDA_VISIBLE_DEVICES=0. If you are on runpod, this is usually a pod issue. Starting a new pod should take care of it.\n\nQ: Received mismatch error on merge adapters / loading adapters between torch.Size of checkpoint and model.\n\nA: This is likely due to vocab size mismatch. By default, Axolotl expands the model’s embeddings if the tokenizer has more tokens than the model. Please use the axolotl merge-lora command to merge the adapters instead of using your own scripts.\n\n\nOn the other hand, if the model has more tokens than the tokenizer, Axolotl does not shrink the model’s embeddings unless shrink_embeddings: true is set in the config.\n\nQ: How to call Axolotl via custom python scripts?\n\nA: Since Axolotl is just Python, please see src/axolotl/cli/main.py on how each command is called.\n\nQ: How to know the value to use for fsdp_transformer_layer_cls_to_wrap?\n\nA: This is the class name of the transformer layer to wrap with FSDP. For example, for LlamaForCausalLM, the value is LlamaDecoderLayer. To find this for a specific model, check the model’s PreTrainedModel definition and look for _no_split_modules variable in the modeling_<model_name>.py file within transformers library.\n\nQ: ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token\n\nA: This is because the tokenizer does not have a padding token. Please add a padding token to the tokenizer via:\n\n\nspecial_tokens:\n # str. If you're not sure, set to same as `eos_token`.\n pad_token: \"...\"\n\n\n\nChat templates\nQ: jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'content' / 'role' / ____\n\nA: This means that the property mapping for the stated attribute does not exist when building chat_template prompt. For example, if no attribute 'content', please check you have added the correct mapping for content under message_property_mappings.\n\nQ: Empty template generated for turn ___\n\nA: The content is empty for that turn.\n\nQ: Could not find content start/end boundary for turn __\n\nA: The specific turn’s start/end could not be detected. Please ensure you have set the eos_token following your chat_template. Otherwise, this could be a chat_template which doesn’t use proper boundaries for each turn (like system). On the rare occurrence, make sure your content is not [[dummy_message]]. Please let us know about this.\n\nQ: Content end boundary is before start boundary for turn ___\n\nA: This is an edge case which should not occur. Please create an Issue if this happens.\n\nQ: Content end boundary is the same as start boundary for turn ___. This is likely an empty turn.\n\nA: This is likely an empty turn.\n\nQ: The EOS token is incorrectly being masked or not being masked / EOS token __ not found in chat template.\n\nA: There can be two reasons:\n\n\n\nThis is because of the mismatch between tokenizer.eos_token and EOS token in template. Please make sure to set eos_token: under special_tokens: to the same EOS token as in template.\n\n\n\n\nThe EOS token is not in the template. Please check if your template is correct. As an example, phi_35 template does not use its dedicated EOS token <|endoftext|> at the end.\n\n\nQ: “chat_template choice is tokenizer_default but tokenizer’s chat_template is null. Please add a chat_template in tokenizer config”\n\nA: This is because the tokenizer does not have a chat template. Please add a chat template in the tokenizer config. See chat_template for more details.\n\nQ: The EOT token(s) are incorrectly being masked or not being masked / EOT token __ not found in chat template.\n\nA: There can be two reasons:\n\n\n\nThe EOT token is different from the EOS token and was not specified under eot_tokens:. Please set eot_tokens: to the same EOT token(s) as in template.\n\n\n\n\nThere is more than one EOT token per turn in the template. Please raise an issue with examples as we recognize this as an edge case.\n\n\nQ: EOT token encoding failed. Please check if the token is valid and can be encoded.\n\nA: There could be some issue with the tokenizer or unicode encoding. Please raise an issue with examples with the EOT token & tokenizer causing the issue.\n\nQ: EOT token __ is encoded as multiple tokens.\n\nA: This is because the EOT token is encoded as multiple tokens which can cause unexpected behavior. Please add it under tokens: or (recommended) override unused added_tokens via added_tokens_overrides:.\n\nQ: Conflict between train_on_eos and train_on_eot. eos_token is in eot_tokens and train_on_eos != train_on_eot\n\nA: This is because the EOS token is in the eot_tokens: while mismatch between train_on_eos: and train_on_eot:. This will cause one to override the other. Please ensure that train_on_eos: and train_on_eot: are the same or remove the EOS token from eot_tokens:.\n\nQ: If eot_tokens: is not provided, what happens?\n\nA: If eot_tokens: is not provided, the default behavior is the same as before. EOS tokens used to delimit turns are masked/unmasked depending on whether the turn is trainable.\n\n\nInternally, eot_tokens: tokenizer.eos_token and train_on_eot: train_on_eos (which defaults to turn). This transition helps clarify the naming and behavior of EOT/EOS tokens.", + "text": "General\nQ: The trainer stopped and hasn’t progressed in several minutes.\n\nA: Usually an issue with the GPUs communicating with each other. See the NCCL doc\n\nQ: Exitcode -9\n\nA: This usually happens when you run out of system RAM.\n\nQ: Exitcode -7 while using deepspeed\n\nA: Try upgrading deepspeed w: pip install -U deepspeed\n\nQ: AttributeError: ‘DummyOptim’ object has no attribute ‘step’\nQ: ModuleNotFoundError: No module named ‘mpi4py’ using single GPU with deepspeed\n\nA: You may be using deepspeed with single gpu. Please remove the deepspeed: section in the yaml file or --deepspeed CLI flag.\n\nQ: The codes is stuck on saving preprocessed datasets.\n\nA: This is usually an issue with the GPU. This can be resolved through setting the os environment variable CUDA_VISIBLE_DEVICES=0. If you are on runpod, this is usually a pod issue. Starting a new pod should take care of it.\n\nQ: Received mismatch error on merge adapters / loading adapters between torch.Size of checkpoint and model.\n\nA: This is likely due to vocab size mismatch. By default, Axolotl expands the model’s embeddings if the tokenizer has more tokens than the model. Please use the axolotl merge-lora command to merge the adapters instead of using your own scripts.\n\n\nOn the other hand, if the model has more tokens than the tokenizer, Axolotl does not shrink the model’s embeddings unless shrink_embeddings: true is set in the config.\n\nQ: How to call Axolotl via custom python scripts?\n\nA: Since Axolotl is just Python, please see src/axolotl/cli/main.py on how each command is called.\n\nQ: How to know the value to use for fsdp_transformer_layer_cls_to_wrap?\n\nA: This is the class name of the transformer layer to wrap with FSDP. For example, for LlamaForCausalLM, the value is LlamaDecoderLayer. To find this for a specific model, check the model’s PreTrainedModel definition and look for _no_split_modules variable in the modeling_<model_name>.py file within transformers library.\n\nQ: ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token\n\nA: This is because the tokenizer does not have a padding token. Please add a padding token to the tokenizer via:\n\n\nspecial_tokens:\n # str. If you're not sure, set to same as `eos_token`.\n pad_token: \"...\"\n\n\n\nChat templates\nQ: jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'content' / 'role' / ____\n\nA: This means that the property mapping for the stated attribute does not exist when building chat_template prompt. For example, if no attribute 'content', please check you have added the correct mapping for content under message_property_mappings.\n\nQ: Empty template generated for turn ___\n\nA: The content is empty for that turn.\n\nQ: Could not find content start/end boundary for turn __\n\nA: The specific turn’s start/end could not be detected. Please ensure you have set the eos_token following your chat_template. Otherwise, this could be a chat_template which doesn’t use proper boundaries for each turn (like system). On the rare occurrence, make sure your content is not [[dummy_message]]. Please let us know about this.\n\nQ: Content end boundary is before start boundary for turn ___\n\nA: This is an edge case which should not occur. Please create an Issue if this happens.\n\nQ: Content end boundary is the same as start boundary for turn ___. This is likely an empty turn.\n\nA: This is likely an empty turn.\n\nQ: The EOS token is incorrectly being masked or not being masked / EOS token __ not found in chat template.\n\nA: There can be two reasons:\n\n\n\nThis is because of the mismatch between tokenizer.eos_token and EOS token in template. Please make sure to set eos_token: under special_tokens: to the same EOS token as in template.\n\n\n\n\nThe EOS token is not in the template. Please check if your template is correct. As an example, phi_35 template does not use its dedicated EOS token <|endoftext|> at the end.\n\n\nQ: “chat_template choice is tokenizer_default but tokenizer’s chat_template is null. Please add a chat_template in tokenizer config”\n\nA: This is because the tokenizer does not have a chat template. Please add a chat template in the tokenizer config. See chat_template for more details.\n\nQ: The EOT token(s) are incorrectly being masked or not being masked / EOT token __ not found in chat template.\n\nA: There can be two reasons:\n\n\n\nThe EOT token is different from the EOS token and was not specified under eot_tokens:. Please set eot_tokens: to the same EOT token(s) as in template.\n\n\n\n\nThere is more than one EOT token per turn in the template. Please raise an issue with examples as we recognize this as an edge case.\n\n\nQ: EOT token encoding failed. Please check if the token is valid and can be encoded.\n\nA: There could be some issue with the tokenizer or unicode encoding. Please raise an issue with examples with the EOT token & tokenizer causing the issue.\n\nQ: EOT token __ is encoded as multiple tokens.\n\nA: This is because the EOT token is encoded as multiple tokens which can cause unexpected behavior. Please add it under tokens: or (recommended) override unused added_tokens via added_tokens_overrides:.\n\nQ: Conflict between train_on_eos and train_on_eot. eos_token is in eot_tokens and train_on_eos != train_on_eot\n\nA: This is because the EOS token is in the eot_tokens: while mismatch between train_on_eos: and train_on_eot:. This will cause one to override the other. Please ensure that train_on_eos: and train_on_eot: are the same or remove the EOS token from eot_tokens:.\n\nQ: If eot_tokens: is not provided, what happens?\n\nA: If eot_tokens: is not provided, the default behavior is the same as before. EOS tokens used to delimit turns are masked/unmasked depending on whether the turn is trainable.\n\n\nInternally, eot_tokens: tokenizer.eos_token and train_on_eot: train_on_eos (which defaults to turn). This transition helps clarify the naming and behavior of EOT/EOS tokens.\n\nQ: Data processing error: CAS service error\n\nA: Try disabling XET with export HF_HUB_DISABLE_XET=1\n\nQ: torch._inductor.exc.LoweringException: NoValidChoicesError: No choices to select, please consider adding ATEN into max_autotune_gemm_backends config (defined in torch/_inductor/config.py) to allow at least one choice.\n\nA: Depending on the version of torch, you may need to include this in your YAML:\n\n\nflex_attn_compile_kwargs:\n dynamic: false\n mode: max-autotune-no-cudagraphs", "crumbs": [ "Troubleshooting", "FAQ" diff --git a/sitemap.xml b/sitemap.xml index 309dded6d..4b10f4d6f 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,754 +2,754 @@ https://docs.axolotl.ai/TODO.html - 2025-05-28T19:00:07.843Z + 2025-05-28T20:20:30.191Z https://docs.axolotl.ai/docs/debugging.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/rlhf.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/input_output.html - 2025-05-28T19:00:07.847Z + 2025-05-28T20:20:30.196Z https://docs.axolotl.ai/docs/lr_groups.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/dataset-formats/template_free.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset-formats/pretraining.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset-formats/conversation.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset-formats/inst_tune.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/fsdp_qlora.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/torchao.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/dataset_preprocessing.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/batch_vs_grad.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/nccl.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/lora_optims.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.196Z https://docs.axolotl.ai/docs/multipack.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/faq.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/quantize.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/sequence_parallelism.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/multi-gpu.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/api/utils.collators.core.html - 2025-05-28T19:00:37.450Z + 2025-05-28T20:21:01.349Z https://docs.axolotl.ai/docs/api/utils.collators.mm_chat.html - 2025-05-28T19:00:37.478Z + 2025-05-28T20:21:01.377Z https://docs.axolotl.ai/docs/api/prompt_strategies.pygmalion.html - 2025-05-28T19:00:36.745Z + 2025-05-28T20:21:00.653Z https://docs.axolotl.ai/docs/api/cli.cloud.base.html - 2025-05-28T19:00:36.492Z + 2025-05-28T20:21:00.397Z https://docs.axolotl.ai/docs/api/common.architectures.html - 2025-05-28T19:00:37.430Z + 2025-05-28T20:21:01.329Z https://docs.axolotl.ai/docs/api/utils.optimizers.adopt.html - 2025-05-28T19:00:37.182Z + 2025-05-28T20:21:01.085Z https://docs.axolotl.ai/docs/api/utils.callbacks.lisa.html - 2025-05-28T19:00:37.530Z + 2025-05-28T20:21:01.428Z https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_cpu.html - 2025-05-28T19:00:37.046Z + 2025-05-28T20:21:00.951Z https://docs.axolotl.ai/docs/api/integrations.lm_eval.args.html - 2025-05-28T19:00:37.425Z + 2025-05-28T20:21:01.324Z https://docs.axolotl.ai/docs/api/utils.model_shard_quant.html - 2025-05-28T19:00:37.099Z + 2025-05-28T20:21:01.003Z https://docs.axolotl.ai/docs/api/utils.schemas.utils.html - 2025-05-28T19:00:37.288Z + 2025-05-28T20:21:01.189Z https://docs.axolotl.ai/docs/api/core.trainers.trl.html - 2025-05-28T19:00:36.535Z + 2025-05-28T20:21:00.439Z https://docs.axolotl.ai/docs/api/utils.collators.batching.html - 2025-05-28T19:00:37.469Z + 2025-05-28T20:21:01.368Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_w_system.html - 2025-05-28T19:00:36.690Z + 2025-05-28T20:21:00.599Z https://docs.axolotl.ai/docs/api/integrations.base.html - 2025-05-28T19:00:37.407Z + 2025-05-28T20:21:01.306Z https://docs.axolotl.ai/docs/api/core.chat.messages.html - 2025-05-28T19:00:36.295Z + 2025-05-28T20:21:00.203Z https://docs.axolotl.ai/docs/api/kernels.lora.html - 2025-05-28T19:00:36.900Z + 2025-05-28T20:21:00.806Z https://docs.axolotl.ai/docs/api/utils.callbacks.perplexity.html - 2025-05-28T19:00:37.524Z + 2025-05-28T20:21:01.422Z https://docs.axolotl.ai/docs/api/utils.schemas.training.html - 2025-05-28T19:00:37.226Z + 2025-05-28T20:21:01.128Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.user_defined.html - 2025-05-28T19:00:36.774Z + 2025-05-28T20:21:00.682Z https://docs.axolotl.ai/docs/api/utils.samplers.multipack.html - 2025-05-28T19:00:37.518Z + 2025-05-28T20:21:01.416Z https://docs.axolotl.ai/docs/api/utils.collators.mamba.html - 2025-05-28T19:00:37.473Z + 2025-05-28T20:21:01.372Z https://docs.axolotl.ai/docs/api/cli.merge_sharded_fsdp_weights.html - 2025-05-28T19:00:36.438Z + 2025-05-28T20:21:00.344Z https://docs.axolotl.ai/docs/api/utils.callbacks.profiler.html - 2025-05-28T19:00:37.528Z + 2025-05-28T20:21:01.426Z https://docs.axolotl.ai/docs/api/loaders.tokenizer.html - 2025-05-28T19:00:36.591Z + 2025-05-28T20:21:00.494Z https://docs.axolotl.ai/docs/api/core.datasets.transforms.chat_builder.html - 2025-05-28T19:00:36.313Z + 2025-05-28T20:21:00.220Z https://docs.axolotl.ai/docs/api/cli.vllm_serve.html - 2025-05-28T19:00:36.489Z + 2025-05-28T20:21:00.394Z https://docs.axolotl.ai/docs/api/cli.config.html - 2025-05-28T19:00:36.405Z + 2025-05-28T20:21:00.310Z https://docs.axolotl.ai/docs/api/utils.schemas.trl.html - 2025-05-28T19:00:37.255Z + 2025-05-28T20:21:01.157Z https://docs.axolotl.ai/docs/api/core.trainers.mamba.html - 2025-05-28T19:00:36.540Z + 2025-05-28T20:21:00.444Z https://docs.axolotl.ai/docs/api/integrations.grokfast.optimizer.html - 2025-05-28T19:00:37.411Z + 2025-05-28T20:21:01.310Z https://docs.axolotl.ai/docs/api/prompt_strategies.chat_template.html - 2025-05-28T19:00:36.663Z + 2025-05-28T20:21:00.572Z https://docs.axolotl.ai/docs/api/utils.schemas.multimodal.html - 2025-05-28T19:00:37.260Z + 2025-05-28T20:21:01.162Z https://docs.axolotl.ai/docs/api/monkeypatch.utils.html - 2025-05-28T19:00:37.013Z + 2025-05-28T20:21:00.919Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.user_defined.html - 2025-05-28T19:00:36.793Z + 2025-05-28T20:21:00.701Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.rng_state_loader.html - 2025-05-28T19:00:36.614Z + 2025-05-28T20:21:00.522Z https://docs.axolotl.ai/docs/api/loaders.adapter.html - 2025-05-28T19:00:36.598Z + 2025-05-28T20:21:00.501Z https://docs.axolotl.ai/docs/api/integrations.liger.args.html - 2025-05-28T19:00:37.422Z + 2025-05-28T20:21:01.321Z https://docs.axolotl.ai/docs/api/utils.data.sft.html - 2025-05-28T19:00:37.185Z + 2025-05-28T20:21:01.088Z https://docs.axolotl.ai/docs/api/utils.freeze.html - 2025-05-28T19:00:37.111Z + 2025-05-28T20:21:01.014Z https://docs.axolotl.ai/docs/api/cli.preprocess.html - 2025-05-28T19:00:36.447Z + 2025-05-28T20:21:00.352Z https://docs.axolotl.ai/docs/api/index.html - 2025-05-28T19:00:36.030Z + 2025-05-28T20:20:59.940Z https://docs.axolotl.ai/docs/api/cli.main.html - 2025-05-28T19:00:36.344Z + 2025-05-28T20:21:00.251Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chatml.html - 2025-05-28T19:00:36.771Z + 2025-05-28T20:21:00.679Z https://docs.axolotl.ai/docs/api/core.trainer_builder.html - 2025-05-28T19:00:36.185Z + 2025-05-28T20:21:00.093Z https://docs.axolotl.ai/docs/api/train.html - 2025-05-28T19:00:36.091Z + 2025-05-28T20:21:00.000Z https://docs.axolotl.ai/docs/api/utils.callbacks.qat.html - 2025-05-28T19:00:37.544Z + 2025-05-28T20:21:01.442Z https://docs.axolotl.ai/docs/api/monkeypatch.stablelm_attn_hijack_flash.html - 2025-05-28T19:00:37.022Z + 2025-05-28T20:21:00.927Z https://docs.axolotl.ai/docs/api/utils.schemas.enums.html - 2025-05-28T19:00:37.283Z + 2025-05-28T20:21:01.184Z https://docs.axolotl.ai/docs/api/loaders.constants.html - 2025-05-28T19:00:36.608Z + 2025-05-28T20:21:00.511Z https://docs.axolotl.ai/docs/api/prompt_strategies.metharme.html - 2025-05-28T19:00:36.734Z + 2025-05-28T20:21:00.643Z https://docs.axolotl.ai/docs/api/kernels.swiglu.html - 2025-05-28T19:00:36.920Z + 2025-05-28T20:21:00.826Z https://docs.axolotl.ai/docs/api/core.trainers.base.html - 2025-05-28T19:00:36.517Z + 2025-05-28T20:21:00.422Z https://docs.axolotl.ai/docs/api/monkeypatch.relora.html - 2025-05-28T19:00:36.979Z + 2025-05-28T20:21:00.885Z https://docs.axolotl.ai/docs/api/cli.cloud.modal_.html - 2025-05-28T19:00:36.498Z + 2025-05-28T20:21:00.403Z https://docs.axolotl.ai/docs/api/core.training_args.html - 2025-05-28T19:00:36.272Z + 2025-05-28T20:21:00.180Z https://docs.axolotl.ai/docs/api/utils.callbacks.comet_.html - 2025-05-28T19:00:37.537Z + 2025-05-28T20:21:01.435Z https://docs.axolotl.ai/docs/api/core.chat.format.shared.html - 2025-05-28T19:00:36.299Z + 2025-05-28T20:21:00.207Z https://docs.axolotl.ai/docs/api/monkeypatch.lora_kernels.html - 2025-05-28T19:00:37.006Z + 2025-05-28T20:21:00.911Z https://docs.axolotl.ai/docs/api/cli.merge_lora.html - 2025-05-28T19:00:36.427Z + 2025-05-28T20:21:00.332Z https://docs.axolotl.ai/docs/api/utils.trainer.html - 2025-05-28T19:00:37.128Z + 2025-05-28T20:21:01.031Z https://docs.axolotl.ai/docs/api/utils.dict.html - 2025-05-28T19:00:37.175Z + 2025-05-28T20:21:01.077Z https://docs.axolotl.ai/docs/api/kernels.quantize.html - 2025-05-28T19:00:36.927Z + 2025-05-28T20:21:00.833Z https://docs.axolotl.ai/docs/api/core.trainers.utils.html - 2025-05-28T19:00:36.573Z + 2025-05-28T20:21:00.477Z https://docs.axolotl.ai/docs/api/monkeypatch.data.batch_dataset_fetcher.html - 2025-05-28T19:00:37.041Z + 2025-05-28T20:21:00.946Z https://docs.axolotl.ai/docs/api/utils.ctx_managers.sequence_parallel.html - 2025-05-28T19:00:36.644Z + 2025-05-28T20:21:00.553Z https://docs.axolotl.ai/docs/api/core.trainers.grpo.sampler.html - 2025-05-28T19:00:36.572Z + 2025-05-28T20:21:00.475Z https://docs.axolotl.ai/docs/api/prompt_strategies.user_defined.html - 2025-05-28T19:00:36.699Z + 2025-05-28T20:21:00.607Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_flash.html - 2025-05-28T19:00:36.955Z + 2025-05-28T20:21:00.861Z https://docs.axolotl.ai/docs/api/cli.train.html - 2025-05-28T19:00:36.352Z + 2025-05-28T20:21:00.259Z https://docs.axolotl.ai/docs/api/cli.quantize.html - 2025-05-28T19:00:36.503Z + 2025-05-28T20:21:00.408Z https://docs.axolotl.ai/docs/api/cli.utils.html - 2025-05-28T19:00:36.484Z + 2025-05-28T20:21:00.389Z https://docs.axolotl.ai/src/axolotl/integrations/LICENSE.html - 2025-05-28T19:00:07.865Z + 2025-05-28T20:20:30.213Z https://docs.axolotl.ai/FAQS.html - 2025-05-28T19:00:07.842Z + 2025-05-28T20:20:30.191Z https://docs.axolotl.ai/examples/colab-notebooks/colab-axolotl-example.html - 2025-05-28T19:00:07.849Z + 2025-05-28T20:20:30.198Z https://docs.axolotl.ai/index.html - 2025-05-28T19:00:07.861Z + 2025-05-28T20:20:30.210Z https://docs.axolotl.ai/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html - 2025-05-28T19:00:07.865Z + 2025-05-28T20:20:30.214Z https://docs.axolotl.ai/docs/api/datasets.html - 2025-05-28T19:00:36.109Z + 2025-05-28T20:21:00.018Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.optimizer.html - 2025-05-28T19:00:36.611Z + 2025-05-28T20:21:00.516Z https://docs.axolotl.ai/docs/api/prompt_strategies.orpo.chat_template.html - 2025-05-28T19:00:36.813Z + 2025-05-28T20:21:00.721Z https://docs.axolotl.ai/docs/api/prompt_strategies.completion.html - 2025-05-28T19:00:36.717Z + 2025-05-28T20:21:00.625Z https://docs.axolotl.ai/docs/api/evaluate.html - 2025-05-28T19:00:36.102Z + 2025-05-28T20:21:00.011Z https://docs.axolotl.ai/docs/api/loaders.processor.html - 2025-05-28T19:00:36.593Z + 2025-05-28T20:21:00.496Z https://docs.axolotl.ai/docs/api/common.const.html - 2025-05-28T19:00:37.431Z + 2025-05-28T20:21:01.330Z https://docs.axolotl.ai/docs/api/prompt_strategies.bradley_terry.llama3.html - 2025-05-28T19:00:36.817Z + 2025-05-28T20:21:00.725Z https://docs.axolotl.ai/docs/api/utils.distributed.html - 2025-05-28T19:00:37.172Z + 2025-05-28T20:21:01.074Z https://docs.axolotl.ai/docs/api/utils.tokenization.html - 2025-05-28T19:00:37.079Z + 2025-05-28T20:21:00.983Z https://docs.axolotl.ai/docs/api/utils.schemas.integrations.html - 2025-05-28T19:00:37.272Z + 2025-05-28T20:21:01.173Z https://docs.axolotl.ai/docs/api/utils.schedulers.html - 2025-05-28T19:00:37.152Z + 2025-05-28T20:21:01.055Z https://docs.axolotl.ai/docs/api/core.trainers.mixins.scheduler.html - 2025-05-28T19:00:36.621Z + 2025-05-28T20:21:00.530Z https://docs.axolotl.ai/docs/api/core.datasets.chat.html - 2025-05-28T19:00:36.305Z + 2025-05-28T20:21:00.212Z https://docs.axolotl.ai/docs/api/utils.callbacks.mlflow_.html - 2025-05-28T19:00:37.533Z + 2025-05-28T20:21:01.431Z https://docs.axolotl.ai/docs/api/monkeypatch.mistral_attn_hijack_flash.html - 2025-05-28T19:00:36.971Z + 2025-05-28T20:21:00.876Z https://docs.axolotl.ai/docs/api/core.chat.format.llama3x.html - 2025-05-28T19:00:36.298Z + 2025-05-28T20:21:00.206Z https://docs.axolotl.ai/docs/api/cli.checks.html - 2025-05-28T19:00:36.387Z + 2025-05-28T20:21:00.293Z https://docs.axolotl.ai/docs/api/monkeypatch.transformers_fa_utils.html - 2025-05-28T19:00:37.031Z + 2025-05-28T20:21:00.937Z https://docs.axolotl.ai/docs/api/prompt_strategies.llama2_chat.html - 2025-05-28T19:00:36.711Z + 2025-05-28T20:21:00.620Z https://docs.axolotl.ai/docs/api/convert.html - 2025-05-28T19:00:36.123Z + 2025-05-28T20:21:00.031Z https://docs.axolotl.ai/docs/api/utils.schemas.datasets.html - 2025-05-28T19:00:37.244Z + 2025-05-28T20:21:01.145Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.llama3.html - 2025-05-28T19:00:36.761Z + 2025-05-28T20:21:00.669Z https://docs.axolotl.ai/docs/api/cli.args.html - 2025-05-28T19:00:36.380Z + 2025-05-28T20:21:00.287Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_xformers.html - 2025-05-28T19:00:36.956Z + 2025-05-28T20:21:00.862Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chat_template.html - 2025-05-28T19:00:36.751Z + 2025-05-28T20:21:00.659Z https://docs.axolotl.ai/docs/api/models.mamba.modeling_mamba.html - 2025-05-28T19:00:37.449Z + 2025-05-28T20:21:01.348Z https://docs.axolotl.ai/docs/api/prompt_strategies.messages.chat.html - 2025-05-28T19:00:36.749Z + 2025-05-28T20:21:00.657Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_patch_multipack.html - 2025-05-28T19:00:37.016Z + 2025-05-28T20:21:00.922Z https://docs.axolotl.ai/docs/api/loaders.patch_manager.html - 2025-05-28T19:00:36.606Z + 2025-05-28T20:21:00.509Z https://docs.axolotl.ai/docs/api/utils.schemas.peft.html - 2025-05-28T19:00:37.252Z + 2025-05-28T20:21:01.153Z https://docs.axolotl.ai/docs/api/monkeypatch.llama_expand_mask.html - 2025-05-28T19:00:36.981Z + 2025-05-28T20:21:00.886Z https://docs.axolotl.ai/docs/api/core.trainers.grpo.trainer.html - 2025-05-28T19:00:36.560Z + 2025-05-28T20:21:00.463Z https://docs.axolotl.ai/docs/api/integrations.spectrum.args.html - 2025-05-28T19:00:37.428Z + 2025-05-28T20:21:01.327Z https://docs.axolotl.ai/docs/api/core.trainers.relora.html - 2025-05-28T19:00:36.544Z + 2025-05-28T20:21:00.448Z https://docs.axolotl.ai/docs/api/loaders.model.html - 2025-05-28T19:00:36.583Z + 2025-05-28T20:21:00.486Z https://docs.axolotl.ai/docs/api/utils.bench.html - 2025-05-28T19:00:37.103Z + 2025-05-28T20:21:01.006Z https://docs.axolotl.ai/docs/api/prompt_strategies.input_output.html - 2025-05-28T19:00:36.723Z + 2025-05-28T20:21:00.631Z https://docs.axolotl.ai/docs/api/cli.inference.html - 2025-05-28T19:00:36.418Z + 2025-05-28T20:21:00.324Z https://docs.axolotl.ai/docs/api/core.trainers.dpo.trainer.html - 2025-05-28T19:00:36.550Z + 2025-05-28T20:21:00.454Z https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_disk.html - 2025-05-28T19:00:37.072Z + 2025-05-28T20:21:00.976Z https://docs.axolotl.ai/docs/api/utils.lora.html - 2025-05-28T19:00:37.094Z + 2025-05-28T20:21:00.998Z https://docs.axolotl.ai/docs/api/monkeypatch.btlm_attn_hijack_flash.html - 2025-05-28T19:00:37.015Z + 2025-05-28T20:21:00.920Z https://docs.axolotl.ai/docs/api/utils.chat_templates.html - 2025-05-28T19:00:37.089Z + 2025-05-28T20:21:00.993Z https://docs.axolotl.ai/docs/api/integrations.cut_cross_entropy.args.html - 2025-05-28T19:00:37.410Z + 2025-05-28T20:21:01.309Z https://docs.axolotl.ai/docs/api/monkeypatch.trainer_fsdp_optim.html - 2025-05-28T19:00:37.025Z + 2025-05-28T20:21:00.931Z https://docs.axolotl.ai/docs/api/prompt_strategies.stepwise_supervised.html - 2025-05-28T19:00:36.728Z + 2025-05-28T20:21:00.636Z https://docs.axolotl.ai/docs/api/common.datasets.html - 2025-05-28T19:00:37.448Z + 2025-05-28T20:21:01.347Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_instruct.html - 2025-05-28T19:00:36.678Z + 2025-05-28T20:21:00.587Z https://docs.axolotl.ai/docs/api/core.chat.format.chatml.html - 2025-05-28T19:00:36.296Z + 2025-05-28T20:21:00.204Z https://docs.axolotl.ai/docs/api/monkeypatch.attention.mllama.html - 2025-05-28T19:00:37.040Z + 2025-05-28T20:21:00.945Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.zephyr.html - 2025-05-28T19:00:36.773Z + 2025-05-28T20:21:00.681Z https://docs.axolotl.ai/docs/api/utils.schemas.config.html - 2025-05-28T19:00:37.214Z + 2025-05-28T20:21:01.116Z https://docs.axolotl.ai/docs/api/monkeypatch.unsloth_.html - 2025-05-28T19:00:37.033Z + 2025-05-28T20:21:00.938Z https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.passthrough.html - 2025-05-28T19:00:36.776Z + 2025-05-28T20:21:00.683Z https://docs.axolotl.ai/docs/api/prompt_strategies.orcamini.html - 2025-05-28T19:00:36.738Z + 2025-05-28T20:21:00.646Z https://docs.axolotl.ai/docs/api/cli.evaluate.html - 2025-05-28T19:00:36.360Z + 2025-05-28T20:21:00.267Z https://docs.axolotl.ai/docs/api/monkeypatch.multipack.html - 2025-05-28T19:00:36.972Z + 2025-05-28T20:21:00.878Z https://docs.axolotl.ai/docs/api/cli.sweeps.html - 2025-05-28T19:00:36.452Z + 2025-05-28T20:21:00.358Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.chatml.html - 2025-05-28T19:00:36.792Z + 2025-05-28T20:21:00.699Z https://docs.axolotl.ai/docs/api/prompt_strategies.base.html - 2025-05-28T19:00:36.645Z + 2025-05-28T20:21:00.554Z https://docs.axolotl.ai/docs/api/kernels.utils.html - 2025-05-28T19:00:36.929Z + 2025-05-28T20:21:00.835Z https://docs.axolotl.ai/docs/api/utils.schemas.model.html - 2025-05-28T19:00:37.221Z + 2025-05-28T20:21:01.123Z https://docs.axolotl.ai/docs/api/utils.data.pretraining.html - 2025-05-28T19:00:37.184Z + 2025-05-28T20:21:01.086Z https://docs.axolotl.ai/docs/api/monkeypatch.mixtral.html - 2025-05-28T19:00:37.043Z + 2025-05-28T20:21:00.948Z https://docs.axolotl.ai/docs/api/logging_config.html - 2025-05-28T19:00:36.170Z + 2025-05-28T20:21:00.078Z https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_chat.html - 2025-05-28T19:00:36.677Z + 2025-05-28T20:21:00.586Z https://docs.axolotl.ai/docs/api/prompt_strategies.kto.llama3.html - 2025-05-28T19:00:36.784Z + 2025-05-28T20:21:00.691Z https://docs.axolotl.ai/docs/api/integrations.kd.trainer.html - 2025-05-28T19:00:37.418Z + 2025-05-28T20:21:01.318Z https://docs.axolotl.ai/docs/api/utils.quantization.html - 2025-05-28T19:00:37.206Z + 2025-05-28T20:21:01.108Z https://docs.axolotl.ai/docs/api/prompt_tokenizers.html - 2025-05-28T19:00:36.165Z + 2025-05-28T20:21:00.073Z https://docs.axolotl.ai/docs/api/kernels.geglu.html - 2025-05-28T19:00:36.910Z + 2025-05-28T20:21:00.816Z https://docs.axolotl.ai/docs/custom_integrations.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/multimodal.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/multi-node.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/docker.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/amd_hpc.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset_loading.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/qat.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/cli.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/config.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/unsloth.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/ray-integration.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/dataset-formats/stepwise_supervised.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset-formats/index.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/dataset-formats/tokenized.html - 2025-05-28T19:00:07.844Z + 2025-05-28T20:20:30.193Z https://docs.axolotl.ai/docs/installation.html - 2025-05-28T19:00:07.847Z + 2025-05-28T20:20:30.196Z https://docs.axolotl.ai/docs/inference.html - 2025-05-28T19:00:07.847Z + 2025-05-28T20:20:30.196Z https://docs.axolotl.ai/docs/mac.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z https://docs.axolotl.ai/docs/getting-started.html - 2025-05-28T19:00:07.845Z + 2025-05-28T20:20:30.194Z https://docs.axolotl.ai/docs/reward_modelling.html - 2025-05-28T19:00:07.848Z + 2025-05-28T20:20:30.197Z