See LM Eval Harness for more details.
+Delinearizes a Llama 4 linearized model into a regular HuggingFace Llama 4 model. This only works with the non-quantized linearized model.
+ +This would be necessary to use with other frameworks. If you have an adapter, merge it with the non-quantized linearized model before delinearizing.
+While the new Click-based CLI is preferred, Axolotl still supports the legacy module-based CLI:
-# Preprocess
-python -m axolotl.cli.preprocess config.yml
-
-# Train
-accelerate launch -m axolotl.cli.train config.yml
-
-# Inference
-accelerate launch -m axolotl.cli.inference config.yml \
- --lora_model_dir="./outputs/lora-out"
-
-# Gradio interface
-accelerate launch -m axolotl.cli.inference config.yml \
- --lora_model_dir="./outputs/lora-out" --gradio# Preprocess
+python -m axolotl.cli.preprocess config.yml
+
+# Train
+accelerate launch -m axolotl.cli.train config.yml
+
+# Inference
+accelerate launch -m axolotl.cli.inference config.yml \
+ --lora_model_dir="./outputs/lora-out"
+
+# Gradio interface
+accelerate launch -m axolotl.cli.inference config.yml \
+ --lora_model_dir="./outputs/lora-out" --gradioCreate a cloud config YAML with your Modal settings:
-# cloud_config.yml
-provider: modal
-gpu: a100 # Supported: l40s, a100-40gb, a100-80gb, a10g, h100, t4, l4
-gpu_count: 1 # Number of GPUs to use
-timeout: 86400 # Maximum runtime in seconds (24 hours)
-branch: main # Git branch to use (optional)
-
-volumes: # Persistent storage volumes
- - name: axolotl-cache
- mount: /workspace/cache
- - name: axolotl-data
- mount: /workspace/data
- - name: axolotl-artifacts
- mount: /workspace/artifacts
-
-secrets: # Secrets to inject
- - WANDB_API_KEY
- - HF_TOKEN# cloud_config.yml
+provider: modal
+gpu: a100 # Supported: l40s, a100-40gb, a100-80gb, a10g, h100, t4, l4
+gpu_count: 1 # Number of GPUs to use
+timeout: 86400 # Maximum runtime in seconds (24 hours)
+branch: main # Git branch to use (optional)
+
+volumes: # Persistent storage volumes
+ - name: axolotl-cache
+ mount: /workspace/cache
+ - name: axolotl-data
+ mount: /workspace/data
+ - name: axolotl-artifacts
+ mount: /workspace/artifacts
+
+secrets: # Secrets to inject
+ - WANDB_API_KEY
+ - HF_TOKENCommands that support the –cloud flag:
-# Preprocess on cloud
-axolotl preprocess config.yml --cloud cloud_config.yml
-
-# Train on cloud
-axolotl train config.yml --cloud cloud_config.yml
-
-# Train without accelerate on cloud
-axolotl train config.yml --cloud cloud_config.yml --no-accelerate
-
-# Run lm-eval on cloud
-axolotl lm-eval config.yml --cloud cloud_config.yml# Preprocess on cloud
+axolotl preprocess config.yml --cloud cloud_config.yml
+
+# Train on cloud
+axolotl train config.yml --cloud cloud_config.yml
+
+# Train without accelerate on cloud
+axolotl train config.yml --cloud cloud_config.yml --no-accelerate
+
+# Run lm-eval on cloud
+axolotl lm-eval config.yml --cloud cloud_config.ymlprovider: # compute provider, currently only `modal` is supported
-gpu: # GPU type to use
-gpu_count: # Number of GPUs (default: 1)
-memory: # RAM in GB (default: 128)
-timeout: # Maximum runtime in seconds
-timeout_preprocess: # Preprocessing timeout
-branch: # Git branch to use
-docker_tag: # Custom Docker image tag
-volumes: # List of persistent storage volumes
-
-# Environment variables to pass. Can be specified in two ways:
-# 1. As a string: Will load the value from the host computer's environment variables
-# 2. As a key-value pair: Will use the specified value directly
-# Example:
-# env:
-# - CUSTOM_VAR # Loads from host's $CUSTOM_VAR
-# - {CUSTOM_VAR: "value"} # Uses "value" directly
-env:
-
-# Secrets to inject. Same input format as `env` but for sensitive data.
-secrets:
- # - HF_TOKEN
- # - WANDB_API_KEYprovider: # compute provider, currently only `modal` is supported
+gpu: # GPU type to use
+gpu_count: # Number of GPUs (default: 1)
+memory: # RAM in GB (default: 128)
+timeout: # Maximum runtime in seconds
+timeout_preprocess: # Preprocessing timeout
+branch: # Git branch to use
+docker_tag: # Custom Docker image tag
+volumes: # List of persistent storage volumes
+
+# Environment variables to pass. Can be specified in two ways:
+# 1. As a string: Will load the value from the host computer's environment variables
+# 2. As a key-value pair: Will use the specified value directly
+# Example:
+# env:
+# - CUSTOM_VAR # Loads from host's $CUSTOM_VAR
+# - {CUSTOM_VAR: "value"} # Uses "value" directly
+env:
+
+# Secrets to inject. Same input format as `env` but for sensitive data.
+secrets:
+ # - HF_TOKEN
+ # - WANDB_API_KEYPlease make sure to have Pytorch installed before installing Axolotl in your local environment.
+Follow the instructions at: https://pytorch.org/get-started/locally/
+pip3 install -U packaging setuptools wheel ninja
diff --git a/search.json b/search.json
index 52a6e3b9c..3be5705ef 100644
--- a/search.json
+++ b/search.json
@@ -1864,7 +1864,7 @@
"href": "docs/installation.html#sec-installation-methods",
"title": "Installation",
"section": "2 Installation Methods",
- "text": "2 Installation Methods\n\n2.1 PyPI Installation (Recommended)\npip3 install -U packaging setuptools wheel ninja\npip3 install --no-build-isolation axolotl[flash-attn,deepspeed]\nWe use --no-build-isolation in order to detect the installed PyTorch version (if\ninstalled) in order not to clobber it, and so that we set the correct version of\ndependencies that are specific to the PyTorch version or other installed\nco-dependencies.\n\n\n2.2 Edge/Development Build\nFor the latest features between releases:\ngit clone https://github.com/axolotl-ai-cloud/axolotl.git\ncd axolotl\npip3 install -U packaging setuptools wheel ninja\npip3 install --no-build-isolation -e '.[flash-attn,deepspeed]'\n\n\n2.3 Docker\ndocker run --gpus '\"all\"' --rm -it axolotlai/axolotl:main-latest\nFor development with Docker:\ndocker compose up -d\n\n\n\n\n\n\nAdvanced Docker Configuration\n\n\n\ndocker run --privileged --gpus '\"all\"' --shm-size 10g --rm -it \\\n --name axolotl --ipc=host \\\n --ulimit memlock=-1 --ulimit stack=67108864 \\\n --mount type=bind,src=\"${PWD}\",target=/workspace/axolotl \\\n -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \\\n axolotlai/axolotl:main-latest\n\n\nPlease refer to the Docker documentation for more information on the different Docker images that are available.",
+ "text": "2 Installation Methods\n\n\n\n\n\n\nImportant\n\n\n\nPlease make sure to have Pytorch installed before installing Axolotl in your local environment.\nFollow the instructions at: https://pytorch.org/get-started/locally/\n\n\n\n2.1 PyPI Installation (Recommended)\npip3 install -U packaging setuptools wheel ninja\npip3 install --no-build-isolation axolotl[flash-attn,deepspeed]\nWe use --no-build-isolation in order to detect the installed PyTorch version (if\ninstalled) in order not to clobber it, and so that we set the correct version of\ndependencies that are specific to the PyTorch version or other installed\nco-dependencies.\n\n\n2.2 Edge/Development Build\nFor the latest features between releases:\ngit clone https://github.com/axolotl-ai-cloud/axolotl.git\ncd axolotl\npip3 install -U packaging setuptools wheel ninja\npip3 install --no-build-isolation -e '.[flash-attn,deepspeed]'\n\n\n2.3 Docker\ndocker run --gpus '\"all\"' --rm -it axolotlai/axolotl:main-latest\nFor development with Docker:\ndocker compose up -d\n\n\n\n\n\n\nAdvanced Docker Configuration\n\n\n\ndocker run --privileged --gpus '\"all\"' --shm-size 10g --rm -it \\\n --name axolotl --ipc=host \\\n --ulimit memlock=-1 --ulimit stack=67108864 \\\n --mount type=bind,src=\"${PWD}\",target=/workspace/axolotl \\\n -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \\\n axolotlai/axolotl:main-latest\n\n\nPlease refer to the Docker documentation for more information on the different Docker images that are available.",
"crumbs": [
"Getting Started",
"Installation"
@@ -1941,7 +1941,7 @@
"href": "docs/cli.html#command-reference",
"title": "Command Line Interface (CLI)",
"section": "Command Reference",
- "text": "Command Reference\n\nfetch\nDownloads example configurations and deepspeed configs to your local machine.\n# Get example YAML files\naxolotl fetch examples\n\n# Get deepspeed config files\naxolotl fetch deepspeed_configs\n\n# Specify custom destination\naxolotl fetch examples --dest path/to/folder\n\n\npreprocess\nPreprocesses and tokenizes your dataset before training. This is recommended for large datasets.\n# Basic preprocessing\naxolotl preprocess config.yml\n\n# Preprocessing with one GPU\nCUDA_VISIBLE_DEVICES=\"0\" axolotl preprocess config.yml\n\n# Debug mode to see processed examples\naxolotl preprocess config.yml --debug\n\n# Debug with limited examples\naxolotl preprocess config.yml --debug --debug-num-examples 5\nConfiguration options:\ndataset_prepared_path: Local folder for saving preprocessed data\npush_dataset_to_hub: HuggingFace repo to push preprocessed data (optional)\n\n\ntrain\nTrains or fine-tunes a model using the configuration specified in your YAML file.\n# Basic training\naxolotl train config.yml\n\n# Train and set/override specific options\naxolotl train config.yml \\\n --learning-rate 1e-4 \\\n --micro-batch-size 2 \\\n --num-epochs 3\n\n# Training without accelerate\naxolotl train config.yml --no-accelerate\n\n# Resume training from checkpoint\naxolotl train config.yml --resume-from-checkpoint path/to/checkpoint\nIt is possible to run sweeps over multiple hyperparameters by passing in a sweeps config.\n# Basic training with sweeps\naxolotl train config.yml --sweep path/to/sweep.yaml\nExample sweep config:\n_:\n # This section is for dependent variables we need to fix\n - load_in_8bit: false\n load_in_4bit: false\n adapter: lora\n - load_in_8bit: true\n load_in_4bit: false\n adapter: lora\n\n# These are independent variables\nlearning_rate: [0.0003, 0.0006]\nlora_r:\n - 16\n - 32\nlora_alpha:\n - 16\n - 32\n - 64\n\n\ninference\nRuns inference using your trained model in either CLI or Gradio interface mode.\n# CLI inference with LoRA\naxolotl inference config.yml --lora-model-dir=\"./outputs/lora-out\"\n\n# CLI inference with full model\naxolotl inference config.yml --base-model=\"./completed-model\"\n\n# Gradio web interface\naxolotl inference config.yml --gradio \\\n --lora-model-dir=\"./outputs/lora-out\"\n\n# Inference with input from file\ncat prompt.txt | axolotl inference config.yml \\\n --base-model=\"./completed-model\"\n\n\nmerge-lora\nMerges trained LoRA adapters into the base model.\n# Basic merge\naxolotl merge-lora config.yml\n\n# Specify LoRA directory (usually used with checkpoints)\naxolotl merge-lora config.yml --lora-model-dir=\"./lora-output/checkpoint-100\"\n\n# Merge using CPU (if out of GPU memory)\nCUDA_VISIBLE_DEVICES=\"\" axolotl merge-lora config.yml\nConfiguration options:\ngpu_memory_limit: Limit GPU memory usage\nlora_on_cpu: Load LoRA weights on CPU\n\n\nmerge-sharded-fsdp-weights\nMerges sharded FSDP model checkpoints into a single combined checkpoint.\n# Basic merge\naxolotl merge-sharded-fsdp-weights config.yml\n\n\nevaluate\nEvaluates a model’s performance (loss etc) on the train and eval datasets.\n# Basic evaluation\naxolotl evaluate config.yml\n\n\nlm-eval\nRuns LM Evaluation Harness on your model.\n# Basic evaluation\naxolotl lm-eval config.yml\nConfiguration options:\n# List of tasks to evaluate\nlm_eval_tasks:\n - arc_challenge\n - hellaswag\nlm_eval_batch_size: # Batch size for evaluation\noutput_dir: # Directory to save evaluation results\nSee LM Eval Harness for more details.",
+ "text": "Command Reference\n\nfetch\nDownloads example configurations and deepspeed configs to your local machine.\n# Get example YAML files\naxolotl fetch examples\n\n# Get deepspeed config files\naxolotl fetch deepspeed_configs\n\n# Specify custom destination\naxolotl fetch examples --dest path/to/folder\n\n\npreprocess\nPreprocesses and tokenizes your dataset before training. This is recommended for large datasets.\n# Basic preprocessing\naxolotl preprocess config.yml\n\n# Preprocessing with one GPU\nCUDA_VISIBLE_DEVICES=\"0\" axolotl preprocess config.yml\n\n# Debug mode to see processed examples\naxolotl preprocess config.yml --debug\n\n# Debug with limited examples\naxolotl preprocess config.yml --debug --debug-num-examples 5\nConfiguration options:\ndataset_prepared_path: Local folder for saving preprocessed data\npush_dataset_to_hub: HuggingFace repo to push preprocessed data (optional)\n\n\ntrain\nTrains or fine-tunes a model using the configuration specified in your YAML file.\n# Basic training\naxolotl train config.yml\n\n# Train and set/override specific options\naxolotl train config.yml \\\n --learning-rate 1e-4 \\\n --micro-batch-size 2 \\\n --num-epochs 3\n\n# Training without accelerate\naxolotl train config.yml --no-accelerate\n\n# Resume training from checkpoint\naxolotl train config.yml --resume-from-checkpoint path/to/checkpoint\nIt is possible to run sweeps over multiple hyperparameters by passing in a sweeps config.\n# Basic training with sweeps\naxolotl train config.yml --sweep path/to/sweep.yaml\nExample sweep config:\n_:\n # This section is for dependent variables we need to fix\n - load_in_8bit: false\n load_in_4bit: false\n adapter: lora\n - load_in_8bit: true\n load_in_4bit: false\n adapter: lora\n\n# These are independent variables\nlearning_rate: [0.0003, 0.0006]\nlora_r:\n - 16\n - 32\nlora_alpha:\n - 16\n - 32\n - 64\n\n\ninference\nRuns inference using your trained model in either CLI or Gradio interface mode.\n# CLI inference with LoRA\naxolotl inference config.yml --lora-model-dir=\"./outputs/lora-out\"\n\n# CLI inference with full model\naxolotl inference config.yml --base-model=\"./completed-model\"\n\n# Gradio web interface\naxolotl inference config.yml --gradio \\\n --lora-model-dir=\"./outputs/lora-out\"\n\n# Inference with input from file\ncat prompt.txt | axolotl inference config.yml \\\n --base-model=\"./completed-model\"\n\n\nmerge-lora\nMerges trained LoRA adapters into the base model.\n# Basic merge\naxolotl merge-lora config.yml\n\n# Specify LoRA directory (usually used with checkpoints)\naxolotl merge-lora config.yml --lora-model-dir=\"./lora-output/checkpoint-100\"\n\n# Merge using CPU (if out of GPU memory)\nCUDA_VISIBLE_DEVICES=\"\" axolotl merge-lora config.yml\nConfiguration options:\ngpu_memory_limit: Limit GPU memory usage\nlora_on_cpu: Load LoRA weights on CPU\n\n\nmerge-sharded-fsdp-weights\nMerges sharded FSDP model checkpoints into a single combined checkpoint.\n# Basic merge\naxolotl merge-sharded-fsdp-weights config.yml\n\n\nevaluate\nEvaluates a model’s performance (loss etc) on the train and eval datasets.\n# Basic evaluation\naxolotl evaluate config.yml\n\n\nlm-eval\nRuns LM Evaluation Harness on your model.\n# Basic evaluation\naxolotl lm-eval config.yml\nConfiguration options:\n# List of tasks to evaluate\nlm_eval_tasks:\n - arc_challenge\n - hellaswag\nlm_eval_batch_size: # Batch size for evaluation\noutput_dir: # Directory to save evaluation results\nSee LM Eval Harness for more details.\n\n\ndelinearize-llama4\nDelinearizes a Llama 4 linearized model into a regular HuggingFace Llama 4 model. This only works with the non-quantized linearized model.\naxolotl delinearize-llama4 --model path/to/model_dir --output path/to/output_dir\nThis would be necessary to use with other frameworks. If you have an adapter, merge it with the non-quantized linearized model before delinearizing.",
"crumbs": [
"Getting Started",
"Command Line Interface (CLI)"
diff --git a/sitemap.xml b/sitemap.xml
index 2438b5169..c54a2bce3 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,682 +2,682 @@
https://docs.axolotl.ai/examples/colab-notebooks/colab-axolotl-example.html
- 2025-04-24T04:40:00.905Z
+ 2025-04-24T05:03:52.797Z
https://docs.axolotl.ai/index.html
- 2025-04-24T04:40:00.917Z
+ 2025-04-24T05:03:52.809Z
https://docs.axolotl.ai/docs/rlhf.html
- 2025-04-24T04:40:00.905Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/unsloth.html
- 2025-04-24T04:40:00.905Z
+ 2025-04-24T05:03:52.797Z
https://docs.axolotl.ai/docs/dataset_preprocessing.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/input_output.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/dataset_loading.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/api/utils.collators.mamba.html
- 2025-04-24T04:40:29.433Z
+ 2025-04-24T05:04:21.627Z
https://docs.axolotl.ai/docs/api/utils.optimizers.adopt.html
- 2025-04-24T04:40:29.213Z
+ 2025-04-24T05:04:21.407Z
https://docs.axolotl.ai/docs/api/prompt_strategies.user_defined.html
- 2025-04-24T04:40:28.722Z
+ 2025-04-24T05:04:20.919Z
https://docs.axolotl.ai/docs/api/utils.chat_templates.html
- 2025-04-24T04:40:29.115Z
+ 2025-04-24T05:04:21.309Z
https://docs.axolotl.ai/docs/api/cli.merge_lora.html
- 2025-04-24T04:40:28.557Z
+ 2025-04-24T05:04:20.755Z
https://docs.axolotl.ai/docs/api/monkeypatch.multipack.html
- 2025-04-24T04:40:28.998Z
+ 2025-04-24T05:04:21.192Z
https://docs.axolotl.ai/docs/api/core.chat.format.shared.html
- 2025-04-24T04:40:28.434Z
+ 2025-04-24T05:04:20.632Z
https://docs.axolotl.ai/docs/api/utils.schemas.integrations.html
- 2025-04-24T04:40:29.286Z
+ 2025-04-24T05:04:21.479Z
https://docs.axolotl.ai/docs/api/utils.freeze.html
- 2025-04-24T04:40:29.140Z
+ 2025-04-24T05:04:21.334Z
https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_w_system.html
- 2025-04-24T04:40:28.714Z
+ 2025-04-24T05:04:20.910Z
https://docs.axolotl.ai/docs/api/monkeypatch.attention.mllama.html
- 2025-04-24T04:40:29.066Z
+ 2025-04-24T05:04:21.260Z
https://docs.axolotl.ai/docs/api/utils.schemas.model.html
- 2025-04-24T04:40:29.234Z
+ 2025-04-24T05:04:21.428Z
https://docs.axolotl.ai/docs/api/core.datasets.transforms.chat_builder.html
- 2025-04-24T04:40:28.447Z
+ 2025-04-24T05:04:20.645Z
https://docs.axolotl.ai/docs/api/monkeypatch.btlm_attn_hijack_flash.html
- 2025-04-24T04:40:29.041Z
+ 2025-04-24T05:04:21.235Z
https://docs.axolotl.ai/docs/api/models.mamba.modeling_mamba.html
- 2025-04-24T04:40:29.404Z
+ 2025-04-24T05:04:21.597Z
https://docs.axolotl.ai/docs/api/core.datasets.chat.html
- 2025-04-24T04:40:28.439Z
+ 2025-04-24T05:04:20.637Z
https://docs.axolotl.ai/docs/api/utils.model_shard_quant.html
- 2025-04-24T04:40:29.129Z
+ 2025-04-24T05:04:21.322Z
https://docs.axolotl.ai/docs/api/monkeypatch.llama_patch_multipack.html
- 2025-04-24T04:40:29.043Z
+ 2025-04-24T05:04:21.236Z
https://docs.axolotl.ai/docs/api/utils.lora_embeddings.html
- 2025-04-24T04:40:29.123Z
+ 2025-04-24T05:04:21.317Z
https://docs.axolotl.ai/docs/api/utils.schemas.peft.html
- 2025-04-24T04:40:29.265Z
+ 2025-04-24T05:04:21.459Z
https://docs.axolotl.ai/docs/api/core.chat.format.llama3x.html
- 2025-04-24T04:40:28.432Z
+ 2025-04-24T05:04:20.630Z
https://docs.axolotl.ai/docs/api/monkeypatch.mistral_attn_hijack_flash.html
- 2025-04-24T04:40:28.996Z
+ 2025-04-24T05:04:21.191Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chat_template.html
- 2025-04-24T04:40:28.776Z
+ 2025-04-24T05:04:20.971Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.user_defined.html
- 2025-04-24T04:40:28.799Z
+ 2025-04-24T05:04:20.994Z
https://docs.axolotl.ai/docs/api/utils.distributed.html
- 2025-04-24T04:40:29.202Z
+ 2025-04-24T05:04:21.396Z
https://docs.axolotl.ai/docs/api/prompt_strategies.pygmalion.html
- 2025-04-24T04:40:28.770Z
+ 2025-04-24T05:04:20.965Z
https://docs.axolotl.ai/docs/api/utils.dict.html
- 2025-04-24T04:40:29.205Z
+ 2025-04-24T05:04:21.399Z
https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_flash.html
- 2025-04-24T04:40:28.980Z
+ 2025-04-24T05:04:21.175Z
https://docs.axolotl.ai/docs/api/prompt_strategies.base.html
- 2025-04-24T04:40:28.672Z
+ 2025-04-24T05:04:20.869Z
https://docs.axolotl.ai/docs/api/monkeypatch.data.batch_dataset_fetcher.html
- 2025-04-24T04:40:29.068Z
+ 2025-04-24T05:04:21.262Z
https://docs.axolotl.ai/docs/api/kernels.swiglu.html
- 2025-04-24T04:40:28.945Z
+ 2025-04-24T05:04:21.141Z
https://docs.axolotl.ai/docs/api/core.chat.messages.html
- 2025-04-24T04:40:28.430Z
+ 2025-04-24T05:04:20.627Z
https://docs.axolotl.ai/docs/api/index.html
- 2025-04-24T04:40:28.158Z
+ 2025-04-24T05:04:20.355Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.zephyr.html
- 2025-04-24T04:40:28.797Z
+ 2025-04-24T05:04:20.993Z
https://docs.axolotl.ai/docs/api/convert.html
- 2025-04-24T04:40:28.252Z
+ 2025-04-24T05:04:20.448Z
https://docs.axolotl.ai/docs/api/utils.schemas.multimodal.html
- 2025-04-24T04:40:29.274Z
+ 2025-04-24T05:04:21.467Z
https://docs.axolotl.ai/docs/api/core.trainers.base.html
- 2025-04-24T04:40:28.643Z
+ 2025-04-24T05:04:20.840Z
https://docs.axolotl.ai/docs/api/evaluate.html
- 2025-04-24T04:40:28.230Z
+ 2025-04-24T05:04:20.426Z
https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_xformers.html
- 2025-04-24T04:40:28.982Z
+ 2025-04-24T05:04:21.176Z
https://docs.axolotl.ai/docs/api/kernels.quantize.html
- 2025-04-24T04:40:28.953Z
+ 2025-04-24T05:04:21.148Z
https://docs.axolotl.ai/docs/api/utils.callbacks.mlflow_.html
- 2025-04-24T04:40:29.462Z
+ 2025-04-24T05:04:21.655Z
https://docs.axolotl.ai/docs/api/utils.callbacks.profiler.html
- 2025-04-24T04:40:29.456Z
+ 2025-04-24T05:04:21.650Z
https://docs.axolotl.ai/docs/api/core.trainers.dpo.trainer.html
- 2025-04-24T04:40:28.667Z
+ 2025-04-24T05:04:20.864Z
https://docs.axolotl.ai/docs/api/cli.vllm_serve.html
- 2025-04-24T04:40:28.619Z
+ 2025-04-24T05:04:20.816Z
https://docs.axolotl.ai/docs/api/train.html
- 2025-04-24T04:40:28.219Z
+ 2025-04-24T05:04:20.416Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chatml.html
- 2025-04-24T04:40:28.796Z
+ 2025-04-24T05:04:20.991Z
https://docs.axolotl.ai/docs/api/utils.schemas.trl.html
- 2025-04-24T04:40:29.268Z
+ 2025-04-24T05:04:21.462Z
https://docs.axolotl.ai/docs/api/kernels.geglu.html
- 2025-04-24T04:40:28.935Z
+ 2025-04-24T05:04:21.131Z
https://docs.axolotl.ai/docs/api/utils.bench.html
- 2025-04-24T04:40:29.132Z
+ 2025-04-24T05:04:21.326Z
https://docs.axolotl.ai/docs/api/monkeypatch.transformers_fa_utils.html
- 2025-04-24T04:40:29.058Z
+ 2025-04-24T05:04:21.252Z
https://docs.axolotl.ai/docs/api/integrations.liger.args.html
- 2025-04-24T04:40:29.377Z
+ 2025-04-24T05:04:21.570Z
https://docs.axolotl.ai/docs/api/core.trainer_builder.html
- 2025-04-24T04:40:28.319Z
+ 2025-04-24T05:04:20.509Z
https://docs.axolotl.ai/docs/api/utils.schemas.utils.html
- 2025-04-24T04:40:29.298Z
+ 2025-04-24T05:04:21.492Z
https://docs.axolotl.ai/docs/api/kernels.lora.html
- 2025-04-24T04:40:28.925Z
+ 2025-04-24T05:04:21.120Z
https://docs.axolotl.ai/docs/api/prompt_strategies.bradley_terry.llama3.html
- 2025-04-24T04:40:28.842Z
+ 2025-04-24T05:04:21.037Z
https://docs.axolotl.ai/docs/api/core.trainers.grpo.trainer.html
- 2025-04-24T04:40:28.670Z
+ 2025-04-24T05:04:20.867Z
https://docs.axolotl.ai/docs/api/prompt_strategies.messages.chat.html
- 2025-04-24T04:40:28.774Z
+ 2025-04-24T05:04:20.970Z
https://docs.axolotl.ai/docs/api/utils.tokenization.html
- 2025-04-24T04:40:29.105Z
+ 2025-04-24T05:04:21.299Z
https://docs.axolotl.ai/docs/api/prompt_strategies.chat_template.html
- 2025-04-24T04:40:28.687Z
+ 2025-04-24T05:04:20.883Z
https://docs.axolotl.ai/docs/api/prompt_strategies.stepwise_supervised.html
- 2025-04-24T04:40:28.752Z
+ 2025-04-24T05:04:20.948Z
https://docs.axolotl.ai/docs/api/utils.samplers.multipack.html
- 2025-04-24T04:40:29.446Z
+ 2025-04-24T05:04:21.640Z
https://docs.axolotl.ai/docs/api/cli.args.html
- 2025-04-24T04:40:28.511Z
+ 2025-04-24T05:04:20.709Z
https://docs.axolotl.ai/docs/api/utils.callbacks.perplexity.html
- 2025-04-24T04:40:29.453Z
+ 2025-04-24T05:04:21.646Z
https://docs.axolotl.ai/docs/api/utils.gradient_checkpointing.unsloth.html
- 2025-04-24T04:40:29.219Z
+ 2025-04-24T05:04:21.413Z
https://docs.axolotl.ai/docs/mac.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/config.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.792Z
https://docs.axolotl.ai/docs/multimodal.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/lr_groups.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/dataset-formats/index.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/dataset-formats/stepwise_supervised.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/dataset-formats/pretraining.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/multi-node.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/sequence_parallelism.html
- 2025-04-24T04:40:00.905Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/batch_vs_grad.html
- 2025-04-24T04:40:00.900Z
+ 2025-04-24T05:03:52.792Z
https://docs.axolotl.ai/docs/amd_hpc.html
- 2025-04-24T04:40:00.900Z
+ 2025-04-24T05:03:52.792Z
https://docs.axolotl.ai/docs/faq.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/custom_integrations.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.792Z
https://docs.axolotl.ai/src/axolotl/integrations/LICENSE.html
- 2025-04-24T04:40:00.920Z
+ 2025-04-24T05:03:52.812Z
https://docs.axolotl.ai/TODO.html
- 2025-04-24T04:40:00.899Z
+ 2025-04-24T05:03:52.791Z
https://docs.axolotl.ai/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html
- 2025-04-24T04:40:00.921Z
+ 2025-04-24T05:03:52.813Z
https://docs.axolotl.ai/docs/getting-started.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/multipack.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/multi-gpu.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/installation.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/cli.html
- 2025-04-24T04:40:00.900Z
+ 2025-04-24T05:03:52.792Z
https://docs.axolotl.ai/docs/dataset-formats/inst_tune.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/dataset-formats/tokenized.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/dataset-formats/conversation.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/dataset-formats/template_free.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/reward_modelling.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/lora_optims.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/nccl.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/api/logging_config.html
- 2025-04-24T04:40:28.304Z
+ 2025-04-24T05:04:20.494Z
https://docs.axolotl.ai/docs/api/utils.trainer.html
- 2025-04-24T04:40:29.157Z
+ 2025-04-24T05:04:21.350Z
https://docs.axolotl.ai/docs/api/monkeypatch.unsloth_.html
- 2025-04-24T04:40:29.059Z
+ 2025-04-24T05:04:21.253Z
https://docs.axolotl.ai/docs/api/cli.evaluate.html
- 2025-04-24T04:40:28.494Z
+ 2025-04-24T05:04:20.692Z
https://docs.axolotl.ai/docs/api/kernels.utils.html
- 2025-04-24T04:40:28.954Z
+ 2025-04-24T05:04:21.150Z
https://docs.axolotl.ai/docs/api/datasets.html
- 2025-04-24T04:40:28.237Z
+ 2025-04-24T05:04:20.434Z
https://docs.axolotl.ai/docs/api/utils.models.html
- 2025-04-24T04:40:29.098Z
+ 2025-04-24T05:04:21.292Z
https://docs.axolotl.ai/docs/api/prompt_strategies.kto.llama3.html
- 2025-04-24T04:40:28.809Z
+ 2025-04-24T05:04:21.004Z
https://docs.axolotl.ai/docs/api/cli.preprocess.html
- 2025-04-24T04:40:28.577Z
+ 2025-04-24T05:04:20.775Z
https://docs.axolotl.ai/docs/api/cli.merge_sharded_fsdp_weights.html
- 2025-04-24T04:40:28.569Z
+ 2025-04-24T05:04:20.767Z
https://docs.axolotl.ai/docs/api/integrations.base.html
- 2025-04-24T04:40:29.362Z
+ 2025-04-24T05:04:21.555Z
https://docs.axolotl.ai/docs/api/prompt_strategies.orpo.chat_template.html
- 2025-04-24T04:40:28.839Z
+ 2025-04-24T05:04:21.034Z
https://docs.axolotl.ai/docs/api/utils.schemas.enums.html
- 2025-04-24T04:40:29.292Z
+ 2025-04-24T05:04:21.486Z
https://docs.axolotl.ai/docs/api/utils.callbacks.comet_.html
- 2025-04-24T04:40:29.465Z
+ 2025-04-24T05:04:21.658Z
https://docs.axolotl.ai/docs/api/prompt_strategies.input_output.html
- 2025-04-24T04:40:28.747Z
+ 2025-04-24T05:04:20.943Z
https://docs.axolotl.ai/docs/api/utils.schedulers.html
- 2025-04-24T04:40:29.181Z
+ 2025-04-24T05:04:21.375Z
https://docs.axolotl.ai/docs/api/cli.cloud.base.html
- 2025-04-24T04:40:28.623Z
+ 2025-04-24T05:04:20.820Z
https://docs.axolotl.ai/docs/api/cli.utils.html
- 2025-04-24T04:40:28.615Z
+ 2025-04-24T05:04:20.812Z
https://docs.axolotl.ai/docs/api/monkeypatch.lora_kernels.html
- 2025-04-24T04:40:29.032Z
+ 2025-04-24T05:04:21.226Z
https://docs.axolotl.ai/docs/api/prompt_strategies.completion.html
- 2025-04-24T04:40:28.741Z
+ 2025-04-24T05:04:20.937Z
https://docs.axolotl.ai/docs/api/utils.schemas.config.html
- 2025-04-24T04:40:29.227Z
+ 2025-04-24T05:04:21.421Z
https://docs.axolotl.ai/docs/api/monkeypatch.llama_expand_mask.html
- 2025-04-24T04:40:29.006Z
+ 2025-04-24T05:04:21.201Z
https://docs.axolotl.ai/docs/api/cli.config.html
- 2025-04-24T04:40:28.535Z
+ 2025-04-24T05:04:20.733Z
https://docs.axolotl.ai/docs/api/utils.collators.core.html
- 2025-04-24T04:40:29.405Z
+ 2025-04-24T05:04:21.599Z
https://docs.axolotl.ai/docs/api/utils.data.sft.html
- 2025-04-24T04:40:29.216Z
+ 2025-04-24T05:04:21.410Z
https://docs.axolotl.ai/docs/api/integrations.spectrum.args.html
- 2025-04-24T04:40:29.383Z
+ 2025-04-24T05:04:21.577Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.llama3.html
- 2025-04-24T04:40:28.786Z
+ 2025-04-24T05:04:20.981Z
https://docs.axolotl.ai/docs/api/cli.inference.html
- 2025-04-24T04:40:28.549Z
+ 2025-04-24T05:04:20.747Z
https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_instruct.html
- 2025-04-24T04:40:28.702Z
+ 2025-04-24T05:04:20.898Z
https://docs.axolotl.ai/docs/api/utils.collators.batching.html
- 2025-04-24T04:40:29.430Z
+ 2025-04-24T05:04:21.623Z
https://docs.axolotl.ai/docs/api/utils.schemas.datasets.html
- 2025-04-24T04:40:29.256Z
+ 2025-04-24T05:04:21.450Z
https://docs.axolotl.ai/docs/api/utils.data.pretraining.html
- 2025-04-24T04:40:29.214Z
+ 2025-04-24T05:04:21.408Z
https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.passthrough.html
- 2025-04-24T04:40:28.800Z
+ 2025-04-24T05:04:20.996Z
https://docs.axolotl.ai/docs/api/monkeypatch.utils.html
- 2025-04-24T04:40:29.040Z
+ 2025-04-24T05:04:21.234Z
https://docs.axolotl.ai/docs/api/utils.callbacks.lisa.html
- 2025-04-24T04:40:29.458Z
+ 2025-04-24T05:04:21.651Z
https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_chat.html
- 2025-04-24T04:40:28.701Z
+ 2025-04-24T05:04:20.897Z
https://docs.axolotl.ai/docs/api/prompt_strategies.kto.user_defined.html
- 2025-04-24T04:40:28.818Z
+ 2025-04-24T05:04:21.013Z
https://docs.axolotl.ai/docs/api/core.training_args.html
- 2025-04-24T04:40:28.407Z
+ 2025-04-24T05:04:20.604Z
https://docs.axolotl.ai/docs/api/utils.collators.mm_chat.html
- 2025-04-24T04:40:29.438Z
+ 2025-04-24T05:04:21.631Z
https://docs.axolotl.ai/docs/api/common.architectures.html
- 2025-04-24T04:40:29.385Z
+ 2025-04-24T05:04:21.578Z
https://docs.axolotl.ai/docs/api/integrations.kd.trainer.html
- 2025-04-24T04:40:29.374Z
+ 2025-04-24T05:04:21.567Z
https://docs.axolotl.ai/docs/api/prompt_strategies.llama2_chat.html
- 2025-04-24T04:40:28.735Z
+ 2025-04-24T05:04:20.931Z
https://docs.axolotl.ai/docs/api/core.trainers.trl.html
- 2025-04-24T04:40:28.661Z
+ 2025-04-24T05:04:20.857Z
https://docs.axolotl.ai/docs/api/prompt_strategies.orcamini.html
- 2025-04-24T04:40:28.763Z
+ 2025-04-24T05:04:20.958Z
https://docs.axolotl.ai/docs/api/integrations.cut_cross_entropy.args.html
- 2025-04-24T04:40:29.365Z
+ 2025-04-24T05:04:21.558Z
https://docs.axolotl.ai/docs/api/utils.lora.html
- 2025-04-24T04:40:29.120Z
+ 2025-04-24T05:04:21.314Z
https://docs.axolotl.ai/docs/api/cli.sweeps.html
- 2025-04-24T04:40:28.583Z
+ 2025-04-24T05:04:20.781Z
https://docs.axolotl.ai/docs/api/core.chat.format.chatml.html
- 2025-04-24T04:40:28.431Z
+ 2025-04-24T05:04:20.629Z
https://docs.axolotl.ai/docs/api/common.const.html
- 2025-04-24T04:40:29.386Z
+ 2025-04-24T05:04:21.580Z
https://docs.axolotl.ai/docs/api/prompt_strategies.metharme.html
- 2025-04-24T04:40:28.759Z
+ 2025-04-24T05:04:20.955Z
https://docs.axolotl.ai/docs/api/monkeypatch.stablelm_attn_hijack_flash.html
- 2025-04-24T04:40:29.048Z
+ 2025-04-24T05:04:21.242Z
https://docs.axolotl.ai/docs/api/integrations.grokfast.optimizer.html
- 2025-04-24T04:40:29.366Z
+ 2025-04-24T05:04:21.559Z
https://docs.axolotl.ai/docs/api/cli.checks.html
- 2025-04-24T04:40:28.517Z
+ 2025-04-24T05:04:20.715Z
https://docs.axolotl.ai/docs/api/common.datasets.html
- 2025-04-24T04:40:29.403Z
+ 2025-04-24T05:04:21.596Z
https://docs.axolotl.ai/docs/api/integrations.lm_eval.args.html
- 2025-04-24T04:40:29.380Z
+ 2025-04-24T05:04:21.574Z
https://docs.axolotl.ai/docs/api/monkeypatch.mixtral.html
- 2025-04-24T04:40:29.069Z
+ 2025-04-24T05:04:21.263Z
https://docs.axolotl.ai/docs/api/prompt_strategies.kto.chatml.html
- 2025-04-24T04:40:28.817Z
+ 2025-04-24T05:04:21.012Z
https://docs.axolotl.ai/docs/api/cli.train.html
- 2025-04-24T04:40:28.486Z
+ 2025-04-24T05:04:20.684Z
https://docs.axolotl.ai/docs/api/prompt_tokenizers.html
- 2025-04-24T04:40:28.299Z
+ 2025-04-24T05:04:20.489Z
https://docs.axolotl.ai/docs/api/utils.schemas.training.html
- 2025-04-24T04:40:29.239Z
+ 2025-04-24T05:04:21.433Z
https://docs.axolotl.ai/docs/api/monkeypatch.relora.html
- 2025-04-24T04:40:29.005Z
+ 2025-04-24T05:04:21.199Z
https://docs.axolotl.ai/docs/api/cli.cloud.modal_.html
- 2025-04-24T04:40:28.629Z
+ 2025-04-24T05:04:20.826Z
https://docs.axolotl.ai/docs/api/cli.main.html
- 2025-04-24T04:40:28.478Z
+ 2025-04-24T05:04:20.676Z
https://docs.axolotl.ai/docs/api/monkeypatch.trainer_fsdp_optim.html
- 2025-04-24T04:40:29.051Z
+ 2025-04-24T05:04:21.245Z
https://docs.axolotl.ai/docs/fsdp_qlora.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/debugging.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/ray-integration.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/docker.html
- 2025-04-24T04:40:00.901Z
+ 2025-04-24T05:03:52.793Z
https://docs.axolotl.ai/docs/inference.html
- 2025-04-24T04:40:00.904Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/docs/torchao.html
- 2025-04-24T04:40:00.905Z
+ 2025-04-24T05:03:52.796Z
https://docs.axolotl.ai/FAQS.html
- 2025-04-24T04:40:00.899Z
+ 2025-04-24T05:03:52.791Z