tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-05-01 16:27:24 +00:00
parent 7cec02149d
commit c6274d0582
4 changed files with 283 additions and 174 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.nojekyll
View File

@@ -1 +1 @@
7eacb345
98a26bb0

104
docs/custom_integrations.html
View File

@@ -480,6 +480,15 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
<li><a href="#usage-5" id="toc-usage-5" class="nav-link" data-scroll-target="#usage-5">Usage</a></li>
<li><a href="#citation-4" id="toc-citation-4" class="nav-link" data-scroll-target="#citation-4">Citation</a></li>
</ul></li>
<li><a href="#llmcompressor" id="toc-llmcompressor" class="nav-link" data-scroll-target="#llmcompressor">LLMCompressor</a>
<ul class="collapse">
<li><a href="#requirements-1" id="toc-requirements-1" class="nav-link" data-scroll-target="#requirements-1">Requirements</a></li>
<li><a href="#usage-6" id="toc-usage-6" class="nav-link" data-scroll-target="#usage-6">Usage</a></li>
<li><a href="#storage-optimization-with-save_compressed" id="toc-storage-optimization-with-save_compressed" class="nav-link" data-scroll-target="#storage-optimization-with-save_compressed">Storage Optimization with save_compressed</a></li>
<li><a href="#example-config" id="toc-example-config" class="nav-link" data-scroll-target="#example-config">Example Config</a></li>
<li><a href="#inference-with-vllm" id="toc-inference-with-vllm" class="nav-link" data-scroll-target="#inference-with-vllm">Inference with vLLM</a></li>
<li><a href="#learn-more" id="toc-learn-more" class="nav-link" data-scroll-target="#learn-more">Learn More</a></li>
</ul></li>
<li><a href="#adding-a-new-integration" id="toc-adding-a-new-integration" class="nav-link" data-scroll-target="#adding-a-new-integration">Adding a new integration</a></li>
</ul>
</nav>
@@ -742,6 +751,95 @@ By identifying the top n% of layers with the highest SNR, you can optimize train
<p>Please see reference <a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl/integrations/spectrum">here</a></p>
</section>
</section>
<section id="llmcompressor" class="level2">
<h2 class="anchored" data-anchor-id="llmcompressor">LLMCompressor</h2>
<p>Fine-tune sparsified models in Axolotl using Neural Magic’s <a href="https://github.com/vllm-project/llm-compressor">LLMCompressor</a>.</p>
<p>This integration enables fine-tuning of models sparsified using LLMCompressor within the Axolotl training framework. By combining LLMCompressor’s model compression capabilities with Axolotl’s distributed training pipelines, users can efficiently fine-tune sparse models at scale.</p>
<p>It uses Axolotl’s plugin system to hook into the fine-tuning flows while maintaining sparsity throughout training.</p>
<hr>
<section id="requirements-1" class="level3">
<h3 class="anchored" data-anchor-id="requirements-1">Requirements</h3>
<ul>
<li><p>Axolotl with <code>llmcompressor</code> extras:</p>
<div class="sourceCode" id="cb14"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ex">pip</span> install <span class="st">"axolotl[llmcompressor]"</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div></li>
<li><p>Requires <code>llmcompressor &gt;= 0.5.1</code></p></li>
</ul>
<p>This will install all necessary dependencies to fine-tune sparsified models using the integration.</p>
<hr>
</section>
<section id="usage-6" class="level3">
<h3 class="anchored" data-anchor-id="usage-6">Usage</h3>
<p>To enable sparse fine-tuning with this integration, include the plugin in your Axolotl config:</p>
<div class="sourceCode" id="cb15"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plugins</span><span class="kw">:</span></span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> axolotl.integrations.llm_compressor.LLMCompressorPlugin</span></span>
<span id="cb15-3"><a href="#cb15-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-4"><a href="#cb15-4" aria-hidden="true" tabindex="-1"></a><span class="fu">llmcompressor</span><span class="kw">:</span></span>
<span id="cb15-5"><a href="#cb15-5" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">recipe</span><span class="kw">:</span></span>
<span id="cb15-6"><a href="#cb15-6" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">finetuning_stage</span><span class="kw">:</span></span>
<span id="cb15-7"><a href="#cb15-7" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">finetuning_modifiers</span><span class="kw">:</span></span>
<span id="cb15-8"><a href="#cb15-8" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">ConstantPruningModifier</span><span class="kw">:</span></span>
<span id="cb15-9"><a href="#cb15-9" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">targets</span><span class="kw">:</span><span class="at"> </span><span class="kw">[</span></span>
<span id="cb15-10"><a href="#cb15-10" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*q_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-11"><a href="#cb15-11" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*k_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-12"><a href="#cb15-12" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*v_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-13"><a href="#cb15-13" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*o_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-14"><a href="#cb15-14" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*gate_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-15"><a href="#cb15-15" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*up_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-16"><a href="#cb15-16" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="st">'re:.*down_proj.weight'</span><span class="kw">,</span></span>
<span id="cb15-17"><a href="#cb15-17" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">]</span></span>
<span id="cb15-18"><a href="#cb15-18" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">start</span><span class="kw">:</span><span class="at"> </span><span class="dv">0</span></span>
<span id="cb15-19"><a href="#cb15-19" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">save_compressed</span><span class="kw">:</span><span class="at"> </span><span class="ch">true</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>This plugin <strong>does not apply pruning or sparsification itself</strong> — it is intended for <strong>fine-tuning models that have already been sparsified</strong>.</p>
<p>Pre-sparsified checkpoints can be:
- Generated using <a href="https://github.com/vllm-project/llm-compressor">LLMCompressor</a>
- Downloaded from <a href="https://huggingface.co/neuralmagic">Neural Magic’s Hugging Face page</a>
- Any custom LLM with compatible sparsity patterns that you’ve created yourself</p>
<p>To learn more about writing and customizing LLMCompressor recipes, refer to the official documentation:
<a href="https://github.com/vllm-project/llm-compressor/blob/main/README.md">https://github.com/vllm-project/llm-compressor/blob/main/README.md</a></p>
</section>
<section id="storage-optimization-with-save_compressed" class="level3">
<h3 class="anchored" data-anchor-id="storage-optimization-with-save_compressed">Storage Optimization with save_compressed</h3>
<p>Setting <code>save_compressed: true</code> in your configuration enables saving models in a compressed format, which:
- Reduces disk space usage by approximately 40%
- Maintains compatibility with vLLM for accelerated inference
- Maintains compatibility with llmcompressor for further optimization (example: quantization)</p>
<p>This option is highly recommended when working with sparse models to maximize the benefits of model compression.</p>
</section>
<section id="example-config" class="level3">
<h3 class="anchored" data-anchor-id="example-config">Example Config</h3>
<p>See <a href="examples/llama-3/sparse-finetuning.yaml"><code>examples/llama-3/sparse-finetuning.yaml</code></a> for a complete example.</p>
<hr>
</section>
<section id="inference-with-vllm" class="level3">
<h3 class="anchored" data-anchor-id="inference-with-vllm">Inference with vLLM</h3>
<p>After fine-tuning your sparse model, you can leverage vLLM for efficient inference.
You can also use LLMCompressor to apply additional quantization to your fine-tuned
sparse model before inference for even greater performance benefits.:</p>
<div class="sourceCode" id="cb16"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> vllm <span class="im">import</span> LLM, SamplingParams</span>
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a>prompts <span class="op">=</span> [</span>
<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> <span class="st">"Hello, my name is"</span>,</span>
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a> <span class="st">"The president of the United States is"</span>,</span>
<span id="cb16-6"><a href="#cb16-6" aria-hidden="true" tabindex="-1"></a> <span class="st">"The capital of France is"</span>,</span>
<span id="cb16-7"><a href="#cb16-7" aria-hidden="true" tabindex="-1"></a> <span class="st">"The future of AI is"</span>,</span>
<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a>]</span>
<span id="cb16-9"><a href="#cb16-9" aria-hidden="true" tabindex="-1"></a>sampling_params <span class="op">=</span> SamplingParams(temperature<span class="op">=</span><span class="fl">0.8</span>, top_p<span class="op">=</span><span class="fl">0.95</span>)</span>
<span id="cb16-10"><a href="#cb16-10" aria-hidden="true" tabindex="-1"></a>llm <span class="op">=</span> LLM(<span class="st">"path/to/your/sparse/model"</span>)</span>
<span id="cb16-11"><a href="#cb16-11" aria-hidden="true" tabindex="-1"></a>outputs <span class="op">=</span> llm.generate(prompts, sampling_params)</span>
<span id="cb16-12"><a href="#cb16-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-13"><a href="#cb16-13" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> output <span class="kw">in</span> outputs:</span>
<span id="cb16-14"><a href="#cb16-14" aria-hidden="true" tabindex="-1"></a> prompt <span class="op">=</span> output.prompt</span>
<span id="cb16-15"><a href="#cb16-15" aria-hidden="true" tabindex="-1"></a> generated_text <span class="op">=</span> output.outputs[<span class="dv">0</span>].text</span>
<span id="cb16-16"><a href="#cb16-16" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(<span class="ss">f"Prompt: </span><span class="sc">{</span>prompt<span class="sc">!r}</span><span class="ss">, Generated text: </span><span class="sc">{</span>generated_text<span class="sc">!r}</span><span class="ss">"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>For more details on vLLM’s capabilities and advanced configuration options, see the <a href="https://docs.vllm.ai/">official vLLM documentation</a>.</p>
</section>
<section id="learn-more" class="level3">
<h3 class="anchored" data-anchor-id="learn-more">Learn More</h3>
<p>For details on available sparsity and quantization schemes, fine-tuning recipes, and usage examples, visit the official LLMCompressor repository:</p>
<p><a href="https://github.com/vllm-project/llm-compressor">https://github.com/vllm-project/llm-compressor</a></p>
<p>Please see reference <a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl/integrations/llm_compressor">here</a></p>
</section>
</section>
<section id="adding-a-new-integration" class="level2">
<h2 class="anchored" data-anchor-id="adding-a-new-integration">Adding a new integration</h2>
<p>Plugins can be used to customize the behavior of the training pipeline through <a href="https://en.wikipedia.org/wiki/Hooking">hooks</a>. See <a href="https://github.com/axolotl-ai-cloud/axolotl/blob/main/src/axolotl/integrations/base.py"><code>axolotl.integrations.BasePlugin</code></a> for the possible hooks.</p>
@@ -782,10 +880,10 @@ Warning
</div>
<div class="callout-body-container callout-body">
<p>If you could not load your integration, please ensure you are pip installing in editable mode.</p>
<div class="sourceCode" id="cb14"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ex">pip</span> install <span class="at">-e</span> .</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb17"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="ex">pip</span> install <span class="at">-e</span> .</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>and correctly spelled the integration name in the config file.</p>
<div class="sourceCode" id="cb15"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plugins</span><span class="kw">:</span></span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> axolotl.integrations.your_integration_name.YourIntegrationPlugin</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb18"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb18-1"><a href="#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="fu">plugins</span><span class="kw">:</span></span>
<span id="cb18-2"><a href="#cb18-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> axolotl.integrations.your_integration_name.YourIntegrationPlugin</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
</div>
<div class="callout callout-style-default callout-note callout-titled">

11
search.json
View File

@@ -1622,6 +1622,17 @@
"Custom Integrations"
]
},
{
"objectID": "docs/custom_integrations.html#llmcompressor",
"href": "docs/custom_integrations.html#llmcompressor",
"title": "Custom Integrations",
"section": "LLMCompressor",
"text": "LLMCompressor\nFine-tune sparsified models in Axolotl using Neural Magic’s LLMCompressor.\nThis integration enables fine-tuning of models sparsified using LLMCompressor within the Axolotl training framework. By combining LLMCompressor’s model compression capabilities with Axolotl’s distributed training pipelines, users can efficiently fine-tune sparse models at scale.\nIt uses Axolotl’s plugin system to hook into the fine-tuning flows while maintaining sparsity throughout training.\n\n\nRequirements\n\nAxolotl with llmcompressor extras:\npip install \"axolotl[llmcompressor]\"\nRequires llmcompressor &gt;= 0.5.1\n\nThis will install all necessary dependencies to fine-tune sparsified models using the integration.\n\n\n\nUsage\nTo enable sparse fine-tuning with this integration, include the plugin in your Axolotl config:\nplugins:\n - axolotl.integrations.llm_compressor.LLMCompressorPlugin\n\nllmcompressor:\n recipe:\n finetuning_stage:\n finetuning_modifiers:\n ConstantPruningModifier:\n targets: [\n 're:.*q_proj.weight',\n 're:.*k_proj.weight',\n 're:.*v_proj.weight',\n 're:.*o_proj.weight',\n 're:.*gate_proj.weight',\n 're:.*up_proj.weight',\n 're:.*down_proj.weight',\n ]\n start: 0\n save_compressed: true\nThis plugin does not apply pruning or sparsification itself — it is intended for fine-tuning models that have already been sparsified.\nPre-sparsified checkpoints can be:\n- Generated using LLMCompressor\n- Downloaded from Neural Magic’s Hugging Face page\n- Any custom LLM with compatible sparsity patterns that you’ve created yourself\nTo learn more about writing and customizing LLMCompressor recipes, refer to the official documentation:\nhttps://github.com/vllm-project/llm-compressor/blob/main/README.md\n\n\nStorage Optimization with save_compressed\nSetting save_compressed: true in your configuration enables saving models in a compressed format, which:\n- Reduces disk space usage by approximately 40%\n- Maintains compatibility with vLLM for accelerated inference\n- Maintains compatibility with llmcompressor for further optimization (example: quantization)\nThis option is highly recommended when working with sparse models to maximize the benefits of model compression.\n\n\nExample Config\nSee examples/llama-3/sparse-finetuning.yaml for a complete example.\n\n\n\nInference with vLLM\nAfter fine-tuning your sparse model, you can leverage vLLM for efficient inference.\nYou can also use LLMCompressor to apply additional quantization to your fine-tuned\nsparse model before inference for even greater performance benefits.:\nfrom vllm import LLM, SamplingParams\n\nprompts = [\n \"Hello, my name is\",\n \"The president of the United States is\",\n \"The capital of France is\",\n \"The future of AI is\",\n]\nsampling_params = SamplingParams(temperature=0.8, top_p=0.95)\nllm = LLM(\"path/to/your/sparse/model\")\noutputs = llm.generate(prompts, sampling_params)\n\nfor output in outputs:\n prompt = output.prompt\n generated_text = output.outputs[0].text\n print(f\"Prompt: {prompt!r}, Generated text: {generated_text!r}\")\nFor more details on vLLM’s capabilities and advanced configuration options, see the official vLLM documentation.\n\n\nLearn More\nFor details on available sparsity and quantization schemes, fine-tuning recipes, and usage examples, visit the official LLMCompressor repository:\nhttps://github.com/vllm-project/llm-compressor\nPlease see reference here",
"crumbs": [
"Advanced Features",
"Custom Integrations"
]
},
{
"objectID": "docs/custom_integrations.html#adding-a-new-integration",
"href": "docs/custom_integrations.html#adding-a-new-integration",

340
sitemap.xml
View File

@@ -2,682 +2,682 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://docs.axolotl.ai/examples/colab-notebooks/colab-axolotl-example.html</loc>
<lastmod>2025-05-01T13:41:42.488Z</lastmod>
<lastmod>2025-05-01T16:25:28.202Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/index.html</loc>
<lastmod>2025-05-01T13:41:42.500Z</lastmod>
<lastmod>2025-05-01T16:25:28.213Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/rlhf.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/unsloth.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset_preprocessing.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/input_output.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset_loading.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.collators.mamba.html</loc>
<lastmod>2025-05-01T13:42:10.206Z</lastmod>
<lastmod>2025-05-01T16:25:57.563Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.optimizers.adopt.html</loc>
<lastmod>2025-05-01T13:42:09.983Z</lastmod>
<lastmod>2025-05-01T16:25:57.336Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.user_defined.html</loc>
<lastmod>2025-05-01T13:42:09.499Z</lastmod>
<lastmod>2025-05-01T16:25:56.846Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.chat_templates.html</loc>
<lastmod>2025-05-01T13:42:09.888Z</lastmod>
<lastmod>2025-05-01T16:25:57.238Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.merge_lora.html</loc>
<lastmod>2025-05-01T13:42:09.329Z</lastmod>
<lastmod>2025-05-01T16:25:56.675Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.multipack.html</loc>
<lastmod>2025-05-01T13:42:09.772Z</lastmod>
<lastmod>2025-05-01T16:25:57.121Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.shared.html</loc>
<lastmod>2025-05-01T13:42:09.207Z</lastmod>
<lastmod>2025-05-01T16:25:56.551Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.integrations.html</loc>
<lastmod>2025-05-01T13:42:10.056Z</lastmod>
<lastmod>2025-05-01T16:25:57.409Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.freeze.html</loc>
<lastmod>2025-05-01T13:42:09.913Z</lastmod>
<lastmod>2025-05-01T16:25:57.264Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_w_system.html</loc>
<lastmod>2025-05-01T13:42:09.491Z</lastmod>
<lastmod>2025-05-01T16:25:56.838Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.attention.mllama.html</loc>
<lastmod>2025-05-01T13:42:09.839Z</lastmod>
<lastmod>2025-05-01T16:25:57.189Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.model.html</loc>
<lastmod>2025-05-01T13:42:10.004Z</lastmod>
<lastmod>2025-05-01T16:25:57.357Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.datasets.transforms.chat_builder.html</loc>
<lastmod>2025-05-01T13:42:09.219Z</lastmod>
<lastmod>2025-05-01T16:25:56.564Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.btlm_attn_hijack_flash.html</loc>
<lastmod>2025-05-01T13:42:09.814Z</lastmod>
<lastmod>2025-05-01T16:25:57.164Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/models.mamba.modeling_mamba.html</loc>
<lastmod>2025-05-01T13:42:10.182Z</lastmod>
<lastmod>2025-05-01T16:25:57.538Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.datasets.chat.html</loc>
<lastmod>2025-05-01T13:42:09.212Z</lastmod>
<lastmod>2025-05-01T16:25:56.557Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.model_shard_quant.html</loc>
<lastmod>2025-05-01T13:42:09.901Z</lastmod>
<lastmod>2025-05-01T16:25:57.252Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_patch_multipack.html</loc>
<lastmod>2025-05-01T13:42:09.816Z</lastmod>
<lastmod>2025-05-01T16:25:57.166Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.lora_embeddings.html</loc>
<lastmod>2025-05-01T13:42:09.896Z</lastmod>
<lastmod>2025-05-01T16:25:57.247Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.peft.html</loc>
<lastmod>2025-05-01T13:42:10.035Z</lastmod>
<lastmod>2025-05-01T16:25:57.388Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.llama3x.html</loc>
<lastmod>2025-05-01T13:42:09.205Z</lastmod>
<lastmod>2025-05-01T16:25:56.550Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.mistral_attn_hijack_flash.html</loc>
<lastmod>2025-05-01T13:42:09.770Z</lastmod>
<lastmod>2025-05-01T16:25:57.120Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chat_template.html</loc>
<lastmod>2025-05-01T13:42:09.551Z</lastmod>
<lastmod>2025-05-01T16:25:56.899Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.user_defined.html</loc>
<lastmod>2025-05-01T13:42:09.575Z</lastmod>
<lastmod>2025-05-01T16:25:56.923Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.distributed.html</loc>
<lastmod>2025-05-01T13:42:09.973Z</lastmod>
<lastmod>2025-05-01T16:25:57.325Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.pygmalion.html</loc>
<lastmod>2025-05-01T13:42:09.546Z</lastmod>
<lastmod>2025-05-01T16:25:56.893Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.dict.html</loc>
<lastmod>2025-05-01T13:42:09.976Z</lastmod>
<lastmod>2025-05-01T16:25:57.328Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_flash.html</loc>
<lastmod>2025-05-01T13:42:09.755Z</lastmod>
<lastmod>2025-05-01T16:25:57.104Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.base.html</loc>
<lastmod>2025-05-01T13:42:09.446Z</lastmod>
<lastmod>2025-05-01T16:25:56.793Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.data.batch_dataset_fetcher.html</loc>
<lastmod>2025-05-01T13:42:09.841Z</lastmod>
<lastmod>2025-05-01T16:25:57.191Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/kernels.swiglu.html</loc>
<lastmod>2025-05-01T13:42:09.720Z</lastmod>
<lastmod>2025-05-01T16:25:57.069Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.chat.messages.html</loc>
<lastmod>2025-05-01T13:42:09.202Z</lastmod>
<lastmod>2025-05-01T16:25:56.547Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/index.html</loc>
<lastmod>2025-05-01T13:42:08.940Z</lastmod>
<lastmod>2025-05-01T16:25:56.280Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.zephyr.html</loc>
<lastmod>2025-05-01T13:42:09.573Z</lastmod>
<lastmod>2025-05-01T16:25:56.921Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/convert.html</loc>
<lastmod>2025-05-01T13:42:09.031Z</lastmod>
<lastmod>2025-05-01T16:25:56.372Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.multimodal.html</loc>
<lastmod>2025-05-01T13:42:10.044Z</lastmod>
<lastmod>2025-05-01T16:25:57.397Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.trainers.base.html</loc>
<lastmod>2025-05-01T13:42:09.414Z</lastmod>
<lastmod>2025-05-01T16:25:56.761Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/evaluate.html</loc>
<lastmod>2025-05-01T13:42:09.011Z</lastmod>
<lastmod>2025-05-01T16:25:56.351Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_xformers.html</loc>
<lastmod>2025-05-01T13:42:09.756Z</lastmod>
<lastmod>2025-05-01T16:25:57.106Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/kernels.quantize.html</loc>
<lastmod>2025-05-01T13:42:09.728Z</lastmod>
<lastmod>2025-05-01T16:25:57.077Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.mlflow_.html</loc>
<lastmod>2025-05-01T13:42:10.235Z</lastmod>
<lastmod>2025-05-01T16:25:57.591Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.profiler.html</loc>
<lastmod>2025-05-01T13:42:10.229Z</lastmod>
<lastmod>2025-05-01T16:25:57.586Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.trainers.dpo.trainer.html</loc>
<lastmod>2025-05-01T13:42:09.441Z</lastmod>
<lastmod>2025-05-01T16:25:56.788Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.vllm_serve.html</loc>
<lastmod>2025-05-01T13:42:09.390Z</lastmod>
<lastmod>2025-05-01T16:25:56.737Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/train.html</loc>
<lastmod>2025-05-01T13:42:09.000Z</lastmod>
<lastmod>2025-05-01T16:25:56.341Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chatml.html</loc>
<lastmod>2025-05-01T13:42:09.572Z</lastmod>
<lastmod>2025-05-01T16:25:56.920Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.trl.html</loc>
<lastmod>2025-05-01T13:42:10.038Z</lastmod>
<lastmod>2025-05-01T16:25:57.391Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/kernels.geglu.html</loc>
<lastmod>2025-05-01T13:42:09.710Z</lastmod>
<lastmod>2025-05-01T16:25:57.059Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.bench.html</loc>
<lastmod>2025-05-01T13:42:09.905Z</lastmod>
<lastmod>2025-05-01T16:25:57.256Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.transformers_fa_utils.html</loc>
<lastmod>2025-05-01T13:42:09.831Z</lastmod>
<lastmod>2025-05-01T16:25:57.181Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.liger.args.html</loc>
<lastmod>2025-05-01T13:42:10.155Z</lastmod>
<lastmod>2025-05-01T16:25:57.510Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.trainer_builder.html</loc>
<lastmod>2025-05-01T13:42:09.094Z</lastmod>
<lastmod>2025-05-01T16:25:56.435Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.utils.html</loc>
<lastmod>2025-05-01T13:42:10.068Z</lastmod>
<lastmod>2025-05-01T16:25:57.422Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/kernels.lora.html</loc>
<lastmod>2025-05-01T13:42:09.700Z</lastmod>
<lastmod>2025-05-01T16:25:57.049Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.bradley_terry.llama3.html</loc>
<lastmod>2025-05-01T13:42:09.618Z</lastmod>
<lastmod>2025-05-01T16:25:56.967Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.trainers.grpo.trainer.html</loc>
<lastmod>2025-05-01T13:42:09.444Z</lastmod>
<lastmod>2025-05-01T16:25:56.791Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.messages.chat.html</loc>
<lastmod>2025-05-01T13:42:09.550Z</lastmod>
<lastmod>2025-05-01T16:25:56.898Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.tokenization.html</loc>
<lastmod>2025-05-01T13:42:09.878Z</lastmod>
<lastmod>2025-05-01T16:25:57.228Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.chat_template.html</loc>
<lastmod>2025-05-01T13:42:09.464Z</lastmod>
<lastmod>2025-05-01T16:25:56.811Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.stepwise_supervised.html</loc>
<lastmod>2025-05-01T13:42:09.528Z</lastmod>
<lastmod>2025-05-01T16:25:56.876Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.samplers.multipack.html</loc>
<lastmod>2025-05-01T13:42:10.219Z</lastmod>
<lastmod>2025-05-01T16:25:57.576Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.args.html</loc>
<lastmod>2025-05-01T13:42:09.283Z</lastmod>
<lastmod>2025-05-01T16:25:56.629Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.perplexity.html</loc>
<lastmod>2025-05-01T13:42:10.226Z</lastmod>
<lastmod>2025-05-01T16:25:57.582Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.gradient_checkpointing.unsloth.html</loc>
<lastmod>2025-05-01T13:42:09.990Z</lastmod>
<lastmod>2025-05-01T16:25:57.342Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/mac.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/config.html</loc>
<lastmod>2025-05-01T13:41:42.483Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/multimodal.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/lr_groups.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/index.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/stepwise_supervised.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/pretraining.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/multi-node.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/sequence_parallelism.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/batch_vs_grad.html</loc>
<lastmod>2025-05-01T13:41:42.483Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/amd_hpc.html</loc>
<lastmod>2025-05-01T13:41:42.483Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/faq.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/custom_integrations.html</loc>
<lastmod>2025-05-01T13:41:42.483Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/src/axolotl/integrations/LICENSE.html</loc>
<lastmod>2025-05-01T13:41:42.503Z</lastmod>
<lastmod>2025-05-01T16:25:28.217Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/TODO.html</loc>
<lastmod>2025-05-01T13:41:42.482Z</lastmod>
<lastmod>2025-05-01T16:25:28.196Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html</loc>
<lastmod>2025-05-01T13:41:42.504Z</lastmod>
<lastmod>2025-05-01T16:25:28.217Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/getting-started.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.198Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/multipack.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/multi-gpu.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/installation.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/cli.html</loc>
<lastmod>2025-05-01T13:41:42.483Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/inst_tune.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/tokenized.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/conversation.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/dataset-formats/template_free.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/reward_modelling.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/lora_optims.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/nccl.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/logging_config.html</loc>
<lastmod>2025-05-01T13:42:09.078Z</lastmod>
<lastmod>2025-05-01T16:25:56.420Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.trainer.html</loc>
<lastmod>2025-05-01T13:42:09.929Z</lastmod>
<lastmod>2025-05-01T16:25:57.281Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.unsloth_.html</loc>
<lastmod>2025-05-01T13:42:09.833Z</lastmod>
<lastmod>2025-05-01T16:25:57.183Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.evaluate.html</loc>
<lastmod>2025-05-01T13:42:09.266Z</lastmod>
<lastmod>2025-05-01T16:25:56.612Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/kernels.utils.html</loc>
<lastmod>2025-05-01T13:42:09.729Z</lastmod>
<lastmod>2025-05-01T16:25:57.078Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/datasets.html</loc>
<lastmod>2025-05-01T13:42:09.018Z</lastmod>
<lastmod>2025-05-01T16:25:56.359Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.models.html</loc>
<lastmod>2025-05-01T13:42:09.871Z</lastmod>
<lastmod>2025-05-01T16:25:57.222Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.llama3.html</loc>
<lastmod>2025-05-01T13:42:09.584Z</lastmod>
<lastmod>2025-05-01T16:25:56.933Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.preprocess.html</loc>
<lastmod>2025-05-01T13:42:09.349Z</lastmod>
<lastmod>2025-05-01T16:25:56.695Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.merge_sharded_fsdp_weights.html</loc>
<lastmod>2025-05-01T13:42:09.341Z</lastmod>
<lastmod>2025-05-01T16:25:56.687Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.base.html</loc>
<lastmod>2025-05-01T13:42:10.140Z</lastmod>
<lastmod>2025-05-01T16:25:57.495Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.orpo.chat_template.html</loc>
<lastmod>2025-05-01T13:42:09.614Z</lastmod>
<lastmod>2025-05-01T16:25:56.963Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.enums.html</loc>
<lastmod>2025-05-01T13:42:10.062Z</lastmod>
<lastmod>2025-05-01T16:25:57.416Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.comet_.html</loc>
<lastmod>2025-05-01T13:42:10.238Z</lastmod>
<lastmod>2025-05-01T16:25:57.595Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.input_output.html</loc>
<lastmod>2025-05-01T13:42:09.524Z</lastmod>
<lastmod>2025-05-01T16:25:56.871Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schedulers.html</loc>
<lastmod>2025-05-01T13:42:09.954Z</lastmod>
<lastmod>2025-05-01T16:25:57.306Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.cloud.base.html</loc>
<lastmod>2025-05-01T13:42:09.393Z</lastmod>
<lastmod>2025-05-01T16:25:56.741Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.utils.html</loc>
<lastmod>2025-05-01T13:42:09.386Z</lastmod>
<lastmod>2025-05-01T16:25:56.733Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.lora_kernels.html</loc>
<lastmod>2025-05-01T13:42:09.805Z</lastmod>
<lastmod>2025-05-01T16:25:57.155Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.completion.html</loc>
<lastmod>2025-05-01T13:42:09.518Z</lastmod>
<lastmod>2025-05-01T16:25:56.865Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.config.html</loc>
<lastmod>2025-05-01T13:42:09.998Z</lastmod>
<lastmod>2025-05-01T16:25:57.350Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_expand_mask.html</loc>
<lastmod>2025-05-01T13:42:09.780Z</lastmod>
<lastmod>2025-05-01T16:25:57.130Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.config.html</loc>
<lastmod>2025-05-01T13:42:09.307Z</lastmod>
<lastmod>2025-05-01T16:25:56.653Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.collators.core.html</loc>
<lastmod>2025-05-01T13:42:10.184Z</lastmod>
<lastmod>2025-05-01T16:25:57.540Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.data.sft.html</loc>
<lastmod>2025-05-01T13:42:09.986Z</lastmod>
<lastmod>2025-05-01T16:25:57.339Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.spectrum.args.html</loc>
<lastmod>2025-05-01T13:42:10.162Z</lastmod>
<lastmod>2025-05-01T16:25:57.517Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.llama3.html</loc>
<lastmod>2025-05-01T13:42:09.562Z</lastmod>
<lastmod>2025-05-01T16:25:56.909Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.inference.html</loc>
<lastmod>2025-05-01T13:42:09.321Z</lastmod>
<lastmod>2025-05-01T16:25:56.667Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_instruct.html</loc>
<lastmod>2025-05-01T13:42:09.479Z</lastmod>
<lastmod>2025-05-01T16:25:56.826Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.collators.batching.html</loc>
<lastmod>2025-05-01T13:42:10.203Z</lastmod>
<lastmod>2025-05-01T16:25:57.559Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.datasets.html</loc>
<lastmod>2025-05-01T13:42:10.027Z</lastmod>
<lastmod>2025-05-01T16:25:57.380Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.data.pretraining.html</loc>
<lastmod>2025-05-01T13:42:09.985Z</lastmod>
<lastmod>2025-05-01T16:25:57.338Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.passthrough.html</loc>
<lastmod>2025-05-01T13:42:09.576Z</lastmod>
<lastmod>2025-05-01T16:25:56.924Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.utils.html</loc>
<lastmod>2025-05-01T13:42:09.813Z</lastmod>
<lastmod>2025-05-01T16:25:57.163Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.lisa.html</loc>
<lastmod>2025-05-01T13:42:10.231Z</lastmod>
<lastmod>2025-05-01T16:25:57.587Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_chat.html</loc>
<lastmod>2025-05-01T13:42:09.477Z</lastmod>
<lastmod>2025-05-01T16:25:56.824Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.user_defined.html</loc>
<lastmod>2025-05-01T13:42:09.594Z</lastmod>
<lastmod>2025-05-01T16:25:56.942Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.training_args.html</loc>
<lastmod>2025-05-01T13:42:09.180Z</lastmod>
<lastmod>2025-05-01T16:25:56.524Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.collators.mm_chat.html</loc>
<lastmod>2025-05-01T13:42:10.211Z</lastmod>
<lastmod>2025-05-01T16:25:57.567Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/common.architectures.html</loc>
<lastmod>2025-05-01T13:42:10.163Z</lastmod>
<lastmod>2025-05-01T16:25:57.519Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.kd.trainer.html</loc>
<lastmod>2025-05-01T13:42:10.152Z</lastmod>
<lastmod>2025-05-01T16:25:57.506Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.llama2_chat.html</loc>
<lastmod>2025-05-01T13:42:09.512Z</lastmod>
<lastmod>2025-05-01T16:25:56.859Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.trainers.trl.html</loc>
<lastmod>2025-05-01T13:42:09.431Z</lastmod>
<lastmod>2025-05-01T16:25:56.778Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.orcamini.html</loc>
<lastmod>2025-05-01T13:42:09.539Z</lastmod>
<lastmod>2025-05-01T16:25:56.886Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.cut_cross_entropy.args.html</loc>
<lastmod>2025-05-01T13:42:10.143Z</lastmod>
<lastmod>2025-05-01T16:25:57.498Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.lora.html</loc>
<lastmod>2025-05-01T13:42:09.892Z</lastmod>
<lastmod>2025-05-01T16:25:57.243Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.sweeps.html</loc>
<lastmod>2025-05-01T13:42:09.354Z</lastmod>
<lastmod>2025-05-01T16:25:56.701Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.chatml.html</loc>
<lastmod>2025-05-01T13:42:09.204Z</lastmod>
<lastmod>2025-05-01T16:25:56.548Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/common.const.html</loc>
<lastmod>2025-05-01T13:42:10.165Z</lastmod>
<lastmod>2025-05-01T16:25:57.521Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.metharme.html</loc>
<lastmod>2025-05-01T13:42:09.535Z</lastmod>
<lastmod>2025-05-01T16:25:56.883Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.stablelm_attn_hijack_flash.html</loc>
<lastmod>2025-05-01T13:42:09.821Z</lastmod>
<lastmod>2025-05-01T16:25:57.171Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.grokfast.optimizer.html</loc>
<lastmod>2025-05-01T13:42:10.145Z</lastmod>
<lastmod>2025-05-01T16:25:57.499Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.checks.html</loc>
<lastmod>2025-05-01T13:42:09.289Z</lastmod>
<lastmod>2025-05-01T16:25:56.635Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/common.datasets.html</loc>
<lastmod>2025-05-01T13:42:10.181Z</lastmod>
<lastmod>2025-05-01T16:25:57.537Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/integrations.lm_eval.args.html</loc>
<lastmod>2025-05-01T13:42:10.159Z</lastmod>
<lastmod>2025-05-01T16:25:57.513Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.mixtral.html</loc>
<lastmod>2025-05-01T13:42:09.842Z</lastmod>
<lastmod>2025-05-01T16:25:57.192Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.chatml.html</loc>
<lastmod>2025-05-01T13:42:09.592Z</lastmod>
<lastmod>2025-05-01T16:25:56.941Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.train.html</loc>
<lastmod>2025-05-01T13:42:09.258Z</lastmod>
<lastmod>2025-05-01T16:25:56.604Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/prompt_tokenizers.html</loc>
<lastmod>2025-05-01T13:42:09.073Z</lastmod>
<lastmod>2025-05-01T16:25:56.415Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.training.html</loc>
<lastmod>2025-05-01T13:42:10.009Z</lastmod>
<lastmod>2025-05-01T16:25:57.362Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.relora.html</loc>
<lastmod>2025-05-01T13:42:09.779Z</lastmod>
<lastmod>2025-05-01T16:25:57.128Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.cloud.modal_.html</loc>
<lastmod>2025-05-01T13:42:09.400Z</lastmod>
<lastmod>2025-05-01T16:25:56.747Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/cli.main.html</loc>
<lastmod>2025-05-01T13:42:09.250Z</lastmod>
<lastmod>2025-05-01T16:25:56.595Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.trainer_fsdp_optim.html</loc>
<lastmod>2025-05-01T13:42:09.825Z</lastmod>
<lastmod>2025-05-01T16:25:57.175Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/fsdp_qlora.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.198Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/debugging.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/ray-integration.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/docker.html</loc>
<lastmod>2025-05-01T13:41:42.484Z</lastmod>
<lastmod>2025-05-01T16:25:28.197Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/inference.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.200Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/docs/torchao.html</loc>
<lastmod>2025-05-01T13:41:42.487Z</lastmod>
<lastmod>2025-05-01T16:25:28.201Z</lastmod>
</url>
<url>
<loc>https://docs.axolotl.ai/FAQS.html</loc>
<lastmod>2025-05-01T13:41:42.482Z</lastmod>
<lastmod>2025-05-01T16:25:28.195Z</lastmod>
</url>
</urlset>