tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2026-03-03 15:13:12 +00:00
parent 35e6a7e228
commit 28bc7f60e2
8 changed files with 899 additions and 899 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.nojekyll
View File

@@ -1 +1 @@
590c8a6b 18088c0d

27
docs/api/loaders.patch_manager.html
View File

@@ -817,40 +817,49 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
</thead> </thead>
<tbody> <tbody>
<tr class="odd"> <tr class="odd">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_post_model_build_patches">apply_post_model_build_patches</a></td>
<td>Apply patches right after model build, before post-load setup.</td>
</tr>
<tr class="even">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches">apply_post_model_load_patches</a></td> <td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches">apply_post_model_load_patches</a></td>
<td>Apply patches that require the model instance.</td> <td>Apply patches that require the model instance.</td>
</tr> </tr>
<tr class="even"> <tr class="odd">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches">apply_post_plugin_pre_model_load_patches</a></td> <td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches">apply_post_plugin_pre_model_load_patches</a></td>
<td>Apply post plugin-pre_model_load load patches based on config.</td> <td>Apply post plugin-pre_model_load load patches based on config.</td>
</tr> </tr>
<tr class="odd"> <tr class="even">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches">apply_pre_config_load_patches</a></td> <td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches">apply_pre_config_load_patches</a></td>
<td>Apply patches that must be set up before config loading.</td> <td>Apply patches that must be set up before config loading.</td>
</tr> </tr>
<tr class="even"> <tr class="odd">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches">apply_pre_model_load_patches</a></td> <td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches">apply_pre_model_load_patches</a></td>
<td>Apply pre-model load patches based on config.</td> <td>Apply pre-model load patches based on config.</td>
</tr> </tr>
<tr class="odd"> <tr class="even">
<td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches">apply_pre_tokenizer_load_patches</a></td> <td><a href="#axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches">apply_pre_tokenizer_load_patches</a></td>
<td>Apply patches that must be set up before tokenizer loading.</td> <td>Apply patches that must be set up before tokenizer loading.</td>
</tr> </tr>
</tbody> </tbody>
</table> </table>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_build_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_build_patches">apply_post_model_build_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_post_model_build_patches(model)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply patches right after model build, before post-load setup.</p>
</section>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches" class="level5"> <section id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches">apply_post_model_load_patches</h5> <h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_post_model_load_patches">apply_post_model_load_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_post_model_load_patches(model)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_post_model_load_patches(model)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply patches that require the model instance.</p> <p>Apply patches that require the model instance.</p>
</section> </section>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches" class="level5"> <section id="axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches">apply_post_plugin_pre_model_load_patches</h5> <h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches">apply_post_plugin_pre_model_load_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_post_plugin_pre_model_load_patches()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply post plugin-pre_model_load load patches based on config.</p> <p>Apply post plugin-pre_model_load load patches based on config.</p>
</section> </section>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches" class="level5"> <section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches">apply_pre_config_load_patches</h5> <h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_config_load_patches">apply_pre_config_load_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_config_load_patches(cfg)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_config_load_patches(cfg)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply patches that must be set up before config loading. <p>Apply patches that must be set up before config loading.
This is for patches that intercept remote code loading from HuggingFace, This is for patches that intercept remote code loading from HuggingFace,
which needs to be in place before AutoConfig.from_pretrained() is called.</p> which needs to be in place before AutoConfig.from_pretrained() is called.</p>
@@ -884,12 +893,12 @@ which needs to be in place before AutoConfig.from_pretrained() is called.</p>
</section> </section>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches" class="level5"> <section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches">apply_pre_model_load_patches</h5> <h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_model_load_patches">apply_pre_model_load_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_model_load_patches()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_model_load_patches()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply pre-model load patches based on config.</p> <p>Apply pre-model load patches based on config.</p>
</section> </section>
<section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches" class="level5"> <section id="axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches" class="level5">
<h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches">apply_pre_tokenizer_load_patches</h5> <h5 class="anchored" data-anchor-id="axolotl.loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches">apply_pre_tokenizer_load_patches</h5>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches(cfg)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a>loaders.patch_manager.PatchManager.apply_pre_tokenizer_load_patches(cfg)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Apply patches that must be set up before tokenizer loading. <p>Apply patches that must be set up before tokenizer loading.
This is for patches that intercept remote code loading from HuggingFace, This is for patches that intercept remote code loading from HuggingFace,
which needs to be in place before AutoTokenizer.from_pretrained() is called.</p> which needs to be in place before AutoTokenizer.from_pretrained() is called.</p>

1256
docs/config-reference.html
View File

File diff suppressed because it is too large Load Diff

7
docs/custom_integrations.html
View File

@@ -954,7 +954,7 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<ul> <ul>
<li>If you are installing from pip</li> <li>If you are installing from pip</li>
</ul> </ul>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="ex">pip3</span> uninstall <span class="at">-y</span> cut-cross-entropy <span class="kw">&amp;&amp;</span> <span class="ex">pip3</span> install <span class="st">"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@58d6572"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="ex">pip3</span> uninstall <span class="at">-y</span> cut-cross-entropy <span class="kw">&amp;&amp;</span> <span class="ex">pip3</span> install <span class="st">"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@a668583"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
</section> </section>
<section id="usage" class="level3"> <section id="usage" class="level3">
<h3 class="anchored" data-anchor-id="usage">Usage</h3> <h3 class="anchored" data-anchor-id="usage">Usage</h3>
@@ -1021,9 +1021,9 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<li>qwen2_vl</li> <li>qwen2_vl</li>
<li>qwen3</li> <li>qwen3</li>
<li>qwen3_5</li> <li>qwen3_5</li>
<li>qwen3_5_text</li>
<li>qwen3_5_moe</li> <li>qwen3_5_moe</li>
<li>qwen3_5_moe_vl</li> <li>qwen3_5_moe_text</li>
<li>qwen3_5_vl</li>
<li>qwen3_moe</li> <li>qwen3_moe</li>
<li>qwen3_next</li> <li>qwen3_next</li>
<li>qwen3_vl</li> <li>qwen3_vl</li>
@@ -1248,6 +1248,7 @@ The quick brown fox jumps over the loud dog</code></pre>
<section id="limitations-1" class="level3"> <section id="limitations-1" class="level3">
<h3 class="anchored" data-anchor-id="limitations-1">Limitations</h3> <h3 class="anchored" data-anchor-id="limitations-1">Limitations</h3>
<p>ScatterMoE uses a softmax -&gt; topk routing, so results may be different for some model arch as baseline (GPT-OSS, GLM_MOE_DSA).</p> <p>ScatterMoE uses a softmax -&gt; topk routing, so results may be different for some model arch as baseline (GPT-OSS, GLM_MOE_DSA).</p>
<p>ScatterMoE does not work for GLM4.7 Flash (glm4_moe_lite) atm.</p>
</section> </section>
<section id="note-on-megablocks" class="level3"> <section id="note-on-megablocks" class="level3">
<h3 class="anchored" data-anchor-id="note-on-megablocks">Note on MegaBlocks</h3> <h3 class="anchored" data-anchor-id="note-on-megablocks">Note on MegaBlocks</h3>

8
docs/models/trinity.html
View File

@@ -753,7 +753,6 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<li><a href="#tips" id="toc-tips" class="nav-link" data-scroll-target="#tips">TIPS</a></li> <li><a href="#tips" id="toc-tips" class="nav-link" data-scroll-target="#tips">TIPS</a></li>
</ul></li> </ul></li>
<li><a href="#optimization-guides" id="toc-optimization-guides" class="nav-link" data-scroll-target="#optimization-guides">Optimization Guides</a></li> <li><a href="#optimization-guides" id="toc-optimization-guides" class="nav-link" data-scroll-target="#optimization-guides">Optimization Guides</a></li>
<li><a href="#limitations" id="toc-limitations" class="nav-link" data-scroll-target="#limitations">Limitations</a></li>
<li><a href="#related-resources" id="toc-related-resources" class="nav-link" data-scroll-target="#related-resources">Related Resources</a></li> <li><a href="#related-resources" id="toc-related-resources" class="nav-link" data-scroll-target="#related-resources">Related Resources</a></li>
</ul> </ul>
</nav> </nav>
@@ -786,10 +785,11 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<h2 class="anchored" data-anchor-id="getting-started">Getting started</h2> <h2 class="anchored" data-anchor-id="getting-started">Getting started</h2>
<ol type="1"> <ol type="1">
<li><p>Install Axolotl following the main from the <a href="https://docs.axolotl.ai/docs/installation.html#sec-edge-build">installation guide</a>.</p></li> <li><p>Install Axolotl following the main from the <a href="https://docs.axolotl.ai/docs/installation.html#sec-edge-build">installation guide</a>.</p></li>
<li><p>Install <a href="https://docs.axolotl.ai/docs/custom_integrations.html#cut-cross-entropy">Cut Cross Entropy</a> to reduce training VRAM usage.</p></li>
<li><p>Run the finetuning example:</p> <li><p>Run the finetuning example:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ex">axolotl</span> train examples/trinity/trinity-nano-preview-qlora.yaml</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div></li> <div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ex">axolotl</span> train examples/trinity/trinity-nano-preview-qlora.yaml</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div></li>
</ol> </ol>
<p>This config uses about 24.9 GiB VRAM.</p> <p>This config uses about 24.9 GiB VRAM (w/o CCE).</p>
<p>Let us know how it goes. Happy finetuning! 🚀</p> <p>Let us know how it goes. Happy finetuning! 🚀</p>
<section id="tips" class="level3"> <section id="tips" class="level3">
<h3 class="anchored" data-anchor-id="tips">TIPS</h3> <h3 class="anchored" data-anchor-id="tips">TIPS</h3>
@@ -805,10 +805,6 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<h2 class="anchored" data-anchor-id="optimization-guides">Optimization Guides</h2> <h2 class="anchored" data-anchor-id="optimization-guides">Optimization Guides</h2>
<p>Please check the <a href="https://docs.axolotl.ai/docs/optimizations.html">Optimizations doc</a>.</p> <p>Please check the <a href="https://docs.axolotl.ai/docs/optimizations.html">Optimizations doc</a>.</p>
</section> </section>
<section id="limitations" class="level2">
<h2 class="anchored" data-anchor-id="limitations">Limitations</h2>
<p><strong>Cut Cross Entropy (CCE)</strong>: Currently not supported. We plan to include CCE support for Trinity in the near future.</p>
</section>
<section id="related-resources" class="level2"> <section id="related-resources" class="level2">
<h2 class="anchored" data-anchor-id="related-resources">Related Resources</h2> <h2 class="anchored" data-anchor-id="related-resources">Related Resources</h2>
<ul> <ul>

2
examples/colab-notebooks/colab-axolotl-example.html
View File

@@ -804,7 +804,7 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="op">%%</span>capture</span> <div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="op">%%</span>capture</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="co"># This step can take ~5-10 minutes to install dependencies</span></span> <span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="co"># This step can take ~5-10 minutes to install dependencies</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>pip install <span class="op">--</span>no<span class="op">-</span>build<span class="op">-</span>isolation axolotl[flash<span class="op">-</span>attn]<span class="op">&gt;=</span><span class="fl">0.9.1</span></span> <span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>pip install <span class="op">--</span>no<span class="op">-</span>build<span class="op">-</span>isolation axolotl[flash<span class="op">-</span>attn]<span class="op">&gt;=</span><span class="fl">0.9.1</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>pip install <span class="st">"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@58d6572"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div> <span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>pip install <span class="st">"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@a668583"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
</div> </div>
<section id="demo-talk-like-a-pirate" class="level2"> <section id="demo-talk-like-a-pirate" class="level2">
<h2 class="anchored" data-anchor-id="demo-talk-like-a-pirate">Demo: Talk Like a Pirate</h2> <h2 class="anchored" data-anchor-id="demo-talk-like-a-pirate">Demo: Talk Like a Pirate</h2>

24
search.json
View File

File diff suppressed because one or more lines are too long

472
sitemap.xml
View File

File diff suppressed because it is too large Load Diff