tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-09-12 10:02:06 +00:00
parent 782d946b5a
commit db626de56e
9 changed files with 1503 additions and 1524 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/quantize.html
View File

@@ -561,6 +561,20 @@ you used to train the model:</p>
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a><span class="fu">output_dir</span><span class="kw">:</span><span class="co"> # The path to the output directory used during training where the final checkpoint has been saved.</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb3"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">axolotl</span> quantize qat.yml</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>This ensures that an identical quantization configuration is used to quantize the model as was used to train it.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
Note
</div>
</div>
<div class="callout-body-container callout-body">
<p>If you have configured pushing to hub with <code>hub_model_id</code>, your model hub name will have the quantization schema appended to it,
e.g.&nbsp;<code>axolotl-ai-cloud/qat-nvfp4-llama3B</code> will become <code>axolotl-ai-cloud/qat-nvfp4-llama3B-nvfp4w</code></p>
</div>
</div>
</section>