tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2024-07-11 13:20:35 +00:00
parent d2a96e9b1d
commit 5ad84a9488
26 changed files with 70 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/fsdp_qlora.html
View File

@@ -85,7 +85,7 @@ ul.task-list li input[type="checkbox"] {
</div>
<div class="quarto-navbar-tools tools-wide tools-end">
<a href="https://twitter.com/axolotl_ai" title="" class="quarto-navigation-tool px-1" aria-label=""><i class="bi bi-twitter"></i></a>
<a href="https://github.com/OpenAccess-AI-Collective/axolotl/" title="" class="quarto-navigation-tool px-1" aria-label=""><i class="bi bi-github"></i></a>
<a href="https://github.com/axolotl-ai-cloud/axolotl/" title="" class="quarto-navigation-tool px-1" aria-label=""><i class="bi bi-github"></i></a>
<a href="https://discord.gg/7m9sfhzaf3" title="" class="quarto-navigation-tool px-1" aria-label=""><i class="bi bi-discord"></i></a>
</div>
<div id="quarto-search" class="" title="Search"></div>
@@ -294,7 +294,7 @@ ul.task-list li input[type="checkbox"] {
</blockquote>
<ol type="1">
<li>Set <code>adapter: qlora</code> in your axolotl config file.</li>
<li>Enable FSDP in your axolotl config, as <a href="https://github.com/OpenAccess-AI-Collective/axolotl?tab=readme-ov-file#fsdp">described here</a>.</li>
<li>Enable FSDP in your axolotl config, as <a href="https://github.com/axolotl-ai-cloud/axolotl?tab=readme-ov-file#fsdp">described here</a>.</li>
<li>Use one of the supported model types: <code>llama</code>, <code>mistral</code> or <code>mixtral</code>.</li>
</ol>
</section>
@@ -305,7 +305,7 @@ ul.task-list li input[type="checkbox"] {
<section id="references" class="level2">
<h2 class="anchored" data-anchor-id="references">References</h2>
<ul>
<li><a href="https://github.com/OpenAccess-AI-Collective/axolotl/pull/1378">PR #1378</a> enabling QLoRA in FSDP in Axolotl.</li>
<li><a href="https://github.com/axolotl-ai-cloud/axolotl/pull/1378">PR #1378</a> enabling QLoRA in FSDP in Axolotl.</li>
<li><a href="https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html">Blog Post</a> from the <a href="https://www.answer.ai/">Answer.AI</a> team describing the work that enabled QLoRA in FSDP.</li>
<li>Related HuggingFace PRs Enabling FDSP + QLoRA:
<ul>