tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-06-14 18:56:28 +00:00
parent 84db47f3c0
commit 5b66b8e86c
6 changed files with 393 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
docs/api/monkeypatch.lora_kernels.html
View File

@@ -477,6 +477,7 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<ul class="collapse">
<li><a href="#axolotl.monkeypatch.lora_kernels.apply_lora_kernel_patches" id="toc-axolotl.monkeypatch.lora_kernels.apply_lora_kernel_patches" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.apply_lora_kernel_patches">apply_lora_kernel_patches</a></li>
<li><a href="#axolotl.monkeypatch.lora_kernels.get_attention_cls_from_config" id="toc-axolotl.monkeypatch.lora_kernels.get_attention_cls_from_config" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.get_attention_cls_from_config">get_attention_cls_from_config</a></li>
<li><a href="#axolotl.monkeypatch.lora_kernels.get_layers" id="toc-axolotl.monkeypatch.lora_kernels.get_layers" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.get_layers">get_layers</a></li>
<li><a href="#axolotl.monkeypatch.lora_kernels.original_apply_o" id="toc-axolotl.monkeypatch.lora_kernels.original_apply_o" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.original_apply_o">original_apply_o</a></li>
<li><a href="#axolotl.monkeypatch.lora_kernels.original_apply_qkv" id="toc-axolotl.monkeypatch.lora_kernels.original_apply_qkv" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.original_apply_qkv">original_apply_qkv</a></li>
<li><a href="#axolotl.monkeypatch.lora_kernels.patch_self_attn_lora" id="toc-axolotl.monkeypatch.lora_kernels.patch_self_attn_lora" class="nav-link" data-scroll-target="#axolotl.monkeypatch.lora_kernels.patch_self_attn_lora">patch_self_attn_lora</a></li>
@@ -536,14 +537,18 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<td>Get the appropriate attention class by inspecting the model config.</td>
</tr>
<tr class="odd">
<td><a href="#axolotl.monkeypatch.lora_kernels.get_layers">get_layers</a></td>
<td>Get the layers of the model. Handles text-only and multimodal models.</td>
</tr>
<tr class="even">
<td><a href="#axolotl.monkeypatch.lora_kernels.original_apply_o">original_apply_o</a></td>
<td>Original implementation of output projection without optimizations.</td>
</tr>
<tr class="even">
<tr class="odd">
<td><a href="#axolotl.monkeypatch.lora_kernels.original_apply_qkv">original_apply_qkv</a></td>
<td>Original implementation of QKV projection without optimizations.</td>
</tr>
<tr class="odd">
<tr class="even">
<td><a href="#axolotl.monkeypatch.lora_kernels.patch_self_attn_lora">patch_self_attn_lora</a></td>
<td>Given an <code>axolotl</code> config, this method patches the inferred attention class forward</td>
</tr>
@@ -740,13 +745,58 @@ the standard transformers naming convention.</p>
</table>
</section>
</section>
<section id="axolotl.monkeypatch.lora_kernels.original_apply_o" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.monkeypatch.lora_kernels.original_apply_o">original_apply_o</h3>
<div class="sourceCode" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.original_apply_o(<span class="va">self</span>, hidden_states)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Original implementation of output projection without optimizations.</p>
<section id="axolotl.monkeypatch.lora_kernels.get_layers" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.monkeypatch.lora_kernels.get_layers">get_layers</h3>
<div class="sourceCode" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.get_layers(model)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Get the layers of the model. Handles text-only and multimodal models.</p>
<section id="parameters-2" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-2">Parameters</h4>
<table class="caption-top table">
<thead>
<tr class="header">
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>model</td>
<td>PeftModelForCausalLM</td>
<td>A PEFT model.</td>
<td><em>required</em></td>
</tr>
</tbody>
</table>
</section>
<section id="returns-2" class="level4 doc-section doc-section-returns">
<h4 class="doc-section doc-section-returns anchored" data-anchor-id="returns-2">Returns</h4>
<table class="caption-top table">
<thead>
<tr class="header">
<th>Name</th>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td></td>
<td>list[nn.Module]</td>
<td>A list of layers.</td>
</tr>
</tbody>
</table>
</section>
</section>
<section id="axolotl.monkeypatch.lora_kernels.original_apply_o" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.monkeypatch.lora_kernels.original_apply_o">original_apply_o</h3>
<div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.original_apply_o(<span class="va">self</span>, hidden_states)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Original implementation of output projection without optimizations.</p>
<section id="parameters-3" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-3">Parameters</h4>
<table class="caption-top table">
<colgroup>
<col style="width: 14%">
<col style="width: 13%">
@@ -777,8 +827,8 @@ the standard transformers naming convention.</p>
</tbody>
</table>
</section>
<section id="returns-2" class="level4 doc-section doc-section-returns">
<h4 class="doc-section doc-section-returns anchored" data-anchor-id="returns-2">Returns</h4>
<section id="returns-3" class="level4 doc-section doc-section-returns">
<h4 class="doc-section doc-section-returns anchored" data-anchor-id="returns-3">Returns</h4>
<table class="caption-top table">
<thead>
<tr class="header">
@@ -799,10 +849,10 @@ the standard transformers naming convention.</p>
</section>
<section id="axolotl.monkeypatch.lora_kernels.original_apply_qkv" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.monkeypatch.lora_kernels.original_apply_qkv">original_apply_qkv</h3>
<div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.original_apply_qkv(<span class="va">self</span>, hidden_states)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.original_apply_qkv(<span class="va">self</span>, hidden_states)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Original implementation of QKV projection without optimizations.</p>
<section id="parameters-3" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-3">Parameters</h4>
<section id="parameters-4" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-4">Parameters</h4>
<table class="caption-top table">
<colgroup>
<col style="width: 15%">
@@ -834,8 +884,8 @@ the standard transformers naming convention.</p>
</tbody>
</table>
</section>
<section id="returns-3" class="level4 doc-section doc-section-returns">
<h4 class="doc-section doc-section-returns anchored" data-anchor-id="returns-3">Returns</h4>
<section id="returns-4" class="level4 doc-section doc-section-returns">
<h4 class="doc-section doc-section-returns anchored" data-anchor-id="returns-4">Returns</h4>
<table class="caption-top table">
<colgroup>
<col style="width: 4%">
@@ -861,13 +911,13 @@ the standard transformers naming convention.</p>
</section>
<section id="axolotl.monkeypatch.lora_kernels.patch_self_attn_lora" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.monkeypatch.lora_kernels.patch_self_attn_lora">patch_self_attn_lora</h3>
<div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.patch_self_attn_lora(cfg)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb7"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a>monkeypatch.lora_kernels.patch_self_attn_lora(cfg)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Given an <code>axolotl</code> config, this method patches the inferred attention class forward
pass with optimized LoRA implementations.</p>
<p>It modifies the attention class to use optimized QKV and output projections. The
original implementation is preserved and can be restored if needed.</p>
<section id="parameters-4" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-4">Parameters</h4>
<section id="parameters-5" class="level4 doc-section doc-section-parameters">
<h4 class="doc-section doc-section-parameters anchored" data-anchor-id="parameters-5">Parameters</h4>
<table class="caption-top table">
<colgroup>
<col style="width: 9%">