tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-12-22 19:07:05 +00:00
parent 33140be573
commit 3411187898
5 changed files with 215 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/api/utils.trainer.html
View File

@@ -587,10 +587,17 @@ remaining in each sample.</p>
</section>
<section id="axolotl.utils.trainer.drop_long_seq" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.utils.trainer.drop_long_seq">drop_long_seq</h3>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a>utils.trainer.drop_long_seq(sample, sequence_len<span class="op">=</span><span class="dv">2048</span>, min_sequence_len<span class="op">=</span><span class="dv">2</span>)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a>utils.trainer.drop_long_seq(</span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> sample,</span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> sequence_len<span class="op">=</span><span class="dv">2048</span>,</span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> min_sequence_len<span class="op">=</span><span class="dv">2</span>,</span>
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> raise_on_drop<span class="op">=</span><span class="va">False</span>,</span>
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a>)</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
<p>Drop samples whose sequence length is either too long (&gt; sequence_len)
or too short (&lt; min_sequence_len).</p>
<p>Works for both single-example (list[int]) or batched (list[list[int]]).</p>
<p>If raise_on_drop is set, the code raises a ValueError if a sample is
encountered that is too long and would have been dropped.</p>
</section>
<section id="axolotl.utils.trainer.setup_trainer" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.utils.trainer.setup_trainer">setup_trainer</h3>

6
docs/config-reference.html
View File

@@ -1285,9 +1285,9 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<span id="cb1-746"><a href="#cb1-746" aria-hidden="true" tabindex="-1"></a><span class="co"># as most models have a token/context limit of 2048</span></span>
<span id="cb1-747"><a href="#cb1-747" aria-hidden="true" tabindex="-1"></a><span class="fu">sequence_len</span><span class="kw">:</span><span class="at"> int = 512</span></span>
<span id="cb1-748"><a href="#cb1-748" aria-hidden="true" tabindex="-1"></a><span class="co"># What to do when a tokenized row exceeds sequence_len. 'drop' removes the row;</span></span>
<span id="cb1-749"><a href="#cb1-749" aria-hidden="true" tabindex="-1"></a><span class="co"># 'truncate' slices tensors to sequence_len. Defaults to 'drop' for backward</span></span>
<span id="cb1-750"><a href="#cb1-750" aria-hidden="true" tabindex="-1"></a><span class="co"># compatibility.</span></span>
<span id="cb1-751"><a href="#cb1-751" aria-hidden="true" tabindex="-1"></a><span class="fu">excess_length_strategy</span><span class="kw">:</span><span class="at"> Literal['drop', 'truncate'] | None</span></span>
<span id="cb1-749"><a href="#cb1-749" aria-hidden="true" tabindex="-1"></a><span class="co"># 'truncate' slices tensors to sequence_len; 'raise' raises a ValueError. Defaults to</span></span>
<span id="cb1-750"><a href="#cb1-750" aria-hidden="true" tabindex="-1"></a><span class="co"># 'drop' for backward compatibility.</span></span>
<span id="cb1-751"><a href="#cb1-751" aria-hidden="true" tabindex="-1"></a><span class="fu">excess_length_strategy</span><span class="kw">:</span><span class="at"> Literal['drop', 'truncate', 'raise'] | None</span></span>
<span id="cb1-752"><a href="#cb1-752" aria-hidden="true" tabindex="-1"></a><span class="co"># The maximum length of an input for evaluation. If not specified, defaults to</span></span>
<span id="cb1-753"><a href="#cb1-753" aria-hidden="true" tabindex="-1"></a><span class="co"># sequence_len</span></span>
<span id="cb1-754"><a href="#cb1-754" aria-hidden="true" tabindex="-1"></a><span class="fu">eval_sequence_len</span><span class="kw">:</span><span class="at"> int | None</span></span>