tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-03-31 21:17:51 +00:00
parent 71afa0897d
commit be8430d321
12 changed files with 2625 additions and 1515 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
docs/api/utils.samplers.multipack.html
View File

@@ -444,6 +444,10 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
<ul class="collapse">
<li><a href="#axolotl.utils.samplers.multipack.MultipackBatchSampler" id="toc-axolotl.utils.samplers.multipack.MultipackBatchSampler" class="nav-link" data-scroll-target="#axolotl.utils.samplers.multipack.MultipackBatchSampler">MultipackBatchSampler</a></li>
</ul></li>
<li><a href="#functions" id="toc-functions" class="nav-link" data-scroll-target="#functions">Functions</a>
<ul class="collapse">
<li><a href="#axolotl.utils.samplers.multipack.allocate_sequentially" id="toc-axolotl.utils.samplers.multipack.allocate_sequentially" class="nav-link" data-scroll-target="#axolotl.utils.samplers.multipack.allocate_sequentially">allocate_sequentially</a></li>
</ul></li>
</ul></li>
</ul>
</nav>
@@ -485,9 +489,41 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> packing_efficiency_estimate<span class="op">=</span><span class="fl">1.0</span>,</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> drop_last<span class="op">=</span><span class="va">False</span>,</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> num_count_samples<span class="op">=</span><span class="dv">16</span>,</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="op">**</span>kwargs,</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> sequential<span class="op">=</span><span class="va">False</span>,</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="op">**</span>kwargs,</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Batch sampler class for multipack</p>
</section>
</section>
<section id="functions" class="level2">
<h2 class="anchored" data-anchor-id="functions">Functions</h2>
<table class="caption-top table">
<thead>
<tr class="header">
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><a href="#axolotl.utils.samplers.multipack.allocate_sequentially">allocate_sequentially</a></td>
<td>Sequential allocator that preserves example order</td>
</tr>
</tbody>
</table>
<section id="axolotl.utils.samplers.multipack.allocate_sequentially" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.utils.samplers.multipack.allocate_sequentially">allocate_sequentially</h3>
<div class="sourceCode" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a>utils.samplers.multipack.allocate_sequentially(lengths, rank, c, n)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Sequential allocator that preserves example order</p>
<p>Parameters:
- lengths: The lengths of all examples
- rank: The current rank (for distributed training)
- c: The capacity of each bin (maximum sequence length)
- n: Number of ranks</p>
<p>Returns:
- result: List of batches for the current rank
- total_used: Number of actual example tokens
- total_slots: Maximum theoretical number of example tokens (number of bins * bin capacity)</p>
</section>