tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-05-30 04:24:18 +00:00
parent dd36fe4391
commit 9304e18f4b
58 changed files with 3955 additions and 2244 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/api/utils.callbacks.perplexity.html
View File

@@ -505,7 +505,7 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
</table>
<section id="axolotl.utils.callbacks.perplexity.Perplexity" class="level3">
<h3 class="anchored" data-anchor-id="axolotl.utils.callbacks.perplexity.Perplexity">Perplexity</h3>
<div class="sourceCode" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a>utils.callbacks.perplexity.Perplexity(<span class="va">self</span>, tokenizer, max_seq_len, stride<span class="op">=</span><span class="dv">512</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="sourceCode" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a>utils.callbacks.perplexity.Perplexity(tokenizer, max_seq_len, stride<span class="op">=</span><span class="dv">512</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Calculate perplexity as defined in https://huggingface.co/docs/transformers/en/perplexity.
This is a custom variant that doesn’t re-tokenize the input or re-load the model.</p>
<section id="methods" class="level4">