Built site for gh-pages
This commit is contained in:
Quarto GHA Workflow Runner
File diff suppressed because it is too large
Load Diff
134
docs/rlhf.html
134
docs/rlhf.html
@@ -554,6 +554,7 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
|
||||
<li><a href="#grpo" id="toc-grpo" class="nav-link" data-scroll-target="#grpo">GRPO</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#reward-functions" id="toc-reward-functions" class="nav-link" data-scroll-target="#reward-functions">Reward functions</a></li>
|
||||
<li><a href="#openenv-rollout-functions" id="toc-openenv-rollout-functions" class="nav-link" data-scroll-target="#openenv-rollout-functions">OpenEnv Rollout Functions</a></li>
|
||||
<li><a href="#grpo-with-dapodr.-grpo-loss" id="toc-grpo-with-dapodr.-grpo-loss" class="nav-link" data-scroll-target="#grpo-with-dapodr.-grpo-loss">GRPO with DAPO/Dr. GRPO loss</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#simpo" id="toc-simpo" class="nav-link" data-scroll-target="#simpo">SimPO</a></li>
|
||||
@@ -1120,39 +1121,140 @@ Note
|
||||
<p>To see other examples of custom reward functions, please see <a href="https://github.com/huggingface/trl/blob/main/docs/source/grpo_trainer.md#using-a-custom-reward-function">TRL GRPO Docs</a>.</p>
|
||||
<p>To see all configs, please see <a href="https://github.com/axolotl-ai-cloud/axolotl/blob/v0.9.2/src/axolotl/utils/schemas/trl.py">TRLConfig</a>.</p>
|
||||
</section>
|
||||
<section id="openenv-rollout-functions" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="openenv-rollout-functions">OpenEnv Rollout Functions</h4>
|
||||
<p>GRPO supports custom rollout functions for OpenEnv-style environments, enabling interactive tasks like web browsing, code execution, or tool use. This allows you to implement custom generation logic that interacts with external environments.</p>
|
||||
<p>For example, to implement a simple math-solving environment with step-by-step verification:</p>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb41"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb41-1"><a href="#cb41-1" aria-hidden="true" tabindex="-1"></a><span class="co"># math_env.py</span></span>
|
||||
<span id="cb41-2"><a href="#cb41-2" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> re</span>
|
||||
<span id="cb41-3"><a href="#cb41-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-4"><a href="#cb41-4" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> math_solver_rollout(model, processing_class, prompts, generation_config<span class="op">=</span><span class="va">None</span>):</span>
|
||||
<span id="cb41-5"><a href="#cb41-5" aria-hidden="true" tabindex="-1"></a> <span class="co">"""</span></span>
|
||||
<span id="cb41-6"><a href="#cb41-6" aria-hidden="true" tabindex="-1"></a><span class="co"> Custom rollout function that generates step-by-step math solutions.</span></span>
|
||||
<span id="cb41-7"><a href="#cb41-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-8"><a href="#cb41-8" aria-hidden="true" tabindex="-1"></a><span class="co"> Args:</span></span>
|
||||
<span id="cb41-9"><a href="#cb41-9" aria-hidden="true" tabindex="-1"></a><span class="co"> model: The language model</span></span>
|
||||
<span id="cb41-10"><a href="#cb41-10" aria-hidden="true" tabindex="-1"></a><span class="co"> processing_class: The tokenizer/processing_class</span></span>
|
||||
<span id="cb41-11"><a href="#cb41-11" aria-hidden="true" tabindex="-1"></a><span class="co"> prompts: List of prompt dicts (with 'messages' key for chat format)</span></span>
|
||||
<span id="cb41-12"><a href="#cb41-12" aria-hidden="true" tabindex="-1"></a><span class="co"> generation_config: Optional generation configuration</span></span>
|
||||
<span id="cb41-13"><a href="#cb41-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-14"><a href="#cb41-14" aria-hidden="true" tabindex="-1"></a><span class="co"> Returns:</span></span>
|
||||
<span id="cb41-15"><a href="#cb41-15" aria-hidden="true" tabindex="-1"></a><span class="co"> List of completion strings</span></span>
|
||||
<span id="cb41-16"><a href="#cb41-16" aria-hidden="true" tabindex="-1"></a><span class="co"> """</span></span>
|
||||
<span id="cb41-17"><a href="#cb41-17" aria-hidden="true" tabindex="-1"></a> completions <span class="op">=</span> []</span>
|
||||
<span id="cb41-18"><a href="#cb41-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-19"><a href="#cb41-19" aria-hidden="true" tabindex="-1"></a> <span class="cf">for</span> prompt <span class="kw">in</span> prompts:</span>
|
||||
<span id="cb41-20"><a href="#cb41-20" aria-hidden="true" tabindex="-1"></a> <span class="co"># Apply chat template to prompt</span></span>
|
||||
<span id="cb41-21"><a href="#cb41-21" aria-hidden="true" tabindex="-1"></a> messages <span class="op">=</span> prompt.get(<span class="st">"messages"</span>, [])</span>
|
||||
<span id="cb41-22"><a href="#cb41-22" aria-hidden="true" tabindex="-1"></a> formatted_prompt <span class="op">=</span> processing_class.apply_chat_template(</span>
|
||||
<span id="cb41-23"><a href="#cb41-23" aria-hidden="true" tabindex="-1"></a> messages, processing_class<span class="op">=</span><span class="va">False</span>, add_generation_prompt<span class="op">=</span><span class="va">True</span></span>
|
||||
<span id="cb41-24"><a href="#cb41-24" aria-hidden="true" tabindex="-1"></a> )</span>
|
||||
<span id="cb41-25"><a href="#cb41-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-26"><a href="#cb41-26" aria-hidden="true" tabindex="-1"></a> <span class="co"># Generate step-by-step solution</span></span>
|
||||
<span id="cb41-27"><a href="#cb41-27" aria-hidden="true" tabindex="-1"></a> full_response <span class="op">=</span> <span class="st">""</span></span>
|
||||
<span id="cb41-28"><a href="#cb41-28" aria-hidden="true" tabindex="-1"></a> <span class="cf">for</span> step <span class="kw">in</span> <span class="bu">range</span>(<span class="dv">5</span>): <span class="co"># Max 5 reasoning steps</span></span>
|
||||
<span id="cb41-29"><a href="#cb41-29" aria-hidden="true" tabindex="-1"></a> current_input <span class="op">=</span> formatted_prompt <span class="op">+</span> full_response <span class="op">+</span> <span class="st">"</span><span class="ch">\n</span><span class="st">Next step:"</span></span>
|
||||
<span id="cb41-30"><a href="#cb41-30" aria-hidden="true" tabindex="-1"></a> inputs <span class="op">=</span> processing_class(current_input, return_tensors<span class="op">=</span><span class="st">"pt"</span>).to(model.device)</span>
|
||||
<span id="cb41-31"><a href="#cb41-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-32"><a href="#cb41-32" aria-hidden="true" tabindex="-1"></a> outputs <span class="op">=</span> model.generate(</span>
|
||||
<span id="cb41-33"><a href="#cb41-33" aria-hidden="true" tabindex="-1"></a> <span class="op">**</span>inputs,</span>
|
||||
<span id="cb41-34"><a href="#cb41-34" aria-hidden="true" tabindex="-1"></a> max_new_tokens<span class="op">=</span><span class="dv">100</span>,</span>
|
||||
<span id="cb41-35"><a href="#cb41-35" aria-hidden="true" tabindex="-1"></a> generation_config<span class="op">=</span>generation_config,</span>
|
||||
<span id="cb41-36"><a href="#cb41-36" aria-hidden="true" tabindex="-1"></a> )</span>
|
||||
<span id="cb41-37"><a href="#cb41-37" aria-hidden="true" tabindex="-1"></a> step_text <span class="op">=</span> processing_class.decode(</span>
|
||||
<span id="cb41-38"><a href="#cb41-38" aria-hidden="true" tabindex="-1"></a> outputs[<span class="dv">0</span>][inputs.input_ids.shape[<span class="dv">1</span>]:],</span>
|
||||
<span id="cb41-39"><a href="#cb41-39" aria-hidden="true" tabindex="-1"></a> skip_special_tokens<span class="op">=</span><span class="va">True</span></span>
|
||||
<span id="cb41-40"><a href="#cb41-40" aria-hidden="true" tabindex="-1"></a> )</span>
|
||||
<span id="cb41-41"><a href="#cb41-41" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-42"><a href="#cb41-42" aria-hidden="true" tabindex="-1"></a> <span class="co"># Check if solution is complete</span></span>
|
||||
<span id="cb41-43"><a href="#cb41-43" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> <span class="st">"FINAL ANSWER:"</span> <span class="kw">in</span> step_text:</span>
|
||||
<span id="cb41-44"><a href="#cb41-44" aria-hidden="true" tabindex="-1"></a> full_response <span class="op">+=</span> step_text</span>
|
||||
<span id="cb41-45"><a href="#cb41-45" aria-hidden="true" tabindex="-1"></a> <span class="cf">break</span></span>
|
||||
<span id="cb41-46"><a href="#cb41-46" aria-hidden="true" tabindex="-1"></a> full_response <span class="op">+=</span> step_text <span class="op">+</span> <span class="st">"</span><span class="ch">\n</span><span class="st">"</span></span>
|
||||
<span id="cb41-47"><a href="#cb41-47" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-48"><a href="#cb41-48" aria-hidden="true" tabindex="-1"></a> completions.append(full_response)</span>
|
||||
<span id="cb41-49"><a href="#cb41-49" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-50"><a href="#cb41-50" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> completions</span>
|
||||
<span id="cb41-51"><a href="#cb41-51" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-52"><a href="#cb41-52" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> math_reward(prompts, completions, answers, <span class="op">**</span>kwargs):</span>
|
||||
<span id="cb41-53"><a href="#cb41-53" aria-hidden="true" tabindex="-1"></a> <span class="co">"""Reward function that checks mathematical correctness"""</span></span>
|
||||
<span id="cb41-54"><a href="#cb41-54" aria-hidden="true" tabindex="-1"></a> rewards <span class="op">=</span> []</span>
|
||||
<span id="cb41-55"><a href="#cb41-55" aria-hidden="true" tabindex="-1"></a> <span class="cf">for</span> completion, correct_answer <span class="kw">in</span> <span class="bu">zip</span>(completions, answers):</span>
|
||||
<span id="cb41-56"><a href="#cb41-56" aria-hidden="true" tabindex="-1"></a> <span class="co"># Extract predicted answer</span></span>
|
||||
<span id="cb41-57"><a href="#cb41-57" aria-hidden="true" tabindex="-1"></a> match <span class="op">=</span> re.search(<span class="vs">r"FINAL ANSWER:</span><span class="dv">\s</span><span class="op">*</span><span class="kw">(</span><span class="dv">.</span><span class="op">+</span><span class="kw">)</span><span class="vs">"</span>, completion)</span>
|
||||
<span id="cb41-58"><a href="#cb41-58" aria-hidden="true" tabindex="-1"></a> predicted <span class="op">=</span> match.group(<span class="dv">1</span>).strip() <span class="cf">if</span> match <span class="cf">else</span> <span class="st">""</span></span>
|
||||
<span id="cb41-59"><a href="#cb41-59" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-60"><a href="#cb41-60" aria-hidden="true" tabindex="-1"></a> <span class="co"># Compare with correct answer</span></span>
|
||||
<span id="cb41-61"><a href="#cb41-61" aria-hidden="true" tabindex="-1"></a> reward <span class="op">=</span> <span class="fl">1.0</span> <span class="cf">if</span> predicted <span class="op">==</span> <span class="bu">str</span>(correct_answer) <span class="cf">else</span> <span class="fl">0.0</span></span>
|
||||
<span id="cb41-62"><a href="#cb41-62" aria-hidden="true" tabindex="-1"></a> rewards.append(reward)</span>
|
||||
<span id="cb41-63"><a href="#cb41-63" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-64"><a href="#cb41-64" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> rewards</span>
|
||||
<span id="cb41-65"><a href="#cb41-65" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb41-66"><a href="#cb41-66" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> math_transform(cfg, <span class="op">*</span>args, <span class="op">**</span>kwargs):</span>
|
||||
<span id="cb41-67"><a href="#cb41-67" aria-hidden="true" tabindex="-1"></a> <span class="co">"""Transform dataset to GRPO format with answer field"""</span></span>
|
||||
<span id="cb41-68"><a href="#cb41-68" aria-hidden="true" tabindex="-1"></a> <span class="kw">def</span> transform_fn(example, processing_class<span class="op">=</span><span class="va">None</span>):</span>
|
||||
<span id="cb41-69"><a href="#cb41-69" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> {</span>
|
||||
<span id="cb41-70"><a href="#cb41-70" aria-hidden="true" tabindex="-1"></a> <span class="st">"prompt"</span>: [{<span class="st">"role"</span>: <span class="st">"user"</span>, <span class="st">"content"</span>: example[<span class="st">"question"</span>]}],</span>
|
||||
<span id="cb41-71"><a href="#cb41-71" aria-hidden="true" tabindex="-1"></a> <span class="st">"answer"</span>: <span class="bu">str</span>(example[<span class="st">"answer"</span>]),</span>
|
||||
<span id="cb41-72"><a href="#cb41-72" aria-hidden="true" tabindex="-1"></a> }</span>
|
||||
<span id="cb41-73"><a href="#cb41-73" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> transform_fn, {<span class="st">"remove_columns"</span>: [<span class="st">"question"</span>]}</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb42"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb42-1"><a href="#cb42-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rl</span><span class="kw">:</span><span class="at"> grpo</span></span>
|
||||
<span id="cb42-2"><a href="#cb42-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb42-3"><a href="#cb42-3" aria-hidden="true" tabindex="-1"></a><span class="fu">trl</span><span class="kw">:</span></span>
|
||||
<span id="cb42-4"><a href="#cb42-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">beta</span><span class="kw">:</span><span class="at"> </span><span class="fl">0.001</span></span>
|
||||
<span id="cb42-5"><a href="#cb42-5" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">max_completion_length</span><span class="kw">:</span><span class="at"> </span><span class="dv">512</span></span>
|
||||
<span id="cb42-6"><a href="#cb42-6" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">num_generations</span><span class="kw">:</span><span class="at"> </span><span class="dv">4</span></span>
|
||||
<span id="cb42-7"><a href="#cb42-7" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">rollout_func</span><span class="kw">:</span><span class="at"> </span><span class="st">"math_env.math_solver_rollout"</span><span class="co"> # Custom rollout function</span></span>
|
||||
<span id="cb42-8"><a href="#cb42-8" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">reward_funcs</span><span class="kw">:</span><span class="at"> </span><span class="kw">[</span><span class="st">"math_env.math_reward"</span><span class="kw">]</span></span>
|
||||
<span id="cb42-9"><a href="#cb42-9" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">reward_weights</span><span class="kw">:</span><span class="at"> </span><span class="kw">[</span><span class="fl">1.0</span><span class="kw">]</span></span>
|
||||
<span id="cb42-10"><a href="#cb42-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb42-11"><a href="#cb42-11" aria-hidden="true" tabindex="-1"></a><span class="fu">datasets</span><span class="kw">:</span></span>
|
||||
<span id="cb42-12"><a href="#cb42-12" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> </span><span class="fu">path</span><span class="kw">:</span><span class="at"> openai/gsm8k</span></span>
|
||||
<span id="cb42-13"><a href="#cb42-13" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">name</span><span class="kw">:</span><span class="at"> main</span></span>
|
||||
<span id="cb42-14"><a href="#cb42-14" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">type</span><span class="kw">:</span><span class="at"> math_env.math_transform</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<p>The <code>rollout_func</code> parameter accepts a fully qualified name (e.g., <code>module_name.function_name</code>) that points to a callable function in your local directory. The function receives:</p>
|
||||
<ul>
|
||||
<li><code>model</code>: The language model</li>
|
||||
<li><code>processing_class</code>: The tokenizer/processing class</li>
|
||||
<li><code>prompts</code>: List of prompt dictionaries</li>
|
||||
<li><code>generation_config</code> (optional): Generation configuration</li>
|
||||
</ul>
|
||||
<p>And should return a list of completion strings.</p>
|
||||
<p>For more OpenEnv examples, see <a href="https://huggingface.co/docs/trl/main/en/openenv">TRL OpenEnv Documentation</a>.</p>
|
||||
</section>
|
||||
<section id="grpo-with-dapodr.-grpo-loss" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="grpo-with-dapodr.-grpo-loss">GRPO with DAPO/Dr. GRPO loss</h4>
|
||||
<p>The DAPO paper and subsequently Dr. GRPO paper proposed an alternative loss function for GRPO to remediate the penalty in longer responses.</p>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb41"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb41-1"><a href="#cb41-1" aria-hidden="true" tabindex="-1"></a><span class="fu">trl</span><span class="kw">:</span></span>
|
||||
<span id="cb41-2"><a href="#cb41-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">loss_type</span><span class="kw">:</span><span class="at"> dr_grpo</span></span>
|
||||
<span id="cb41-3"><a href="#cb41-3" aria-hidden="true" tabindex="-1"></a><span class="co"> # Normalizes loss based on max completion length (default: 256)</span></span>
|
||||
<span id="cb41-4"><a href="#cb41-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">max_completion_length</span><span class="kw">:</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb43"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb43-1"><a href="#cb43-1" aria-hidden="true" tabindex="-1"></a><span class="fu">trl</span><span class="kw">:</span></span>
|
||||
<span id="cb43-2"><a href="#cb43-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">loss_type</span><span class="kw">:</span><span class="at"> dr_grpo</span></span>
|
||||
<span id="cb43-3"><a href="#cb43-3" aria-hidden="true" tabindex="-1"></a><span class="co"> # Normalizes loss based on max completion length (default: 256)</span></span>
|
||||
<span id="cb43-4"><a href="#cb43-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">max_completion_length</span><span class="kw">:</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<p>For more information, see <a href="https://huggingface.co/docs/trl/v0.17.0/en/grpo_trainer#loss-types">GRPO docs</a>.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="simpo" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="simpo">SimPO</h3>
|
||||
<p>SimPO uses <a href="https://huggingface.co/docs/trl/main/en/cpo_trainer">CPOTrainer</a> but with alternative loss function.</p>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb42"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb42-1"><a href="#cb42-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rl</span><span class="kw">:</span><span class="at"> simpo</span></span>
|
||||
<span id="cb42-2"><a href="#cb42-2" aria-hidden="true" tabindex="-1"></a><span class="fu">rl_beta</span><span class="kw">:</span><span class="at"> </span><span class="fl">0.1</span><span class="co"> # default in CPOTrainer</span></span>
|
||||
<span id="cb42-3"><a href="#cb42-3" aria-hidden="true" tabindex="-1"></a><span class="fu">cpo_alpha</span><span class="kw">:</span><span class="at"> </span><span class="fl">1.0</span><span class="co"> # default in CPOTrainer</span></span>
|
||||
<span id="cb42-4"><a href="#cb42-4" aria-hidden="true" tabindex="-1"></a><span class="fu">simpo_gamma</span><span class="kw">:</span><span class="at"> </span><span class="fl">0.5</span><span class="co"> # default in CPOTrainer</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb44"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb44-1"><a href="#cb44-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rl</span><span class="kw">:</span><span class="at"> simpo</span></span>
|
||||
<span id="cb44-2"><a href="#cb44-2" aria-hidden="true" tabindex="-1"></a><span class="fu">rl_beta</span><span class="kw">:</span><span class="at"> </span><span class="fl">0.1</span><span class="co"> # default in CPOTrainer</span></span>
|
||||
<span id="cb44-3"><a href="#cb44-3" aria-hidden="true" tabindex="-1"></a><span class="fu">cpo_alpha</span><span class="kw">:</span><span class="at"> </span><span class="fl">1.0</span><span class="co"> # default in CPOTrainer</span></span>
|
||||
<span id="cb44-4"><a href="#cb44-4" aria-hidden="true" tabindex="-1"></a><span class="fu">simpo_gamma</span><span class="kw">:</span><span class="at"> </span><span class="fl">0.5</span><span class="co"> # default in CPOTrainer</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<p>This method uses the same dataset format as <a href="#dpo">DPO</a>.</p>
|
||||
</section>
|
||||
<section id="using-local-dataset-files" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="using-local-dataset-files">Using local dataset files</h3>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb43"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb43-1"><a href="#cb43-1" aria-hidden="true" tabindex="-1"></a><span class="fu">datasets</span><span class="kw">:</span></span>
|
||||
<span id="cb43-2"><a href="#cb43-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> </span><span class="fu">ds_type</span><span class="kw">:</span><span class="at"> json</span></span>
|
||||
<span id="cb43-3"><a href="#cb43-3" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">data_files</span><span class="kw">:</span></span>
|
||||
<span id="cb43-4"><a href="#cb43-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> orca_rlhf.jsonl</span></span>
|
||||
<span id="cb43-5"><a href="#cb43-5" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">split</span><span class="kw">:</span><span class="at"> train</span></span>
|
||||
<span id="cb43-6"><a href="#cb43-6" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">type</span><span class="kw">:</span><span class="at"> chatml.intel</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb45"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb45-1"><a href="#cb45-1" aria-hidden="true" tabindex="-1"></a><span class="fu">datasets</span><span class="kw">:</span></span>
|
||||
<span id="cb45-2"><a href="#cb45-2" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> </span><span class="fu">ds_type</span><span class="kw">:</span><span class="at"> json</span></span>
|
||||
<span id="cb45-3"><a href="#cb45-3" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">data_files</span><span class="kw">:</span></span>
|
||||
<span id="cb45-4"><a href="#cb45-4" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="kw">-</span><span class="at"> orca_rlhf.jsonl</span></span>
|
||||
<span id="cb45-5"><a href="#cb45-5" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">split</span><span class="kw">:</span><span class="at"> train</span></span>
|
||||
<span id="cb45-6"><a href="#cb45-6" aria-hidden="true" tabindex="-1"></a><span class="at"> </span><span class="fu">type</span><span class="kw">:</span><span class="at"> chatml.intel</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
</section>
|
||||
<section id="trl-auto-unwrapping-for-peft" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="trl-auto-unwrapping-for-peft">TRL auto-unwrapping for PEFT</h3>
|
||||
<p>TRL supports auto-unwrapping PEFT models for RL training paradigms which rely on a reference model. This significantly reduces memory pressure as an additional refreference model does not need to be loaded, and reference model log-probabilities can be obtained by disabling PEFT adapters. This is enabled by default. To turn it off, pass the following config:</p>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb44"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb44-1"><a href="#cb44-1" aria-hidden="true" tabindex="-1"></a><span class="co"># load ref model when adapter training.</span></span>
|
||||
<span id="cb44-2"><a href="#cb44-2" aria-hidden="true" tabindex="-1"></a><span class="fu">rl_adapter_ref_model</span><span class="kw">:</span><span class="at"> </span><span class="ch">true</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb46"><pre class="sourceCode yaml code-with-copy"><code class="sourceCode yaml"><span id="cb46-1"><a href="#cb46-1" aria-hidden="true" tabindex="-1"></a><span class="co"># load ref model when adapter training.</span></span>
|
||||
<span id="cb46-2"><a href="#cb46-2" aria-hidden="true" tabindex="-1"></a><span class="fu">rl_adapter_ref_model</span><span class="kw">:</span><span class="at"> </span><span class="ch">true</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
|
||||
|
||||
|
||||
</section>
|
||||
|
||||
File diff suppressed because one or more lines are too long
398
sitemap.xml
398
sitemap.xml
@@ -2,798 +2,798 @@
|
||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html</loc>
|
||||
<lastmod>2025-11-06T21:06:11.016Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.059Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/mac.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/cli.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/nccl.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/getting-started.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/lr_groups.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/qat.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/multipack.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/streaming.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.995Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.033Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/lora_optims.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/amd_hpc.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/debugging.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/conversation.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/inst_tune.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/index.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/config-reference.html</loc>
|
||||
<lastmod>2025-11-06T21:10:13.564Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:45.096Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/multimodal.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/ray-integration.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/faq.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset_preprocessing.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/torchao.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.995Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.033Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/optimizers.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schedulers.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.688Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.351Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.sweeps.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.896Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.546Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/datasets.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.495Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.136Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.tokenization.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.607Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.265Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.tokenizer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.015Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.664Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_expand_mask.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.482Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.138Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_cpu.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.567Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.225Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.data.sft.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.738Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.402Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.transformers_fa_utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.548Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.205Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.patch_manager.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.036Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.686Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.liger.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.061Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.724Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.peft.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.828Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.493Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.pygmalion.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.227Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.880Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_instruct.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.146Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.798Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.cloud.base.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.852Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.501Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.gradient_checkpointing.offload_disk.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.599Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.257Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/kernels.swiglu.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.453Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.109Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.cut_cross_entropy.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.046Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.709Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.user_defined.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.293Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.947Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.526Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.183Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.builders.rl.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.592Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.234Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.processor.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.017Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.666Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.lisa.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.192Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.854Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.training_args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.607Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.250Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.adapter.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.023Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.673Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.merge_sharded_fsdp_weights.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.823Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.472Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.train.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.707Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.354Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.mixins.rng_state_loader.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.048Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.699Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.completion.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.193Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.846Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.stepwise_supervised.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.206Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.858Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.lora_kernels.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.516Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.173Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.messages.chat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.232Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.885Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.user_defined.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.170Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.822Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.chat.messages.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.635Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.279Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.mixins.scheduler.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.056Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.707Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.user_defined.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.269Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.923Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.llama3.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.281Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.935Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.integrations.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.856Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.521Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/convert.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.511Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.152Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.passthrough.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.271Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.925Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.config.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.778Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.442Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.enums.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.867Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.532Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.btlm_attn_hijack_flash.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.528Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.185Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chat_template.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.240Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.894Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.grpo.trainer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.976Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.625Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.lm_eval.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.065Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.728Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.collators.core.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.094Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.757Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.shared.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.640Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.285Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.orpo.chat_template.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.317Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.973Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.samplers.multipack.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.177Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.840Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.qat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.209Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.871Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.chat_template.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.127Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.779Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.multimodal.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.838Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.504Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.comet_.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.200Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.863Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.base.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.087Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.738Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/kernels.utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.464Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.119Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.merge_lora.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.808Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.458Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.861Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.511Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.ctx_managers.sequence_parallel.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.085Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.736Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/index.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.396Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.038Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.llama3.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.253Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.907Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.mixtral.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.563Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.220Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.orcamini.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.219Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.872Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.grpo.sampler.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.639Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.lora.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.615Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.274Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.mixins.optimizer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.044Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.694Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.config.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.775Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.423Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.multipack.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.476Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.132Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.collators.batching.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.117Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.780Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.quantization.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.762Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.427Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.dict.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.719Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.383Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/kernels.quantize.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.462Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.118Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.training.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.795Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.460Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/train.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.475Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.116Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.datasets.transforms.chat_builder.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.656Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.301Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/inference.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.993Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/FAQS.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.988Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.024Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/examples/colab-notebooks/colab-axolotl-example.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.999Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.039Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/index.html</loc>
|
||||
<lastmod>2025-11-06T21:06:11.011Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.054Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/custom_integrations.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.874Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.539Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/kernels.geglu.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.441Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.096Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.builders.causal.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.586Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.229Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.mamba.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.954Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.603Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.bradley_terry.llama3.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.322Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.977Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.datasets.chat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.647Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.291Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.collators.mm_chat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.127Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.790Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.llama2_chat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.186Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.838Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/common.const.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.073Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.736Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.quantize.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.839Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.489Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.trainer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.655Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.317Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.delinearize_llama4.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.781Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.429Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/evaluate.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.487Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.129Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.mistral_attn_hijack_flash.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.474Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.130Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.model.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.005Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.654Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.distributed.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.713Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.376Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.model_shard_quant.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.621Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.281Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/kernels.lora.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.428Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.084Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.main.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.697Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.343Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.spectrum.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.069Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.732Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.optimizers.adopt.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.729Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.393Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.cloud.modal_.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.859Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.509Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_flash.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.470Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.126Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.builders.base.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.581Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.223Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.trl.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.832Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.497Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.876Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.525Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.base.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.929Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.578Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_patch_multipack.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.529Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.186Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.llama_attn_hijack_xformers.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.472Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.128Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.model.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.787Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.451Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.kto.chatml.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.291Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.945Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.mlflow_.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.196Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.858Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/common.datasets.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.091Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.754Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.schemas.datasets.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.817Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.482Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.fetch.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.882Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.532Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.chatml.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.266Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.919Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.relora.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.480Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.136Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.evaluate.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.717Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.364Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.dpo.zephyr.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.268Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.921Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.utils.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.993Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.641Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_w_system.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.161Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.812Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.chat_templates.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.609Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.267Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.data.streaming.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.730Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.395Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.bench.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.626Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.286Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/common.architectures.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.071Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.734Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.checks.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.753Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.401Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.dpo.trainer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.963Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.611Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.base.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.042Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.705Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.train.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.911Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.560Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.collators.mamba.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.122Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.785Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.art.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.745Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.393Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.trainer_fsdp_optim.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.540Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.198Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/logging_config.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.573Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.215Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.freeze.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.635Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.296Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.metharme.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.214Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.867Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.alpaca_chat.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.144Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.796Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.stablelm_attn_hijack_flash.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.536Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.193Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/models.mamba.modeling_mamba.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.093Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.755Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.trainers.trl.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.947Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.596Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_strategies.input_output.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.201Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.853Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/loaders.constants.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.037Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.687Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.data.batch_dataset_fetcher.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.561Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.219Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.vllm_serve.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.848Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.497Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/prompt_tokenizers.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.561Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.203Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.args.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.741Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.389Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.inference.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.798Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.447Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.utils.load.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.889Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.539Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/cli.preprocess.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.833Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.483Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.profiler.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.190Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.852Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/utils.callbacks.perplexity.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.185Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.847Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.chatml.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.637Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.281Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.grokfast.optimizer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.048Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.710Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/integrations.kd.trainer.html</loc>
|
||||
<lastmod>2025-11-06T21:09:58.057Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.720Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/monkeypatch.unsloth_.html</loc>
|
||||
<lastmod>2025-11-06T21:09:57.550Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:28.207Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/api/core.chat.format.llama3x.html</loc>
|
||||
<lastmod>2025-11-06T21:09:56.639Z</lastmod>
|
||||
<lastmod>2025-11-07T17:21:27.283Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/reward_modelling.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/quantize.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/fsdp_qlora.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/nd_parallelism.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/batch_vs_grad.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/multi-node.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/rlhf.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/stepwise_supervised.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/pretraining.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.026Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/tokenized.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset-formats/template_free.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/multi-gpu.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/input_output.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.993Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/docker.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/gradient_checkpointing.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.991Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/optimizations.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/sequence_parallelism.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.995Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.032Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/dataset_loading.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.990Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.027Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/installation.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/mixed_precision.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.994Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.031Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/docs/unsloth.html</loc>
|
||||
<lastmod>2025-11-06T21:06:10.995Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.033Z</lastmod>
|
||||
</url>
|
||||
<url>
|
||||
<loc>https://docs.axolotl.ai/src/axolotl/integrations/LICENSE.html</loc>
|
||||
<lastmod>2025-11-06T21:06:11.016Z</lastmod>
|
||||
<lastmod>2025-11-07T17:17:37.059Z</lastmod>
|
||||
</url>
|
||||
</urlset>
|
||||
|
||||
Reference in New Issue
Block a user