tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2026-04-23 04:33:48 +00:00
parent c7ad3c8e22
commit ac48bfadba
249 changed files with 8526 additions and 8515 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/agents/preference_tuning.html
View File

@@ -42,7 +42,7 @@ ul.task-list li input[type="checkbox"] {
<link href="../../site_libs/quarto-html/quarto-syntax-highlighting-dark-d0ae9245876894da5ac7e18953ecc5cc.css" rel="stylesheet" id="quarto-text-highlighting-styles">
<script src="../../site_libs/bootstrap/bootstrap.min.js"></script>
<link href="../../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
<link href="../../site_libs/bootstrap/bootstrap-b7aea7e464dd78f23decae44cf02da44.min.css" rel="stylesheet" append-hash="true" id="quarto-bootstrap" data-mode="dark">
<link href="../../site_libs/bootstrap/bootstrap-ab6ebd6eb475c4578b58908bc314f719.min.css" rel="stylesheet" append-hash="true" id="quarto-bootstrap" data-mode="dark">
<script id="quarto-search-options" type="application/json">{
"location": "navbar",
"copy-button": false,
@@ -842,7 +842,7 @@ Exception: ORPO and SimPO do NOT use a reference model (~50% less VRAM).</code><
<li>Paired preference data (chosen + rejected)?
<ul>
<li>Default → <code>rl: dpo</code></li>
<li>Overfitting → <code>rl: ipo</code></li>
<li>Overfitting → <code>rl: dpo, dpo_loss_type: ["ipo"]</code></li>
<li>VRAM-limited → <code>rl: orpo</code> (no ref model)</li>
<li>Length-sensitive → <code>rl: simpo</code> (no ref model)</li>
</ul></li>