tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Built site for gh-pages

Browse Source
This commit is contained in:
Quarto GHA Workflow Runner
2025-08-18 12:50:03 +00:00
parent 866a618cb4
commit ee80de3cfb
5 changed files with 808 additions and 803 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
docs/api/cli.args.html
View File

@@ -605,16 +605,17 @@ gtag('config', 'G-9KYCVJBNMQ', { 'anonymize_ip': true});
<h3 class="anchored" data-anchor-id="axolotl.cli.args.VllmServeCliArgs">VllmServeCliArgs</h3>
<div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>cli.args.VllmServeCliArgs(</span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> tensor_parallel_size<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> host<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> port<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> gpu_memory_utilization<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> dtype<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> max_model_len<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> enable_prefix_caching<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> serve_module<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> enable_reasoning<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a> reasoning_parser<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> data_parallel_size<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> host<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> port<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> gpu_memory_utilization<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> dtype<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> max_model_len<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> enable_prefix_caching<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> serve_module<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a> enable_reasoning<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a> reasoning_parser<span class="op">=</span><span class="va">None</span>,</span>
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Dataclass with CLI arguments for <code>axolotl vllm-serve</code> command.</p>