# cli.inference – Axolotl

# cli.inference

`cli.inference`

CLI to run inference on a trained model.

## Functions

Name | Description
--- | ---
do_cli | Parses axolotl config, CLI args, and calls `do_inference` or `do_inference_gradio`.
do_inference | Runs inference on the command line in a loop. User input is accepted, a chat template
do_inference_gradio | Runs inference in a Gradio interface. User input is accepted, a chat template is
get_multi_line_input | Gets multi-line input from terminal.

### do_cli

    cli.inference.do_cli(config=Path('examples/'), gradio=False, **kwargs)

Parses axolotl config, CLI args, and calls `do_inference` or `do_inference_gradio`.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
config | Union[Path, str] | Path to `axolotl` config YAML file. | `Path('examples/')`
kwargs |  | Additional keyword arguments to override config file values. | `{}`

### do_inference

    cli.inference.do_inference(cfg, cli_args)

Runs inference on the command line in a loop. User input is accepted, a chat template is (optionally) applied, and the model specified in the `axolotl` config is used to generate completions according to a default generation config.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
cfg | DictDefault | Dictionary mapping `axolotl` config keys to values. | required
cli_args | InferenceCliArgs | Inference-specific CLI arguments. | required

### do_inference_gradio

    cli.inference.do_inference_gradio(cfg, cli_args)

Runs inference in a Gradio interface. User input is accepted, a chat template is (optionally) applied, and the model specified in the `axolotl` config is used to generate completions according to a default generation config.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
cfg | DictDefault | Dictionary mapping `axolotl` config keys to values. | required
cli_args | InferenceCliArgs | Inference-specific CLI arguments. | required

### get_multi_line_input

    cli.inference.get_multi_line_input()

Gets multi-line input from terminal.

#### Returns

Name | Type | Description
--- | --- | ---
| str | Possibly multi-line, possibly empty stdin input as a string.