# cli.utils

`cli.utils`

Utility methods for axolotl CLI.

## Functions

Name | Description
--- | ---
add_options_from_config | Create Click options from the fields of a Pydantic model.
add_options_from_dataclass | Create Click options from the fields of a dataclass.
build_command | Build command list from base command and options.
download_file | Download a single file and return its processing status.
fetch_from_github | Sync files from a specific directory in the GitHub repository.
filter_none_kwargs | Wraps function to remove `None`-valued `kwargs`.
load_model_and_tokenizer | Helper function for loading a model, tokenizer, and processor specified in the given `axolotl`
strip_optional_type | Extracts the non-`None` type from an `Optional` / `Union` type.

### add_options_from_config

    cli.utils.add_options_from_config(config_class)

Create Click options from the fields of a Pydantic model.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
config_class | Type[BaseModel] | PyDantic model with fields to parse from the CLI | required

#### Returns

Name | Type | Description
--- | --- | ---
| Callable | Function decorator for Axolotl CLI command.

### add_options_from_dataclass

    cli.utils.add_options_from_dataclass(config_class)

Create Click options from the fields of a dataclass.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
config_class | Type[Any] | Dataclass with fields to parse from the CLI. | required

#### Returns

Name | Type | Description
--- | --- | ---
| Callable | Function decorator for Axolotl CLI command.

### build_command

    cli.utils.build_command(base_cmd, options)

Build command list from base command and options.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
base_cmd | list[str] | Command without options. | required
options | dict[str, Any] | Options to parse and append to base command. | required

#### Returns

Name | Type | Description
--- | --- | ---
| list[str] | List of strings giving shell command.

### download_file

    cli.utils.download_file(file_info, raw_base_url, dest_path, dir_prefix)

Download a single file and return its processing status.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
file_info | tuple | Tuple of (file_path, remote_sha). | required
raw_base_url | str | Base URL for raw GitHub content. | required
dest_path | Path | Local destination directory. | required
dir_prefix | str | Directory prefix to filter files. | required

#### Returns

Name | Type | Description
--- | --- | ---
| tuple[str, str] | Tuple of (file_path, status) where status is ‘new’, ‘updated’, or ‘unchanged’.

### fetch_from_github

    cli.utils.fetch_from_github(dir_prefix, dest_dir=None, max_workers=5)

Sync files from a specific directory in the GitHub repository. Only downloads files that don’t exist locally or have changed.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
dir_prefix | str | Directory prefix to filter files (e.g., ‘examples/’, ‘deepspeed_configs/’). | required
dest_dir | str | None | Local destination directory. | `None`
max_workers | int | Maximum number of concurrent downloads. | `5`

### filter_none_kwargs

    cli.utils.filter_none_kwargs(func)

Wraps function to remove `None`-valued `kwargs`.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
func | Callable | Function to wrap. | required

#### Returns

Name | Type | Description
--- | --- | ---
| Callable | Wrapped function.

### load_model_and_tokenizer

    cli.utils.load_model_and_tokenizer(cfg, inference=False)

Helper function for loading a model, tokenizer, and processor specified in the given `axolotl` config.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
cfg | DictDefault | Dictionary mapping `axolotl` config keys to values. | required
inference | bool | Boolean denoting inference mode. | `False`

#### Returns

Name | Type | Description
--- | --- | ---
| tuple[PreTrainedModel, PreTrainedTokenizer | PreTrainedTokenizerFast | Any, ProcessorMixin | None] | Tuple of (PreTrainedModel, PreTrainedTokenizer, ProcessorMixin).

### strip_optional_type

    cli.utils.strip_optional_type(field_type)

Extracts the non-`None` type from an `Optional` / `Union` type.

#### Parameters

Name | Type | Description | Default
--- | --- | --- | ---
field_type | type | str | None | Type of field for Axolotl CLI command. | required

#### Returns

Name | Type | Description
--- | --- | ---
|  | If the input type is `Union[T, None]` or `Optional[T]`, returns `T`. Otherwise returns the input type unchanged.