# utils.models { #axolotl.utils.models }

`utils.models`

Module for models and model loading

## Classes

| Name | Description |
| --- | --- |
| [ModelLoader](#axolotl.utils.models.ModelLoader) | ModelLoader: managing all the config and monkey patches while loading model |

### ModelLoader { #axolotl.utils.models.ModelLoader }

```python
utils.models.ModelLoader(
    self,
    cfg,
    tokenizer,
    *,
    processor=None,
    inference=False,
    reference_model=False,
    **kwargs,
)
```

ModelLoader: managing all the config and monkey patches while loading model

#### Attributes

| Name | Description |
| --- | --- |
| [has_flash_attn](#axolotl.utils.models.ModelLoader.has_flash_attn) | Check if flash attention is installed |

#### Methods

| Name | Description |
| --- | --- |
| [patch_llama_derived_model](#axolotl.utils.models.ModelLoader.patch_llama_derived_model) | Modify all llama derived models in one block |
| [patch_loss_llama](#axolotl.utils.models.ModelLoader.patch_loss_llama) | Patch loss functions and other optimizations |
| [set_attention_config](#axolotl.utils.models.ModelLoader.set_attention_config) | sample packing uses custom FA2 patch |
| [set_auto_model_loader](#axolotl.utils.models.ModelLoader.set_auto_model_loader) | set self.AutoModelLoader |

##### patch_llama_derived_model { #axolotl.utils.models.ModelLoader.patch_llama_derived_model }

```python
utils.models.ModelLoader.patch_llama_derived_model()
```

Modify all llama derived models in one block

##### patch_loss_llama { #axolotl.utils.models.ModelLoader.patch_loss_llama }

```python
utils.models.ModelLoader.patch_loss_llama()
```

Patch loss functions and other optimizations

##### set_attention_config { #axolotl.utils.models.ModelLoader.set_attention_config }

```python
utils.models.ModelLoader.set_attention_config()
```

sample packing uses custom FA2 patch

##### set_auto_model_loader { #axolotl.utils.models.ModelLoader.set_auto_model_loader }

```python
utils.models.ModelLoader.set_auto_model_loader()
```

set self.AutoModelLoader
- default value: AutoModelForCausalLM (set at __init__)
- when using a multi modality model, self.AutoModelLoader should
  be set according to model type of the model

## Functions

| Name | Description |
| --- | --- |
| [get_module_class_from_name](#axolotl.utils.models.get_module_class_from_name) | Gets a class from a module by its name. |
| [load_model](#axolotl.utils.models.load_model) | Load a model for a given configuration and tokenizer. |
| [load_tokenizer](#axolotl.utils.models.load_tokenizer) | Load and configure the tokenizer based on the provided config. |
| [modify_tokenizer_files](#axolotl.utils.models.modify_tokenizer_files) | Modify tokenizer files to replace added_tokens strings, save to output directory, and return the path to the modified tokenizer. |
| [setup_quantized_meta_for_peft](#axolotl.utils.models.setup_quantized_meta_for_peft) | Replaces `quant_state.to` with a dummy function to prevent PEFT from moving `quant_state` to meta device |
| [setup_quantized_peft_meta_for_training](#axolotl.utils.models.setup_quantized_peft_meta_for_training) | Replaces dummy `quant_state.to` method with the original function to allow training to continue |

### get_module_class_from_name { #axolotl.utils.models.get_module_class_from_name }

```python
utils.models.get_module_class_from_name(module, name)
```

Gets a class from a module by its name.

Args:
    module (`torch.nn.Module`): The module to get the class from.
    name (`str`): The name of the class.

### load_model { #axolotl.utils.models.load_model }

```python
utils.models.load_model(
    cfg,
    tokenizer,
    *,
    processor=None,
    inference=False,
    reference_model=False,
    **kwargs,
)
```

Load a model for a given configuration and tokenizer.

### load_tokenizer { #axolotl.utils.models.load_tokenizer }

```python
utils.models.load_tokenizer(cfg)
```

Load and configure the tokenizer based on the provided config.

### modify_tokenizer_files { #axolotl.utils.models.modify_tokenizer_files }

```python
utils.models.modify_tokenizer_files(tokenizer_path, token_mappings, output_dir)
```

Modify tokenizer files to replace added_tokens strings, save to output directory, and return the path to the modified tokenizer.

This only works with reserved tokens that were added to the tokenizer, not tokens already part of the vocab.

Args:
    tokenizer_path: Path or name of the original tokenizer
    token_mappings: Dict mapping {token_id (int): new_token_string}
    output_dir: Directory to save the modified tokenizer

Returns:
    Path to the modified tokenizer directory

Ref: https://github.com/huggingface/transformers/issues/27974#issuecomment-1854188941

### setup_quantized_meta_for_peft { #axolotl.utils.models.setup_quantized_meta_for_peft }

```python
utils.models.setup_quantized_meta_for_peft(model)
```

Replaces `quant_state.to` with a dummy function to prevent PEFT from moving `quant_state` to meta device

### setup_quantized_peft_meta_for_training { #axolotl.utils.models.setup_quantized_peft_meta_for_training }

```python
utils.models.setup_quantized_peft_meta_for_training(model)
```

Replaces dummy `quant_state.to` method with the original function to allow training to continue
