quartodoc integration

docs/api/utils.models.qmd

@@ -0,0 +1,161 @@
+# utils.models { #axolotl.utils.models }
+
+`utils.models`
+
+Module for models and model loading
+
+## Classes
+
+| Name | Description |
+| --- | --- |
+| [ModelLoader](#axolotl.utils.models.ModelLoader) | ModelLoader: managing all the config and monkey patches while loading model |
+
+### ModelLoader { #axolotl.utils.models.ModelLoader }
+
+```python
+utils.models.ModelLoader(
+    self,
+    cfg,
+    tokenizer,
+    *,
+    processor=None,
+    inference=False,
+    reference_model=False,
+    **kwargs,
+)
+```
+
+ModelLoader: managing all the config and monkey patches while loading model
+
+#### Attributes
+
+| Name | Description |
+| --- | --- |
+| [has_flash_attn](#axolotl.utils.models.ModelLoader.has_flash_attn) | Check if flash attention is installed |
+
+#### Methods
+
+| Name | Description |
+| --- | --- |
+| [patch_llama_derived_model](#axolotl.utils.models.ModelLoader.patch_llama_derived_model) | Modify all llama derived models in one block |
+| [patch_loss_llama](#axolotl.utils.models.ModelLoader.patch_loss_llama) | Patch loss functions and other optimizations |
+| [set_attention_config](#axolotl.utils.models.ModelLoader.set_attention_config) | sample packing uses custom FA2 patch |
+| [set_auto_model_loader](#axolotl.utils.models.ModelLoader.set_auto_model_loader) | set self.AutoModelLoader |
+
+##### patch_llama_derived_model { #axolotl.utils.models.ModelLoader.patch_llama_derived_model }
+
+```python
+utils.models.ModelLoader.patch_llama_derived_model()
+```
+
+Modify all llama derived models in one block
+
+##### patch_loss_llama { #axolotl.utils.models.ModelLoader.patch_loss_llama }
+
+```python
+utils.models.ModelLoader.patch_loss_llama()
+```
+
+Patch loss functions and other optimizations
+
+##### set_attention_config { #axolotl.utils.models.ModelLoader.set_attention_config }
+
+```python
+utils.models.ModelLoader.set_attention_config()
+```
+
+sample packing uses custom FA2 patch
+
+##### set_auto_model_loader { #axolotl.utils.models.ModelLoader.set_auto_model_loader }
+
+```python
+utils.models.ModelLoader.set_auto_model_loader()
+```
+
+set self.AutoModelLoader
+- default value: AutoModelForCausalLM (set at __init__)
+- when using a multi modality model, self.AutoModelLoader should
+  be set according to model type of the model
+
+## Functions
+
+| Name | Description |
+| --- | --- |
+| [get_module_class_from_name](#axolotl.utils.models.get_module_class_from_name) | Gets a class from a module by its name. |
+| [load_model](#axolotl.utils.models.load_model) | Load a model for a given configuration and tokenizer. |
+| [load_tokenizer](#axolotl.utils.models.load_tokenizer) | Load and configure the tokenizer based on the provided config. |
+| [modify_tokenizer_files](#axolotl.utils.models.modify_tokenizer_files) | Modify tokenizer files to replace added_tokens strings, save to output directory, and return the path to the modified tokenizer. |
+| [setup_quantized_meta_for_peft](#axolotl.utils.models.setup_quantized_meta_for_peft) | Replaces `quant_state.to` with a dummy function to prevent PEFT from moving `quant_state` to meta device |
+| [setup_quantized_peft_meta_for_training](#axolotl.utils.models.setup_quantized_peft_meta_for_training) | Replaces dummy `quant_state.to` method with the original function to allow training to continue |
+
+### get_module_class_from_name { #axolotl.utils.models.get_module_class_from_name }
+
+```python
+utils.models.get_module_class_from_name(module, name)
+```
+
+Gets a class from a module by its name.
+
+Args:
+    module (`torch.nn.Module`): The module to get the class from.
+    name (`str`): The name of the class.
+
+### load_model { #axolotl.utils.models.load_model }
+
+```python
+utils.models.load_model(
+    cfg,
+    tokenizer,
+    *,
+    processor=None,
+    inference=False,
+    reference_model=False,
+    **kwargs,
+)
+```
+
+Load a model for a given configuration and tokenizer.
+
+### load_tokenizer { #axolotl.utils.models.load_tokenizer }
+
+```python
+utils.models.load_tokenizer(cfg)
+```
+
+Load and configure the tokenizer based on the provided config.
+
+### modify_tokenizer_files { #axolotl.utils.models.modify_tokenizer_files }
+
+```python
+utils.models.modify_tokenizer_files(tokenizer_path, token_mappings, output_dir)
+```
+
+Modify tokenizer files to replace added_tokens strings, save to output directory, and return the path to the modified tokenizer.
+
+This only works with reserved tokens that were added to the tokenizer, not tokens already part of the vocab.
+
+Args:
+    tokenizer_path: Path or name of the original tokenizer
+    token_mappings: Dict mapping {token_id (int): new_token_string}
+    output_dir: Directory to save the modified tokenizer
+
+Returns:
+    Path to the modified tokenizer directory
+
+Ref: https://github.com/huggingface/transformers/issues/27974#issuecomment-1854188941
+
+### setup_quantized_meta_for_peft { #axolotl.utils.models.setup_quantized_meta_for_peft }
+
+```python
+utils.models.setup_quantized_meta_for_peft(model)
+```
+
+Replaces `quant_state.to` with a dummy function to prevent PEFT from moving `quant_state` to meta device
+
+### setup_quantized_peft_meta_for_training { #axolotl.utils.models.setup_quantized_peft_meta_for_training }
+
+```python
+utils.models.setup_quantized_peft_meta_for_training(model)
+```
+
+Replaces dummy `quant_state.to` method with the original function to allow training to continue