diff --git a/.nojekyll b/.nojekyll index cb54a878f..01b3f75e6 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -e07abf35 \ No newline at end of file +4fb42e45 \ No newline at end of file diff --git a/docs/api/index.html b/docs/api/index.html index 13d3a976a..3f1f280c0 100644 --- a/docs/api/index.html +++ b/docs/api/index.html @@ -416,6 +416,7 @@ ul.task-list li input[type="checkbox"] {
  • Core
  • CLI
  • Trainers
    • +
  • Model Loading
  • Mixins
  • Context Managers
  • Prompt Strategies
    • @@ -612,6 +613,38 @@ ul.task-list li input[type="checkbox"] { +
    +

    Model Loading

    +

    Functionality for loading and patching models, tokenizers, etc.

    + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    loaders.modelModel loader class implementation for loading, configuring, and patching various
    loaders.tokenizerTokenizer loading functionality and associated utils
    loaders.processorProcessor loading functionality for multi-modal models
    loaders.adapterAdapter loading functionality, including LoRA / QLoRA and associated utils
    loaders.patch_managerPatch manager class implementation to complement axolotl.loaders.ModelLoader.
    loaders.constantsShared constants for axolotl.loaders module
    +

    Mixins

    Mixin classes for augmenting trainers

    @@ -853,6 +886,14 @@ ul.task-list li input[type="checkbox"] { monkeypatch.mixtral Patches to support multipack for mixtral + +monkeypatch.gradient_checkpointing.offload_cpu +CPU offloaded checkpointing + + +monkeypatch.gradient_checkpointing.offload_disk +DISCO - DIsk-based Storage and Checkpointing with Optimized prefetching +
    @@ -862,25 +903,17 @@ ul.task-list li input[type="checkbox"] { - - - - - + - + - - - - @@ -921,14 +954,6 @@ ul.task-list li input[type="checkbox"] { - - - - - - - -
    utils.modelsModule for models and model loading
    utils.tokenization Module for tokenization utilities
    utils.chat_templates This module provides functionality for selecting chat templates based on user choices.
    utils.lora module to get the state dict of a merged lora model
    utils.lora_embeddingshelpers for lora embeddings
    utils.model_shard_quant module to handle loading model on cpu/meta device for FSDPutils.data.sft data handling specific to SFT
    utils.gradient_checkpointing.offload_cpuCPU offloaded checkpointing
    utils.gradient_checkpointing.offload_diskDISCO - DIsk-based Storage and Checkpointing with Optimized prefetching
    diff --git a/docs/api/integrations.base.html b/docs/api/integrations.base.html index 90c7994fe..7024243d2 100644 --- a/docs/api/integrations.base.html +++ b/docs/api/integrations.base.html @@ -495,7 +495,7 @@ Plugins can be used to integrate third-party models, modify the training process PluginManager -The PluginManager class is responsible for loading and managing plugins. +The PluginManager class is responsible for loading and managing plugins. It @@ -508,21 +508,33 @@ Plugins can be used to integrate third-party models, modify the training process

    BasePlugin

    integrations.base.BasePlugin(self)

    Base class for all plugins. Defines the interface for plugin methods.

    -

    Attributes: -None

    -

    Methods: -register(cfg): Registers the plugin with the given configuration. -load_datasets(cfg): Loads and preprocesses the dataset for training. -pre_model_load(cfg): Performs actions before the model is loaded. -post_model_build(cfg, model): Performs actions after the model is loaded, but before LoRA adapters are applied. -pre_lora_load(cfg, model): Performs actions before LoRA weights are loaded. -post_lora_load(cfg, model): Performs actions after LoRA weights are loaded. -post_model_load(cfg, model): Performs actions after the model is loaded, inclusive of any adapters. -post_trainer_create(cfg, trainer): Performs actions after the trainer is created. -create_optimizer(cfg, trainer): Creates and returns an optimizer for training. -create_lr_scheduler(cfg, trainer, optimizer, num_training_steps): Creates and returns a learning rate scheduler. -add_callbacks_pre_trainer(cfg, model): Adds callbacks to the trainer before training. -add_callbacks_post_trainer(cfg, trainer): Adds callbacks to the trainer after training.

    +

    A plugin is a reusable, modular, and self-contained piece of code that extends +the functionality of Axolotl. Plugins can be used to integrate third-party models, +modify the training process, or add new features.

    +

    To create a new plugin, you need to inherit from the BasePlugin class and +implement the required methods.

    +
    +

    Note

    +

    Plugin methods include: +- register(cfg): Registers the plugin with the given configuration. +- load_datasets(cfg): Loads and preprocesses the dataset for training. +- pre_model_load(cfg): Performs actions before the model is loaded. +- post_model_build(cfg, model): Performs actions after the model is loaded, but +before LoRA adapters are applied. +- pre_lora_load(cfg, model): Performs actions before LoRA weights are loaded. +- post_lora_load(cfg, model): Performs actions after LoRA weights are loaded. +- post_model_load(cfg, model): Performs actions after the model is loaded, +inclusive of any adapters. +- post_trainer_create(cfg, trainer): Performs actions after the trainer is +created. +- create_optimizer(cfg, trainer): Creates and returns an optimizer for training. +- create_lr_scheduler(cfg, trainer, optimizer, num_training_steps): Creates and +returns a learning rate scheduler. +- add_callbacks_pre_trainer(cfg, model): Adds callbacks to the trainer before +training. +- add_callbacks_post_trainer(cfg, trainer): Adds callbacks to the trainer after +training.

    +

    Methods

    @@ -535,11 +547,11 @@ add_callbacks_post_trainer(cfg, trainer): Adds callbacks to the trainer after tr - + - + @@ -602,11 +614,17 @@ add_callbacks_post_trainer(cfg, trainer): Adds callbacks to the trainer after tr
    add_callbacks_post_trainer
    integrations.base.BasePlugin.add_callbacks_post_trainer(cfg, trainer)
    -

    Adds callbacks to the trainer after creating the trainer. -This is useful for callbacks that require access to the model or trainer.

    +

    Adds callbacks to the trainer after creating the trainer. This is useful for +callbacks that require access to the model or trainer.

    Parameters
    add_callbacks_post_trainerAdds callbacks to the trainer after creating the trainer.Adds callbacks to the trainer after creating the trainer. This is useful for
    add_callbacks_pre_trainersetup callbacks before creating the trainer.Set up callbacks before creating the trainer.
    create_lr_scheduler
    ++++++ @@ -618,13 +636,13 @@ This is useful for callbacks that require access to the model or trainer.

    - + - + @@ -634,11 +652,6 @@ This is useful for callbacks that require access to the model or trainer.

    Returns
    Name
    cfgdictDictDefault The configuration for the plugin. required
    trainerobjectTrainer The trainer object for training. required
    ----- @@ -649,8 +662,8 @@ This is useful for callbacks that require access to the model or trainer.

    - - + +
    Name
    List[callable]: A list of callback functions to be addedlist[Callable]A list of callback functions to be added
    @@ -659,10 +672,16 @@ This is useful for callbacks that require access to the model or trainer.

    add_callbacks_pre_trainer
    integrations.base.BasePlugin.add_callbacks_pre_trainer(cfg, model)
    -

    setup callbacks before creating the trainer.

    +

    Set up callbacks before creating the trainer.

    Parameters
    ++++++ @@ -674,13 +693,13 @@ This is useful for callbacks that require access to the model or trainer.

    - + - + @@ -692,8 +711,8 @@ This is useful for callbacks that require access to the model or trainer.

    Name
    cfgdictDictDefault The configuration for the plugin. required
    modelobjectPreTrainedModel The loaded model. required
    --++ @@ -705,8 +724,8 @@ This is useful for callbacks that require access to the model or trainer.

    - - + +
    List[callable]: A list of callback functions to be added to the TrainingArgslist[Callable]A list of callback functions to be added to the TrainingArgs.
    @@ -725,10 +744,10 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ---+++ @@ -741,19 +760,19 @@ This is useful for callbacks that require access to the model or trainer.

    - + - + - + @@ -778,8 +797,8 @@ This is useful for callbacks that require access to the model or trainer.

    - - + + @@ -793,6 +812,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    cfgdictDictDefault The configuration for the plugin. required
    trainerobjectTrainer The trainer object for training. required
    optimizerobjectOptimizer The optimizer for training. required
    objectLRSchedulerLRScheduler | None The created learning rate scheduler.
    ++++++ @@ -804,13 +829,13 @@ This is useful for callbacks that require access to the model or trainer.

    - + - + @@ -829,8 +854,8 @@ This is useful for callbacks that require access to the model or trainer.

    - + @@ -849,6 +874,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    Name
    cfgdictDictDefault The configuration for the plugin. required
    trainerobjectTrainer The trainer object for training. required
    object Optimizer | None The created optimizer.
    ++++++ @@ -860,7 +891,7 @@ This is useful for callbacks that require access to the model or trainer.

    - + @@ -870,6 +901,11 @@ This is useful for callbacks that require access to the model or trainer.

    Returns
    Name
    cfgdictDictDefault The global axolotl configuration. required
    +++++ @@ -879,9 +915,9 @@ This is useful for callbacks that require access to the model or trainer.

    - - + +
    Name
    class The class for the trainer.Trainer | NoneThe first non-None trainer class returned by a plugin.
    @@ -927,6 +963,11 @@ This is useful for callbacks that require access to the model or trainer.

    Returns
    +++++ @@ -937,7 +978,7 @@ This is useful for callbacks that require access to the model or trainer.

    - + @@ -951,6 +992,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    Name
    dataset_metaUnion['TrainDatasetMeta', None] The metadata for the training dataset.
    ++++++ @@ -962,38 +1009,19 @@ This is useful for callbacks that require access to the model or trainer.

    - + - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    modelobjectPreTrainedModel | PeftModel The loaded model. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    post_model_build
    @@ -1002,6 +1030,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1013,7 +1047,7 @@ This is useful for callbacks that require access to the model or trainer.

    - + @@ -1028,6 +1062,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    Name
    cfgdictDictDefault The configuration for the plugin. required
    ++++++ @@ -1039,38 +1079,19 @@ This is useful for callbacks that require access to the model or trainer.

    - + - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    modelobjectPreTrainedModel | PeftModel The loaded model. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    post_train
    @@ -1079,6 +1100,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1090,38 +1117,19 @@ This is useful for callbacks that require access to the model or trainer.

    - - + + - +
    Name
    cfgdictThe axolotl configurationDictDefaultThe axolotl configuration. required
    modelobjectPreTrainedModel | PeftModel The loaded model. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    post_train_unload
    @@ -1130,6 +1138,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1141,32 +1155,13 @@ This is useful for callbacks that require access to the model or trainer.

    - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    post_trainer_create
    @@ -1175,6 +1170,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1186,38 +1187,19 @@ This is useful for callbacks that require access to the model or trainer.

    - + - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    trainerobjectTrainer The trainer object for training. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    pre_lora_load
    @@ -1226,6 +1208,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1237,38 +1225,19 @@ This is useful for callbacks that require access to the model or trainer.

    - + - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    modelobjectPreTrainedModel The loaded model. required
    -
    -
    Returns
    - - - - - - - - - - - - - - - -
    NameTypeDescription
    None
    -
    pre_model_load
    @@ -1277,6 +1246,12 @@ This is useful for callbacks that require access to the model or trainer.

    Parameters
    ++++++ @@ -1288,15 +1263,55 @@ This is useful for callbacks that require access to the model or trainer.

    - +
    Name
    cfgdictDictDefault The configuration for the plugin. required
    -
    -
    Returns
    +
    +
    +
    register
    +
    integrations.base.BasePlugin.register(cfg)
    +

    Registers the plugin with the given configuration.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugin.required
    +
    +
    +
    +
    +
    +

    PluginManager

    +
    integrations.base.PluginManager()
    +

    The PluginManager class is responsible for loading and managing plugins. It +should be a singleton so it can be accessed from anywhere in the codebase.

    +
    +

    Attributes

    @@ -1307,36 +1322,20 @@ This is useful for callbacks that require access to the model or trainer.

    - - - + + +
    NonepluginsOrderedDict[str, BasePlugin]A list of loaded plugins.
    +
    +

    Note

    +

    Key methods include: +- get_instance(): Static method to get the singleton instance of PluginManager. +- register(plugin_name: str): Registers a new plugin by its name. +- pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.

    -
    -
    register
    -
    integrations.base.BasePlugin.register(cfg)
    -

    Registers the plugin with the given configuration.

    -

    Parameters: -cfg (dict): The configuration for the plugin.

    -

    Returns: -None

    -
    -
    -
    -
    -

    PluginManager

    -
    integrations.base.PluginManager()
    -

    The PluginManager class is responsible for loading and managing plugins. -It should be a singleton so it can be accessed from anywhere in the codebase.

    -

    Attributes: -plugins (ListBasePlugin): A list of loaded plugins.

    -

    Methods: -get_instance(): Static method to get the singleton instance of PluginManager. -register(plugin_name: str): Registers a new plugin by its name. -pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.

    Methods

    @@ -1349,7 +1348,7 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.< - + @@ -1357,11 +1356,11 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.< - + - + @@ -1369,11 +1368,11 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.< - + - + @@ -1381,15 +1380,15 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.< - + - + - + @@ -1401,11 +1400,11 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.< - + - + @@ -1420,80 +1419,15 @@ pre_model_load(cfg): Calls the pre_model_load method of all registered plugins.<
    add_callbacks_post_trainer
    integrations.base.PluginManager.add_callbacks_post_trainer(cfg, trainer)
    -

    Calls the add_callbacks_post_trainer method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -trainer (object): The trainer object for training.

    -

    Returns: -List[callable]: A list of callback functions to be added to the TrainingArgs.

    -
    -
    -
    add_callbacks_pre_trainer
    -
    integrations.base.PluginManager.add_callbacks_pre_trainer(cfg, model)
    -

    Calls the add_callbacks_pre_trainer method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -List[callable]: A list of callback functions to be added to the TrainingArgs.

    -
    -
    -
    create_lr_scheduler
    -
    integrations.base.PluginManager.create_lr_scheduler(
-    trainer,
-    optimizer,
-    num_training_steps,
-)
    -

    Calls the create_lr_scheduler method of all registered plugins and returns the first non-None scheduler.

    -

    Parameters: -trainer (object): The trainer object for training. -optimizer (object): The optimizer for training.

    -

    Returns: -object: The created learning rate scheduler, or None if none was found.

    -
    -
    -
    create_optimizer
    -
    integrations.base.PluginManager.create_optimizer(trainer)
    -

    Calls the create_optimizer method of all registered plugins and returns the first non-None optimizer.

    -

    Parameters: -trainer (object): The trainer object for training.

    -

    Returns: -object: The created optimizer, or None if none was found.

    -
    -
    -
    get_input_args
    -
    integrations.base.PluginManager.get_input_args()
    -

    Returns a list of Pydantic classes for all registered plugins’ input arguments.’

    -

    Returns: -list[str]: A list of Pydantic classes for all registered plugins’ input arguments.’

    -
    -
    -
    get_instance
    -
    integrations.base.PluginManager.get_instance()
    -

    Returns the singleton instance of PluginManager. -If the instance doesn’t exist, it creates a new one.

    -
    -
    -
    get_trainer_cls
    -
    integrations.base.PluginManager.get_trainer_cls(cfg)
    -

    Calls the get_trainer_cls method of all registered plugins and returns the first non-None trainer class.

    -

    Parameters: -cfg (dict): The configuration for the plugins.

    -

    Returns: -object: The trainer class, or None if none was found.

    -
    -
    -
    load_datasets
    -
    integrations.base.PluginManager.load_datasets(cfg, preprocess=False)
    -

    Calls the load_datasets method of each registered plugin.

    -
    -
    Parameters
    +

    Calls the add_callbacks_post_trainer method of all registered plugins.

    +
    +
    Parameters
    add_callbacks_post_trainerCalls the add_callbacks_post_trainer method of all registered plugins.Calls the add_callbacks_post_trainer method of all registered plugins.
    add_callbacks_pre_trainer
    create_lr_schedulerCalls the create_lr_scheduler method of all registered plugins and returns the first non-None scheduler.Calls the create_lr_scheduler method of all registered plugins and returns
    create_optimizerCalls the create_optimizer method of all registered plugins and returns the first non-None optimizer.Calls the create_optimizer method of all registered plugins and returns
    get_input_args
    get_instanceReturns the singleton instance of PluginManager.Returns the singleton instance of PluginManager. If the instance doesn’t
    get_trainer_clsCalls the get_trainer_cls method of all registered plugins and returns the first non-None trainer class.Calls the get_trainer_cls method of all registered plugins and returns the
    load_datasets
    post_lora_loadCalls the post_lora_load method of all registered plugins.Calls the post_lora_load method of all registered plugins.
    post_model_buildCalls the post_model_build method of all registered plugins after the model has been built/loaded,Calls the post_model_build method of all registered plugins after the
    post_model_loadCalls the post_model_load method of all registered plugins after the model has been loadedCalls the post_model_load method of all registered plugins after the model
    post_train
    post_trainer_createCalls the post_trainer_create method of all registered plugins.Calls the post_trainer_create method of all registered plugins.
    pre_lora_loadCalls the pre_lora_load method of all registered plugins.Calls the pre_lora_load method of all registered plugins.
    pre_model_load
    ----++++ @@ -1506,26 +1440,26 @@ object: The trainer class, or None if none was found.

    - + - - - - + + + +
    cfgDictDefault The configuration for the plugins. required
    preprocessWhether this is preprocess step of the datasets.FalsetrainerTrainerThe trainer object for training.required
    -
    -
    Returns
    +
    +
    Returns
    ---+++ @@ -1536,8 +1470,342 @@ object: The trainer class, or None if none was found.

    - + + + + +
    dataset_meta list[Callable]A list of callback functions to be added to the TrainingArgs.
    +
    +
    +
    +
    add_callbacks_pre_trainer
    +
    integrations.base.PluginManager.add_callbacks_pre_trainer(cfg, model)
    +

    Calls the add_callbacks_pre_trainer method of all registered plugins.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    modelPreTrainedModelThe loaded model.required
    +
    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    list[Callable]A list of callback functions to be added to the TrainingArgs.
    +
    +
    +
    +
    create_lr_scheduler
    +
    integrations.base.PluginManager.create_lr_scheduler(
+    trainer,
+    optimizer,
+    num_training_steps,
+)
    +

    Calls the create_lr_scheduler method of all registered plugins and returns +the first non-None scheduler.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    trainerTrainerThe trainer object for training.required
    optimizerOptimizerThe optimizer for training.required
    +
    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    LRScheduler | NoneThe created learning rate scheduler, or None if not found.
    +
    +
    +
    +
    create_optimizer
    +
    integrations.base.PluginManager.create_optimizer(trainer)
    +

    Calls the create_optimizer method of all registered plugins and returns +the first non-None optimizer.

    +
    +
    Parameters
    + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    trainerTrainerThe trainer object for training.required
    +
    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    Optimizer | NoneThe created optimizer, or None if none was found.
    +
    +
    +
    +
    get_input_args
    +
    integrations.base.PluginManager.get_input_args()
    +

    Returns a list of Pydantic classes for all registered plugins’ input arguments.’

    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    list[str]A list of Pydantic classes for all registered plugins’ input arguments.’
    +
    +
    +
    +
    get_instance
    +
    integrations.base.PluginManager.get_instance()
    +

    Returns the singleton instance of PluginManager. If the instance doesn’t +exist, it creates a new one.

    +
    +
    +
    get_trainer_cls
    +
    integrations.base.PluginManager.get_trainer_cls(cfg)
    +

    Calls the get_trainer_cls method of all registered plugins and returns the +first non-None trainer class.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    +
    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    Trainer | NoneThe first non-None trainer class returned by a plugin.
    +
    +
    +
    +
    load_datasets
    +
    integrations.base.PluginManager.load_datasets(cfg, preprocess=False)
    +

    Calls the load_datasets method of each registered plugin.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    preprocessboolWhether this is preprocess step of the datasets.False
    +
    +
    +
    Returns
    + +++++ + + + + + + + + + + + @@ -1547,21 +1815,16 @@ object: The trainer class, or None if none was found.

    post_lora_load
    integrations.base.PluginManager.post_lora_load(cfg, model)
    -

    Calls the post_lora_load method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -None

    -
    -
    -
    post_model_build
    -
    integrations.base.PluginManager.post_model_build(cfg, model)
    -

    Calls the post_model_build method of all registered plugins after the model has been built/loaded, -but before any adapters have been applied.

    -
    -
    Parameters
    +

    Calls the post_lora_load method of all registered plugins.

    +
    +
    Parameters
    NameTypeDescription
    Union['TrainDatasetMeta', None] The dataset metadata loaded from all registered plugins.
    ++++++ @@ -1573,13 +1836,52 @@ but before any adapters have been applied.

    - + - + + + + + +
    Name
    cfgdictDictDefault The configuration for the plugins. required
    modelobjectPreTrainedModel | PeftModelThe loaded model.required
    +
    +
    +
    +
    post_model_build
    +
    integrations.base.PluginManager.post_model_build(cfg, model)
    +

    Calls the post_model_build method of all registered plugins after the +model has been built / loaded, but before any adapters have been applied.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + @@ -1590,73 +1892,270 @@ but before any adapters have been applied.

    post_model_load
    integrations.base.PluginManager.post_model_load(cfg, model)
    -

    Calls the post_model_load method of all registered plugins after the model has been loaded -inclusive of any adapters

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -None

    +

    Calls the post_model_load method of all registered plugins after the model +has been loaded inclusive of any adapters.

    +
    +
    Parameters
    +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    modelPreTrainedModel The loaded model. required
    ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    modelPreTrainedModel | PeftModelThe loaded model.required
    +
    post_train
    integrations.base.PluginManager.post_train(cfg, model)

    Calls the post_train method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -None

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    modelPreTrainedModel | PeftModelThe loaded model.required
    +
    post_train_unload
    integrations.base.PluginManager.post_train_unload(cfg)

    Calls the post_train_unload method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -None

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    +
    post_trainer_create
    integrations.base.PluginManager.post_trainer_create(cfg, trainer)
    -

    Calls the post_trainer_create method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -trainer (object): The trainer object for training.

    -

    Returns: -None

    +

    Calls the post_trainer_create method of all registered plugins.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    trainerTrainerThe trainer object for training.required
    +
    pre_lora_load
    integrations.base.PluginManager.pre_lora_load(cfg, model)
    -

    Calls the pre_lora_load method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins. -model (object): The loaded model.

    -

    Returns: -None

    +

    Calls the pre_lora_load method of all registered plugins.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    modelPreTrainedModelThe loaded model.required
    +
    pre_model_load
    integrations.base.PluginManager.pre_model_load(cfg)

    Calls the pre_model_load method of all registered plugins.

    -

    Parameters: -cfg (dict): The configuration for the plugins.

    -

    Returns: -None

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    cfgDictDefaultThe configuration for the plugins.required
    +
    register
    integrations.base.PluginManager.register(plugin_name)

    Registers a new plugin by its name.

    -

    Parameters: -plugin_name (str): The name of the plugin to be registered.

    -

    Returns: -None

    -

    Raises: -ImportError: If the plugin module cannot be imported.

    +
    +
    Parameters
    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    plugin_namestrThe name of the plugin to be registered.required
    +
    +
    +
    Raises
    + + + + + + + + + + + + + + + +
    NameTypeDescription
    ImportErrorIf the plugin module cannot be imported.
    +
    @@ -1681,20 +2180,79 @@ ImportError: If the plugin module cannot be imported.

    load_plugin

    integrations.base.load_plugin(plugin_name)

    Loads a plugin based on the given plugin name.

    -

    The plugin name should be in the format “module_name.class_name”. -This function splits the plugin name into module and class, imports the module, -retrieves the class from the module, and creates an instance of the class.

    -

    Parameters: -plugin_name (str): The name of the plugin to be loaded. The name should be in the format “module_name.class_name”.

    -

    Returns: -BasePlugin: An instance of the loaded plugin.

    -

    Raises: -ImportError: If the plugin module cannot be imported.

    +

    The plugin name should be in the format “module_name.class_name”. This function +splits the plugin name into module and class, imports the module, retrieves the +class from the module, and creates an instance of the class.

    +
    +

    Parameters

    + ++++++ + + + + + + + + + + + + + + + + +
    NameTypeDescriptionDefault
    plugin_namestrThe name of the plugin to be loaded. The name should be in the format “module_name.class_name”.required
    +
    +
    +

    Returns

    + + + + + + + + + + + + + + + +
    NameTypeDescription
    BasePluginAn instance of the loaded plugin.
    +
    +
    +

    Raises

    + + + + + + + + + + + + + + + +
    NameTypeDescription
    ImportErrorIf the plugin module cannot be imported.
    + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + +
    + +
    + + +
    + + + +
    + + + + +
    +

    loaders.constants

    +

    loaders.constants

    +

    Shared constants for axolotl.loaders module

    + + +
    + +
    + +
    + + + + + \ No newline at end of file diff --git a/docs/api/loaders.model.html b/docs/api/loaders.model.html new file mode 100644 index 000000000..32bbadc89 --- /dev/null +++ b/docs/api/loaders.model.html @@ -0,0 +1,1020 @@ + + + + + + + + + +loaders.model – Axolotl + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + +
    + +
    + + +
    + + + +
    + + + + +
    +

    loaders.model

    +

    loaders.model

    +

    Model loader class implementation for loading, configuring, and patching various +models.

    +
    +

    Classes

    + + + + + + + + + + + + + +
    NameDescription
    ModelLoaderManages model configuration, initialization and application of patches during
    +
    +

    ModelLoader

    +
    loaders.model.ModelLoader(
+    self,
+    cfg,
+    tokenizer,
+    *,
+    inference=False,
+    reference_model=False,
+    **kwargs,
+)
    +

    Manages model configuration, initialization and application of patches during +model loading.

    +

    This class orchestrates the entire process of loading a model from configuration to +final preparation. It handles device mapping, quantization, attention mechanisms, +adapter integration, and various optimizations.

    +
    +

    The loading process includes

    +
      +
    • Loading and validating model configuration
      • +
    • Applying monkey patches for optimizations / fixes
      • +
    • Setting up device mapping (including multi-GPU configurations)
      • +
    • Configuring quantization
      • +
    • Setting attention mechanisms (Flash Attention, SDPA, etc.)
      • +
    • Loading and initializing the model
      • +
    • Applying adapters (LoRA, QLoRA, etc.)
      • +
    +
    +
    +

    Attributes

    + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    NameTypeDescription
    modelPreTrainedModel | PeftModel | PeftMixedModelThe loaded model instance (available after load() is called).
    model_kwargsdict[str, Any]Dictionary of keyword arguments passed to model initialization.
    base_modelName or path of the base model to load.
    model_typeType of model to load (e.g., AutoModelForCausalLM).
    model_configConfiguration object for the model.
    auto_model_loaderclass used for loading the model (default: AutoModelForCausalLM).
    +
    +
    +

    Methods

    + + + + + + + + + + + + + +
    NameDescription
    loadLoad and prepare the model with all configurations and patches.
    +
    +
    load
    +
    loaders.model.ModelLoader.load()
    +

    Load and prepare the model with all configurations and patches.

    +
    +
    Returns
    + +++++ + + + + + + + + + + + + + + +
    NameTypeDescription
    tuple[PreTrainedModel, PeftConfig | None]A tuple with the loaded model and its LoRA configuration (if applicable).
    + + +
    +
    +
    +
    +
    +
    + +
    + +
    + + + + + \ No newline at end of file diff --git a/docs/api/loaders.patch_manager.html b/docs/api/loaders.patch_manager.html new file mode 100644 index 000000000..37d00f0db --- /dev/null +++ b/docs/api/loaders.patch_manager.html @@ -0,0 +1,949 @@ + + + + + + + + + +loaders.patch_manager – Axolotl + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + +
    + +
    + + +
    + + + +
    + + + + +
    +

    loaders.patch_manager

    +

    loaders.patch_manager

    +

    Patch manager class implementation to complement axolotl.loaders.ModelLoader.

    +

    Applies pre- and post-model load patches for various fixes and optimizations.

    +
    +

    Classes

    + + + + + + + + + + + + + +
    NameDescription
    PatchManagerManages the application of patches during the model loading process.
    +
    +

    PatchManager

    +
    loaders.patch_manager.PatchManager(self, cfg, model_config, inference=False)
    +

    Manages the application of patches during the model loading process.

    +
    +

    Attributes

    + + + + + + + + + + + + + +
    NameDescription
    has_flash_attnCheck if flash attention is installed.
    +
    +
    +

    Methods

    + + + + + + + + + + + + + + + + + +
    NameDescription
    apply_post_model_load_patchesApply patches that require the model instance.
    apply_pre_model_load_patchesApply pre-model load patches based on config.
    +
    +
    apply_post_model_load_patches
    +
    loaders.patch_manager.PatchManager.apply_post_model_load_patches(model)
    +

    Apply patches that require the model instance.

    +
    +
    +
    apply_pre_model_load_patches
    +
    loaders.patch_manager.PatchManager.apply_pre_model_load_patches()
    +

    Apply pre-model load patches based on config.

    + + +
    +
    +
    +
    +
    + +
    + +
    + + + + + \ No newline at end of file diff --git a/docs/api/loaders.processor.html b/docs/api/loaders.processor.html new file mode 100644 index 000000000..dc8a0bf67 --- /dev/null +++ b/docs/api/loaders.processor.html @@ -0,0 +1,837 @@ + + + + + + + + + +loaders.processor – Axolotl + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + +
    + +
    + + +
    + + + +
    + + + + +
    +

    loaders.processor

    +

    loaders.processor

    +

    Processor loading functionality for multi-modal models

    + + +
    + +
    + +
    + + + + + \ No newline at end of file diff --git a/docs/api/utils.models.html b/docs/api/loaders.tokenizer.html similarity index 72% rename from docs/api/utils.models.html rename to docs/api/loaders.tokenizer.html index f1b080952..6e70ab7f3 100644 --- a/docs/api/utils.models.html +++ b/docs/api/loaders.tokenizer.html @@ -7,7 +7,7 @@ -utils.models – Axolotl +loaders.tokenizer – Axolotl