# utils.model_shard_quant – Axolotl

# utils.model_shard_quant

`utils.model_shard_quant`

module to handle loading model on cpu/meta device for FSDP

## Functions

Name | Description
--- | ---
load_and_quantize | Loads `value` tensor into submodule of `module`, optionally skipping `skip_names` and converting to `dtype`.

### load_and_quantize

    utils.model_shard_quant.load_and_quantize(
        module,
        name,
        value,
        device=None,
        dtype=None,
        skip_names=None,
        to_cpu=False,
        to_meta=False,
        verbose=False,
        quant_method='bnb',
    )

Loads `value` tensor into submodule of `module`, optionally skipping `skip_names` and converting to `dtype`.

Quantizes `Params4bit` on `device` then places on “cpu” if to_cpu=True or “meta” if to_meta=True.