diff --git a/.nojekyll b/.nojekyll index 2279700b3..152c19a6f 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -60422756 \ No newline at end of file +bef07254 \ No newline at end of file diff --git a/docs/api/core.training_args.html b/docs/api/core.training_args.html index c12feb863..2a48a1e55 100644 --- a/docs/api/core.training_args.html +++ b/docs/api/core.training_args.html @@ -545,8 +545,10 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, - simpo_gamma=None, -) + image_size=None, + image_resize_algorithm=None, + simpo_gamma=None, +)

CPO config for CPO training

@@ -596,7 +598,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

KTO config for KTO training

@@ -646,7 +650,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

ORPO config for ORPO training

@@ -696,7 +702,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

PRM config for PRM training

@@ -746,7 +754,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

Reward config for Reward training

@@ -796,7 +806,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

Training arguments for Causal trainer

This code is duplicated due to HF TrainingArguments not setting output_dir with a default value so it can’t be used as a mixin.

@@ -848,7 +860,9 @@ default value so it can’t be used as a mixin.

kd_zscore_base_temp=None, kd_top_k_before_softmax=None, sequence_parallel_degree=1, -) + image_size=None, + image_resize_algorithm=None, +)

Mixin class for the Axolotl training args.

diff --git a/docs/api/index.html b/docs/api/index.html index 24d96c050..d3c33b818 100644 --- a/docs/api/index.html +++ b/docs/api/index.html @@ -891,14 +891,18 @@ ul.task-list li input[type="checkbox"] { Pydantic models for TRL trainer configuration +utils.schemas.multimodal +Pydantic models for multimodal-related configuration + + utils.schemas.integrations Pydantic models for Axolotl integrations - + utils.schemas.enums Enums for Axolotl input config - + utils.schemas.utils Utilities for Axolotl Pydantic models diff --git a/docs/api/utils.collators.mm_chat.html b/docs/api/utils.collators.mm_chat.html index a136e354e..d9918f76a 100644 --- a/docs/api/utils.collators.mm_chat.html +++ b/docs/api/utils.collators.mm_chat.html @@ -473,14 +473,12 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin 
utils.collators.mm_chat.MultiModalChatDataCollator(
     self,
     tokenizer,
-    processor,
-    return_tensors='pt',
-    chat_template=None,
-    packing=False,
-    max_images=-1,
-    padding=True,
-    pad_to_multiple_of=None,
-)
+ processing_strategy, + packing=False, + return_tensors='pt', + padding=True, + pad_to_multiple_of=None, +)

Collator for multi-modal chat messages

diff --git a/docs/api/utils.schemas.multimodal.html b/docs/api/utils.schemas.multimodal.html new file mode 100644 index 000000000..2e784bd1e --- /dev/null +++ b/docs/api/utils.schemas.multimodal.html @@ -0,0 +1,928 @@ + + + + + + + + + +utils.schemas.multimodal – Axolotl + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ + + + +
+

utils.schemas.multimodal

+

utils.schemas.multimodal

+

Pydantic models for multimodal-related configuration

+
+

Classes

+ + + + + + + + + + + + + +
NameDescription
MultiModalConfigMulti-modal configuration subset
+
+

MultiModalConfig

+
utils.schemas.multimodal.MultiModalConfig()
+

Multi-modal configuration subset

+
+

Methods

+ + + + + + + + + + + + + +
NameDescription
convert_image_resize_algorithmConvert the image resize algorithm to a PIL.Image.Resampling enum.
+
+
convert_image_resize_algorithm
+
utils.schemas.multimodal.MultiModalConfig.convert_image_resize_algorithm(
+    image_resize_algorithm,
+)
+

Convert the image resize algorithm to a PIL.Image.Resampling enum.

+ + +
+
+
+
+
+ +
+ +
+ + + + + \ No newline at end of file diff --git a/docs/config.html b/docs/config.html index f0ccac4ab..4d4527515 100644 --- a/docs/config.html +++ b/docs/config.html @@ -1040,61 +1040,69 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin # Be careful with this being turned on between different models. auto_resume_from_checkpoints: false -# Don't mess with this, it's here for accelerate and torchrun -local_rank: - -# Add or change special tokens. -# If you add tokens here, you don't need to add them to the `tokens` list. -special_tokens: - # bos_token: "<s>" - # eos_token: "</s>" - # unk_token: "<unk>" - # pad_token: "[PAD]" +## Multimodal section +# int | tuple[int, int] | None . Size to resize images to, width x height. +# Will read from model/processor config if not set. +image_size: +# str. Algorithm to use for image resizing. "bilinear", "bicubic", "lanczos". Default is "bilinear". +image_resize_algorithm: 'bilinear' +## End of multimodal section + +# Don't mess with this, it's here for accelerate and torchrun +local_rank: -# Add extra tokens. -tokens: - -# Mapping token_id to new_token_string to override reserved added_tokens in the tokenizer. -# Only works for tokens that are not part of the base vocab (aka are added_tokens). -# Can be checked if they exist in tokenizer.json added_tokens. -added_tokens_overrides: # Dict[int, str] -# 128041: "<|im_start|>" -# 128042: "<|im_end|>" - -# FSDP -fsdp: -fsdp_config: - -# Deepspeed config path. e.g., deepspeed_configs/zero3.json -deepspeed: - -# Advanced DDP Arguments -ddp_timeout: -ddp_bucket_cap_mb: -ddp_broadcast_buffers: +# Add or change special tokens. +# If you add tokens here, you don't need to add them to the `tokens` list. +special_tokens: + # bos_token: "<s>" + # eos_token: "</s>" + # unk_token: "<unk>" + # pad_token: "[PAD]" + +# Add extra tokens. +tokens: + +# Mapping token_id to new_token_string to override reserved added_tokens in the tokenizer. +# Only works for tokens that are not part of the base vocab (aka are added_tokens). +# Can be checked if they exist in tokenizer.json added_tokens. +added_tokens_overrides: # Dict[int, str] +# 128041: "<|im_start|>" +# 128042: "<|im_end|>" + +# FSDP +fsdp: +fsdp_config: -# Sequence parallelism -# Set to a divisor of the number of GPUs available to split sequences into chunks of equal size. -# Use in long context training to prevent OOM when sequences cannot fit into a single GPU's VRAM. -# E.g., if 4 GPUs are available, set this value to 2 to split each sequence into two equal-sized -# subsequences, or set to 4 to split into four equal-sized subsequences. -# See https://axolotl-ai-cloud.github.io/axolotl/docs/sequence_parallelism.html for more details. -sequence_parallel_degree: +# Deepspeed config path. e.g., deepspeed_configs/zero3.json +deepspeed: + +# Advanced DDP Arguments +ddp_timeout: +ddp_bucket_cap_mb: +ddp_broadcast_buffers: -# Path to torch distx for optim 'adamw_anyprecision' -torchdistx_path: - -# Set to HF dataset for type: 'completion' for streaming instead of pre-tokenize -pretraining_dataset: - -# Debug mode -debug: - -# Seed -seed: - -# Allow overwrite yml config using from cli -strict: +# Sequence parallelism +# Set to a divisor of the number of GPUs available to split sequences into chunks of equal size. +# Use in long context training to prevent OOM when sequences cannot fit into a single GPU's VRAM. +# E.g., if 4 GPUs are available, set this value to 2 to split each sequence into two equal-sized +# subsequences, or set to 4 to split into four equal-sized subsequences. +# See https://axolotl-ai-cloud.github.io/axolotl/docs/sequence_parallelism.html for more details. +sequence_parallel_degree: + +# Path to torch distx for optim 'adamw_anyprecision' +torchdistx_path: + +# Set to HF dataset for type: 'completion' for streaming instead of pre-tokenize +pretraining_dataset: + +# Debug mode +debug: + +# Seed +seed: + +# Allow overwrite yml config using from cli +strict: diff --git a/docs/multimodal.html b/docs/multimodal.html index 66f9d4a61..310be4e4f 100644 --- a/docs/multimodal.html +++ b/docs/multimodal.html @@ -7,7 +7,7 @@ -multimodal – Axolotl +MultiModal / Vision Language Models (BETA) – Axolotl