Wing Lian 130ef7c51a Various fixes for VLMs (#3063) ...

* fix to not use batch feature indexing

* more vlm fixes

* use AutoModelForImageTextToText

* add example yaml and need num2words for chat template

* improve handling of adding image tokens to conversation

* add lfm2-vl support

* update the lfm readme

* fix markdown and add rtol for loss checks

* feat: add smolvlm2 processing strat

* fix: check for causal-conv1d in lfm models

* feat: add docs for lfm2

* feat: add new models and tips to docs

* feat: add smolvlm2 docs and remove extra dep

* chore: update docs

* feat: add video instructions

* chore: cleanup

* chore: comments

* fix: typo

* feat: add usage stats

* chore: refactor

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>

2025-08-15 10:52:57 -04:00

..

cli

use exec instead of subprocess to make ctrl+c nicer for cli (#3044)

2025-08-10 20:22:20 -04:00

core

update training args check for new defaults (#3051) [skip ci]

2025-08-10 11:26:22 -04:00

e2e

Various fixes for VLMs (#3063)

2025-08-15 10:52:57 -04:00

fixtures

Respect sequence_len in config for type: llama2_chat (#926)

2023-12-12 09:39:22 -08:00

integrations

remove deprecated wandb env var (#2751)

2025-06-03 14:04:15 -07:00

monkeypatch

use nanmean for loss aggregation (CP fix) (#3033)

2025-08-08 08:15:17 -04:00

patched

release v0.11.0 (#2875)

2025-07-09 09:22:35 -04:00

prompt_strategies

Feat: Add voxtral, magistral small 1.1, and misc gemma3n fixes (#2979)

2025-07-30 15:57:05 +07:00

utils

allow custom trainer_cls to be defined as a module reference in the YAML (#3024) [skip ci]

2025-08-06 22:49:19 -04:00

__init__.py

fix: minor patches for multimodal (#2441)

2025-03-31 13:40:12 +07:00

conftest.py

refactor dupes from merge/rebase (#2919) [skip ci]

2025-07-14 10:05:26 -04:00

constants.py

Add Exact Deduplication Feature to Preprocessing Pipeline (#2072)

2024-12-02 08:47:10 -05:00

hf_offline_utils.py

fix tokenizer overrides w gemma3 (#2488)

2025-04-05 01:25:44 -04:00

test_chunked_xentropy.py

chunked cross entropy loss (#2625)

2025-06-23 23:08:46 -04:00

test_data.py

fix: minor patches for multimodal (#2441)

2025-03-31 13:40:12 +07:00

test_datasets.py

limit num_proc when saving datasets to disk (#2948) [skip ci]

2025-07-21 11:39:38 -04:00

test_dict.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_exact_deduplication.py

limit num_proc when saving datasets to disk (#2948) [skip ci]

2025-07-21 11:39:38 -04:00

test_expand_mask.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_freeze.py

Train parameters exclusively in specific ranges (#1390)

2024-03-14 11:05:42 -04:00

test_loaders.py

Add support for Accelerate CP, ND examples, and fix for parallel config w fsdp (#3019)

2025-08-07 21:22:15 -04:00

test_lora.py

models.py -> loaders/ module refactor (#2680)

2025-05-23 15:51:11 -04:00

test_normalize_config.py

FSDP2 fix validation and add tests (#2910)

2025-07-14 09:25:44 -04:00

test_packed_batch_sampler.py

allow for different sequence_len for evaluations (#2836) [skip ci]

2025-06-27 11:02:51 -04:00

test_packed_dataset.py

limit num_proc when saving datasets to disk (#2948) [skip ci]

2025-07-21 11:39:38 -04:00

test_packed_pretraining.py

fix streaming packing test (#2454)

2025-03-29 08:30:06 -04:00

test_perplexity.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_prompt_tokenizers.py

remove deprecated wandb env var (#2751)

2025-06-03 14:04:15 -07:00

test_prompters.py

fix: prompt phi (#1845) [skip ci]

2024-08-22 11:46:57 -04:00

test_schedulers.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_tokenizers.py

models.py -> loaders/ module refactor (#2680)

2025-05-23 15:51:11 -04:00

test_train.py

refactor dupes from merge/rebase (#2919) [skip ci]

2025-07-14 10:05:26 -04:00

test_validation_dataset.py

release v0.11.0 (#2875)

2025-07-09 09:22:35 -04:00