Dan Saunders
78a039e1be
add depr warning for preprocess --iterable
2025-08-22 16:02:30 +00:00
Dan Saunders
69f356163e
fix
2025-08-22 16:02:30 +00:00
Dan Saunders
53bbca2591
bugfix for sample packing
2025-08-22 16:02:30 +00:00
Dan Saunders
49bd6ece4a
remove unused
2025-08-22 16:02:30 +00:00
Dan Saunders
42b38a718a
remove eval streaming (not HF supported)
2025-08-22 16:02:30 +00:00
Dan Saunders
4121bcbc33
fix kd test
2025-08-22 16:02:30 +00:00
Dan Saunders
0caa24eab0
comments
2025-08-22 16:02:30 +00:00
Dan Saunders
68bb70bbae
fix test
2025-08-22 16:02:30 +00:00
Dan Saunders
5d8d7ef327
lint
2025-08-22 16:02:30 +00:00
Dan Saunders
7836da9ed9
remove unuse
2025-08-22 16:02:30 +00:00
Dan Saunders
7eba3795fe
fixes
2025-08-22 16:02:30 +00:00
Dan Saunders
1b7b67d06e
smoke test
2025-08-22 16:02:30 +00:00
Dan Saunders
0843dc678a
separate out train and eval datasets streaming; cleanup
2025-08-22 16:02:30 +00:00
Dan Saunders
067158e24a
nits
2025-08-22 16:02:30 +00:00
Dan Saunders
aa5a497a2c
nits
2025-08-22 16:02:30 +00:00
Dan Saunders
2176962231
separate out train and eval dataset streaming
2025-08-22 16:02:30 +00:00
Dan Saunders
10335d5df9
add multidata strats
2025-08-22 16:02:30 +00:00
Dan Saunders
e4e8ffd40c
nits
2025-08-22 16:02:30 +00:00
Dan Saunders
846aa41baa
nits
2025-08-22 16:02:30 +00:00
Dan Saunders
7bb52d00bb
progress on streaming
2025-08-22 16:02:30 +00:00
Dan Saunders
3b2dd05798
remove iterable CLI arg
2025-08-22 16:02:30 +00:00
Dan Saunders
b6431083be
nit
2025-08-22 16:02:30 +00:00
Dan Saunders
16ff01df85
separate streaming and pretraining
2025-08-22 16:02:30 +00:00
Wing Lian
ab4d604a8f
upgrade peft for 0.17.1 ( #3094 )
...
* upgrade peft to 0.17.1
* upgrade for transformers too
2025-08-22 07:26:30 -04:00
Wing Lian
0fa752e58b
upgrade flash-attn to 2.8.3 for gpt-oss attn sink support ( #3082 )
2025-08-21 15:04:10 -04:00
Dan Saunders
08e517ea48
Update .coderabbit.yaml ( #3091 ) [skip ci]
2025-08-20 22:14:13 -04:00
Wing Lian
07fd22f39b
better handling of lora w bias with fsdp2 and handling of files when saving model checkpoint ( #3090 )
2025-08-20 15:17:48 -04:00
Wing Lian
06eaf6c448
misc fixes ( #3085 )
2025-08-20 08:52:26 -04:00
goggle
050210e637
fix: Sweep runs overwrite each other because output_dir from base config is reused ( #3080 )
...
* refactor: improve output_dir handling in generate_config_files
* fix typo
* cli: harden sweep output_dir handling with base fallback
- Ensure sweep permutations always resolve a valid output_dir
- Default to ./model-out if neither permutation nor base config sets output_dir
- Append sweepXXXX suffix consistently for each permutation
- Prevent Path(None) TypeError and improve robustness of sweep config generation
* fix typo
* chore: lint
---------
Co-authored-by: Wing Lian <wing@axolotl.ai >
2025-08-19 20:25:20 -04:00
Wing Lian
05cedbfb1e
add baseten info for gpt-oss recipe ( #3078 )
...
* add bsaeten info for gpt-oss recipe
* incorporate PR review
2025-08-19 13:30:37 -04:00
VED
c10eb811fa
data_parallel_size in in VllmserveCliArgs ( #3074 )
...
* data_parallel_size in in VllmserveCliArgs
* moved to 43
2025-08-18 08:44:37 -04:00
VED
0eef385b1a
[feat] truncation support with excess_length_strategy ( #3068 ) [skip ci]
...
* feat:truncation support with excess_len
* pre-commit
* excess_length_strategy
* requested changes
* lint
* added handle_long_seq_in_dataset in sft
* comments improved
2025-08-18 08:39:13 -04:00
Wing Lian
ecbe8b2b61
[GPT-OSS] improve FSDP shard merging and documentation for GPT-OSS ( #3073 )
...
* improve fsdp shard merging
* improve logging
* update information on merging and inferencing GPT-OSS
* cleanup readme
* automate cleanup of FSDP prefix
* import GRPO only if necessary
* only modify config.json on rank0
* merge final checkpoint at end of training
* prevent circular import
* Fix saving for sharded state dict
* devx, move merged to output dir
* move import back to top
* Fix stuck merge
* fix conditionals from pr feedback and add test
2025-08-15 21:25:01 -04:00
Wing Lian
130ef7c51a
Various fixes for VLMs ( #3063 )
...
* fix to not use batch feature indexing
* more vlm fixes
* use AutoModelForImageTextToText
* add example yaml and need num2words for chat template
* improve handling of adding image tokens to conversation
* add lfm2-vl support
* update the lfm readme
* fix markdown and add rtol for loss checks
* feat: add smolvlm2 processing strat
* fix: check for causal-conv1d in lfm models
* feat: add docs for lfm2
* feat: add new models and tips to docs
* feat: add smolvlm2 docs and remove extra dep
* chore: update docs
* feat: add video instructions
* chore: cleanup
* chore: comments
* fix: typo
* feat: add usage stats
* chore: refactor
---------
Co-authored-by: NanoCode012 <nano@axolotl.ai >
2025-08-15 10:52:57 -04:00
salman
d1de6f5f3d
Add option to skip slow tests in PRs ( #3060 ) [skip ci]
...
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* testing e2e skip [skip-e2e]
* stop running multigpu [skip-e2e]
* should work now [skip-e2e]
* reverting [skip-e2e]
* testing [skip-e2e]
* debug [skip-e2e]
* debug [skip-e2e]
* round 2[skip-e2e]
* removing debug [skip-e2e]
* support skipping whole PR [skip-e2e]
* use script for e2e skip [skip-e2e]
* contributing [skip-e2e]
* contributing [skip-e2e]
---------
Co-authored-by: Wing Lian <wing@axolotl.ai >
2025-08-13 22:57:51 -04:00
Wing Lian
48b7ae1677
use updated patch releasE ( #3066 )
2025-08-13 21:23:05 -04:00
NanoCode012
506e3a3907
fix: fsdp_config validation being None ( #3061 ) [skip ci]
...
* fix: fsdp_config validation being None
* fix: handling
---------
Co-authored-by: salman <salman.mohammadi@outlook.com >
2025-08-13 21:21:50 -04:00
Wing Lian
09145de8fa
upgrade transformers==4.55.1 and bitsandbytes==0.47.0 ( #3064 )
...
* upgrade transformers==4.55.1
* also upgrade bnb
* remove bnb params4bit patch (upstreamed)
* use latest causal-conv1d
* fix patching ring-flash-attn with now missing imports
---------
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
2025-08-13 19:41:07 -04:00
Wing Lian
e0a2523a3b
Workaround to unblock docs build in main ( #3055 )
...
Co-authored-by: Salman Mohammadi <salman.mohammadi@outlook.com >
2025-08-13 11:39:39 +01:00
Wing Lian
3d45620008
remove prepare-from-posids patch ( #3052 ) [skip ci]
2025-08-11 09:34:41 -04:00
github-actions[bot]
ce20e838b5
chore: update pre-commit hooks ( #3050 ) [skip ci]
...
Co-authored-by: djsaunde <1245942+djsaunde@users.noreply.github.com >
2025-08-11 09:32:21 -04:00
Wing Lian
d4d84d48af
fix ray train and add fsdp2 smoke test for ray trainer ( #3053 )
...
* add fsdp2 smokle test for ray trainer
* fix raytrain with fsdp2
2025-08-11 09:31:54 -04:00
Wing Lian
9b12c05660
use exec instead of subprocess to make ctrl+c nicer for cli ( #3044 )
...
* use exec instead of subprocess to make ctrl+c nicer for cli
* change var name to use_exec
* simplify to bool
* flush std*
* patch subprocess as mock in test
* fix tests
* more test fixes
2025-08-10 20:22:20 -04:00
Wing Lian
686933194e
fix vllm tagging and add cloud images w/o tmux ( #3049 ) [skip ci]
2025-08-10 20:21:56 -04:00
Wing Lian
d12b461d19
follow up fix for plugin registration ( #3054 ) [skip ci]
2025-08-10 20:21:38 -04:00
Wing Lian
d6b81b3683
update training args check for new defaults ( #3051 ) [skip ci]
...
* update training args check for new defaults
* skip check for now
2025-08-10 11:26:22 -04:00
Wing Lian
05f1b4b2e8
run monkeypatch tests in seperate runner ( #3047 )
2025-08-09 14:34:07 -04:00
Wing Lian
7cfc80ec77
set dev version ( #3045 ) [skip ci]
2025-08-08 13:56:53 -04:00
salman
0da6a95efa
Add citation.tff ( #3043 ) [skip ci]
2025-08-08 16:18:42 +01:00
Wing Lian
2c8497e489
tag for v0.12.0 release ( #3041 )
ci-cd / build-axolotl (<nil>, 126, 12.6.3, 3.11, 2.6.0) (push) Has been cancelled
ci-cd / build-axolotl (<nil>, 126, 12.6.3, 3.11, 2.7.0) (push) Has been cancelled
ci-cd / build-axolotl (<nil>, 128, 12.8.1, 3.11, 2.7.1) (push) Has been cancelled
ci-cd / build-axolotl (vllm, 126, 12.6.3, true, 3.11, 2.7.1) (push) Has been cancelled
publish pypi / Create Release (push) Has been cancelled
ci-cd / build-axolotl-cloud (<nil>, 126, 12.6.3, 3.11, 2.6.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud (<nil>, 126, 12.6.3, 3.11, 2.7.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud (<nil>, 126, 12.6.3, true, 3.11, 2.7.1) (push) Has been cancelled
ci-cd / build-axolotl-cloud (<nil>, 128, 12.8.1, 3.11, 2.7.1) (push) Has been cancelled
ci-cd / build-axolotl-cloud-no-tmux (<nil>, 126, 12.6.3, 3.11, 2.6.0) (push) Has been cancelled
publish pypi / Upload release to PyPI (push) Has been cancelled
2025-08-08 08:24:09 -04:00
