tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
2,664 Commits 319 Branches 32 Tags
c2bd75aff610f257e00a7f4e7e6649f39d770757
Commit Graph

15 Commits

Author SHA1 Message Date
Dan Saunders
79ddaebe9a Add ruff, remove black, isort, flake8, pylint (#3092) 
* black, isort, flake8 -> ruff

* remove unused

* add back needed import

* fix
 2025-08-23 23:37:33 -04:00
Dan Saunders
10ba1622f7 checkpoint model on first step callback (#2906) 
* checkpoint model on first step callback

* remove debug

* add test cases; update existing tests not to save on first step

* move test out of solo

* delete

* default to False

* typo
 2025-07-15 15:00:48 -04:00
Wing Lian
a85efffbef bump transformers==4.52.4 (#2800) [skip ci] 
* bump transformers==4.52.4

* don't use hf offline for qwen tokenizer

* increase timeout

* don't use methodtype

* increase timeout

* better assertion logging

* upgrade deepspeed version too
 2025-06-18 15:46:14 -04:00
Dan Saunders
00cda8cc70 Data loader refactor (#2707) 
* data loading refactor (wip)

* updates

* progress

* pytest

* pytest fix

* lint

* zero_first -> filelock, more simplifications

* small simplification

* import change

* nit

* lint

* simplify dedup

* couldnt resist

* review comments WIP

* continued wip

* minor changes

* fix; remove contrived test

* further refactor

* set default seed in pydantic config

* lint

* continued simplication

* lint

* renaming and nits

* filelock tests

* fix

* fix

* lint

* remove nullable arg

* remove unnecessary code

* moving dataset save fn to shared module

* remove debug print

* matching var naming

* fn name change

* coderabbit comments

* naming nit

* fix test
 2025-06-10 19:53:07 -04:00
Dan Saunders
1d91d905c9 remove deprecated wandb env var (#2751) 
* remove deprecated wandb env var

* remove os.environ wandb setting; unused loggers

* remove os.environ wandb setting; unused loggers
 2025-06-03 14:04:15 -07:00
salman
65c5481120 Rank 0-only logging (#2608) 
Co-authored-by: Wing Lian <wing@axolotl.ai>
 2025-05-28 14:57:30 +01:00
Wing Lian
1447beb132 make sure to validate the config before normalizing so defaults get set (#2554) 
* make sure to validate the config before normalizing so defaults get set

* validation not needed for particular test

* remove duplicate validations

* set qlora correctly
 2025-04-24 13:01:43 -04:00
Dan Saunders
c907ac173e adding pre-commit auto-update GH action and bumping plugin versions (#2428) 
* adding pre-commit auto-update GH action and bumping plugin versions

* running updated pre-commit plugins

* sorry to revert, but pylint complained

* Update .pre-commit-config.yaml

Co-authored-by: Wing Lian <wing.lian@gmail.com>

---------

Co-authored-by: Dan Saunders <dan@axolotl.ai>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2025-03-21 11:02:43 -04:00
Dan Saunders
1ed4de73b6 CLI cleanup and documentation (#2244) 
* CLI init refactor

* fix

* cleanup and (partial) docs

* Adding documentation and continuing cleanup (in progress)

* remove finetune.py script

* continued cleanup and documentation

* pytest fixes

* review comments

* fix

* Fix

* typing fixes

* make sure the batch dataset patcher for multipack is always loaded when handling datasets

* review comments

* fix

---------

Co-authored-by: Dan Saunders <dan@axolotl.ai>
Co-authored-by: Wing Lian <wing@axolotl.ai>
 2025-01-13 17:55:29 +00:00
Wing Lian
dd26cc3c0f add helper to verify the correct model output file exists (#2245) 
* add helper to verify the correct model output file exists

* more checks using helper

* chore: lint

* fix import and relora model check

* workaround for trl trainer saves

* remove stray print
 2025-01-13 10:43:29 -05:00
Wing Lian
7669a03fb4 update upstream HF deps (#2239) 
* bump axolotl contribs for upstream main conflicts:

* bump datasets, tokenizer, trl

* remove log workarounds in trl

* bump lm-eval

* remove unsloth_ import from critical path

* remove llama fa2 from conftest

* unsloth breaks with latest upstream
 2025-01-09 21:01:59 +00:00
Wing Lian
743ba62bd5 Transformers 4.47.0 (#2138) 
* bump transformers and trl

* fix: update trainer.log signature

* fix trl trainer.log interfaces

* broken 🦥 with latest transformers

* skip parent, call grandparent - yeah, super janky

* update HF HUB env var and fix reward trainer log since it doesn't directly override log

* also bump accelerate

* patches for llama ga

* detab the code to check

* fix whitespace for patch check

* play nicely with CI tests since we patch everytime

* fix pop default in case it doesn't exist

* more tweaks to make patches nicer in CI

* fix detab for when there are possibly multiple patches

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>
 2024-12-07 05:03:01 -05:00
Wing Lian
a1790f2652 replace tensorboard checks with helper function (#2120) [skip ci] 
* replace tensorboard checks with helper function

* move helper function

* use relative
 2024-12-03 21:06:20 -05:00
Wing Lian
ce5bcff750 various tests fixes for flakey tests (#2110) 
* add mhenrichsen/alpaca_2k_test with revision dataset download fixture for flaky tests

* log slowest tests

* pin pynvml==11.5.3

* fix load local hub path

* optimize for speed w smaller models and val_set_size

* replace pynvml

* make the resume from checkpoint e2e faster

* make tests smaller
 2024-12-02 17:28:58 -05:00
Wing Lian
5f1d98e8fc add e2e tests for Unsloth qlora and test the builds (#2093) 
* see if unsloth installs cleanly in ci

* check unsloth install on regular tests, not sdist

* fix ampere check exception for ci

* use cached_property instead

* add an e2e test for unsloth qlora

* reduce seq len and mbsz to prevent oom in ci

* add checks for fp16 and sdp_attention

* pin unsloth to a specific release

* add unsloth to docker image too

* fix flash attn xentropy patch

* fix loss, add check for loss when using fa_xentropy

* fix special tokens for test

* typo

* test fa xentropy with and without gradient accum

* pr feedback changes
 2024-11-29 20:38:49 -05:00