NanoCode012 5c7e89105d Fix: modelloader handling of model_kwargs load_in*bit (#1999) ...

* fix: load_in_*bit not properly read

* fix: load_*bit check

* fix: typo

* refactor: load * bit handling

* feat: add test dpo lora multi-gpu

* fix: turn off sample packing for dpo

* fix: missing warmup_steps

* fix: test to load in 8bit for lora

* skip 8bit lora on h100, add 4bit lora on h100 to multi gpu tests

* chore: reduce max_steps

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-10-30 14:41:34 -04:00

..

__init__.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_4d_multipack_llama.py

Fix: modelloader handling of model_kwargs load_in*bit (#1999)

2024-10-30 14:41:34 -04:00

test_fa_xentropy.py

bump flash attention 2.5.8 -> 2.6.1 (#1738)

2024-07-14 19:11:31 -04:00

test_falcon_samplepack.py

Falcon embeddings (#1149) [skip docker]

2024-01-22 21:01:42 -05:00

test_fused_llama.py

make mlflow optional (#1317)

2024-02-26 11:41:33 -05:00

test_llama_s2_attention.py

support for gemma2 w sample packing (#1718)

2024-06-29 01:38:55 -04:00

test_lora_llama_multipack.py

more fixes 20240228 (#1342) [skip ci]

2024-02-28 12:57:45 -05:00

test_mistral_samplepack.py

relora: magnitude pruning of the optimizer (#1245)

2024-02-06 00:35:30 -05:00

test_mixtral_samplepack.py

fix broken linting (#1541)

2024-04-19 01:03:04 -04:00

test_model_patches.py

swaps to use newer sample packing for mistral (#1773)

2024-07-23 01:41:11 -04:00

test_phi_multipack.py

Phi2 multipack (#1173)

2024-01-23 12:54:36 -05:00

test_resume.py

make sure the CI fails when pytest script fails (#1669)

2024-05-29 10:12:11 -04:00

test_unsloth_integration.py

Fix: Gradient Accumulation issue (#1980)

2024-10-25 11:28:23 -04:00