NanoCode012 5c7e89105d Fix: modelloader handling of model_kwargs load_in*bit (#1999) ...

* fix: load_in_*bit not properly read

* fix: load_*bit check

* fix: typo

* refactor: load * bit handling

* feat: add test dpo lora multi-gpu

* fix: turn off sample packing for dpo

* fix: missing warmup_steps

* fix: test to load in 8bit for lora

* skip 8bit lora on h100, add 4bit lora on h100 to multi gpu tests

* chore: reduce max_steps

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-10-30 14:41:34 -04:00

..

integrations

add e2e smoke tests for llama liger integration (#1884)

2024-09-01 19:29:37 -04:00

multigpu

Fix: modelloader handling of model_kwargs load_in*bit (#1999)

2024-10-30 14:41:34 -04:00

patched

Fix: modelloader handling of model_kwargs load_in*bit (#1999)

2024-10-30 14:41:34 -04:00

__init__.py

missing dunder-init

2023-11-06 18:33:01 -05:00

.gitignore

Support Sample packing for phi arch (#586)

2023-09-15 15:46:54 -04:00

test_dpo.py

bump trl and accelerate for latest releases (#1730)

2024-07-10 11:15:44 -04:00

test_falcon.py

Falcon embeddings (#1149) [skip docker]

2024-01-22 21:01:42 -05:00

test_imports.py

Bump deepspeed 20240727 (#1790)

2024-07-27 10:24:11 -04:00

test_llama_pretrain.py

fixes to accelerator so that iterable pretraining datasets work (#1759)

2024-07-17 10:58:38 -04:00

test_load_model.py

Refactor func load_model to class ModelLoader (#1909)

2024-10-25 09:06:56 -04:00

test_lora_llama.py

add basic support for the optimi adamw optimizer (#1727)

2024-07-14 19:12:57 -04:00

test_mamba.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_mistral.py

use temp_dir kwarg instead

2023-11-06 18:33:01 -05:00

test_mixtral.py

fix broken linting (#1541)

2024-04-19 01:03:04 -04:00

test_optimizers.py

add basic support for the optimi adamw optimizer (#1727)

2024-07-14 19:12:57 -04:00

test_packing_loss.py

Fix: Gradient Accumulation issue (#1980)

2024-10-25 11:28:23 -04:00

test_phi.py

re-enable phi for tests in modal ci (#1373)

2024-05-29 15:41:46 -04:00

test_relora_llama.py

relora: magnitude pruning of the optimizer (#1245)

2024-02-06 00:35:30 -05:00

test_reward_model_llama.py

Reward model (#1879)

2024-10-13 15:11:13 -04:00

utils.py

Fix: modelloader handling of model_kwargs load_in*bit (#1999)

2024-10-30 14:41:34 -04:00