NanoCode012 5c7e89105d Fix: modelloader handling of model_kwargs load_in*bit (#1999) ...

* fix: load_in_*bit not properly read

* fix: load_*bit check

* fix: typo

* refactor: load * bit handling

* feat: add test dpo lora multi-gpu

* fix: turn off sample packing for dpo

* fix: missing warmup_steps

* fix: test to load in 8bit for lora

* skip 8bit lora on h100, add 4bit lora on h100 to multi gpu tests

* chore: reduce max_steps

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-10-30 14:41:34 -04:00

1.3 KiB

Raw Blame History

View Raw