Sunny Liu f62e23737b memoize dataset length for eval sample packing (#1974) ...

* wip on multimodal sample packing support

* wip on multimodal packing support

* llama-1b-yml

* setup logging for test

* yml

* yml

* yml

* fix for __len__ for eval sample packing

* reverted irrelavant changes

* reformatted, reverted log message

* reverted unnecessary changes

* added e2e multigpu testing for eval sample packing

* formatting

* fixed e2e test_eval params

* fix test_eval e2e multigpu

* fix test_eval e2e multigpu

* Update tests/e2e/multigpu/test_eval.py

Co-authored-by: Wing Lian <wing.lian@gmail.com>

* Update tests/e2e/multigpu/test_eval.py

Co-authored-by: Wing Lian <wing.lian@gmail.com>

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-10-17 15:15:29 -04:00

..

__init__.py

Attempt to run multigpu in PR CI for now to ensure it works (#1815) [skip ci]

2024-08-09 11:50:13 -04:00

test_eval.py

memoize dataset length for eval sample packing (#1974)

2024-10-17 15:15:29 -04:00

test_llama.py

add ds zero3 to multigpu biweekly tests (#1900)

2024-10-13 17:34:37 -04:00

test_qwen2.py

ensure that the bias is also in the correct dtype (#1848) [skip ci]

2024-08-22 11:45:00 -04:00