Dan Saunders 00cda8cc70 Data loader refactor (#2707) ...

* data loading refactor (wip)

* updates

* progress

* pytest

* pytest fix

* lint

* zero_first -> filelock, more simplifications

* small simplification

* import change

* nit

* lint

* simplify dedup

* couldnt resist

* review comments WIP

* continued wip

* minor changes

* fix; remove contrived test

* further refactor

* set default seed in pydantic config

* lint

* continued simplication

* lint

* renaming and nits

* filelock tests

* fix

* fix

* lint

* remove nullable arg

* remove unnecessary code

* moving dataset save fn to shared module

* remove debug print

* matching var naming

* fn name change

* coderabbit comments

* naming nit

* fix test

2025-06-10 19:53:07 -04:00

..

lora_kernels

Lora kernels fix (#2732)

2025-05-28 10:03:43 -04:00

__init__.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_4d_multipack_llama.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_activation_checkpointing.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_cli_integrations.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_fa_xentropy.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_falcon_samplepack.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_fused_llama.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_llama_s2_attention.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_lora_llama_multipack.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_mistral_samplepack.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_mixtral_samplepack.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_model_patches.py

models.py -> loaders/ module refactor (#2680)

2025-05-23 15:51:11 -04:00

test_peft_embeddings.py

Configurable embeddings upcast (#2621)

2025-05-06 23:40:44 -04:00

test_phi_multipack.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_resume.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00

test_sp.py

SP context manager update (#2699)

2025-05-22 11:18:32 -04:00

test_unsloth_integration.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_unsloth_qlora.py

Data loader refactor (#2707)

2025-06-10 19:53:07 -04:00