Wing Lian 8c2e05ade3 relora: magnitude pruning of the optimizer (#1245) ...

* magnitude pruning of the optimizer

* add alpaca chat template and fix relora patch

* fix handling of lora adapter for relora

* fix merge and save call

* fixes for 8-bit lora merge

* save intermediate checkpoint adapters

* auto merge

* fix eval check

* handle relora annealing

* fix anneal step logic

* chore: lint

* misx fix

* fix types

* Update tests/e2e/test_relora_llama.py

* check for safetensors saved from relora

2024-02-06 00:35:30 -05:00

..

__init__.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_4d_multipack_llama.py

support for true batches with multipack (#1230)

2024-02-01 10:18:42 -05:00

test_falcon_samplepack.py

Falcon embeddings (#1149) [skip docker]

2024-01-22 21:01:42 -05:00

test_fused_llama.py

support for true batches with multipack (#1230)

2024-02-01 10:18:42 -05:00

test_llama_s2_attention.py

Add shifted sparse attention (#973) [skip-ci]

2024-01-18 10:16:07 -05:00

test_lora_llama_multipack.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_mistral_samplepack.py

relora: magnitude pruning of the optimizer (#1245)

2024-02-06 00:35:30 -05:00

test_mixtral_samplepack.py

Falcon embeddings (#1149) [skip docker]

2024-01-22 21:01:42 -05:00

test_model_patches.py

Multipack simplify for Mixtral (#1142)

2024-01-18 16:23:49 -05:00

test_phi_multipack.py

Phi2 multipack (#1173)

2024-01-23 12:54:36 -05:00

test_resume.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00