Wing Lian 8c2e05ade3 relora: magnitude pruning of the optimizer (#1245) ...

* magnitude pruning of the optimizer

* add alpaca chat template and fix relora patch

* fix handling of lora adapter for relora

* fix merge and save call

* fixes for 8-bit lora merge

* save intermediate checkpoint adapters

* auto merge

* fix eval check

* handle relora annealing

* fix anneal step logic

* chore: lint

* misx fix

* fix types

* Update tests/e2e/test_relora_llama.py

* check for safetensors saved from relora

2024-02-06 00:35:30 -05:00

3.6 KiB

Raw Blame History

View Raw