VED
b55706b9f6
feat:merge-lora iterate through bins without loading (#3095)
* merge_method added
* merge_efficient core implement
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Wing Lian <wing.lian@gmail.com>
* Update src/axolotl/utils/lora_merge_efficient.py
Co-authored-by: Wing Lian <wing.lian@gmail.com>
* standard to leagcy + rstrip + try/except for do_merge_lora_efficient(cfg=cfg)
* fix: 'dict' object has no attribute 'lora_alpha'
* into -> debug
* lint
* lint2
* moved everythign to cpu + peformance improvments
* lint
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com>
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com>
* string handeling + try except remove
* merge_method -> merge_lora_methods
* remove duplicate cal + safetensor + move to lora_merge.py
* lint
* handle quant-dequant, handle experts
* fix parameter merging and prefer peft's native merge logic per module
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Dan Saunders <danjsaund@gmail.com>