Wing Lian
37acb28d02
fix einsum dims
2026-04-20 23:09:47 +00:00
Wing Lian
66c3e5a3fd
better handling of dora merge on Conv layers in Qwen 3.5 ( #3599 )
...
* better handling of dora merge on Conv layers in Qwen 3.5
* address issues from code review
* stricter efficient merges for dora since we now have meta model to reference
2026-04-12 10:57:45 -04:00
VED
b55706b9f6
feat:merge-lora iterate through bins without loading ( #3095 )
...
* merge_method added
* merge_efficient core implement
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Wing Lian <wing.lian@gmail.com >
* Update src/axolotl/utils/lora_merge_efficient.py
Co-authored-by: Wing Lian <wing.lian@gmail.com >
* standard to leagcy + rstrip + try/except for do_merge_lora_efficient(cfg=cfg)
* fix: 'dict' object has no attribute 'lora_alpha'
* into -> debug
* lint
* lint2
* moved everythign to cpu + peformance improvments
* lint
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
* string handeling + try except remove
* merge_method -> merge_lora_methods
* remove duplicate cal + safetensor + move to lora_merge.py
* lint
* handle quant-dequant, handle experts
* fix parameter merging and prefer peft's native merge logic per module
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com >
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
2026-03-25 08:41:32 -04:00
VED
e7f0d4ba5b
Increased test coverage for lora/qlora ( #3147 )
...
* config_val tests
* remove config val(not needed)
* config validation
* parameter freeze validation
* merge/unmerge tests
* removal unwanted
* rename
* lint
* updated lint
* Update tests/utils/lora/test_config_validation_lora.py
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com >
* pytest skip + mock fix
* nitpicks
* revert some nitpicks
---------
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com >
2026-01-06 11:44:48 -05:00
