Wing Lian
37acb28d02
fix einsum dims
2026-04-20 23:09:47 +00:00
Wing Lian
66c3e5a3fd
better handling of dora merge on Conv layers in Qwen 3.5 ( #3599 )
...
* better handling of dora merge on Conv layers in Qwen 3.5
* address issues from code review
* stricter efficient merges for dora since we now have meta model to reference
2026-04-12 10:57:45 -04:00
VED
b55706b9f6
feat:merge-lora iterate through bins without loading ( #3095 )
...
* merge_method added
* merge_efficient core implement
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Wing Lian <wing.lian@gmail.com >
* Update src/axolotl/utils/lora_merge_efficient.py
Co-authored-by: Wing Lian <wing.lian@gmail.com >
* standard to leagcy + rstrip + try/except for do_merge_lora_efficient(cfg=cfg)
* fix: 'dict' object has no attribute 'lora_alpha'
* into -> debug
* lint
* lint2
* moved everythign to cpu + peformance improvments
* lint
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
* Update src/axolotl/cli/merge_lora.py
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
* string handeling + try except remove
* merge_method -> merge_lora_methods
* remove duplicate cal + safetensor + move to lora_merge.py
* lint
* handle quant-dequant, handle experts
* fix parameter merging and prefer peft's native merge logic per module
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com >
Co-authored-by: Dan Saunders <danjsaund@gmail.com >
2026-03-25 08:41:32 -04:00
Wing Lian
b3289fd190
feat: LoRA kernel support for bias, dropout, dora, embeddings ( #3528 ) [skip ci]
...
* feat: LoRA kernel support for bias, dropout, dora, embeddings
* chore: lint
* chore: lint
* address PR feedback, add regression tests, add fsdp2 tests for lora kernels
* update tests for new sigs
* update tests now that bias and dropout are supported
2026-03-22 13:53:19 -04:00
NanoCode012
a2da852576
fix: improve lora kernels failure message and handle trust_remote_code ( #3378 ) [skip ci]
...
* fix: improve lora kernels failure message and handle trust_remote_code
* chore: re-order model guides
2026-02-10 17:58:40 +07:00
VED
e7f0d4ba5b
Increased test coverage for lora/qlora ( #3147 )
...
* config_val tests
* remove config val(not needed)
* config validation
* parameter freeze validation
* merge/unmerge tests
* removal unwanted
* rename
* lint
* updated lint
* Update tests/utils/lora/test_config_validation_lora.py
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com >
* pytest skip + mock fix
* nitpicks
* revert some nitpicks
---------
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com >
2026-01-06 11:44:48 -05:00
