tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

fix: disable auto lora kernel if dropout nonzero (#2655) [skip ci]

Browse Source 
* fix: disable auto lora kernel if dropout nonzero

* Add comment from PR feedback

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>
This commit is contained in:
NanoCode012
2025-05-13 03:23:53 +07:00
committed by GitHub
parent 526ddb886d
commit 67c4ea9c7c
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/axolotl/utils/schemas/config.py
View File

@@ -1345,6 +1345,10 @@ class AxolotlConfigWCapabilities(AxolotlInputConfig):
): ):
return data return data
# Skip if dropout is not 0, as auto enabling it would just disable it during runtime patch checks
if data.get("lora_dropout") != 0:
return data
# Check multi-GPU compatibility # Check multi-GPU compatibility
capabilities = data.get("capabilities") capabilities = data.get("capabilities")
is_multi_gpu = capabilities and capabilities.get("n_gpu", 0) > 1 is_multi_gpu = capabilities and capabilities.get("n_gpu", 0) > 1