tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add FSDP v2 swap memory support + QLoRA compatibility fixes (#3167)

Browse Source 
Co-authored-by: salman <salman.mohammadi@outlook.com>
This commit is contained in:
Grant Holmes (Ren)
2025-09-26 04:23:59 -05:00
committed by GitHub
parent 7fa8ac40cd
commit 850c1a5f8d
6 changed files with 71 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/llama-2/qlora-fsdp.yml
View File

@@ -66,6 +66,7 @@ fsdp_config:
fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
fsdp_transformer_layer_cls_to_wrap: LlamaDecoderLayer
fsdp_state_dict_type: FULL_STATE_DICT
# fsdp_cpu_offload_pin_memory: false # uncomment to enable swap memory usage when RAM is insufficient
special_tokens:
# save_first_step: true # uncomment this to validate checkpoint saving works with your config