tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add layers_to_transform for lora_config (#1118)

Browse Source
This commit is contained in:
xzuyn
2024-01-15 21:29:55 -05:00
committed by GitHub
parent 9cd27b2f91
commit 8487b97cf3
4 changed files with 23 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -677,7 +677,8 @@ lora_target_modules:
# - gate_proj
# - down_proj
# - up_proj
lora_target_linear: # If true, will target all linear layers
lora_target_linear: # If true, will target all linear modules
peft_layers_to_transform: # The layer indices to transform, otherwise, apply to all layers
# If you added new tokens to the tokenizer, you may need to save some LoRA modules because they need to know the new tokens.
# For LLaMA and Mistral, you need to save `embed_tokens` and `lm_head`. It may vary for other models.