tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Fix Lora config error for Llama3 (#1659)

Browse Source 
The current yml code throws an error: ValueError: Please set lora_modules_to_save to [`embed_tokens`, `lm_head`] when using an adapter and changing the special tokens.

I added the required changes to resolve it
This commit is contained in:
Faria Huq
2024-05-28 11:25:08 -04:00
committed by GitHub
parent cc11c6bce2
commit 230e0ac363
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
examples/llama-3/lora-8b.yml
View File

@@ -24,6 +24,9 @@ lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
lora_modules_to_save:
- embed_tokens
- lm_head
wandb_project:
wandb_entity: