tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

add llama 7b config and fiz lora_fan_in_fan_out for llama (copy pasta bug)

Browse Source
This commit is contained in:
Wing Lian
2023-04-15 14:26:52 -04:00
parent d33a975747
commit d060c803ce
2 changed files with 42 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
configs/llama_65B_alpaca.yml
View File

@@ -22,7 +22,7 @@ lora_dropout: 0.05
lora_target_modules:
- q_proj
- w_proj
lora_fan_in_fan_out: true # pythia/GPTNeoX lora specific
lora_fan_in_fan_out: false
wandb_project: llama-65b-lora
wandb_watch:
wandb_run_id: