tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

update tinyllama to use final instead of checkpoints (#1820) [skip ci]

Browse Source
This commit is contained in:
Wing Lian
2024-08-09 10:58:19 -04:00
committed by GitHub
parent 5ee4b7325f
commit 3e2b269d06
3 changed files with 4 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/tiny-llama/lora-mps.yml
View File

@@ -1,4 +1,4 @@
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T base_model: TinyLlama/TinyLlama_v1.1
model_type: LlamaForCausalLM model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer tokenizer_type: LlamaTokenizer

5
examples/tiny-llama/lora.yml
View File

@@ -1,6 +1,5 @@
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T base_model: TinyLlama/TinyLlama_v1.1
model_type: LlamaForCausalLM tokenizer_type: AutoTokenizer
tokenizer_type: LlamaTokenizer
load_in_8bit: true load_in_8bit: true
load_in_4bit: false load_in_4bit: false

2
examples/tiny-llama/qlora.yml
View File

@@ -1,4 +1,4 @@
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T base_model: TinyLlama/TinyLlama_v1.1
model_type: LlamaForCausalLM model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer tokenizer_type: LlamaTokenizer