tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

various bugfixes (#856)

Browse Source 
* various bugfixes

use latest tinyllama release
check if val_set_size is empty first
update sdp and xformers llama patches for updated upstream transformers
fix system prompt when no input
calculate total and total supervised tokens even when not sample packing

* add fix for when eval size is estimated to be too small

* should be len 1 for dataset length

* add catchall kwargs
This commit is contained in:
Wing Lian
2023-11-15 12:23:18 -05:00
committed by GitHub
parent 501b4d1379
commit 14706504e3
7 changed files with 45 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/llama-2/tiny-llama.yml
View File

@@ -1,4 +1,4 @@
base_model: PY007/TinyLlama-1.1B-step-50K-105b
base_model: PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer