tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

add trust_remote_code, explicit bfloat16 and bnb dtype settings

Browse Source
This commit is contained in:
tocmo0nlord
2026-05-13 13:32:46 +00:00
parent 3f29fa017b
commit 38adf5cd37
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
human_chat_qlora.yml
View File

@@ -18,6 +18,13 @@ tokenizer_type: AutoTokenizer
load_in_4bit: true load_in_4bit: true
strict: false strict: false
trust_remote_code: true
# Explicit dtype settings to avoid weight conversion errors
torch_dtype: bfloat16
bnb_4bit_use_double_quant: true
bnb_4bit_quant_type: nf4
bnb_4bit_compute_dtype: bfloat16
# --- System prompt baked into every conversation --- # --- System prompt baked into every conversation ---
# This is the primary lever for "no error correcting, more human-like" # This is the primary lever for "no error correcting, more human-like"