From 11c48c5e039b1b08941b02d356d814bc62e8ac1f Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Tue, 10 Oct 2023 21:08:17 +0900 Subject: [PATCH] fix(doc): Add note on inference w sample packing (#712) --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index e277120e6..1610f7bda 100644 --- a/README.md +++ b/README.md @@ -876,6 +876,10 @@ Pass the appropriate flag to the train command: --base_model="./completed-model" --prompter=None --load_in_8bit=True ``` +Please use `--sample_packing False` if you have it on and receive the error similar to below: + +> RuntimeError: stack expects each tensor to be equal size, but got [1, 32, 1, 128] at entry 0 and [1, 32, 8, 128] at entry 1 + ### Merge LORA to base Add below flag to train command above