pass a prompt in from stdin for inference

2023-06-10 15:07:40 -04:00
parent 01248253a3
commit c4e4f8115c
2 changed files with 23 additions and 5 deletions
--- a/README.md
+++ b/README.md
@@ -495,6 +495,11 @@ Pass the appropriate flag to the train command:
  ```bash
  --inference --base_model ./completed-model
  ```
+- Full weights finetune w/ a prompt from a text file:
+  ```bash
+  cat /tmp/prompt.txt | python scripts/finetune.py configs/your_config.yml \
+    --base_model ./completed-model --inference --prompter=None --load_in_8bit=True
+  ```

 ### Merge LORA to base