Add ORPO example and e2e test (#1572)

* add example for mistral orpo * sample_packing: false for orpo * go to load_dataset (since load_rl_datasets require a transfom_fn, which only dpo uses currently)
2024-04-27 09:07:06 -07:00
parent 68601ec6ad
commit 98c25e15cb
4 changed files with 131 additions and 1 deletions
--- a/docs/rlhf.qmd
+++ b/docs/rlhf.qmd
@@ -49,7 +49,7 @@ remove_unused_columns: false
 chat_template: chatml
 datasets:
  - path: argilla/ultrafeedback-binarized-preferences-cleaned
-    type: orpo.chat_template
+    type: chat_template.argilla
 ```

 #### Using local dataset files