Wing Lian 521e62daf1 remove the bos token from dpo outputs (#1733) [skip ci] ...

* remove the bos token from dpo outputs

* don't forget to fix prompt_input_ids too

* use processing_class instead of tokenizer

* fix for processing class

2024-11-15 19:09:20 -05:00

2.7 KiB

Raw Blame History

View Raw