Wing Lian 2ea70ebbd8 ORPO (#1419) ...

* orpo trainer

* rl handling for orpo

* support for remove_unused_columns

* orpo fixes

* fix loader for orpo

* chore: lint

* fix default for remove_unused_columns

* roll ORPO into the main AxolotlTrainer so it can be compatible with some of the other techniques like relora

* better handling of system message for orpo

* revert system prompt changes for chat templtes

* no need for else condition

* split dataset parsing into it's own component

2024-03-18 13:10:00 -04:00

19 KiB

Raw Permalink Blame History

View Raw