Wing Lian 4d09b42ee3 plain input/output prompt strategy w/o chat templates (#1346) ...

* plain input/output prompt strategy w/o chat templates

* disable duplicate code check

* make sure to add an eos/eot token to the end of the output so it will stop

* multi turn segement support and test

2024-03-04 16:25:16 -05:00

..

test_alpaca.py

Feat(test): Add tests for alpaca chatml prompt tokenizer (#1088)

2024-01-23 13:30:26 +09:00

test_raw_io.py

plain input/output prompt strategy w/o chat templates (#1346)

2024-03-04 16:25:16 -05:00

test_sharegpt.py

Feat/chatml add system message (#1117)

2024-01-25 08:24:27 +01:00