NanoCode012 7099343c56 feat: add eos_tokens and train_on_eot for chat_template EOT parsing (#2364) ...

* feat: add eos_tokens and train_on_eot for chat_template EOT parsing

* fix: comments

* chore: add some examples of tokens

* feat: add new potential errors for chat_template to faq

* feat: add examples for EOT handling

* fix: change error to warning for missing EOS

* fix: warning typo

* feat: add tests for eot token handling

* fix: remove broken caplog capture in test

* fix: chattemplate strategy with kd missing eot changes

2025-04-28 10:11:20 -04:00

..

conversation.qmd

feat: add eos_tokens and train_on_eot for chat_template EOT parsing (#2364)

2025-04-28 10:11:20 -04:00

index.qmd

chore: update doc links (#2509)

2025-04-11 09:53:18 -04:00

inst_tune.qmd

Feat: update doc (#1475) [skip ci]

2024-04-04 13:43:40 +09:00

pretraining.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

stepwise_supervised.qmd

Misc fixes 20250130 (#2301)

2025-01-31 08:58:04 -05:00

template_free.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

tokenized.qmd

Fix documentation for pre-tokenized dataset (#1894)

2024-09-05 23:11:31 +09:00