Wing Lian c7cf3810bd Pretrain transforms (#1261) ...

* wip for pretraining/iterable data with arbitrary prompt strategies

* more fixes, wip

* more fixes for custom pretraining

* iterable ds wrapper not needed

* remove extra features

* chore: lint

* update pretraning example yml

* fix order for partials

* fixup for tests

2024-02-06 00:37:03 -05:00

2.6 KiB

Raw Blame History

View Raw