2 Commits

Author	SHA1	Message	Date
Wing Lian	418ad2b586	add missing fixture decorator for predownload dataset (#2117) [skip ci] ... * add missing fixture decorator for predownload dataset * also pre download the tokenizer files	2024-12-03 18:08:46 -05:00
Keith Stevens	7b9f669a3a	Trigger the original tokenization behavior when no advanced turn settings are provided (#1915)	2024-09-14 08:22:54 -04:00