Wing Lian a07f432d9c calculate cum seq lens with pos_ids instead of mask, simplify packing params, fix distributed barrier

2023-08-10 17:16:01 -04:00

..

fixtures

experimental llama 2 chat support (#296)

2023-08-06 17:40:52 -04:00

monkeypatch

add unit tests for cum seq lens, add ability to build cu_seq_lens from positional ids, fix prompt test

2023-08-07 09:38:04 -04:00

test_dict.py

Ignore unsupported-binary-operation

2023-05-31 02:53:53 +09:00

test_expand_mask.py

calculate cum seq lens with pos_ids instead of mask, simplify packing params, fix distributed barrier

2023-08-10 17:16:01 -04:00

test_packed_dataset.py

set position ids and use block diagonal attn mask

2023-08-07 09:38:03 -04:00

test_prompt_tokenizers.py

add unit tests for cum seq lens, add ability to build cu_seq_lens from positional ids, fix prompt test

2023-08-07 09:38:04 -04:00

test_prompters.py

fix test and pylint checks

2023-08-07 09:38:05 -04:00

test_tokenizers.py

optionally define whether to use_fast tokenizer

2023-06-25 10:19:49 -04:00

test_validation.py

validation for sample packing and doc

2023-08-07 09:38:03 -04:00