3 Commits

Author	SHA1	Message	Date
Wing Lian	a07f432d9c	calculate cum seq lens with pos_ids instead of mask, simplify packing params, fix distributed barrier	2023-08-10 17:16:01 -04:00
Wing Lian	f93f0017cd	fix flash-attn, xformers, packing, support chatml	2023-08-07 09:38:04 -04:00
Wing Lian	e74eab6e73	add a test for the mask expansion for sequence packing	2023-08-07 09:38:04 -04:00