4 Commits

Author	SHA1	Message	Date
Wing Lian	4964b0d345	set position ids and use block diagonal attn mask	2023-08-07 09:38:03 -04:00
Wing Lian	36b0e30a9d	fix attetion mask with packing	2023-08-07 09:38:03 -04:00
Wing Lian	0136f510f2	don't worry about duplicate code here	2023-05-31 12:05:43 -04:00
Wing Lian	9b8585dc70	fix packing so that concatenated sequences reset the attention	2023-05-31 11:38:52 -04:00