Wing Lian 87455e7f32 swaps to use newer sample packing for mistral (#1773) ...

* swaps to use newer sample packing for mistral

* fix multipack patch test

* patch the common fa utils

* update for refactor of flash attn unpad

* remove un-needed drop attn mask for mistral

* bump transformers to main to pick up latest mistral fix for 12b and refactor of fa2

* update test

2024-07-23 01:41:11 -04:00

3.1 KiB

Raw Blame History

View Raw