Wing Lian cae5cebb59 xformers attention with packing (#2619) ...

* xformers attention with packing

* wire up the patch

* fix xformers + packing validation

* fix warning

* reorder the packing check

* fix fp16 / bf16 reset when using fp16 with bf16 auto

* fix seq lens calc to drop hanging sequences

* handle xformers patch for inference too

* fix batch size setter

* fix xformers inference

* add colab callback to fix inference post train

* PR feedback

2025-05-07 16:10:16 -04:00

34 KiB

Raw Blame History

View Raw