Mixtral multipack (#928)

* mixtral multipack * use mixtral model * sample yml * calculate cu_seqlens properly * use updated flash ettention setting * attn var checks * force use of flash attention 2 for packing * lint * disable future fix for now * update support table
2023-12-09 21:26:30 -05:00
parent 03c6318ba3
commit 68b227a7d8
8 changed files with 1793 additions and 25 deletions
--- a/.mypy.ini
+++ b/.mypy.ini
@@ -8,6 +8,9 @@ ignore_missing_imports = True
 [mypy-axolotl.monkeypatch.*]
 ignore_errors = True

+[mypy-axolotl.models.mixtral.*]
+ignore_errors = True
+
 [mypy-axolotl.models.phi.*]
 ignore_errors = True