Sequence parallel training context manager (#2553)

* ctx manager for SP * updates * update * further simplifying * accommodate both training context managers * simplifying * simplifying * nit * reorg * tweak codecov yaml * add gather post hook, simplify, fixes * pytest * pytest fix
2025-04-25 10:33:54 -04:00
parent 1447beb132
commit ae1c7ace63
12 changed files with 608 additions and 207 deletions
--- a/tests/e2e/patched/test_mixtral_samplepack.py
+++ b/tests/e2e/patched/test_mixtral_samplepack.py
@@ -99,6 +99,7 @@ class TestMixtral(unittest.TestCase):
                "bf16": "auto",
            }
        )
+        cfg = validate_config(cfg)
        normalize_config(cfg)
        cli_args = TrainerCliArgs()
        dataset_meta = load_datasets(cfg=cfg, cli_args=cli_args)