run tests again on Modal (#1289) [skip ci]

* run tests again on Modal

* make sure to run the full suite of tests on modal

* run cicd steps via shell script

* run tests in different runs

* increase timeout

* split tests into steps on modal

* increase workflow timeout

* retry doing this with only a single script

* fix yml launch for modal ci

* reorder tests to run on modal

* skip dpo tests on modal

* run on L4s, A10G takes too long

* increase CPU and RAM for modal test

* run modal tests on A100s

* skip phi test on modal

* env not arg in modal dockerfile

* upgrade pydantic and fastapi for modal tests

* cleanup stray character

* use A10s instead of A100 for modal

requirements.txt

@@ -6,7 +6,7 @@ tokenizers==0.15.0
 bitsandbytes>=0.41.1
 accelerate==0.26.1
 deepspeed==0.13.1
-pydantic>=2.5.3
+pydantic==2.6.3
 addict
 fire
 PyYAML>=6.0