use pythia-12b, neox-20b is flaky

2023-05-27 19:37:24 -04:00
parent 28acebac36
commit 8a37b43678
2 changed files with 22 additions and 8 deletions
--- a/examples/pythia-12b/README.md
+++ b/examples/pythia-12b/README.md
@@ -0,0 +1,10 @@
+# Python 12B
+
+- Single-GPU A100 only (?)
+
+```shell
+python scripts/finetune.py examples/pythia-12b/config.yml
+```
+
+⚠️ Multiple-GPU A100 - Doesn't seem to work with multi-gpu without causing OOM! ⚠️
+