9f824ef76a
* simplify the example configs to be more minimal and less daunting * drop empty s2_attention from example yamls
Pythia 12B
- Single-GPU A100 only (?)
python scripts/finetune.py examples/pythia-12b/config.yml
⚠️ Multiple-GPU A100 - Doesn't seem to work with multi-gpu without causing OOM! ⚠️