Fix(config): Update handling of deepspeed config (#404)

* Fix(config): Update handling of deepspeed config * feat: auto set deepspeed env if deepspeed passed * fix: update new deepspeed instructions
2023-08-16 01:22:43 +09:00
parent 72fe3f8e3d
commit c01015f33f
3 changed files with 13 additions and 15 deletions
--- a/README.md
+++ b/README.md
@@ -519,7 +519,7 @@ tokens:
 fsdp:
 fsdp_config:

-# Deepspeed
+# Deepspeed config path
 deepspeed:

 # Path to torch distx for optim 'adamw_anyprecision'
@@ -570,7 +570,10 @@ fsdp_config:
  fsdp_transformer_layer_cls_to_wrap: LlamaDecoderLayer
 ```

- llama Deepspeed: append `ACCELERATE_USE_DEEPSPEED=true` in front of finetune command
+- llama Deepspeed
+```yaml
+deepspeed: # path to config
+```

 ##### Weights & Biases Logging