tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Added advanced DDP args (#515)

Browse Source 
* add ddp_config

* add advanced ddp config

* add ddp_config

* add advanced ddp config

---------

Co-authored-by: Jan Philipp Harries <jphme@users.noreply.github.com>
This commit is contained in:
Jan Philipp Harries
2023-08-31 19:37:47 +02:00
committed by GitHub
parent b21e4a20fe
commit 396a7a74fc
2 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@@ -623,6 +623,11 @@ fsdp_config:
# Deepspeed config path
deepspeed:
# Advanced DDP Arguments
ddp_timeout:
ddp_bucket_cap_mb:
ddp_broadcast_buffers:
# Path to torch distx for optim 'adamw_anyprecision'
torchdistx_path: