tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Merge pull request #130 from OpenAccess-AI-Collective/gas

Browse Source 
swap batch size for gradient accumulation steps to decouple from num gpu
This commit is contained in:
Wing Lian
2023-05-31 13:03:51 -04:00
committed by GitHub
parent 5079753b7a c2a0792680
commit f94dd626f0
16 changed files with 17 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -265,7 +265,7 @@ wandb_log_model: # 'checkpoint'
output_dir: ./completed-model
# training hyperparameters
batch_size: 8
gradient_accumulation_steps: 1
micro_batch_size: 2
eval_batch_size: 2
num_epochs: 3