* use warmup_ratio as a better default than warmup steps since it's data dependent * replace remainder of warmup_steps
RedPajama 3B preview release
accelerate launch scripts/finetune.py examples/redpajama/config-3b.yml
* use warmup_ratio as a better default than warmup steps since it's data dependent * replace remainder of warmup_steps
accelerate launch scripts/finetune.py examples/redpajama/config-3b.yml