More hints on what to do with CUDA Out of memory errors (#925)
This commit is contained in:
@@ -1022,6 +1022,10 @@ Please reduce any below
|
|||||||
- `gradient_accumulation_steps`
|
- `gradient_accumulation_steps`
|
||||||
- `sequence_len`
|
- `sequence_len`
|
||||||
|
|
||||||
|
If it does not help, try running without deepspeed and without accelerate (replace "accelerate launch" with "python") in the command.
|
||||||
|
|
||||||
|
Using adamw_bnb_8bit might also save you some memory.
|
||||||
|
|
||||||
> `failed (exitcode: -9)`
|
> `failed (exitcode: -9)`
|
||||||
|
|
||||||
Usually means your system has run out of system memory.
|
Usually means your system has run out of system memory.
|
||||||
|
|||||||
Reference in New Issue
Block a user