From b0cf397ecbde58866f8d0afe5c2a379de947aeb0 Mon Sep 17 00:00:00 2001
From: Juraj Bednar <juraj@bednar.io>
Date: Wed, 13 Dec 2023 08:38:38 +0100
Subject: [PATCH] More hints on what to do with CUDA Out of memory errors
 (#925)

---
 README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README.md b/README.md
index 7f092d308..1dfa524b1 100644
--- a/README.md
+++ b/README.md
@@ -1022,6 +1022,10 @@ Please reduce any below
   - `gradient_accumulation_steps`
   - `sequence_len`
 
+If it does not help, try running without deepspeed and without accelerate (replace "accelerate launch" with "python") in the command.
+
+Using adamw_bnb_8bit might also save you some memory.
+
 > `failed (exitcode: -9)`
 
 Usually means your system has run out of system memory.