diff --git a/FAQS.md b/FAQS.md
new file mode 100644
index 000000000..7eb741482
--- /dev/null
+++ b/FAQS.md
@@ -0,0 +1,4 @@
+# FAQs
+
+- Can you train StableLM with this? Yes, but only with a single GPU atm. Multi GPU support is coming soon! Just waiting on this [PR](https://github.com/huggingface/transformers/pull/22874)
+-