Add QLoRA + FSDP Docs (#1403)

* pre commit * Update fsdp_qlora.md
2024-03-14 08:04:51 -07:00
parent 0976781e15
commit 8b12468230
2 changed files with 41 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -1080,6 +1080,10 @@ fsdp_config:
  fsdp_transformer_layer_cls_to_wrap: LlamaDecoderLayer
 ```

+##### FSDP + QLoRA
+
+Axolotl supports training with FSDP and QLoRA, see [these docs](docs/fsdp_qlora.md) for more information.
+
 ##### Weights & Biases Logging

 Make sure your `WANDB_API_KEY` environment variable is set (recommended) or you login to wandb with `wandb login`.