From c3d4f6e29553385d9c3d653bd61c4611e5fb34d7 Mon Sep 17 00:00:00 2001 From: Dan Saunders Date: Tue, 18 Feb 2025 10:06:31 -0500 Subject: [PATCH] Doc fix: TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL not necessary to use Triton kernel patches (#2343) * removing note about TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL * suggest using TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL for memory efficient attn --- docs/lora_optims.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/lora_optims.qmd b/docs/lora_optims.qmd index a57aa854b..3f8276bc5 100644 --- a/docs/lora_optims.qmd +++ b/docs/lora_optims.qmd @@ -82,7 +82,7 @@ lora_o_kernel: true ## Requirements - One or more NVIDIA or AMD GPUs (in order to use the Triton kernels) - - AMD can be used with experimental Triton support by setting the environment variable `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` + - Note: Set `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` to enable [memory-efficient attention on AMD GPUs](https://github.com/ROCm/aotriton/issues/16#issuecomment-2346675491) - Targeted LoRA adapters cannot use Dropout - This may limit model expressivity / cause overfitting - Targeted LoRA adapters cannot have bias terms