From 810ecd4e81a93a8cd8d740c4e59f12fa05424f69 Mon Sep 17 00:00:00 2001
From: Wing Lian <wing.lian@gmail.com>
Date: Fri, 23 Aug 2024 14:34:03 -0400
Subject: [PATCH] add liger to readme (#1865)

* add liger to readme

* updates from PR feedback
---
 README.md | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/README.md b/README.md
index 8c70da015..af604fad5 100644
--- a/README.md
+++ b/README.md
@@ -55,6 +55,7 @@ Features:
         - [FSDP + QLoRA](#fsdp--qlora)
         - [Weights \& Biases Logging](#weights--biases-logging)
         - [Special Tokens](#special-tokens)
+      - [Liger Kernel](#liger-kernel)
     - [Inference Playground](#inference-playground)
     - [Merge LORA to base](#merge-lora-to-base)
   - [Common Errors 🧰](#common-errors-)
@@ -530,6 +531,25 @@ tokens: # these are delimiters
 
 When you include these tokens in your axolotl config, axolotl adds these tokens to the tokenizer's vocabulary.
 
+##### Liger Kernel
+
+Liger Kernel: Efficient Triton Kernels for LLM Training
+
+https://github.com/linkedin/Liger-Kernel
+
+Liger (LinkedIn GPU Efficient Runtime) Kernel is a collection of Triton kernels designed specifically for LLM training.
+It can effectively increase multi-GPU training throughput by 20% and reduces memory usage by 60%. The Liger Kernel
+composes well and is compatible with both FSDP and Deepspeed.
+
+```yaml
+plugins:
+  - axolotl.integrations.liger.LigerPlugin
+liger_rope: true
+liger_rms_norm: true
+liger_swiglu: true
+liger_fused_linear_cross_entropy: true
+```
+
 ### Inference Playground
 
 Axolotl allows you to load your model in an interactive terminal playground for quick experimentation.