From e3a38450ded395feb8ef1bc227d09b8b476d18f9 Mon Sep 17 00:00:00 2001 From: Byron Hsu Date: Thu, 29 Aug 2024 05:19:18 -0700 Subject: [PATCH] Add liger kernel to features (#1881) [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index af604fad5..c84f1cb8c 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Features: - Supports fullfinetune, lora, qlora, relora, and gptq - Customize configurations using a simple yaml file or CLI overwrite - Load different dataset formats, use custom formats, or bring your own tokenized datasets -- Integrated with xformer, flash attention, rope scaling, and multipacking +- Integrated with xformer, flash attention, [liger kernel](https://github.com/linkedin/Liger-Kernel), rope scaling, and multipacking - Works with single GPU or multiple GPUs via FSDP or Deepspeed - Easily run with Docker locally or on the cloud - Log results and optionally checkpoints to wandb or mlflow