From d883b11b6f0ef42d90595db256087f7c1fdd592d Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Wed, 5 Mar 2025 22:00:39 +0700 Subject: [PATCH] fix(doc): add installation for cce to docs (#2375) [skip ci] * fix(doc): add installation for cce to docs * fix: format --- .../integrations/cut_cross_entropy/README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/axolotl/integrations/cut_cross_entropy/README.md b/src/axolotl/integrations/cut_cross_entropy/README.md index 625cbc0ec..14449a144 100644 --- a/src/axolotl/integrations/cut_cross_entropy/README.md +++ b/src/axolotl/integrations/cut_cross_entropy/README.md @@ -4,6 +4,22 @@ Cut Cross Entropy reduces VRAM usage through optimization on the cross-entropy o See https://github.com/apple/ml-cross-entropy +## Requirements + +- PyTorch 2.4.0 or higher + +## Installation + +Run the following command to install `cut_cross_entropy[transformers]` if you don't have it already. + +```bash +# if you are in dev environment +python scripts/cutcrossentropy_install.py | sh + +# if you are not in dev environment +pip3 uninstall -y cut-cross-entropy && pip3 install "cut-cross-entropy @ git+https://github.com/apple/ml-cross-entropy.git@9c297c905f55b73594b5d650722d1e78183b77bd"' +``` + ## Usage ```yaml