recast loralayer, norm, lmhead + embed token weights per original qlora (#393)

* recast loralayer, norm, lmhead + embed token weights per original qlora * try again for the fix * refactor torch dtype picking * linter fixes * missing import for LoraLayer * fix install for tests now that peft is involved
2023-08-21 18:41:12 -04:00
parent 50682a3c06
commit 96deb6bd67
4 changed files with 33 additions and 22 deletions
--- a/setup.py
+++ b/setup.py
@@ -32,5 +32,8 @@ setup(
        "extras": [
            "deepspeed",
        ],
+        "peft": [
+            "peft @ git+https://github.com/huggingface/peft.git",
+        ],
    },
 )