Wing Lian b3289fd190 feat: LoRA kernel support for bias, dropout, dora, embeddings (#3528) [skip ci] ...

* feat: LoRA kernel support for bias, dropout, dora, embeddings

* chore: lint

* chore: lint

* address PR feedback, add regression tests, add fsdp2 tests for lora kernels

* update tests for new sigs

* update tests now that bias and dropout are supported

2026-03-22 13:53:19 -04:00

..

lora_kernels

feat: LoRA kernel support for bias, dropout, dora, embeddings (#3528) [skip ci]

2026-03-22 13:53:19 -04:00

__init__.py

attempt to also run e2e tests that needs gpus (#1070)

2024-01-09 21:23:23 -05:00

test_4d_multipack_llama.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_activation_checkpointing.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_cli_integrations.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_fa_xentropy.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_falcon_samplepack.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_flattening.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_fsdp2_qlora.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_fused_llama.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_llama_s2_attention.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_lora_llama_multipack.py

Scattermoe LoRA optimizations (#3513)

2026-03-19 23:07:42 -04:00

test_mistral_samplepack.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_mixtral_samplepack.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_model_patches.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_peft_embeddings.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_phi_multipack.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_resume.py

fix token state json and mistral tokenizer issue (#3522) [skip ci]

2026-03-21 22:46:10 -04:00

test_unsloth_integration.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_unsloth_qlora.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00