NanoCode012
|
45bf634d17
|
feat: add support for multimodal in lora kernels (#2472) [skip ci]
* feat: add support for multimodal in lora kernels
* fix: improve multimodal checks
* fix: add fallback for model config
* chor: add gemma3 to docs
|
2025-04-02 09:33:46 -04:00 |
|
NanoCode012
|
7235123d44
|
chore(docs): add cookbook/blog link to docs (#2410) [skip ci]
|
2025-03-17 08:38:19 -04:00 |
|
NanoCode012
|
2efe1b4c09
|
Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)
* feat(doc): organize docs, add to menu bar, fix broken formatting
* feat: add link to custom integrations
* feat: update readme for integrations to include citations and repo link
* chore: update lm_eval info
* chore: use fullname
* Update docs/cli.qmd per suggestion
Co-authored-by: Dan Saunders <danjsaund@gmail.com>
* feat: add sweep doc
* feat: add kd doc
* fix: remove toc
* fix: update deprecation
* feat: add more info about chat_template issues
* fix: heading level
* fix: shell->bash code block
* fix: ray link
* fix(doc): heading level, header links, formatting
* feat: add grpo docs
* feat: add style changes
* fix: wrong cli arg for lm-eval
* fix: remove old run method
* feat: load custom integration doc dynamically
* fix: remove old cli way
* fix: toc
* fix: minor formatting
---------
Co-authored-by: Dan Saunders <danjsaund@gmail.com>
|
2025-02-25 16:09:37 +07:00 |
|
Dan Saunders
|
954e192f38
|
quick formatting fix for LoRA optims doc (#2349)
|
2025-02-19 09:23:31 -05:00 |
|
Dan Saunders
|
c3d4f6e295
|
Doc fix: TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL not necessary to use Triton kernel patches (#2343)
* removing note about TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL
* suggest using TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL for memory efficient attn
|
2025-02-18 10:06:31 -05:00 |
|
Dan Saunders
|
3d8425fa91
|
Activation function Triton kernels, LoRA custom autograd functions (#2324)
* LoRA + activation fn Triton kernels: initial commit
* implementing optims
* finalizing MLP LoRA kernels and progress on QKV / W kernels
* updates
* O projection optim
* adding monkey patching logic
* doc strings, typing, pre-commit fixes
* updates
* adding lora 8b kernels example
* working on fsdp support
* tests and fixes
* small fixes, getting tests to pass, adding doc strings
* integration tests for LoRA patching
* config.qmd
* remove unneeded pytest fixture
* fix
* review comments first pass
* improving tests, attention class agnostic patching
* adding support for more archs
* wip SiLU / GELU impls
* improved testing, small updates, etc.
* slightly updating docs
* rebase
* fixing test_attention_patching_integration
* additional review comments, fixing test in CI (hopefully)
* isolating problematic patching test
* relaxing allclose threshold to reduce flakiness
* fixing accidental change
* adding model arch agnostic attention class fetching
* removing unused activations
|
2025-02-17 14:23:15 -05:00 |
|