Wing Lian
|
bcb59c70e2
|
automatically set pad_to_sequence_len when use packing (#2607)
* automatically set pad_to_sequence_len when use packing
* update tests
|
2025-05-01 13:24:38 -04:00 |
|
Dan Saunders
|
113e9cd193
|
Autodoc generation with quartodoc (#2419)
* quartodoc integration
* quartodoc progress
* deletions
* Update docs/.gitignore to exclude auto-generated API documentation files
* Fix
* more autodoc progress
* moving reference up near the top of the sidebar
* fix broken link
* update to reflect recent changes
* pydantic models refactor + add to autodoc + fixes
* fix
* shrinking header sizes
* fix accidental change
* include quartodoc build step
* update pre-commit version
* update pylint
* pre-commit
---------
Co-authored-by: Dan Saunders <dan@axolotl.ai>
|
2025-03-21 12:26:47 -04:00 |
|
Wing Lian
|
bd2a594b89
|
use DataCollatorWithFlattening when not sample packing (#2167)
|
2024-12-17 17:46:44 -05:00 |
|
Wing Lian
|
3798229d85
|
handle torch_compile set to auto (#2172) [skip ci]
* handle torch_compile set to auto
* update docs [skip ci]
* add tests
|
2024-12-17 16:42:41 -05:00 |
|
Dan Saunders
|
fc973f4322
|
CLI Implementation with Click (#2107)
* Initial CLI implementation with click package
* Adding fetch command for pulling examples and deepspeed configs
* Automating default options for CliArgs classes
* Mimicking existing no config behavior
* bugfix in choose_config
* Updating fetch to sync instead of re-download
* bugfix
* isort fix
* fixing yaml isort order
* pre-commit fixes
* simplifying argument parsing -- pass through kwargs to do_cli
* make accelerate launch default for non-preprocess commands
* fixing arg handling
* testing None placeholder approach
* removing hacky --use-gpu argument to preprocess command
* Adding brief README documentation for CLI
* remove (New)
* Initial CLI pytest tests
* progress on CLI pytest
* adding inference CLI tests; cleanup
* Refactor train CLI tests to remove various mocking
* Major CLI test refator; adding remaining CLI codepath test coverage
* pytest fixes
* remove integration markers
* parallelizing examples, deepspeed config downloads; rename test to match other CLI test naming
* moving cli pytest due to isolation issues; cleanup
* testing fixes; various minor improvements
* fix
* tests fix
* Update tests/cli/conftest.py
Co-authored-by: Wing Lian <wing.lian@gmail.com>
---------
Co-authored-by: Dan Saunders <dan@axolotl.ai>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
|
2024-12-05 22:11:48 -05:00 |
|