NanoCode012
28e7e444ee
fix: update bradleyterry to use new chat_template
2024-10-16 20:42:14 +07:00
NanoCode012
207e7627f9
fix(doc): formatting
2024-10-15 00:41:50 +07:00
NanoCode012
7eb62ae5a9
fix: update dummy message to prevent potential overlap with real content
2024-10-14 23:50:35 +07:00
NanoCode012
95805cf850
chore: lint
2024-10-14 23:43:30 +07:00
NanoCode012
4aafb7e600
fix: imported name incorrectly updated on merge
2024-10-14 23:41:17 +07:00
NanoCode012
17bc4c8b36
fix: update test based on new defaults
2024-10-14 18:03:35 +07:00
NanoCode012
d101cfc125
feat: handles chat_template requiring specific user/assistant order
2024-10-14 14:00:55 +07:00
NanoCode012
e5cd55cff9
feat: add example using fallback
2024-10-14 12:22:22 +07:00
NanoCode012
24aa6b15a0
feat: handle sharegpt deprecation better in docs
2024-10-14 12:21:58 +07:00
NanoCode012
9dfc5fa8b8
fix: remove default setting on edge case where chat template overriden in dataset section
2024-10-14 11:48:40 +07:00
NanoCode012
0c3255288f
Merge branch 'main' into cj_tokenizer_default_prompt_template
2024-10-14 10:36:08 +07:00
Wing Lian
335027f155
upgrade accelerate to 1.0.1 ( #1969 )
2024-10-13 20:04:30 -04:00
Wing Lian
ec4272c3a0
add ds zero3 to multigpu biweekly tests ( #1900 )
...
* add ds zero3 to multigpu biweekly tests
* fix for upstream api change
* use updated accelerate and fix deepspeed tests
* stringify the Path, and run multigpu tests if the multigpu tests change for a PR
* use correct json rather than yaml
* revert accelerate for deepspeed
2024-10-13 17:34:37 -04:00
Wing Lian
68b1369de9
Reward model ( #1879 )
2024-10-13 15:11:13 -04:00
Wing Lian
cd2d89f467
wip add new proposed message structure ( #1904 )
...
* wip add new proposed message structure
* tokenization
* wip
* wip transform builder
* wip make the chat dataset loadable
* wip chatml + llama 3 new chat objects
* chore: lint
* chore: lint
* fix tokenization
* remove dacite dependency since we're using pydantic now
* fix handling when already correctly split in messages
* make sure to remove chat features from tokenized ds
* move chat to be a input transform for messages
* make sure llama3 has the bos token
* remove non-working special token code
* fix messages strat loader
2024-10-13 12:15:18 -04:00
Chirag Jain
82b5dc9328
Merge branch 'main' into cj_tokenizer_default_prompt_template
2024-10-13 16:27:10 +05:30
Vincent Haines
1834cdc364
Add support for qwen 2.5 chat template ( #1934 )
2024-10-12 21:41:43 -04:00
NanoCode012
ac128b7b1d
fix: update eval causal lm metrics to add perplexity ( #1951 ) [skip ci]
2024-10-12 21:41:13 -04:00
pandora
31591bd94c
Fixing Validation - Mistral Templates ( #1962 )
2024-10-12 21:40:39 -04:00
Wing Lian
d20b48a61e
only install torchao for torch versions >= 2.4.0 ( #1963 )
2024-10-12 20:53:48 -04:00
Wing Lian
09bf1ceacc
update hf deps ( #1964 )
...
* update hf deps
* remove deprecated set_caching_enabled
2024-10-12 18:19:48 -04:00
Afrizal Hasbi Azizy
df359c8a6e
Handle image input as string paths for MMLMs ( #1958 )
...
* Update mm_chat.py
Handle string image (paths)
* chore: lint
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com >
2024-10-11 13:34:13 -04:00
Wing Lian
76883851d2
add warning that sharegpt will be deprecated ( #1957 )
...
* add warning that sharegpt will be deprecated
* add helper script for chat_templates and document deprecation
* Update src/axolotl/prompt_strategies/sharegpt.py
Co-authored-by: NanoCode012 <nano@axolotl.ai >
---------
Co-authored-by: NanoCode012 <nano@axolotl.ai >
2024-10-11 13:33:20 -04:00
Adam Hazell
922db77521
Add MLFlow run name option in config ( #1961 )
...
Co-authored-by: Adam Hazell <adam.hazell@mindfoundry.ai >
2024-10-11 13:33:06 -04:00
Thomas Cleberg
e73b8dff8d
Add Support for revision Dataset Parameter to specify reading from Huggingface Dataset Revision ( #1912 )
...
* Add support for `revision` dataset parameter
* only use revision on hf hub backed datasets
* use revision tied to head
* set download to use revision
* feat: add config to model validator class
* feat: add revision config to RL and tests for it
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com >
Co-authored-by: NanoCode012 <nano@axolotl.ai >
2024-10-11 13:32:50 -04:00
Chirag Jain
ec57918fcd
Merge pull request #7 from NanoCode012/cj_tokenizer_default_prompt_template
...
Feat: merge latest, update docs, fix dropped config bug, added unit test
2024-10-11 14:44:25 +05:30
NanoCode012
dd87d8c438
feat: add test for levy's dpo case
2024-10-11 12:56:46 +07:00
NanoCode012
ef942b6efc
fix: rename var after merge
2024-10-11 12:30:43 +07:00
NanoCode012
3c6a6c61be
Merge branch 'main' into cj_tokenizer_default_prompt_template
2024-10-11 12:29:34 +07:00
NanoCode012
7b4b665e99
chore: skip duplicate
2024-10-11 11:42:36 +07:00
NanoCode012
21326e4ef3
chore: lint
2024-10-11 11:40:42 +07:00
NanoCode012
de23dab4fc
fix: config being dropped and unittest to catch that
2024-10-11 11:40:32 +07:00
NanoCode012
e3efa29cf5
fix: test
2024-10-11 11:11:19 +07:00
Wing Lian
2fbc6b0c64
Axo logo new ( #1956 )
...
* update axolotl ascii art
* spacing for logo
* cleanup dithering
* cleanup ascii logo a bit
2024-10-10 15:57:37 -04:00
Wing Lian
8159cbd1ab
lm_eval harness post train ( #1926 )
...
* wip, lm_eval harness post train
* include latex parser
* add dtype and doc
* add validation when doing bench evals
* automatically add test dataset when doing benches
2024-10-10 15:04:17 -04:00
NanoCode012
2038255052
Merge branch 'main' into cj_tokenizer_default_prompt_template
2024-10-10 20:25:37 +07:00
pandora
979534c851
add mistral templates ( #1927 )
...
Co-authored-by: Wing Lian <wing.lian@gmail.com >
2024-10-10 09:22:53 -04:00
NanoCode012
dab2590e4d
chore: refactor
2024-10-10 18:07:00 +07:00
NanoCode012
e5162b7a41
chore: added example for non-default template
2024-10-10 18:04:33 +07:00
NanoCode012
b6321d2220
chore: clarify doc
2024-10-10 18:01:33 +07:00
NanoCode012
6b3cdfdb8e
feat(doc): updated config with chat template options and clarified examples
2024-10-10 17:57:11 +07:00
NanoCode012
203ae28704
fix: refactor artifact left from main merge
2024-10-10 17:16:41 +07:00
NanoCode012
ed3a33c9fb
fix: re-arrange enum declaration position
2024-10-10 16:18:15 +07:00
NanoCode012
f61e2fc7dc
chore: remove redundant function
2024-10-10 16:15:15 +07:00
NanoCode012
b8056d04d9
Merge branch 'main' into cj_tokenizer_default_prompt_template
2024-10-10 16:11:07 +07:00
NanoCode012
88658c0570
fix: set default to tokenizer template
2024-10-10 15:38:19 +07:00
Boris Feld
6d3caadf90
Comet integration ( #1939 )
...
* Add first version of a Comet integration
* Remove debug prints
* Add test for Comet Configuration transformation to env variables
* Fix last lint warning
* Update Readme for Comet logging documentation
* Update Comet integration to be optional, update code and tests
* Add documentation for Comet configuration
* Add missing check
2024-10-09 16:03:37 -04:00
aarush gupta
dee77232fe
fix type annotations ( #1941 ) [skip ci]
2024-10-09 16:03:16 -04:00
NanoCode012
a560593b1d
fix(log): update perplexity log to clarify from eval split ( #1952 ) [skip ci]
2024-10-09 16:02:32 -04:00
Wing Lian
e8d3da0081
upgrade pytorch from 2.4.0 => 2.4.1 ( #1950 )
...
* upgrade pytorch from 2.4.0 => 2.4.1
* update xformers for updated pytorch version
* handle xformers version case for torch==2.3.1
2024-10-09 11:53:56 -04:00
