tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
1,432 Commits 319 Branches 32 Tags
50421c8b1dd63cceef0a06e9700a1c3e834685f2
Commit Graph

9 Commits

Author SHA1 Message Date
Ram
50421c8b1d feat: Add LLaMA-3 instruct prompt strategies for fine-tuning (#1553) 
* Add prompt strategies

* Update modified URL

* Update modified URL

* Update fastchat_conversation_turns.py

* Update register function

* Remove extra /n for system prompt

* Fix return

* Fix BOS

* Update requirements, pylint

* Linting

* Linting

* fix tuples, make sure to set system message in template

* tests for llama3 tokenization

* fix conditionals for loading chat template

---------

Co-authored-by: Ram <ram@Rams-MacBook-Pro.local>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2024-05-11 00:08:04 -04:00
Wing Lian
c10563c444 fix broken linting (#1541) 
* chore: lint

* include examples in yaml check

* mistral decided to gate their models...

* more mistral models that were gated
 2024-04-19 01:03:04 -04:00
NanoCode012
40a88e8c4a Feat: Add sharegpt multirole (#1137) 
* feat(prompt): support multiple roles for sharegpt

* fix: add handling of empty role back

* feat: rebased and allowed more dynamic roles via config

* fix: variable

* chore: update message

* feat: add vicuna format

* fix: JSON serializable error

* fix: typing

* fix: don't remap for unknown keys

* fix: add roles to pydantic

* feat: add test

* chore: remove leftover print

* chore: remove leftover comment

* chore: remove print

* fix: update test to use chatml
 2024-03-19 20:51:49 +09:00
Brian Fitzgerald
b7d8a7dc4d Add Glaive conversation format support (#1365) 
* Add Glaive conversation format support

* fix black formatting errors

* Fix black and pylint formatting errors

* only set role_key_tool if provided in the dataset constructor

* Update src/axolotl/prompt_strategies/sharegpt.py

Co-authored-by: Wing Lian <wing.lian@gmail.com>

* sharegpt test

* tokenizer test

* fix formatting

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2024-03-10 20:50:25 -04:00
Wing Lian
4d09b42ee3 plain input/output prompt strategy w/o chat templates (#1346) 
* plain input/output prompt strategy w/o chat templates

* disable duplicate code check

* make sure to add an eos/eot token to the end of the output so it will stop

* multi turn segement support and test
 2024-03-04 16:25:16 -05:00
mhenrichsen
98b4762077 Feat/chatml add system message (#1117) 
* add system message to template

* readme update

* added code to register new system message

* register chatml template for test

---------

Co-authored-by: Mads Henrichsen <mads@BrbartiendeMads.lan>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2024-01-25 08:24:27 +01:00
JohanWork
5439707489 Feat(test): Add tests for alpaca chatml prompt tokenizer (#1088) 
* draft for adding test for tokenizer

* clean up

* clean up

* fix pre commit

* fix pylint

* Revert "fix pylint"

This reverts commit cd2cda3cda.

* add pylint exception for pytest fixture

* update comments

* Apply suggestions from code review

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

* update spelling and import promptstyle

* reaname, restrucure

* clean up

* add fmt:on

---------

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
 2024-01-23 13:30:26 +09:00
NanoCode012
043c3860cd fix: train_on_inputs: true ignored for sharegpt (#1045) [skip ci] 
* fix: `train_on_inputs: true` ignored for sharegpt

* enable unit test for train_on_inputs for sharegpt

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2024-01-09 23:00:09 -05:00
Wing Lian
651b7a31fc fix double eos token for chatml (#1054) [skip ci] 
* fix double eos token for chatml

* isolate fix to chatml conversation

* fix add special tokens to include rstrip

* add test for train_on_inputs for sharegpt

* don't use rstrip for chatml
 2024-01-09 09:33:38 -05:00