Wing Lian
394a65f11f
add unit tests for cum seq lens, add ability to build cu_seq_lens from positional ids, fix prompt test
2023-08-07 09:38:04 -04:00
Jan Philipp Harries
3392270544
experimental llama 2 chat support ( #296 )
...
* experimental llama 2 chat support
* few small fixes
* llama2_chat
* small fix to follow original implementation
* small fixes and added fixtures/tests
* fix -mixed up inference and finetuning conversations
* args - small fix
* small fix
* small adjustment and warning
* fix with pre-commit
---------
Co-authored-by: Jan Philipp Harries <jpdus@users.noreply.github.com >
2023-08-06 17:40:52 -04:00
Wing Lian
3d4984b9a5
update prompts for open orca to match the paper ( #317 )
...
fix the test for the updated system tokenizer
2023-07-22 13:49:11 -04:00
theobjectivedad
b1f4f7a34d
Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var
2023-07-15 12:29:35 +00:00
theobjectivedad
553a86b52c
Adding logging enhancement
2023-07-14 07:26:19 -05:00
Wing Lian
3a38271276
add tests and supoort for loader for sys prompt data
2023-06-25 22:28:07 -04:00
Wing Lian
1ab3bf3e67
fix test name
2023-06-15 02:09:33 -04:00
Wing Lian
baed440fa1
ingore duplicate code in tests
2023-06-15 02:03:53 -04:00
Wing Lian
7925ddce86
bugfix for potential off by one
2023-06-15 01:59:33 -04:00
Wing Lian
0136f510f2
don't worry about duplicate code here
2023-05-31 12:05:43 -04:00
NanoCode012
b81c97ff76
Fix pre-commit for rebased files
2023-05-31 03:01:38 +09:00
Wing Lian
cfcc549f6b
fix relative path for fixtures
2023-05-31 02:55:21 +09:00
Wing Lian
e65aeedce7
fix relative path for fixtures
2023-05-30 10:38:20 -04:00
Wing Lian
e6fdeb087f
add unit test for sharegpt tokenization
2023-05-30 10:28:17 -04:00
