Wing Lian 40a53ff181 Merge pull request #307 from OpenAccess-AI-Collective/xgen-user-sharegpt-tokens ...

better handling since xgen tokenizer breaks with convert_tokens_to_ids

2023-07-22 04:10:38 -04:00

..

monkeypatch

fix sdp attention to use the flash/mem-efficient context manaager

2023-07-20 01:05:48 -04:00

prompt_strategies

Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var

2023-07-15 12:29:35 +00:00

utils

don't resize embeddings to multiples of 32x by default

2023-07-22 01:52:38 -04:00

__init__.py

WIP for axolotl trainer

2023-04-14 00:20:05 -04:00

convert.py

Lint convert.py

2023-05-31 02:53:53 +09:00

datasets.py

feat: use multi-core

2023-07-19 10:16:54 +09:00

flash_attn.py

flash attention 2

2023-07-21 08:17:46 -04:00

logging_config.py

misc fixes

2023-07-17 03:00:27 -04:00

prompt_tokenizers.py

better handling since xgen tokenizer breaks with convert_tokens_to_ids

2023-07-21 09:24:11 -04:00

prompters.py

Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var

2023-07-15 12:29:35 +00:00