add new sharegpt, refactor prompt so it can be customized later, add exception if no data is processed

This commit is contained in:
Wing Lian
2023-06-11 18:46:26 -04:00
parent f31a338cbb
commit aac4b7691e
6 changed files with 90 additions and 17 deletions

View File

@@ -219,6 +219,14 @@ Have dataset(s) in one of the following format (JSONL recommended):
```json
{"conversations": [{"role": "...", "value": "..."}]}
```
- `sharegpt_simple.load_role`: conversations where `role` is used instead of `from`
```json
{"conversations": [{"role": "...", "value": "..."}]}
```
- `sharegpt_jokes`: creates a chat where bot is asked to tell a joke, then explain why the joke is funny
```json
{"conversations": [{"title": "...", "text": "...", "explanation": "..."}]}
```
</details>
@@ -530,7 +538,7 @@ Try set `fp16: true`
Try to turn off xformers.
## Need help? 🙋♂️
## Need help? 🙋♂️
Join our [Discord server](https://discord.gg/HhrNrHJPRb) where we can help you