tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

chore(readme): update instruction to set config to load from cache (#1030)

Browse Source
This commit is contained in:
NanoCode012
2024-01-03 11:56:19 +09:00
committed by GitHub
parent c75f916745
commit b31038aae9
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@@ -935,8 +935,9 @@ accelerate launch -m axolotl.cli.train your_config.yml
You can optionally pre-tokenize dataset with the following before finetuning. You can optionally pre-tokenize dataset with the following before finetuning.
This is recommended for large datasets. This is recommended for large datasets.
- Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface. - Set `dataset_prepared_path:` to a local folder for saving and loading pre-tokenized dataset.
- Use `--debug` to see preprocessed examples. - (Optional): Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface.
- (Optional): Use `--debug` to see preprocessed examples.
```bash ```bash
python -m axolotl.cli.preprocess your_config.yml python -m axolotl.cli.preprocess your_config.yml