tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

chore(doc): add clarification about mpi4py error on single gpu deepspeed (#2383) [skip ci]

Browse Source 
* chore(doc): add clarification about mpi4py error on single gpu deepspeed

* fix: lint
This commit is contained in:
NanoCode012
2025-03-05 22:01:28 +07:00
committed by GitHub
parent 9ed4f6b3aa
commit f18231c653
2 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/dataset-formats/index.qmd
View File

@@ -129,6 +129,7 @@ You can mix and match within each approach or across approaches to train a model
We suggest this approach when you want to bring your own tokenized dataset. We suggest this approach when you want to bring your own tokenized dataset.
Axolotl expects the dataset to have three keys: Axolotl expects the dataset to have three keys:
- `input_ids`: from tokenizing formatted prompt - `input_ids`: from tokenizing formatted prompt
- `attention_mask`: for masking padding. If you don't add padding, it would be equal to `len(input_ids) * [1]` - `attention_mask`: for masking padding. If you don't add padding, it would be equal to `len(input_ids) * [1]`
- `labels`: this is the same as `input_ids`, however, if you want to mask certain tokens, you would set those indices to `-100`. - `labels`: this is the same as `input_ids`, however, if you want to mask certain tokens, you would set those indices to `-100`.

4
docs/faq.qmd
View File

@@ -19,7 +19,9 @@ description: Frequently asked questions
**Q: AttributeError: 'DummyOptim' object has no attribute 'step'** **Q: AttributeError: 'DummyOptim' object has no attribute 'step'**
> A: You may be using deepspeed with single gpu. Please don't set `deepspeed:` in yaml or cli. **Q: ModuleNotFoundError: No module named 'mpi4py' using single GPU with deepspeed**
> A: You may be using deepspeed with single gpu. Please remove the `deepspeed:` section in the yaml file or `--deepspeed` CLI flag.
**Q: The codes is stuck on saving preprocessed datasets.** **Q: The codes is stuck on saving preprocessed datasets.**