diff --git a/.nojekyll b/.nojekyll index 4397e558d..d6e8d99bd 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -80747424 \ No newline at end of file +d69e1193 \ No newline at end of file diff --git a/FAQS.html b/FAQS.html index 382bb20a3..04584d57b 100644 --- a/FAQS.html +++ b/FAQS.html @@ -2,7 +2,7 @@ - + diff --git a/TODO.html b/TODO.html index 0dd71c021..13bd0e233 100644 --- a/TODO.html +++ b/TODO.html @@ -2,7 +2,7 @@ - + diff --git a/docs/batch_vs_grad.html b/docs/batch_vs_grad.html index c73b47238..4c5ad49da 100644 --- a/docs/batch_vs_grad.html +++ b/docs/batch_vs_grad.html @@ -2,7 +2,7 @@ - + diff --git a/docs/config.html b/docs/config.html index 57e0c3c3d..af4fd2cfa 100644 --- a/docs/config.html +++ b/docs/config.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset-formats/conversation.html b/docs/dataset-formats/conversation.html index ef115d6ab..6bef9fe6f 100644 --- a/docs/dataset-formats/conversation.html +++ b/docs/dataset-formats/conversation.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset-formats/index.html b/docs/dataset-formats/index.html index ffbcf8437..ae6eecadf 100644 --- a/docs/dataset-formats/index.html +++ b/docs/dataset-formats/index.html @@ -2,7 +2,7 @@ - + @@ -351,7 +351,7 @@ Description - + Pre-training @@ -359,7 +359,7 @@ Description Data format for a pre-training completion task. - + Instruction Tuning @@ -367,7 +367,7 @@ Description Instruction tuning formats for supervised fine-tuning. - + Conversation @@ -375,7 +375,7 @@ Description Conversation format for supervised fine-tuning. - + Template-Free @@ -383,7 +383,7 @@ Description Construct prompts without a template. - + Custom Pre-Tokenized Dataset diff --git a/docs/dataset-formats/inst_tune.html b/docs/dataset-formats/inst_tune.html index 6f7ffa182..f7a6a41e3 100644 --- a/docs/dataset-formats/inst_tune.html +++ b/docs/dataset-formats/inst_tune.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset-formats/pretraining.html b/docs/dataset-formats/pretraining.html index abf29fb35..b5f4dc3f0 100644 --- a/docs/dataset-formats/pretraining.html +++ b/docs/dataset-formats/pretraining.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset-formats/template_free.html b/docs/dataset-formats/template_free.html index 2726b5865..0f9b74bc0 100644 --- a/docs/dataset-formats/template_free.html +++ b/docs/dataset-formats/template_free.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset-formats/tokenized.html b/docs/dataset-formats/tokenized.html index 47884dd61..2cc470746 100644 --- a/docs/dataset-formats/tokenized.html +++ b/docs/dataset-formats/tokenized.html @@ -2,7 +2,7 @@ - + diff --git a/docs/dataset_preprocessing.html b/docs/dataset_preprocessing.html index a8a9697a7..61726f0ff 100644 --- a/docs/dataset_preprocessing.html +++ b/docs/dataset_preprocessing.html @@ -2,7 +2,7 @@ - + diff --git a/docs/debugging.html b/docs/debugging.html index 071f89618..ba6074d54 100644 --- a/docs/debugging.html +++ b/docs/debugging.html @@ -2,7 +2,7 @@ - + diff --git a/docs/faq.html b/docs/faq.html index 530726000..cab6eb8c5 100644 --- a/docs/faq.html +++ b/docs/faq.html @@ -2,7 +2,7 @@ - + diff --git a/docs/fsdp_qlora.html b/docs/fsdp_qlora.html index b023daa99..7c3903000 100644 --- a/docs/fsdp_qlora.html +++ b/docs/fsdp_qlora.html @@ -2,7 +2,7 @@ - + diff --git a/docs/input_output.html b/docs/input_output.html index ade7d18a8..bf3bee994 100644 --- a/docs/input_output.html +++ b/docs/input_output.html @@ -2,7 +2,7 @@ - + @@ -466,7 +466,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin 
import pandas as pd
 pd.DataFrame([{'token': tok.decode(i), 'label': l, 'id':i} for i,l in
               zip(row['input_ids'], row['labels'])])
- +
diff --git a/docs/mac.html b/docs/mac.html index 6c42bbf84..86e8c478a 100644 --- a/docs/mac.html +++ b/docs/mac.html @@ -2,7 +2,7 @@ - + diff --git a/docs/multi-node.html b/docs/multi-node.html index 3ed328d90..3bdf34745 100644 --- a/docs/multi-node.html +++ b/docs/multi-node.html @@ -2,7 +2,7 @@ - + diff --git a/docs/multipack.html b/docs/multipack.html index 0d6024c8a..54e316302 100644 --- a/docs/multipack.html +++ b/docs/multipack.html @@ -2,7 +2,7 @@ - + diff --git a/docs/nccl.html b/docs/nccl.html index 510cec70a..85d6326d3 100644 --- a/docs/nccl.html +++ b/docs/nccl.html @@ -2,7 +2,7 @@ - + @@ -280,7 +280,7 @@ ul.task-list li input[type="checkbox"] {

To force NCCL to use NVLink, simply set this in the environment:

export NCCL_P2P_LEVEL=NVL

If NVLink is not available in your environment there are other options for NCCL_P2P_LEVEL in the table below:

-
token
+
diff --git a/docs/rlhf.html b/docs/rlhf.html index 470e75839..0492db692 100644 --- a/docs/rlhf.html +++ b/docs/rlhf.html @@ -2,7 +2,7 @@ - + diff --git a/examples/colab-notebooks/colab-axolotl-example.html b/examples/colab-notebooks/colab-axolotl-example.html index 8abb857db..82f020f4f 100644 --- a/examples/colab-notebooks/colab-axolotl-example.html +++ b/examples/colab-notebooks/colab-axolotl-example.html @@ -2,7 +2,7 @@ - + diff --git a/index.html b/index.html index 97661a03b..a5f7b8470 100644 --- a/index.html +++ b/index.html @@ -2,7 +2,7 @@ - + @@ -355,7 +355,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin

phorm.ai

Axolotl supports

-
+
@@ -929,7 +929,7 @@ cd skypilot/llm/axolotl
  • Make sure the inference string from #2 looks exactly like the data you fine tuned on from #1, including spaces and new lines. If they aren’t the same, adjust your inference server accordingly.
  • As an additional troubleshooting step, you can look at the token ids between 1 and 2 to make sure they are identical.
    • -

    Having misalignment between your prompts during training and inference can cause models to perform very poorly, so it is worth checking this. See this blog post for a concrete example.

    +

    Having misalignment between your prompts during training and inference can cause models to perform very poorly, so it is worth checking this. See this blog post for a concrete example.

    diff --git a/sitemap.xml b/sitemap.xml index 8c12e77b6..ca7644ee5 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,90 +2,90 @@ https://OpenAccess-AI-Collective.github.io/axolotl/examples/colab-notebooks/colab-axolotl-example.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/FAQS.html - 2024-05-31T17:13:25.351Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/multi-node.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset_preprocessing.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/config.html - 2024-05-31T17:13:25.351Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/inst_tune.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/pretraining.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/tokenized.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/batch_vs_grad.html - 2024-05-31T17:13:25.351Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/debugging.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/faq.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/fsdp_qlora.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/input_output.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/multipack.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/mac.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/index.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/conversation.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/dataset-formats/template_free.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.150Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/rlhf.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/docs/nccl.html - 2024-05-31T17:13:25.355Z + 2024-06-03T16:50:56.154Z https://OpenAccess-AI-Collective.github.io/axolotl/index.html - 2024-05-31T17:13:25.367Z + 2024-06-03T16:50:56.166Z https://OpenAccess-AI-Collective.github.io/axolotl/TODO.html - 2024-05-31T17:13:25.351Z + 2024-06-03T16:50:56.150Z