!pip install torch=="2.1.2"
-!pip install -e git+https://github.com/OpenAccess-AI-Collective/axolotl#egg=axolotl
+!pip install -e git+https://github.com/axolotl-ai-cloud/axolotl#egg=axolotl
!pip install flash-attn=="2.5.0"
!pip install deepspeed=="0.13.1"!pip install mlflow=="2.13.0"Setup
On the host that is running axolotl (ex: if you are using a remote host), clone the axolotl repo and change your current directory to the root:
-git clone https://github.com/OpenAccess-AI-Collective/axolotl
+git clone https://github.com/axolotl-ai-cloud/axolotl
cd axolotl
[!Tip] If you already have axolotl cloned on your host, make sure you have the latest changes and change into the root of the project.
diff --git a/docs/faq.html b/docs/faq.html
index aebca5a63..15769f94c 100644
--- a/docs/faq.html
+++ b/docs/faq.html
@@ -85,7 +85,7 @@ ul.task-list li input[type="checkbox"] {
- Set
adapter: qlorain your axolotl config file.
- - Enable FSDP in your axolotl config, as described here. +
- Enable FSDP in your axolotl config, as described here.
- Use one of the supported model types:
llama,mistralormixtral.
References
-
-
- PR #1378 enabling QLoRA in FSDP in Axolotl. +
- PR #1378 enabling QLoRA in FSDP in Axolotl.
- Blog Post from the Answer.AI team describing the work that enabled QLoRA in FSDP.
- Related HuggingFace PRs Enabling FDSP + QLoRA:
-
diff --git a/docs/input_output.html b/docs/input_output.html
index 5adf6176e..706861ab5 100644
--- a/docs/input_output.html
+++ b/docs/input_output.html
@@ -119,7 +119,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
@@ -345,9 +345,9 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
Masking Inputs
-One of the most popular features of axolotl is setting the following configuration value:
+One of the most popular features of axolotl is setting the following configuration value:
-train_on_inputs: falseIf you declare a dataset formats such as
+alpacaorchatml, axolotl knows what is an input (i.e. human) vs. an output (i.e. the assistant) and masks the input labels so that your model can focus on predicting the outputs only.If you declare a dataset formats such as
alpacaorchatml, axolotl knows what is an input (i.e. human) vs. an output (i.e. the assistant) and masks the input labels so that your model can focus on predicting the outputs only.diff --git a/docs/mac.html b/docs/mac.html index d4d37f225..2169d8674 100644 --- a/docs/mac.html +++ b/docs/mac.html @@ -85,7 +85,7 @@ ul.task-list li input[type="checkbox"] { diff --git a/docs/multi-node.html b/docs/multi-node.html index d5db7dd46..e979e3c1e 100644 --- a/docs/multi-node.html +++ b/docs/multi-node.html @@ -119,7 +119,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin diff --git a/docs/multipack.html b/docs/multipack.html index b82027295..ae333c995 100644 --- a/docs/multipack.html +++ b/docs/multipack.html @@ -85,7 +85,7 @@ ul.task-list li input[type="checkbox"] { diff --git a/docs/nccl.html b/docs/nccl.html index 043cc3840..d4dacb1ff 100644 --- a/docs/nccl.html +++ b/docs/nccl.html @@ -85,7 +85,7 @@ ul.task-list li input[type="checkbox"] { diff --git a/docs/rlhf.html b/docs/rlhf.html index 9894151af..b1edf43b6 100644 --- a/docs/rlhf.html +++ b/docs/rlhf.html @@ -119,7 +119,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin diff --git a/examples/colab-notebooks/colab-axolotl-example.html b/examples/colab-notebooks/colab-axolotl-example.html index a4e56483e..3d72138ee 100644 --- a/examples/colab-notebooks/colab-axolotl-example.html +++ b/examples/colab-notebooks/colab-axolotl-example.html @@ -118,7 +118,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin @@ -318,7 +318,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin Install Axolotl and dependencies
diff --git a/index.html b/index.html index c146636f4..0d1e49533 100644 --- a/index.html +++ b/index.html @@ -118,7 +118,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin @@ -537,7 +537,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlinQuickstart ⚡
Get started with Axolotl in just a few steps! This quickstart guide will walk you through setting up and running a basic fine-tuning task.
Requirements: Python >=3.10 and Pytorch >=2.1.1.
-git clone https://github.com/OpenAccess-AI-Collective/axolotl ++accelerate launch -m axolotl.cli.train https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/examples/openllama-3b/lora.ymlgit clone https://github.com/axolotl-ai-cloud/axolotl cd axolotl pip3 install packaging ninja @@ -560,7 +560,7 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin # remote yaml files - the yaml config can be hosted on a public URL # Note: the yaml config must directly link to the **raw** yaml -accelerate launch -m axolotl.cli.train https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/examples/openllama-3b/lora.yml
Badge ❤🏷️
Building something cool with Axolotl? Consider adding a badge to your model card.
-[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
Community Showcase
@@ -956,7 +956,7 @@ cd skypilot/llm/axolotlContributing 🤝
Please read the contributing guide
-Bugs? Please check the open issues else create a new Issue.
+Bugs? Please check the open issues else create a new Issue.
PRs are greatly welcome!
Please run the quickstart instructions followed by the below to setup env:
pip3 install -r requirements-dev.txt -r requirements-tests.txt
@@ -968,7 +968,7 @@ cd skypilot/llm/axolotlThanks to all of our contributors to date. Help drive open source AI progress forward by contributing to Axolotl.
-