2efe1b4c09
* feat(doc): organize docs, add to menu bar, fix broken formatting * feat: add link to custom integrations * feat: update readme for integrations to include citations and repo link * chore: update lm_eval info * chore: use fullname * Update docs/cli.qmd per suggestion Co-authored-by: Dan Saunders <danjsaund@gmail.com> * feat: add sweep doc * feat: add kd doc * fix: remove toc * fix: update deprecation * feat: add more info about chat_template issues * fix: heading level * fix: shell->bash code block * fix: ray link * fix(doc): heading level, header links, formatting * feat: add grpo docs * feat: add style changes * fix: wrong cli arg for lm-eval * fix: remove old run method * feat: load custom integration doc dynamically * fix: remove old cli way * fix: toc * fix: minor formatting --------- Co-authored-by: Dan Saunders <danjsaund@gmail.com>
119 lines
2.9 KiB
Plaintext
119 lines
2.9 KiB
Plaintext
---
|
|
title: "Installation"
|
|
format:
|
|
html:
|
|
toc: true
|
|
toc-depth: 3
|
|
number-sections: true
|
|
execute:
|
|
enabled: false
|
|
---
|
|
|
|
This guide covers all the ways you can install and set up Axolotl for your environment.
|
|
|
|
## Requirements {#sec-requirements}
|
|
|
|
- NVIDIA GPU (Ampere architecture or newer for `bf16` and Flash Attention) or AMD GPU
|
|
- Python ≥3.10
|
|
- PyTorch ≥2.4.1
|
|
|
|
## Installation Methods {#sec-installation-methods}
|
|
|
|
### PyPI Installation (Recommended) {#sec-pypi}
|
|
|
|
```{.bash}
|
|
pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]
|
|
```
|
|
|
|
We use `--no-build-isolation` in order to detect the installed PyTorch version (if
|
|
installed) in order not to clobber it, and so that we set the correct version of
|
|
dependencies that are specific to the PyTorch version or other installed
|
|
co-dependencies.
|
|
|
|
### Edge/Development Build {#sec-edge-build}
|
|
|
|
For the latest features between releases:
|
|
|
|
```{.bash}
|
|
git clone https://github.com/axolotl-ai-cloud/axolotl.git
|
|
cd axolotl
|
|
pip3 install packaging ninja
|
|
pip3 install --no-build-isolation -e '.[flash-attn,deepspeed]'
|
|
```
|
|
|
|
### Docker {#sec-docker}
|
|
|
|
```{.bash}
|
|
docker run --gpus '"all"' --rm -it axolotlai/axolotl:main-latest
|
|
```
|
|
|
|
For development with Docker:
|
|
|
|
```{.bash}
|
|
docker compose up -d
|
|
```
|
|
|
|
::: {.callout-tip}
|
|
### Advanced Docker Configuration
|
|
```{.bash}
|
|
docker run --privileged --gpus '"all"' --shm-size 10g --rm -it \
|
|
--name axolotl --ipc=host \
|
|
--ulimit memlock=-1 --ulimit stack=67108864 \
|
|
--mount type=bind,src="${PWD}",target=/workspace/axolotl \
|
|
-v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
|
|
axolotlai/axolotl:main-latest
|
|
```
|
|
:::
|
|
|
|
## Cloud Environments {#sec-cloud}
|
|
|
|
### Cloud GPU Providers {#sec-cloud-gpu}
|
|
|
|
For providers supporting Docker:
|
|
|
|
- Use `axolotlai/axolotl-cloud:main-latest`
|
|
- Available on:
|
|
- [Latitude.sh](https://latitude.sh/blueprint/989e0e79-3bf6-41ea-a46b-1f246e309d5c)
|
|
- [JarvisLabs.ai](https://jarvislabs.ai/templates/axolotl)
|
|
- [RunPod](https://runpod.io/gsc?template=v2ickqhz9s&ref=6i7fkpdz)
|
|
|
|
### Google Colab {#sec-colab}
|
|
|
|
Use our [example notebook](../examples/colab-notebooks/colab-axolotl-example.ipynb).
|
|
|
|
## Platform-Specific Instructions {#sec-platform-specific}
|
|
|
|
### macOS {#sec-macos}
|
|
|
|
```{.bash}
|
|
pip3 install --no-build-isolation -e '.'
|
|
```
|
|
|
|
See @sec-troubleshooting for Mac-specific issues.
|
|
|
|
### Windows {#sec-windows}
|
|
|
|
::: {.callout-important}
|
|
We recommend using WSL2 (Windows Subsystem for Linux) or Docker.
|
|
:::
|
|
|
|
## Environment Managers {#sec-env-managers}
|
|
|
|
### Conda/Pip venv {#sec-conda}
|
|
|
|
1. Install Python ≥3.10
|
|
2. Install PyTorch: https://pytorch.org/get-started/locally/
|
|
3. Install Axolotl:
|
|
```{.bash}
|
|
pip3 install packaging
|
|
pip3 install --no-build-isolation -e '.[flash-attn,deepspeed]'
|
|
```
|
|
4. (Optional) Login to Hugging Face:
|
|
```{.bash}
|
|
huggingface-cli login
|
|
```
|
|
|
|
## Troubleshooting {#sec-troubleshooting}
|
|
|
|
If you encounter installation issues, see our [FAQ](faq.qmd) and [Debugging Guide](debugging.qmd).
|