9de5b76336
* feat: move to uv first * fix: update doc to uv first * fix: merge dev/tests into uv pyproject * fix: update docker docs to match current config * fix: migrate examples to readme * fix: add llmcompressor to conflict * feat: rec uv sync with lockfile for dev/ci * fix: update docker docs to clarify how to use uv images * chore: docs * fix: use system python, no venv * fix: set backend cpu * fix: only set for installing pytorch step * fix: remove unsloth kernel and installs * fix: remove U in tests * fix: set backend in deps too * chore: test * chore: comments * fix: attempt to lock torch * fix: workaround torch cuda and not upgraded * fix: forgot to push * fix: missed source * fix: nightly upstream loralinear config * fix: nightly phi3 long rope not work * fix: forgot commit * fix: test phi3 template change * fix: no more requirements * fix: carry over changes from new requirements to pyproject * chore: remove lockfile per discussion * fix: set match-runtime * fix: remove unneeded hf hub buildtime * fix: duplicate cache delete on nightly * fix: torchvision being overridden * fix: migrate to uv images * fix: leftover from merge * fix: simplify base readme * fix: update assertion message to be clearer * chore: docs * fix: change fallback for cicd script * fix: match against main exactly * fix: peft 0.19.1 change * fix: e2e test * fix: ci * fix: e2e test
149 lines
4.0 KiB
Plaintext
149 lines
4.0 KiB
Plaintext
---
|
|
title: "Docker"
|
|
format:
|
|
html:
|
|
toc: true
|
|
toc-depth: 4
|
|
---
|
|
|
|
This section describes the different Docker images that are released by AxolotlAI at
|
|
[Docker Hub](https://hub.docker.com/u/axolotlai).
|
|
|
|
::: {.callout-important}
|
|
For Blackwell GPUs, please use the tags with PyTorch 2.9.1 and CUDA 12.8.
|
|
:::
|
|
|
|
::: {.callout-tip}
|
|
Each image below is available in a **uv variant** that uses [uv](https://docs.astral.sh/uv/) with
|
|
a relocatable venv (`/workspace/axolotl-venv`) instead of Miniconda + pip. Append `-uv` to the image name
|
|
(e.g. `axolotlai/axolotl-base-uv`). Tags follow the same format. We recommend the uv images for new deployments.
|
|
:::
|
|
|
|
## Base
|
|
|
|
The base image is the most minimal image that can install Axolotl. It is based on the `nvidia/cuda` image.
|
|
It includes python, torch, git, git-lfs, awscli, pydantic, and more.
|
|
|
|
#### Image
|
|
|
|
| Variant | Image | Docker Hub |
|
|
|---------|-------|------------|
|
|
| pip | `axolotlai/axolotl-base` | [Link](https://hub.docker.com/r/axolotlai/axolotl-base) |
|
|
| uv | `axolotlai/axolotl-base-uv` | [Link](https://hub.docker.com/r/axolotlai/axolotl-base-uv) |
|
|
|
|
#### Tags format
|
|
|
|
```bash
|
|
main-base-py{python_version}-cu{cuda_version}-{pytorch_version}
|
|
```
|
|
|
|
Tags examples:
|
|
|
|
- `main-base-py3.11-cu128-2.9.1`
|
|
- `main-base-py3.12-cu128-2.10.0`
|
|
- `main-base-py3.12-cu130-2.9.1`
|
|
- `main-base-py3.12-cu130-2.10.0`
|
|
|
|
## Main
|
|
|
|
The main image is the image that is used to run Axolotl. It is based on the `axolotlai/axolotl-base` image and includes the Axolotl codebase, dependencies, and more.
|
|
|
|
#### Image
|
|
|
|
| Variant | Image | Docker Hub |
|
|
|---------|-------|------------|
|
|
| pip | `axolotlai/axolotl` | [Link](https://hub.docker.com/r/axolotlai/axolotl) |
|
|
| uv | `axolotlai/axolotl-uv` | [Link](https://hub.docker.com/r/axolotlai/axolotl-uv) |
|
|
|
|
#### Tags format {#sec-main-tags}
|
|
|
|
```bash
|
|
# on push to main
|
|
main-py{python_version}-cu{cuda_version}-{pytorch_version}
|
|
|
|
# latest main (currently torch 2.9.1, python 3.11, cuda 12.8)
|
|
main-latest
|
|
|
|
# nightly build
|
|
{branch}-{date_in_YYYYMMDD}-py{python_version}-cu{cuda_version}-{pytorch_version}
|
|
|
|
# tagged release
|
|
{version}
|
|
```
|
|
|
|
:::{.callout-tip}
|
|
|
|
There may be some extra tags appended to the image, like `-vllm` which installs those packages.
|
|
|
|
:::
|
|
|
|
Tags examples:
|
|
|
|
- `main-py3.11-cu128-2.9.1`
|
|
- `main-py3.12-cu128-2.10.0`
|
|
- `main-py3.12-cu130-2.9.1`
|
|
- `main-py3.12-cu130-2.10.0`
|
|
- `main-latest`
|
|
- `main-20260315-py3.11-cu128-2.9.1`
|
|
- `0.12.0`
|
|
|
|
## Cloud
|
|
|
|
The cloud image is the image that is used to run Axolotl in the cloud. It is based on the `axolotlai/axolotl` image and sets ENV variables like HuggingFace cache directories for volume mounts, tmux, and more for different cloud providers.
|
|
|
|
:::{.callout-tip}
|
|
|
|
Jupyter lab is run by default. Set `JUPYTER_DISABLE=1` in the environment variables to disable it.
|
|
|
|
:::
|
|
|
|
#### Image
|
|
|
|
| Variant | Image | Docker Hub |
|
|
|---------|-------|------------|
|
|
| pip | `axolotlai/axolotl-cloud` | [Link](https://hub.docker.com/r/axolotlai/axolotl-cloud) |
|
|
| uv | `axolotlai/axolotl-cloud-uv` | [Link](https://hub.docker.com/r/axolotlai/axolotl-cloud-uv) |
|
|
|
|
#### Tags format
|
|
|
|
This uses the same tags as the [`main` image](#sec-main-tags).
|
|
|
|
#### Environment variables
|
|
|
|
- `JUPYTER_DISABLE`: Disable Jupyter lab.
|
|
- `JUPYTER_PASSWORD`: Set a password for the Jupyter lab.
|
|
- `PUBLIC_KEY` / `SSH_KEY`: Add a public key for the SSH service.
|
|
|
|
#### Volume mounts
|
|
|
|
:::{.callout-tip}
|
|
|
|
We recommend mounting volumes to `/workspace/data` for data persistence. `/workspace/axolotl` contains the source code and is ephemeral.
|
|
|
|
:::
|
|
|
|
- `/workspace/data/axolotl-artifacts`: Directory to store Axolotl artifacts.
|
|
- `/workspace/data/huggingface-cache`: Directory to store HuggingFace cache.
|
|
|
|
## Cloud-no-tmux
|
|
|
|
This is the same as the [`cloud` image](#sec-cloud) but without tmux.
|
|
|
|
#### Image
|
|
|
|
```
|
|
axolotlai/axolotl-cloud-term
|
|
```
|
|
|
|
Link: [Docker Hub](https://hub.docker.com/r/axolotlai/axolotl-cloud-term)
|
|
|
|
:::{.callout-note}
|
|
|
|
The naming may be a bit confusing as it has `-term` appended to the end.
|
|
|
|
:::
|
|
|
|
#### Tags format
|
|
|
|
This uses the same tags as the [`cloud` image](#sec-cloud-tags).
|