tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Mixtral multipack (#928)

Browse Source 
* mixtral multipack

* use mixtral model

* sample yml

* calculate cu_seqlens properly

* use updated flash ettention setting

* attn var checks

* force use of flash attention 2 for packing

* lint

* disable future fix for now

* update support table
This commit is contained in:
Wing Lian
2023-12-09 21:26:30 -05:00
committed by GitHub
parent 03c6318ba3
commit 68b227a7d8
8 changed files with 1793 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docker/Dockerfile-runpod
View File

@@ -4,6 +4,7 @@ FROM winglian/axolotl:$BASE_TAG
ENV HF_DATASETS_CACHE="/workspace/data/huggingface-cache/datasets"
ENV HUGGINGFACE_HUB_CACHE="/workspace/data/huggingface-cache/hub"
ENV TRANSFORMERS_CACHE="/workspace/data/huggingface-cache/hub"
ENV HF_HOME="/workspace/data/huggingface-cache/hub"
COPY scripts/runpod-entrypoint.sh /root/runpod-entrypoint.sh