Migrate QAT API; fix axolotl quantize for QAT-ed models; add NVFP4 (#3107)

2025-09-12 10:55:50 +01:00
parent 0401a15888
commit 58d67bf98d
16 changed files with 554 additions and 339 deletions
--- a/docs/quantize.qmd
+++ b/docs/quantize.qmd
@@ -51,3 +51,11 @@ axolotl quantize qat.yml
 ```

 This ensures that an identical quantization configuration is used to quantize the model as was used to train it.
+
+
+::: {.callout-note}
+
+If you have configured pushing to hub with `hub_model_id`, your model hub name will have the quantization schema appended to it,
+e.g. `axolotl-ai-cloud/qat-nvfp4-llama3B` will become `axolotl-ai-cloud/qat-nvfp4-llama3B-nvfp4w`
+
+:::