From 00364ad07afb83d9c9aef9ebbd7f317083e6d434 Mon Sep 17 00:00:00 2001 From: Quarto GHA Workflow Runner Date: Mon, 7 Apr 2025 18:50:15 +0000 Subject: [PATCH] Built site for gh-pages --- .nojekyll | 2 +- docs/api/utils.collators.batching.html | 28 -- search.json | 9 +- sitemap.xml | 340 ++++++++++++------------- 4 files changed, 172 insertions(+), 207 deletions(-) diff --git a/.nojekyll b/.nojekyll index ed2b46d64..7daede940 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -be620dc5 \ No newline at end of file +293e0ad4 \ No newline at end of file diff --git a/docs/api/utils.collators.batching.html b/docs/api/utils.collators.batching.html index 9b9e0ee28..ddb7bb0f6 100644 --- a/docs/api/utils.collators.batching.html +++ b/docs/api/utils.collators.batching.html @@ -453,10 +453,6 @@ pre > code.sourceCode > span > a:first-child::before { text-decoration: underlin
  • PretrainingBatchSamplerDataCollatorForSeq2Seq
  • V2BatchSamplerDataCollatorForSeq2Seq
    • -
  • Functions -
    • @@ -696,30 +692,6 @@ includes logic for handling sequence parallelism collation.

    sequence_parallel_degree=1, )

    Collator for multipack specific to the using the BatchSampler

    - - -
    -

    Functions

    - - - - - - - - - - - - - -
    NameDescription
    adjust_position_ids_for_sliceAdjust position IDs for a sliced sequence to maintain proper relative positions.
    -
    -

    adjust_position_ids_for_slice

    -
    utils.collators.batching.adjust_position_ids_for_slice(position_ids, start_idx)
    -

    Adjust position IDs for a sliced sequence to maintain proper relative positions. -This handles the case where position IDs might not be contiguous due to sample -packing.

    diff --git a/search.json b/search.json index be1b9ec67..1640867b8 100644 --- a/search.json +++ b/search.json @@ -2422,7 +2422,7 @@ "href": "docs/api/utils.collators.batching.html", "title": "utils.collators.batching", "section": "", - "text": "utils.collators.batching\nData collators for axolotl to pad labels and position_ids for packed sequences. Also\nincludes logic for handling sequence parallelism collation.\n\n\n\n\n\nName\nDescription\n\n\n\n\nBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nDataCollatorForSeq2Seq\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\nPretrainingBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nV2BatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\n\n\n\nutils.collators.batching.BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntokenizer\n[PreTrainedTokenizer] or [PreTrainedTokenizerFast]\nThe tokenizer used for encoding the data.\nrequired\n\n\nmodel\n[PreTrainedModel]\nThe model that is being trained. If set and has the prepare_decoder_input_ids_from_labels, use it to prepare the decoder_input_ids This is useful when using label_smoothing to avoid calculating loss twice.\nNone\n\n\npadding\nbool, str or [~utils.PaddingStrategy], optional, defaults to True\nSelect a strategy to pad the returned sequences (according to the model’s padding side and padding index) among: - True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single sequence is provided). - 'max_length': Pad to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided. - False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).\nTrue\n\n\nmax_length\nint, optional\nMaximum length of the returned list and optionally padding length (see above).\nNone\n\n\npad_to_multiple_of\nint, optional\nIf set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).\nNone\n\n\nlabel_pad_token_id\nint, optional, defaults to -100\nThe id to use when padding the labels (-100 will be automatically ignored by PyTorch loss functions).\n-100\n\n\nreturn_tensors\nstr\nThe type of Tensor to return. Allowable values are “np”, “pt” and “tf”.\n'pt'\n\n\nsequence_parallel_degree\nint\nThe degree of sequence parallelism. Default to 1 for no sequence parallelism.\n1\n\n\n\n\n\n\n\n\n\nName\nDescription\n\n\n\n\napply_sequence_parallelism\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq.apply_sequence_parallelism(\n batch,\n)\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\nbatch\ndict[str, torch.Tensor]\nBatch dictionary from parent collator.\nrequired\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\n\ntorch.Tensor\nSliced batch dictionary.\n\n\n\n\n\n\n\n\n\nutils.collators.batching.PretrainingBatchSamplerDataCollatorForSeq2Seq(\n self,\n *args,\n multipack_attn=True,\n **kwargs,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.V2BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\n\n\n\n\nName\nDescription\n\n\n\n\nadjust_position_ids_for_slice\nAdjust position IDs for a sliced sequence to maintain proper relative positions.\n\n\n\n\n\nutils.collators.batching.adjust_position_ids_for_slice(position_ids, start_idx)\nAdjust position IDs for a sliced sequence to maintain proper relative positions.\nThis handles the case where position IDs might not be contiguous due to sample\npacking." + "text": "utils.collators.batching\nData collators for axolotl to pad labels and position_ids for packed sequences. Also\nincludes logic for handling sequence parallelism collation.\n\n\n\n\n\nName\nDescription\n\n\n\n\nBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nDataCollatorForSeq2Seq\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\nPretrainingBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nV2BatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\n\n\n\nutils.collators.batching.BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntokenizer\n[PreTrainedTokenizer] or [PreTrainedTokenizerFast]\nThe tokenizer used for encoding the data.\nrequired\n\n\nmodel\n[PreTrainedModel]\nThe model that is being trained. If set and has the prepare_decoder_input_ids_from_labels, use it to prepare the decoder_input_ids This is useful when using label_smoothing to avoid calculating loss twice.\nNone\n\n\npadding\nbool, str or [~utils.PaddingStrategy], optional, defaults to True\nSelect a strategy to pad the returned sequences (according to the model’s padding side and padding index) among: - True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single sequence is provided). - 'max_length': Pad to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided. - False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).\nTrue\n\n\nmax_length\nint, optional\nMaximum length of the returned list and optionally padding length (see above).\nNone\n\n\npad_to_multiple_of\nint, optional\nIf set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).\nNone\n\n\nlabel_pad_token_id\nint, optional, defaults to -100\nThe id to use when padding the labels (-100 will be automatically ignored by PyTorch loss functions).\n-100\n\n\nreturn_tensors\nstr\nThe type of Tensor to return. Allowable values are “np”, “pt” and “tf”.\n'pt'\n\n\nsequence_parallel_degree\nint\nThe degree of sequence parallelism. Default to 1 for no sequence parallelism.\n1\n\n\n\n\n\n\n\n\n\nName\nDescription\n\n\n\n\napply_sequence_parallelism\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq.apply_sequence_parallelism(\n batch,\n)\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\nbatch\ndict[str, torch.Tensor]\nBatch dictionary from parent collator.\nrequired\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\n\ntorch.Tensor\nSliced batch dictionary.\n\n\n\n\n\n\n\n\n\nutils.collators.batching.PretrainingBatchSamplerDataCollatorForSeq2Seq(\n self,\n *args,\n multipack_attn=True,\n **kwargs,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.V2BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler" }, { "objectID": "docs/api/utils.collators.batching.html#classes", @@ -2431,13 +2431,6 @@ "section": "", "text": "Name\nDescription\n\n\n\n\nBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nDataCollatorForSeq2Seq\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\nPretrainingBatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\nV2BatchSamplerDataCollatorForSeq2Seq\nCollator for multipack specific to the using the BatchSampler\n\n\n\n\n\nutils.collators.batching.BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nData collator that will dynamically pad the inputs received, as well as the labels and position_ids\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\ntokenizer\n[PreTrainedTokenizer] or [PreTrainedTokenizerFast]\nThe tokenizer used for encoding the data.\nrequired\n\n\nmodel\n[PreTrainedModel]\nThe model that is being trained. If set and has the prepare_decoder_input_ids_from_labels, use it to prepare the decoder_input_ids This is useful when using label_smoothing to avoid calculating loss twice.\nNone\n\n\npadding\nbool, str or [~utils.PaddingStrategy], optional, defaults to True\nSelect a strategy to pad the returned sequences (according to the model’s padding side and padding index) among: - True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single sequence is provided). - 'max_length': Pad to a maximum length specified with the argument max_length or to the maximum acceptable input length for the model if that argument is not provided. - False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).\nTrue\n\n\nmax_length\nint, optional\nMaximum length of the returned list and optionally padding length (see above).\nNone\n\n\npad_to_multiple_of\nint, optional\nIf set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).\nNone\n\n\nlabel_pad_token_id\nint, optional, defaults to -100\nThe id to use when padding the labels (-100 will be automatically ignored by PyTorch loss functions).\n-100\n\n\nreturn_tensors\nstr\nThe type of Tensor to return. Allowable values are “np”, “pt” and “tf”.\n'pt'\n\n\nsequence_parallel_degree\nint\nThe degree of sequence parallelism. Default to 1 for no sequence parallelism.\n1\n\n\n\n\n\n\n\n\n\nName\nDescription\n\n\n\n\napply_sequence_parallelism\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\nutils.collators.batching.DataCollatorForSeq2Seq.apply_sequence_parallelism(\n batch,\n)\nApply sequence parallelism slicing to a batch.\n\n\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\nDefault\n\n\n\n\nbatch\ndict[str, torch.Tensor]\nBatch dictionary from parent collator.\nrequired\n\n\n\n\n\n\n\n\n\nName\nType\nDescription\n\n\n\n\n\ntorch.Tensor\nSliced batch dictionary.\n\n\n\n\n\n\n\n\n\nutils.collators.batching.PretrainingBatchSamplerDataCollatorForSeq2Seq(\n self,\n *args,\n multipack_attn=True,\n **kwargs,\n)\nCollator for multipack specific to the using the BatchSampler\n\n\n\nutils.collators.batching.V2BatchSamplerDataCollatorForSeq2Seq(\n self,\n tokenizer,\n model=None,\n padding=True,\n max_length=None,\n pad_to_multiple_of=None,\n label_pad_token_id=-100,\n position_pad_token_id=0,\n return_tensors='pt',\n sequence_parallel_degree=1,\n)\nCollator for multipack specific to the using the BatchSampler" }, - { - "objectID": "docs/api/utils.collators.batching.html#functions", - "href": "docs/api/utils.collators.batching.html#functions", - "title": "utils.collators.batching", - "section": "", - "text": "Name\nDescription\n\n\n\n\nadjust_position_ids_for_slice\nAdjust position IDs for a sliced sequence to maintain proper relative positions.\n\n\n\n\n\nutils.collators.batching.adjust_position_ids_for_slice(position_ids, start_idx)\nAdjust position IDs for a sliced sequence to maintain proper relative positions.\nThis handles the case where position IDs might not be contiguous due to sample\npacking." - }, { "objectID": "docs/api/prompt_strategies.completion.html", "href": "docs/api/prompt_strategies.completion.html", diff --git a/sitemap.xml b/sitemap.xml index c7811e7c5..f863e6a88 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,682 +2,682 @@ https://axolotl-ai-cloud.github.io/axolotl/examples/colab-notebooks/colab-axolotl-example.html - 2025-04-07T16:41:27.525Z + 2025-04-07T18:48:09.453Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/stepwise_supervised.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/template_free.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/tokenized.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/nccl.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/amd_hpc.html - 2025-04-07T16:41:27.520Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/config.html - 2025-04-07T16:41:27.520Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/multi-gpu.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/installation.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/torchao.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/reward_modelling.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/input_output.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/multimodal.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.callbacks.mlflow_.html - 2025-04-07T16:42:04.119Z + 2025-04-07T18:48:41.444Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.trainer_fsdp_optim.html - 2025-04-07T16:42:03.710Z + 2025-04-07T18:48:41.034Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.data.batch_dataset_fetcher.html - 2025-04-07T16:42:03.727Z + 2025-04-07T18:48:41.050Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.stepwise_supervised.html - 2025-04-07T16:42:03.418Z + 2025-04-07T18:48:40.744Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.mistral_attn_hijack_flash.html - 2025-04-07T16:42:03.659Z + 2025-04-07T18:48:40.983Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.user_defined.html - 2025-04-07T16:42:03.464Z + 2025-04-07T18:48:40.789Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.liger.args.html - 2025-04-07T16:42:04.034Z + 2025-04-07T18:48:41.353Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.training.html - 2025-04-07T16:42:03.897Z + 2025-04-07T18:48:41.218Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/datasets.html - 2025-04-07T16:42:02.857Z + 2025-04-07T18:48:40.249Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/kernels.geglu.html - 2025-04-07T16:42:03.600Z + 2025-04-07T18:48:40.922Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.llama_attn_hijack_flash.html - 2025-04-07T16:42:03.643Z + 2025-04-07T18:48:40.968Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.sweeps.html - 2025-04-07T16:42:03.251Z + 2025-04-07T18:48:40.579Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.freeze.html - 2025-04-07T16:42:03.799Z + 2025-04-07T18:48:41.121Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.multipack.html - 2025-04-07T16:42:03.660Z + 2025-04-07T18:48:40.985Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.main.html - 2025-04-07T16:42:03.106Z + 2025-04-07T18:48:40.477Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.trainers.trl.html - 2025-04-07T16:42:03.328Z + 2025-04-07T18:48:40.655Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.passthrough.html - 2025-04-07T16:42:03.466Z + 2025-04-07T18:48:40.791Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.chat.format.llama3x.html - 2025-04-07T16:42:03.061Z + 2025-04-07T18:48:40.432Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.datasets.transforms.chat_builder.html - 2025-04-07T16:42:03.075Z + 2025-04-07T18:48:40.446Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.kto.user_defined.html - 2025-04-07T16:42:03.483Z + 2025-04-07T18:48:40.808Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.collators.mamba.html - 2025-04-07T16:42:04.091Z + 2025-04-07T18:48:41.414Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.base.html - 2025-04-07T16:42:04.019Z + 2025-04-07T18:48:41.339Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.bench.html - 2025-04-07T16:42:03.791Z + 2025-04-07T18:48:41.114Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/kernels.swiglu.html - 2025-04-07T16:42:03.609Z + 2025-04-07T18:48:40.932Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.chat.format.shared.html - 2025-04-07T16:42:03.062Z + 2025-04-07T18:48:40.434Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.cut_cross_entropy.args.html - 2025-04-07T16:42:04.022Z + 2025-04-07T18:48:41.342Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.datasets.chat.html - 2025-04-07T16:42:03.067Z + 2025-04-07T18:48:40.439Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.callbacks.lisa.html - 2025-04-07T16:42:04.116Z + 2025-04-07T18:48:41.441Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.grokfast.optimizer.html - 2025-04-07T16:42:04.023Z + 2025-04-07T18:48:41.343Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.alpaca_chat.html - 2025-04-07T16:42:03.367Z + 2025-04-07T18:48:40.694Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.alpaca_instruct.html - 2025-04-07T16:42:03.369Z + 2025-04-07T18:48:40.695Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.kto.chatml.html - 2025-04-07T16:42:03.481Z + 2025-04-07T18:48:40.806Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.integrations.html - 2025-04-07T16:42:03.943Z + 2025-04-07T18:48:41.264Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.trl.html - 2025-04-07T16:42:03.926Z + 2025-04-07T18:48:41.247Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_tokenizers.html - 2025-04-07T16:42:02.913Z + 2025-04-07T18:48:40.304Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.data.sft.html - 2025-04-07T16:42:03.874Z + 2025-04-07T18:48:41.196Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schedulers.html - 2025-04-07T16:42:03.840Z + 2025-04-07T18:48:41.162Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.chat_templates.html - 2025-04-07T16:42:03.774Z + 2025-04-07T18:48:41.096Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.models.html - 2025-04-07T16:42:03.757Z + 2025-04-07T18:48:41.080Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.chatml.html - 2025-04-07T16:42:03.461Z + 2025-04-07T18:48:40.786Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.distributed.html - 2025-04-07T16:42:03.860Z + 2025-04-07T18:48:41.182Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.utils.html - 2025-04-07T16:42:03.699Z + 2025-04-07T18:48:41.023Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.utils.html - 2025-04-07T16:42:03.955Z + 2025-04-07T18:48:41.276Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.llama_expand_mask.html - 2025-04-07T16:42:03.669Z + 2025-04-07T18:48:40.993Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/common.datasets.html - 2025-04-07T16:42:04.060Z + 2025-04-07T18:48:41.380Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/logging_config.html - 2025-04-07T16:42:02.920Z + 2025-04-07T18:48:40.308Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/kernels.quantize.html - 2025-04-07T16:42:03.617Z + 2025-04-07T18:48:40.941Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.llama_patch_multipack.html - 2025-04-07T16:42:03.702Z + 2025-04-07T18:48:41.025Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.model.html - 2025-04-07T16:42:03.892Z + 2025-04-07T18:48:41.213Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.stablelm_attn_hijack_flash.html - 2025-04-07T16:42:03.707Z + 2025-04-07T18:48:41.031Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.mixtral.html - 2025-04-07T16:42:03.728Z + 2025-04-07T18:48:41.052Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.tokenization.html - 2025-04-07T16:42:03.764Z + 2025-04-07T18:48:41.087Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.kd.trainer.html - 2025-04-07T16:42:04.030Z + 2025-04-07T18:48:41.350Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.datasets.html - 2025-04-07T16:42:03.914Z + 2025-04-07T18:48:41.236Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.collators.core.html - 2025-04-07T16:42:04.062Z + 2025-04-07T18:48:41.383Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.btlm_attn_hijack_flash.html - 2025-04-07T16:42:03.700Z + 2025-04-07T18:48:41.024Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.optimizers.adopt.html - 2025-04-07T16:42:03.871Z + 2025-04-07T18:48:41.193Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.input_output.html - 2025-04-07T16:42:03.413Z + 2025-04-07T18:48:40.739Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/index.html - 2025-04-07T16:42:02.778Z + 2025-04-07T18:48:40.172Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.cloud.modal_.html - 2025-04-07T16:42:03.297Z + 2025-04-07T18:48:40.624Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.llama3.html - 2025-04-07T16:42:03.451Z + 2025-04-07T18:48:40.776Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.train.html - 2025-04-07T16:42:03.114Z + 2025-04-07T18:48:40.485Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.trainer_builder.html - 2025-04-07T16:42:02.943Z + 2025-04-07T18:48:40.323Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.callbacks.perplexity.html - 2025-04-07T16:42:04.110Z + 2025-04-07T18:48:41.436Z https://axolotl-ai-cloud.github.io/axolotl/docs/getting-started.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset_loading.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/batch_vs_grad.html - 2025-04-07T16:41:27.520Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/faq.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/debugging.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/lr_groups.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/TODO.html - 2025-04-07T16:41:27.519Z + 2025-04-07T18:48:09.447Z https://axolotl-ai-cloud.github.io/axolotl/src/axolotl/integrations/LICENSE.html - 2025-04-07T16:41:27.540Z + 2025-04-07T18:48:09.467Z https://axolotl-ai-cloud.github.io/axolotl/index.html - 2025-04-07T16:41:27.537Z + 2025-04-07T18:48:09.464Z https://axolotl-ai-cloud.github.io/axolotl/src/axolotl/integrations/cut_cross_entropy/ACKNOWLEDGEMENTS.html - 2025-04-07T16:41:27.540Z + 2025-04-07T18:48:09.468Z https://axolotl-ai-cloud.github.io/axolotl/FAQS.html - 2025-04-07T16:41:27.519Z + 2025-04-07T18:48:09.447Z https://axolotl-ai-cloud.github.io/axolotl/docs/multi-node.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/sequence_parallelism.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/multipack.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/inference.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/lora_optims.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.lora_embeddings.html - 2025-04-07T16:42:03.782Z + 2025-04-07T18:48:41.104Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/kernels.utils.html - 2025-04-07T16:42:03.618Z + 2025-04-07T18:48:40.942Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.chat_template.html - 2025-04-07T16:42:03.354Z + 2025-04-07T18:48:40.680Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/convert.html - 2025-04-07T16:42:02.870Z + 2025-04-07T18:48:40.263Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/common.const.html - 2025-04-07T16:42:04.043Z + 2025-04-07T18:48:41.363Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.cloud.base.html - 2025-04-07T16:42:03.291Z + 2025-04-07T18:48:40.618Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.relora.html - 2025-04-07T16:42:03.667Z + 2025-04-07T18:48:40.991Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.lora.html - 2025-04-07T16:42:03.779Z + 2025-04-07T18:48:41.101Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.merge_lora.html - 2025-04-07T16:42:03.225Z + 2025-04-07T18:48:40.554Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.bradley_terry.llama3.html - 2025-04-07T16:42:03.507Z + 2025-04-07T18:48:40.831Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.merge_sharded_fsdp_weights.html - 2025-04-07T16:42:03.237Z + 2025-04-07T18:48:40.566Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.spectrum.args.html - 2025-04-07T16:42:04.040Z + 2025-04-07T18:48:41.360Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/models.mamba.modeling_mamba.html - 2025-04-07T16:42:04.061Z + 2025-04-07T18:48:41.381Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/common.architectures.html - 2025-04-07T16:42:04.042Z + 2025-04-07T18:48:41.361Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.trainer.html - 2025-04-07T16:42:03.815Z + 2025-04-07T18:48:41.138Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.callbacks.comet_.html - 2025-04-07T16:42:04.123Z + 2025-04-07T18:48:41.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.vllm_serve.html - 2025-04-07T16:42:03.287Z + 2025-04-07T18:48:40.614Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.multimodal.html - 2025-04-07T16:42:03.931Z + 2025-04-07T18:48:41.252Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.gradient_checkpointing.unsloth.html - 2025-04-07T16:42:03.877Z + 2025-04-07T18:48:41.199Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.trainers.base.html - 2025-04-07T16:42:03.311Z + 2025-04-07T18:48:40.638Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.unsloth_.html - 2025-04-07T16:42:03.718Z + 2025-04-07T18:48:41.042Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.samplers.multipack.html - 2025-04-07T16:42:04.104Z + 2025-04-07T18:48:41.429Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.callbacks.profiler.html - 2025-04-07T16:42:04.114Z + 2025-04-07T18:48:41.439Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/integrations.lm_eval.args.html - 2025-04-07T16:42:04.037Z + 2025-04-07T18:48:41.357Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.data.pretraining.html - 2025-04-07T16:42:03.872Z + 2025-04-07T18:48:41.194Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/evaluate.html - 2025-04-07T16:42:02.849Z + 2025-04-07T18:48:40.242Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.dict.html - 2025-04-07T16:42:03.863Z + 2025-04-07T18:48:41.185Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.utils.html - 2025-04-07T16:42:03.283Z + 2025-04-07T18:48:40.610Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.pygmalion.html - 2025-04-07T16:42:03.435Z + 2025-04-07T18:48:40.761Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.training_args.html - 2025-04-07T16:42:03.035Z + 2025-04-07T18:48:40.407Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.inference.html - 2025-04-07T16:42:03.217Z + 2025-04-07T18:48:40.546Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/kernels.lora.html - 2025-04-07T16:42:03.589Z + 2025-04-07T18:48:40.912Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.evaluate.html - 2025-04-07T16:42:03.122Z + 2025-04-07T18:48:40.493Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.collators.batching.html - 2025-04-07T16:42:04.088Z + 2025-04-07T18:48:41.410Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.completion.html - 2025-04-07T16:42:03.407Z + 2025-04-07T18:48:40.733Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.zephyr.html - 2025-04-07T16:42:03.463Z + 2025-04-07T18:48:40.788Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.metharme.html - 2025-04-07T16:42:03.425Z + 2025-04-07T18:48:40.750Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.orpo.chat_template.html - 2025-04-07T16:42:03.504Z + 2025-04-07T18:48:40.828Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.alpaca_w_system.html - 2025-04-07T16:42:03.381Z + 2025-04-07T18:48:40.707Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.model_shard_quant.html - 2025-04-07T16:42:03.787Z + 2025-04-07T18:48:41.110Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.config.html - 2025-04-07T16:42:03.192Z + 2025-04-07T18:48:40.532Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.enums.html - 2025-04-07T16:42:03.949Z + 2025-04-07T18:48:41.271Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.preprocess.html - 2025-04-07T16:42:03.245Z + 2025-04-07T18:48:40.574Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.chat.messages.html - 2025-04-07T16:42:03.058Z + 2025-04-07T18:48:40.429Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.dpo.chat_template.html - 2025-04-07T16:42:03.441Z + 2025-04-07T18:48:40.766Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.peft.html - 2025-04-07T16:42:03.922Z + 2025-04-07T18:48:41.244Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/train.html - 2025-04-07T16:42:02.839Z + 2025-04-07T18:48:40.232Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.messages.chat.html - 2025-04-07T16:42:03.440Z + 2025-04-07T18:48:40.765Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.orcamini.html - 2025-04-07T16:42:03.429Z + 2025-04-07T18:48:40.754Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.collators.mm_chat.html - 2025-04-07T16:42:04.096Z + 2025-04-07T18:48:41.420Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.kto.llama3.html - 2025-04-07T16:42:03.473Z + 2025-04-07T18:48:40.799Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.attention.mllama.html - 2025-04-07T16:42:03.725Z + 2025-04-07T18:48:41.049Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.checks.html - 2025-04-07T16:42:03.161Z + 2025-04-07T18:48:40.515Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.transformers_fa_utils.html - 2025-04-07T16:42:03.717Z + 2025-04-07T18:48:41.041Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.llama_attn_hijack_xformers.html - 2025-04-07T16:42:03.645Z + 2025-04-07T18:48:40.969Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.trainers.dpo.trainer.html - 2025-04-07T16:42:03.334Z + 2025-04-07T18:48:40.661Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.user_defined.html - 2025-04-07T16:42:03.389Z + 2025-04-07T18:48:40.715Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/cli.args.html - 2025-04-07T16:42:03.150Z + 2025-04-07T18:48:40.509Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.llama2_chat.html - 2025-04-07T16:42:03.401Z + 2025-04-07T18:48:40.728Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/utils.schemas.config.html - 2025-04-07T16:42:03.885Z + 2025-04-07T18:48:41.207Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.trainers.grpo.trainer.html - 2025-04-07T16:42:03.338Z + 2025-04-07T18:48:40.664Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/core.chat.format.chatml.html - 2025-04-07T16:42:03.059Z + 2025-04-07T18:48:40.431Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/monkeypatch.lora_kernels.html - 2025-04-07T16:42:03.691Z + 2025-04-07T18:48:41.015Z https://axolotl-ai-cloud.github.io/axolotl/docs/api/prompt_strategies.base.html - 2025-04-07T16:42:03.339Z + 2025-04-07T18:48:40.666Z https://axolotl-ai-cloud.github.io/axolotl/docs/rlhf.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/cli.html - 2025-04-07T16:41:27.520Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/unsloth.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/fsdp_qlora.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset_preprocessing.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/custom_integrations.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/mac.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/docker.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/ray-integration.html - 2025-04-07T16:41:27.524Z + 2025-04-07T18:48:09.452Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/index.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/conversation.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.448Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/pretraining.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.449Z https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/inst_tune.html - 2025-04-07T16:41:27.521Z + 2025-04-07T18:48:09.448Z