It is typically recommended to save your dataset as .jsonl due to its flexibility and simplicity.
Axolotl supports loading from a Hugging Face hub repo or from local files.
-For pre-training only, Axolotl would split texts if it exceeds the context length into multiple smaller prompts.
-As an example, to train using a Hugging Face dataset hf_org/name, you can pass the following config:
datasets:
- path: hf_org/name
type: completionFrom local files (either example works):
+From local files:
datasets:
- path: A.jsonl
type: completion
- - path: json
- data_files: ["A.jsonl", "B.jsonl", "C.jsonl"]
- type: completionFor completion only, Axolotl would split texts if it exceeds the context length into multiple smaller prompts. If you are interested in having this for pretraining_dataset too, please let us know or help make a PR!
Usually, to load a JSON file, you would do something like this:
+To load a JSON file, you would do something like this:
from datasets import load_dataset
dataset = load_dataset("json", data_files="data.json")Which translates to the following config:
datasets:
- - path: json
- data_files: /path/to/your/file.jsonlHowever, to make things easier, we have added a few shortcuts for loading local dataset files.
-You can just point the path to the file or directory along with the ds_type to load the dataset. The below example shows for a JSON file:
datasets:
- - path: /path/to/your/file.jsonl
- ds_type: jsonIn the example above, it can be seen that we can just point the path to the file or directory along with the ds_type to load the dataset.
This works for CSV, JSON, Parquet, and Arrow files.
We will attempt to load in the following order:
- datasets saved with datasets.save_to_disk
- loading entire directory of files (such as with parquet/arrow files)
datasets:
- - path: /path/to/your/directorydatasets:
+ - path: /path/to/your/directoryProvide data_files with a list of files to load.
datasets:
- # single file
- - path: /path/to/your/directory
- ds_type: csv
- data_files: file1.csv
-
- # multiple files
- - path: /path/to/your/directory
- ds_type: json
- data_files:
- - file1.jsonl
- - file2.jsonl
-
- # multiple files for parquet
- - path: /path/to/your/directory
- ds_type: parquet
- data_files:
- - file1.parquet
- - file2.parquetdatasets:
+ # single file
+ - path: /path/to/your/directory
+ ds_type: csv
+ data_files: file1.csv
+
+ # multiple files
+ - path: /path/to/your/directory
+ ds_type: json
+ data_files:
+ - file1.jsonl
+ - file2.jsonl
+
+ # multiple files for parquet
+ - path: /path/to/your/directory
+ ds_type: parquet
+ data_files:
+ - file1.parquet
+ - file2.parquetThis would mean that the dataset is a single file or file(s) uploaded to the Hub.
-datasets:
- - path: org/dataset-name
- data_files:
- - file1.jsonl
- - file2.jsonldatasets:
+ - path: org/dataset-name
+ data_files:
+ - file1.jsonl
+ - file2.jsonlThis means that the dataset is created as a HuggingFace Dataset and pushed to the Hub via datasets.push_to_hub.
datasets:
- - path: org/dataset-namedatasets:
+ - path: org/dataset-nameThe only difference between the providers is that you need to prepend the path with the respective protocols.
-datasets:
- # Single file
- - path: s3://bucket-name/path/to/your/file.jsonl
-
- # Directory
- - path: s3://bucket-name/path/to/your/directorydatasets:
+ # Single file
+ - path: s3://bucket-name/path/to/your/file.jsonl
+
+ # Directory
+ - path: s3://bucket-name/path/to/your/directoryFor directory, we load via load_from_disk.
The path should start with https://.
datasets:
- - path: https://path/to/your/dataset/file.jsonldatasets:
+ - path: https://path/to/your/dataset/file.jsonlThis must be publically accessible.
lora_mlp_kernel: true
lora_qkv_kernel: true
lora_o_kernel: trueCurrently, LoRA kernels are not supported for RLHF training, only SFT.
+To see other examples of custom reward functions, please see TRL GRPO Docs.
-To see description of the configs, please see TRLConfig.
+To see all configs, please see TRLConfig.
+The DAPO paper and subsequently Dr. GRPO paper proposed an alternative loss function for GRPO to remediate the penalty in longer responses.
+trl:
+ loss_type: dr_grpo
+ # Normalizes loss based on max completion length (default: 256)
+ max_completion_length:For more information, see GRPO docs.
SimPO uses CPOTrainer but with alternative loss function.
-rl: simpo
-rl_beta: 0.1 # default in CPOTrainer
-cpo_alpha: 1.0 # default in CPOTrainer
-simpo_gamma: 0.5 # default in CPOTrainerrl: simpo
+rl_beta: 0.1 # default in CPOTrainer
+cpo_alpha: 1.0 # default in CPOTrainer
+simpo_gamma: 0.5 # default in CPOTrainerThis method uses the same dataset format as DPO.
datasets:
- - ds_type: json
- data_files:
- - orca_rlhf.jsonl
- split: train
- type: chatml.inteldatasets:
+ - ds_type: json
+ data_files:
+ - orca_rlhf.jsonl
+ split: train
+ type: chatml.intelTRL supports auto-unwrapping PEFT models for RL training paradigms which rely on a reference model. This significantly reduces memory pressure as an additional refreference model does not need to be loaded, and reference model log-probabilities can be obtained by disabling PEFT adapters. This is enabled by default. To turn it off, pass the following config:
-# load ref model when adapter training.
-rl_adapter_ref_model: true# load ref model when adapter training.
+rl_adapter_ref_model: true