# prompt_strategies.llama2_chat – Axolotl

# prompt_strategies.llama2_chat

`prompt_strategies.llama2_chat`

Prompt Strategy for finetuning Llama2 chat models see also https://github.com/facebookresearch/llama/blob/6c7fe276574e78057f917549435a2554000a876d/llama/generation.py#L213 for ma reference implementation.

This implementation is based on the Vicuna PR and the fastchat repo, see also: https://github.com/lm-sys/FastChat/blob/cdd7730686cb1bf9ae2b768ee171bdf7d1ff04f3/fastchat/conversation.py#L847

Use dataset type: “llama2_chat” in conig.yml to use this prompt style.

E.g. in the config.yml:

    datasets:
      - path: llama_finetune_train.jsonl
        type: llama2_chat

The dataset itself should look like this:

    {'conversations':[{"from": "human", "value": "Who are you?"}, {"from": "gpt", "value": "I am Vicuna"},...]}

in a jsonl file. The first message should be from the human, the second from gpt. For a custom system message, the first “from” can be “system” (followed by alternating “human” and “gpt” turns).

Important: Don’t use “special_tokens:” in your config.yml if you are not sure what you are doing!

## Classes

Name | Description
--- | ---
LLama2ChatTokenizingStrategy | Tokenizing strategy for Llama2 prompts.
Llama2ChatConversation | A class that manages prompt templates and keeps all conversation history.
Llama2ChatPrompter | A prompter that generates prompts for Llama2 models.

### LLama2ChatTokenizingStrategy

    prompt_strategies.llama2_chat.LLama2ChatTokenizingStrategy(*args, **kwargs)

Tokenizing strategy for Llama2 prompts. adapted from https://github.com/lm-sys/FastChat/blob/main/fastchat/train/train.py

### Llama2ChatConversation

    prompt_strategies.llama2_chat.Llama2ChatConversation(
        name='llama2',
        system="[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<</SYS>>\n\n",
        roles=('[INST]', '[/INST]'),
        messages=list(),
        offset=0,
    )

A class that manages prompt templates and keeps all conversation history. copied from https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py

#### Methods

Name | Description
--- | ---
append_message | Append a new message.
get_prompt | Get the prompt for generation.

##### append_message

    prompt_strategies.llama2_chat.Llama2ChatConversation.append_message(
        role,
        message,
    )

Append a new message.

##### get_prompt

    prompt_strategies.llama2_chat.Llama2ChatConversation.get_prompt()

Get the prompt for generation.

### Llama2ChatPrompter

    prompt_strategies.llama2_chat.Llama2ChatPrompter()

A prompter that generates prompts for Llama2 models.