# utils.callbacks.perplexity – Axolotl

# utils.callbacks.perplexity

`utils.callbacks.perplexity`

callback to calculate perplexity as an evaluation metric.

## Classes

Name | Description
--- | ---
Perplexity | Calculate perplexity as defined in https://huggingface.co/docs/transformers/en/perplexity.

### Perplexity

    utils.callbacks.perplexity.Perplexity(tokenizer, max_seq_len, stride=512)

Calculate perplexity as defined in https://huggingface.co/docs/transformers/en/perplexity. This is a custom variant that doesn’t re-tokenize the input or re-load the model.

#### Methods

Name | Description
--- | ---
compute | Compute perplexity in a fixed length sliding window across the sequence.

##### compute

    utils.callbacks.perplexity.Perplexity.compute(model, references=None)

Compute perplexity in a fixed length sliding window across the sequence.