Phi-3 conversation format, example training script and perplexity metric (#1582)
* phi-3 support and perplexity metric * phi-3 chat template * metrics updates * chore: lint * fix assertion on Tensor * fix tests since tokenization happens in the metric * fix perplexity value of shorter passage --------- Co-authored-by: Wing Lian <wing.lian@gmail.com>
This commit is contained in:
6
.gitignore
vendored
6
.gitignore
vendored
@@ -176,3 +176,9 @@ qlora-out/*
|
||||
mlruns/*
|
||||
|
||||
/.quarto/
|
||||
prepared-datasets/
|
||||
submit.sh
|
||||
*.out*
|
||||
|
||||
typings/
|
||||
out/
|
||||
|
||||
Reference in New Issue
Block a user