Abhishek Mishra d4a88e4eca Adding qlora config for Mistral (#675) ...

* Adding qlora config for Mistral

Contains fix for Mistral FA issue - ValueError: You are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to  call tokenizer.padding_side  = 'left' before tokenizing the input.

Fix for now is to set sample_packing: true and pad_to_sequence_len: true

* Renamed to qlora.yml

2023-10-06 21:05:56 +09:00

1.3 KiB

Raw Blame History

View Raw