tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Whitespace bug fix

Browse Source 
Command had accidentally been moved out of if-else block.
This commit is contained in:
ssmi153
2023-08-05 15:08:44 +12:00
parent 1fed74b1d9
commit 64852ae15a
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/axolotl/monkeypatch/llama_attn_hijack_xformers.py
View File

@@ -156,7 +156,9 @@ def xformers_forward(
f" {attn_output.size()}" f" {attn_output.size()}"
) )
attn_output = attn_output.transpose(1, 2).contiguous() attn_output = attn_output.transpose(1, 2).contiguous()
#end x-formers vs. not x-formers if-else block
attn_output = attn_output.reshape(bsz, q_len, self.hidden_size) attn_output = attn_output.reshape(bsz, q_len, self.hidden_size)
if self.pretraining_tp > 1: if self.pretraining_tp > 1: