tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

fix whitespace for patch check

Browse Source
This commit is contained in:
Wing Lian
2024-12-06 16:43:44 -05:00
parent 2e9e423dfd
commit 31723ac523
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
src/axolotl/monkeypatch/trainer_grad_accum.py
View File

@@ -49,6 +49,7 @@ ORIGINAL_LLAMA_FCLM_CODE = """
cache_position=cache_position, cache_position=cache_position,
**kwargs, **kwargs,
) )
hidden_states = outputs[0] hidden_states = outputs[0]
# Only compute necessary logits, and do not upcast them to float if we are not computing the loss # Only compute necessary logits, and do not upcast them to float if we are not computing the loss
logits = self.lm_head(hidden_states[:, -num_logits_to_keep:, :]) logits = self.lm_head(hidden_states[:, -num_logits_to_keep:, :])