Files
axolotl/src
Seungduk Kim 97a2fa2781 Select input_ids explicitly after panda conversion (#2335)
Without selecting the column, applying `len` counts the whole row as 1 which resulting the total number of the samples instead of the token counts.
2025-02-17 00:07:27 -05:00
..