Wrong tensor index for roll and truncate in DPOTrainer fn concatenated_forward( ).

[yanghh2000]

System Info

it is a tensor index error

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

# Get the first column idx that is all zeros and remove every column after that
empty_cols = torch.sum(attention_mask, dim=0) == 0
first_empty_col = torch.nonzero(empty_cols)[0].item() if empty_cols.any() else attention_mask.size(1) + 1
input_ids = input_ids[:, : first_empty_col - 1]
attention_mask = attention_mask[:, : first_empty_col - 1]
loss_mask = loss_mask[:, : first_empty_col - 1]

Expected behavior

The returns of torch.nonzero is the index (starts from 0) of non-zero elements, so there is no need to add -1 to first_empty_col.
The correct code should be:

empty_cols = torch.sum(attention_mask, dim=0) == 0
first_empty_col = torch.nonzero(empty_cols)[0].item() if empty_cols.any() else attention_mask.size(1)
input_ids = input_ids[:, : first_empty_col]
attention_mask = attention_mask[:, : first_empty_col]
loss_mask = loss_mask[:, : first_empty_col]

[qgallouedec]

Good catch! Thanks! Do you mind opening a PR to fix that?