Bug in loss?

[rachtibat]

Hey,

thanks for the nice work!
I've noticed that you subtract in https://github.com/Aleph-Alpha/AtMan/blob/847f09078c6c5614ac89866d28c3a25787d4782d/atman-magma/atman_magma/logit_parsing.py#L73
the ground truth loss from the suppressed loss.

However, in your paper you do it the other way around as far as I understand: subtracting the suppressed loss from the ground truth. In my experiments, the latter also makes more sense because the suppressed loss should be bigger than the ground truth loss if you flip the correct token and if you want to assign a positive importance value to it.

Is this a bug?

Thanks!