Empty candidate sentence

[achingacham]

Hi,

Could someone help me understand why is the BERTScore recall is non zero when there is an empty candidate sentence. Isn't zero the right score for precision as well as recall over here?

score([''], ['hello how are you'], lang='en')

This is what the model returns now:

Warning: Empty candidate sentence detected; setting precision to be 0.
(tensor([0.]), tensor([0.7738]), tensor([0.]))

best,
Anupama

Originally posted by @achingacham in #62 (comment)

[felixgwu]

Hi @achingacham,

Good catch! Thanks a lot! It comes from the fact that the recall uses the [CLS] and [EOS] tokens added to the empty sentences, so the scores are not zero. However, I believe it is more reasonable to set it as zero. I have fixed this bug in a new commit b1a346.

Best,
Felix