Questions regarding the metric for link prediction tasks.

[VeritasYin]

Hello,

For collab, ppa and ddi datasets, OGB chooses 'Hits' as the evaluation metric; while for citiation2, it adopts MRR. May I know why the metric benchmark is designed this way? Are there any concerns regarding using MRR metric for all datasets, which is more widely applied in practice?

Thanks!

[weihua916]

Hi! Good point. There is no deep reason to be honest, but we use MRR for datasets with reasonably high Hits@1 baseline performance.