Skip to content

AUROC reported wrong value when called differently #1989

Open
@SinChee

Description

@SinChee

🐛 Bug

AUROC calculation is different when being called differently.

To Reproduce

Copy the following code and run.

Note that when .update() is called, values that are greater than 1.0 will be scaled using the sigmoid activation, values that is less than 1.0 will not be scaled. This result in the compute step to compute with the wrong input values. However, directly calling the auroc will not rescale the value

from torchmetrics import AUROC
from torch import tensor

pp = tensor([tensor([1.2]), tensor([0.9678]), tensor([1.7])])
tt = tensor([tensor([1]), tensor([0]), tensor([1])])

auroc = AUROC(num_classes=1, task="binary")
print("Forward:", auroc(pp, tt))


for p, t in zip(pp, tt):
    auroc.update(p, t)
    print(auroc.preds, auroc.target)

print("Update and Compute:", auroc.compute())

Expected behavior

The reported AUROC metrics for both cases should be the same.

Environment

  • TorchMetrics 1.0.3
  • Python 3.9.16
  • PyTorch 1.13.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions