Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import path torchmetrics.Precision does not correspond to a subclass of <class 'torchmetrics.metric.Metric'> #1953

Closed
Toekan opened this issue Jul 31, 2023 · 1 comment · Fixed by #1963
Labels
bug / fix Something isn't working help wanted Extra attention is needed

Comments

@Toekan
Copy link

Toekan commented Jul 31, 2023

🐛 Bug

Thanks for creating this great package! I've encountered an issue (bug?) when trying to integrate torchmetrics' metrics with the LightningCLI of pytorch lightning.

When using the Precision class (and I suspect many others have the same behaviour (?), I'm hindered by the following (see further below why):

>>> issubclass(torchmetrics.Precision, torchmetrics.Metric)
False

while of course the actual class you get back after dispatching for the right task (multilabel in my case):

>>> issubclass(torchmetrics.classification.precision_recall.MultilabelPrecision, torchmetrics.Metric)
True

Why it matters: I'm trying to use metrics together with the LightningCLI (which I assume could be considered a standard use-case?), as I want to define my metrics in my yaml configuration file. Ignoring everything else, this is an example structure in my yaml:

model:
  train_metrics:
      - precision:  
          class_path: torchmetrics.Precision
          init_args:
            task: multilabel
            num_labels: 4
            average: "micro"
            threshold: 0.5

and this is the type hint in my LightningModule:

train_metrics: List[Dict[str, tm.Metric]]

Unfortunately, in my error message I get the following: Import path torchmetrics.Precision does not correspond to a subclass of <class 'torchmetrics.metric.Metric'>

To Reproduce

See above.

Expected behavior

Considering the documentation encourages to use the top-level class with the task= argument and class types matter in the Lightning ecosphere, I would expect torchmetrics' metrics to work with the LightningCLI, meaning that torchmetrics.Precision is considered a Metric by the CLI type-checker.

Environment

python 3.8, set-up with conda, but pip installed the following:

pytorch-lightning==2.0.3
torch==2.0.1
torchaudio==2.0.2
torchmetrics==1.0.1
torchvision==0.15.2
  • Any other relevant information such as OS (e.g., Linux): linux
@Toekan Toekan added bug / fix Something isn't working help wanted Extra attention is needed labels Jul 31, 2023
@github-actions
Copy link

Hi! thanks for your contribution!, great first issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant