Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very Basic 'How to Use' Accuracy example is not runnable if I only install PyTorch; error may relate to multi-dispatch behaviour #115

Closed
REIGN12 opened this issue Mar 19, 2023 · 1 comment
Labels

Comments

@REIGN12
Copy link

REIGN12 commented Mar 19, 2023

Describe the bug
I try to run example in doc of 'How to use', I only install np and torch in my env.

from mmeval import Accuracy
import numpy as np

accuracy = Accuracy()
labels = np.asarray([0, 1, 2, 3])
preds = np.asarray([0, 2, 1, 3])
accuracy(preds, labels)
# {'top1': 0.5}

And I got the error msg:

Patch `plum.type.TypeMeta` with singleton failed, raise error: module 'plum.type' has no attribute 'TypeMeta'. The multiple dispatch speed may be slow.
Patch plum Type with hash value cache failed, raise error: cannot import name 'Dict' from 'plum.parametric' (/anaconda/envs/onemodel/lib/python3.8/site-packages/plum/parametric.py). The multiple dispatch speed may be slow.
Traceback (most recent call last):
  File "test_mmeval.py", line 8, in <module>
    accuracy(preds, labels)
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/mmeval/core/base_metric.py", line 105, in __call__
    self.add(*args, **kwargs)
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/mmeval/metrics/accuracy.py", line 191, in add
    corrects = self._compute_corrects(predictions, labels)
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/plum/function.py", line 419, in __call__
    return self._f(self._instance, *args, **kw_args)
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/plum/function.py", line 342, in __call__
    self._resolve_pending_registrations()
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/plum/function.py", line 220, in _resolve_pending_registrations
    signature = extract_signature(f, precedence=precedence)
  File "/anaconda/envs/onemodel/lib/python3.8/site-packages/plum/signature.py", line 187, in extract_signature
    for k, v in typing.get_type_hints(f).items():
  File "/anaconda/envs/onemodel/lib/python3.8/typing.py", line 1264, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/anaconda/envs/onemodel/lib/python3.8/typing.py", line 270, in _eval_type
    return t._evaluate(globalns, localns)
  File "/anaconda/envs/onemodel/lib/python3.8/typing.py", line 518, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'oneflow' is not defined

Indicating oneflow is not defined.
And I use my debugger try to debug, and found the problem is in accuracy.py L248

@overload  # type: ignore
@dispatch
def _compute_corrects(  # type: ignore
    self, predictions: Union['oneflow.Tensor', Sequence['oneflow.Tensor']],
    labels: Union['oneflow.Tensor',
                  Sequence['oneflow.Tensor']]) -> 'oneflow.Tensor':

when the dispatch try to decorate the _compute_corrects related to oneflow.
And I go deeper to dispatch, found that in dispatcher.py L176,

try:
    module = importlib.import_module(module_name)
    resolved_type = getattr(module, module_attr_basename)
except Exception as e: #L176
    if importable_name not in self._unimportable_types:
        logger.debug(
            f"Unimportable: '{importable_name}', raise error: {e}.")
        resolved_type = type(importable_name, (), {})
        self._unimportable_types[importable_name] = resolved_type
    else:
        resolved_type = self._unimportable_types[importable_name]
return resolved_type

this except should have caught the ModuleNotFoundError and solve this, but the error is not caught and I totally have no idea why.

Reproduction

  1. What command or script did you run?
from mmeval import Accuracy
import numpy as np

accuracy = Accuracy()
labels = np.asarray([0, 1, 2, 3])
preds = np.asarray([0, 2, 1, 3])
accuracy(preds, labels)
# {'top1': 0.5}
@C1rN09
Copy link
Collaborator

C1rN09 commented Mar 20, 2023

plum-dispatch released its 2.0.0 version, which breaks something. We have temporarily fixed this issue in PR #108 and you can reinstall plum-dispatch < 2.0 to solve this for now.
We will try to figure it out and adapt to plum-dispatch >= 2.0.0 in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants