Skip to content

require numpy>1.26#994

Merged
rkingsbury merged 2 commits intomainfrom
rkingsbury-patch-2
Aug 23, 2024
Merged

require numpy>1.26#994
rkingsbury merged 2 commits intomainfrom
rkingsbury-patch-2

Conversation

@rkingsbury
Copy link
Collaborator

@rkingsbury rkingsbury commented Aug 23, 2024

This seems necessary to prevent an unusual pandas error in open data (which surfaced in the Auto Dependency Upgrades workflow after allowing numpy>2

_________________________________ test_update __________________________________

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
>               return self.resolvers[key]

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:231: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/collections/__init__.py:1006: in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = DeepChainMap({'last_updated': 0   2024-08-23 18:30:08.900661
1   2024-08-23 18:30:08.900663
Name: last_updated, dtype:...ast_updated          last_updated
task_id                    task_id
group_level_two    group_level_two
dtype: object})
key = 'np'

    def __missing__(self, key):
>       raise KeyError(key)
E       KeyError: 'np'

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/collections/__init__.py:998: KeyError

During handling of the above exception, another exception occurred:

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
                return self.resolvers[key]
    
            # if we're here that means that we have no locals and we also have
            # no resolvers
            assert not is_local and not self.has_resolvers
            return self.scope[key]
        except KeyError:
            try:
                # last ditch effort we look in temporaries
                # these are created when parsing indexing expressions
                # e.g., df[df > 0]
>               return self.temps[key]
E               KeyError: 'np'

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:242: KeyError

The above exception was the direct cause of the following exception:

s3store = <maggma.stores.open_data.OpenDataStore object at 0x7f18dfb08150>

    def test_update(s3store):
        assert len(s3store.index_data) == 2
>       s3store.update(
            pd.DataFrame(
                [
                    {
                        "task_id": "mp-199999",
                        "data": "asd",
                        "group": {"level_two": 4},
                        s3store.last_updated_field: datetime.utcnow(),
                    }
                ]
            )
        )

tests/stores/test_open_data.py:395: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/maggma/stores/open_data.py:531: in update
    sub_existing = existing_index.query(query_str)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/frame.py:4823: in query
    res = self.eval(expr, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/frame.py:4949: in eval
    return _eval(expr, inplace=inplace, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/eval.py:336: in eval
    parsed_expr = Expr(expr, engine=engine, parser=parser, env=env)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:809: in __init__
    self.terms = self.parse()
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:828: in parse
    return self._visitor.visit(self.expr)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:418: in visit_Module
    return self.visit(expr, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:421: in visit_Expr
    return self.visit(node.value, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:746: in visit_BoolOp
    return reduce(visitor, operands)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:739: in visitor
    lhs = self._try_visit_binop(x)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:735: in _try_visit_binop
    return self.visit(bop)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:719: in visit_Compare
    return self.visit(binop)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:532: in visit_BinOp
    op, op_class, left, right = self._maybe_transform_eq_ne(node)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:454: in _maybe_transform_eq_ne
    right = self.visit(node.right, side="right")
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:659: in visit_Call
    res = self.visit_Attribute(node.func)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:644: in visit_Attribute
    resolved = self.visit(value).value
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:545: in visit_Name
    return self.term_type(node.id, self.env, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/ops.py:91: in __init__
    self._value = self._resolve_name()
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/ops.py:115: in _resolve_name
    res = self.env.resolve(local_name, is_local=is_local)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
                return self.resolvers[key]
    
            # if we're here that means that we have no locals and we also have
            # no resolvers
            assert not is_local and not self.has_resolvers
            return self.scope[key]
        except KeyError:
            try:
                # last ditch effort we look in temporaries
                # these are created when parsing indexing expressions
                # e.g., df[df > 0]
                return self.temps[key]
            except KeyError as err:
>               raise UndefinedVariableError(key, is_local) from err
E               pandas.errors.UndefinedVariableError: name 'np' is not defined

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:244: UndefinedVariableError

@rkingsbury rkingsbury added the dependencies Pull requests that update a dependency file label Aug 23, 2024
@codecov
Copy link

codecov bot commented Aug 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.83%. Comparing base (10689d6) to head (9f924ee).
Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #994   +/-   ##
=======================================
  Coverage   88.83%   88.83%           
=======================================
  Files          46       46           
  Lines        3985     3985           
=======================================
  Hits         3540     3540           
  Misses        445      445           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rkingsbury rkingsbury changed the title require numpy>1.26.4 require numpy>1.26 Aug 23, 2024
@rkingsbury rkingsbury merged commit e553944 into main Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant