Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cumprod returns errors #807

Closed
pwolfram opened this issue Mar 28, 2016 · 3 comments
Closed

cumprod returns errors #807

pwolfram opened this issue Mar 28, 2016 · 3 comments

Comments

@pwolfram
Copy link
Contributor

The xarray implementation of cumprod returns an assertion error, presumably because of bottleneck, e.g., https://github.com/pydata/xarray/blob/master/xarray/core/ops.py#L333. The error is

└─▪ ./test_cumprod.py 
[ 0.8841785   0.54181236  0.29075258  0.28883015  0.1137352   0.09909713
  0.03570122  0.0304542   0.01578143  0.01496195  0.01442681  0.00980845]
Traceback (most recent call last):
  File "./test_cumprod.py", line 13, in <module>
    foo.cumprod()
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/common.py", line 16, in wrapped_func
    skipna=skipna, allow_lazy=True, **kwargs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/dataarray.py", line 991, in reduce
    var = self.variable.reduce(func, dim, axis, keep_attrs, **kwargs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/variable.py", line 871, in reduce
    axis=axis, **kwargs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/ops.py", line 346, in f
    assert using_numpy_nan_func
AssertionError

If bottleneck is uninstalled then a value error is returned:

└─▪ ./test_cumprod.py 
[  2.99508768e-01   2.80142920e-01   1.56389242e-01   1.10791301e-01
   4.58372649e-02   4.10865622e-02   9.91362500e-03   6.76033435e-03
   3.83574249e-03   9.54972340e-04   1.56846616e-04   6.44088547e-05]
Traceback (most recent call last):
  File "./test_cumprod.py", line 13, in <module>
    foo.cumprod()
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/common.py", line 16, in wrapped_func
    skipna=skipna, allow_lazy=True, **kwargs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/dataarray.py", line 991, in reduce
    var = self.variable.reduce(func, dim, axis, keep_attrs, **kwargs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/variable.py", line 880, in reduce
    return Variable(dims, data, attrs=attrs)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/variable.py", line 213, in __init__
    self._dims = self._parse_dimensions(dims)
  File "/Users/pwolfram/anaconda/lib/python2.7/site-packages/xarray/core/variable.py", line 321, in _parse_dimensions
    % (dims, self.ndim))
ValueError: dimensions () must have the same length as the number of data dimensions, ndim=1

No error occurs if the data array is converted to a numpy array prior to use of cumprod.

This can easily be reproduced by https://gist.github.com/c32f231b773ecc4b0ccf, excerpted below

import numpy as np
import pandas as pd
import xarray as xr                                                                      


data = np.random.rand(4, 3)
locs = ['IA', 'IL', 'IN']
times = pd.date_range('2000-01-01', periods=4)
foo = xr.DataArray(data, coords=[times, locs], dims=['time', 'space'])
print foo.values.cumprod()
foo.cumprod()
@pwolfram pwolfram changed the title cumprod returns assertion error cumprod returns errors Mar 28, 2016
@shoyer
Copy link
Member

shoyer commented Mar 30, 2016

Could you please open a pull request with your branch so we can take a look at what's going on?

The current code will need some adjustments to put cumprod/cumsum in (namely, a variant of reduce that doesn't need to drop dimensions)

@pwolfram
Copy link
Contributor Author

Thanks @shoyer for the interest. The PR is at #812. I haven't had a ton of time to work on this but it is on my to do list.

@shoyer
Copy link
Member

shoyer commented Mar 31, 2016

I'm going to close this because it isn't a bug in any merged code. Let's discuss in the PR instead.

@shoyer shoyer closed this as completed Mar 31, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants