Issue with explanation of shared memory in 01_numpy_performance

[mdboom]

Thanks for writing the cookbook and sharing it online. This is certain to become a great resource for our users.

In cookbook 01_numpy_performance, it is recommended to use x.__array_interface__['data'][0] to determine if an array is sharing data with another array. This is only useful if the offset of the arrays are the same, not if one array is a subarray/slice/view of another.

For example, here, two arrays are sharing the same data but they have different starting pointers.

In [1]: import numpy as np

In [2]: x = np.arange(10)

In [3]: y = x[1::2]

In [4]: x.__array_interface__['data'][0]
Out[4]: 46090608

In [5]: y.__array_interface__['data'][0]                                                                                                                                                                                            
Out[5]: 46090616

You could probably figure out that their data areas are overlapping, but that’s kind of expensive/complex.

The best way I’ve found to find out if two arrays share the same data:

def get_data_base(arr):
    base = arr.base
    while base is not None:
        base = base.base
    return base

>>> get_data_base(x) is get_data_base(y)
True

[rossant]

Thanks for the precision! I'll add it in the book. I'll also double-check whether all my results about array copies are true (I may have false positives).

I tried your get_data_base function but it seems to always return None (which seems to be expected looking at the code)?

print(get_data_base(np.zeros(4)))
print(get_data_base(np.zeros(4)[1::2]))
None
None

tested on Python 3.4.1, Anaconda 2.0.1 (64-bit), Windows 8.1 64-bit, NumPy 1.8.1.

[mdboom]

Sorry -- I should have copied it from the code I use rather than from memory:

def get_data_base(arr):
    """
    For a given Numpy array, finds the base array that "owns" the
    actual data.
    """
    base = arr
    while isinstance(base.base, np.ndarray):
        base = base.base
    return base

[rossant]

Thanks, it works great! I've added this precision in the recipe. Closing.