to_device() -- any way to force back to host "portably?"

[tylerjereddy]

This came up in data-apis/array-api-compat#40.

My particular use case is something like this:

x.to_device("cpu") to allow downstream library test suites to guarantee that an array is returned to host (or no-op if already there) for usage in i.e., assert_allclose(x, expected) where expected is a NumPy array (always on host) but x could be on some other device depending on the array lib/backend.

I think specification to that level of detail is left to the libs at the moment. CuPy doesn't even have to_device() at the moment let alone cpu, instead requiring usage of i.e., arr.get() to return to host, but even if they did, I'm not so sure the current array API spec would mandate that they need to allow cpu as a valid argument.

Maybe the decision is that we can't really require that level of detail, in which case I can at least point to this issue when justifying shims to get around it when testing. If it were possible to mandate a common call signature for returning to host someday, hopefully the benefit of that would be fairly obvious re: testing multiple backends.

[oleksandr-pavlyk]

I would like to propose to refrain from using "cpu" as an alias for the host (in SYCL terminology), i.e. where the application is started (usually a CPU). At the same time SYCL allows for, and oneAPI DPC++ implements, exposing multi-core CPU as a SYCL device, allowing user to offload data-parallel computation to CPU, as well as to GPUs, or other accelerators known to SYCL runtime.

Bringing data to the host is left out of scope by the standard. I memory serves, most array library implementation provide standalone asnumpy function, but the standard wants to consider NumPy on the same footings as other array Python libraries.

Would it be agreeable to propose free function asbuffer(array) which would convert an array object to a Python object that implements Python buffer protocol?

[tylerjereddy]

Kokkos also has parallel backends for CPU. I think I'm just asking for any portable way to do it across the libs eventually.

[tylerjereddy]

As discussed in the cross-linked PR, another complication is that i.e., cupy.get() actually gives you a NumPy array rather than staying in the same array namespace like torch switching from GPU to CPU. Both conversions ultimately give me something that I can use directly with NumPy assert_allclose(), but that distinction may of course be relevant for design.

[rgommers]

The two .. note::'s under https://data-apis.org/array-api/latest/design_topics/device_support.html#intended-usage have quite a bit to say about this exact question.

I think that a utility can indeed be added to array_api_compat in order to facilitate testing, but it should be clear there that it's separate from the things in the namespaces of implementing array libraries, and the utility will have to be library-specific.

Choosing cupy->numpy seems reasonable for a pragmatic testing utility, but can't be part of the standard I'd say.

[kgryte]

This issue came up during the most recent dataframe workgroup meeting (2023-05-23). cc @kkraus14

[asmeurer]

By the way, it occured to me that we should probably remove 'cpu' from numpy.array_api, since it might give people the impression that it is portable.

[rgommers]

Comment from @fcharras, copied from #710 (comment):

Adding to

@betatim

This issue made me wonder about converting from one namespace to another. Say from PyTorch to Numpy. This works:

x = array_api_compat.torch.asarray([1,2,3])
array_api_compat.numpy.asarray(x) # -> array([1, 2, 3])

The reason I was thinking about this was that it would be nice to have a consistent way of converting things. Of course, there is no asarray for normal Python, so this is more of a thought experiment.

and

@oleksandr-pavlyk

We did discuss a possibility to standardize bringing data from any array object to Python. It would make sense to have a function that would transfer content of array into another type that exposes Python buffer protocol. From here the content could be converted to NumPy, or passed to xp.asarray in another library.

remarks,

should we use a separate issue that covers inter-namespace conversion specifically rather than tolist ?

I want to emphasize with this usecase I have when trying to adapt code for Array API compliance. There is some code that can't compromise on numerical accuracy and absolutly requires at least float64 precision, but I could use an integrated GPU that supports at most float32 (e.g using mps or xpu backends using pytorch) for everything else. For this I would have to transfer data from device to cpu, run the float64 compute, and transfer back to device. But .to_device("cpu"), this is not part of the standard and some array libraries might not support it (like cupy arrays) so I can't rely on it. .from_dlpack does not support inter-device conversion so it's not appropriate either.

For this usecase an intermediate object that enable inter-device and inter-namespace conversion surely would be practical.

tolist have been mentionned but also conversion to and from numpy is commonly supported:

• torch have Tensor.numpy and torch.from_numpy
• cupy have cupy.asnumpy and cupy.asarray works with numpy arrays
• jax have jax.numpy.array and np.asarray works with jax arrays
• dask from_array supports numpy inputs and np.asarray works with dask arrays
• tensorflow have Tensor.numpy and tf.convert_to_tensor()
• mxnet have NDArray.asnumpy and array supports numpy inputs
• dpctl.tensor have asnumpy and dpctl.tensor.asarray supports numpy inputs

wouldn't it be practical to add to the Array API a conversion to numpy, e.g to_numpy or asnumpy ? (from_numpy doesn't seem as necessary since asarray or from_dlpack commonly already works with numpy inputs)

[lucascolley]

wouldn't it be practical to add to the Array API a conversion to numpy, e.g to_numpy or asnumpy ?

There was some discussion about this (I think on the SciPy repo?) and the conclusion was that the standard is supposed to be completely agnostic - requiring a library to know about and interact with NumPy places NumPy on a pedestal which doesn't make sense for an agnostic standard. Then it would just be "why not to_torch etc.", no?