Unable to create numpy int array from an `EdgeList`

[mtreinish]

Information

• retworkx version: main branch (since 71e34d1)
• Python version: 3.10
• Rust version: 1.60
• Operating system: Linux

What is the current behavior?

Trying to create a numpy array from an EdgeList object fails

TypeError: int() argument must be a string, a bytes-like object or a real number, not 'retworkx.EdgeList'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/test_failure.py", line 5, in <module>
    print(np.asarray(res, dtype=np.uintp))
ValueError: setting an array element with a sequence.

I bisected this to 71e34d1 (#569). I'm not sure if this is caused by changes in the newer version of pyo3 or our minimal changes to the custom_vec_iter_impl macro (I'm leaning towards a pyo3 change).

What is the expected behavior?

calling np.asarray(g.edge_list, dtype=np.uintp) will create a numpy array for the edge list like:

[[0 1]
 [0 2]
 [0 3]
 [0 4]]

Steps to reproduce the problem

import retworkx
import numpy as np
g = retworkx.generators.directed_star_graph(5)
res = g.edge_list()
print(np.asarray(res, dtype=np.uintp))

[mtreinish]

This is going to be a release blocking issue because this causes errors in qiskit-terra as it relies on being able to np.asarray(res, dtype=np.uintp) from graph objects.

[IvanIsCoding]

A possible fix without downgrading PyO3 is to implement the __array__ method as described in the NumPy API. That way we can guarantee that np.asarray will always works for our custom types.

It is puzzling that bumping the PyO3 version broke our code, especially considering we did not update iterators.rs. We should add some tests for conversions that are important to us

[mtreinish]

That's a good idea, that also would potentially avoid the iteration overhead too because we could go straight to a numpy array via the c api (not that array creation from our custom return types is a bottleneck for anything).

[mtreinish]

I also reported this upstream to pyo3 in PyO3/pyo3#2392 one option we might want to look at is converting back to use pyproto with the PySequenceProtocol trait for the custom_vec_iter_impl macro https://github.com/Qiskit/retworkx/blob/7dc5182a61fdc67425038f5f8cf9bf4474c080ca/src/iterators.rs#L416-L533 and suppress the deprecation warnings. That is probably the simplest fix to start.

[georgios-ts]

The new pyo3 implementation fills mp_length slot but not sq_length. I'm guessing this causes numpy some trouble and can't identify our custom return types as sequence objects.