Add support for dpctl.dparray. #50

PokhodenkoSA · 2020-12-04T12:35:49Z

Update with main
Transform unit tests test_usmarray.py
Need to get rid of relative paths in setup.py

DrTodd13

Looks correct at first glance. I didn't look to see why the tests were failing but that would be expected even if this is using the dpctl dparray branch since we removed the section of code there that looks for Numba meminfo objects. Once this is merged then I will add the code here to monkey-patch the meminfo objects with sycl_usm_array_interface.

Co-authored-by: etotmeni <elena.totmenina@intel.com>

* Del dppl dir in tests * Del unused var Co-authored-by: etotmeni <elena.totmenina@intel.com>

This reverts commit 776bf22.

Co-authored-by: etotmeni <elena.totmenina@intel.com> Co-authored-by: Diptorup Deb <diptorup.deb@intel.com>

…Numba-dppy pipeline (#52) * Sum example * Moved from infer_type, lower_builtin to overload * Added two level module name functions * Remove cython generated file * Module name fix for moving to new extension * Incomplete linalg.eig implementation * Updted all dppl to dppy and moved rewrite_numpy_function_pass to it's own file * Import module at correct locations * Added comments * Added test and updated comments * Revert unneeded changes * Update Eigen implementation * Remove eig implementation * Add checking equivalent IR Co-authored-by: reazul.hoque <reazul.hoque@intel.com>

…ation

diptorupd · 2020-12-10T17:44:29Z

numba_dppy/numpy_usm_shared.py

+    todo_getattr = []
+
+    # For all Numpy identifiers that have been registered for typing in Numba...
+    # this registry contains functions, getattrs, setattrs, casts and constants...need to do them all? FIX FIX FIX


Please convert this comment into a ticket.

PokhodenkoSA · 2020-12-10T17:52:14Z

This code was moved to dpctl.dptensor.

Is it relate to some line?

…ect is a USM MemInfo.

PokhodenkoSA · 2020-12-11T11:50:51Z

This PR requires IntelPython/dpctl#214.

…ass and the Numba integration were in the same file. I changed those to explicitly refer to the usmarray module in dpctl.

…n numpy_usm_shared in dpctl.dptensor. This fixes the ndindex issue.

PokhodenkoSA · 2020-12-15T09:24:15Z

Assertion error in last case:

print("------------------- Testing Numba usmarray.T")
sys.stdout.flush()
dp4 = f12(dp3)
print("dp4:", dp4, type(dp4))
assert isinstance(dp4, usmarray.ndarray)
assert usmarray.has_array_interface(dp4)
del dp3
del dp4

------------------- Testing Numba usmarray.T
...
<class 'numpy.ndarray'>
Traceback (most recent call last):
  File "numba_dppy/tests/test_usmarray.py", line 220, in <module>
    assert isinstance(dp4, usmarray.ndarray)
AssertionError

Fixed.

* Split tests for usmarray in separate unittest test cases * Remove prints and use unittest assertions * Move functions to tests * Give names to functions * Add expectedFailure for failed tests * Clean code

setup.py

PokhodenkoSA · 2020-12-15T13:55:10Z

3 tests failed and marked as expected failure:

test_numba_usmarray_mul_add - strange, it passed if runs not in batch
test_numba_usmarray_constructor_from_numpy_ndarray - was commented previously
test_numba_usmarray_T - did not work as I started

DrTodd13 · 2020-12-22T22:14:31Z

I would like to merge it. @DrTodd13 what is not ready yet?

I would like to merge it before #135 - black formatter. However I can do it in any order.

There are some parts of the USM array attribute class that use bound_function that don't seem to work with the current configuration and I'm not sure why. I say we merge what we currently have while I investigate this issue I described and we can have another PR for that.

numba_dppy/numpy_usm_shared.py

numba_dppy/tests/test_usmarray.py

PokhodenkoSA · 2020-12-23T10:12:52Z

@DrTodd13
I have find interesting tests dependency and possibly an issue. Have you any idea about it?

import numba
import numpy
import dpctl.dptensor.numpy_usm_shared as usmarray

@numba.njit()
def numba_mul(a, b):  # a is usmarray, b is numpy
    return a * b

@numba.njit()
def numba_mul_add(a):
    return a * 2.0 + 13

assert isinstance(numba_mul(usmarray.ones(10), numpy.ones(10)), usmarray.ndarray)  # 1
assert isinstance(numba_mul_add(numpy.ones(10)), numpy.ndarray)  # 2
assert isinstance(numba_mul_add(usmarray.ones(10)), usmarray.ndarray)  # 3 - error

Traceback (most recent call last):
  File "./usmarray_and_ndarray.py", line 27, in <module>
    assert isinstance(numba_mul_add(usmarray.ones(10)), usmarray.ndarray)
AssertionError

If any of 1 or 2 are commented then 3 passed.
If change order of 1 or 2 then 3 passed.

numba_dppy/tests/test_usmarray.py

numba_dppy/numpy_usm_shared.py

PokhodenkoSA · 2020-12-23T15:36:17Z

@DrTodd13
I have made formatting for the code with black. If you will face merge conflicts do following:

pip install black
black .

PokhodenkoSA · 2021-01-11T11:25:42Z

@DrTodd13 is it ready to be merged? How can I help with it?

DrTodd13 · 2021-01-11T16:06:29Z

@DrTodd13 is it ready to be merged? How can I help with it?

Yes. We can merge. There are some known issues to be fixed but let's get the stuff that is working checked in and fix the rest later.

numba_dppy/numpy_usm_shared.py

PokhodenkoSA · 2021-01-12T13:08:13Z

@diptorupd @DrTodd13
Could somebody of you please approve this PR ?

Add support for dpctl.dparray.

7dd904d

DrTodd13 approved these changes Dec 4, 2020

View reviewed changes

This was referenced Dec 9, 2020

Initial support for Numpy subclasses (for dparray) IntelPython/numba#116

Closed

numba-dppy extension IntelPython/numba#76

Closed

DrTodd13 and others added 11 commits December 9, 2020 15:41

Fix build issues for dppy_rt.c

26d5e2e

Name changes from dparray things to usmarray.

a320b50

Delete old backup file (#45)

f9de97a

Co-authored-by: etotmeni <elena.totmenina@intel.com>

Del dppl dir in tests (#43)

b8c11f7

* Del dppl dir in tests * Del unused var Co-authored-by: etotmeni <elena.totmenina@intel.com>

Revert "numba-dppy requires cffi"

16fb9b8

This reverts commit 776bf22.

Remove use of cffi

56e969a

Rename dppl to dppy (#42)

0a56e08

Co-authored-by: etotmeni <elena.totmenina@intel.com> Co-authored-by: Diptorup Deb <diptorup.deb@intel.com>

Store allocation queue on a per-object basis.

059a5a3

Merge branch 'main' into spokhode/dparray

f895e1d

Add imports to usmarray module and fixed setup.py extension initializ…

4e99a55

…ation

diptorupd reviewed Dec 10, 2020

View reviewed changes

Register is_usm_callback with dpctl to say whether a given Python obj…

595f94b

…ect is a USM MemInfo.

DrTodd13 added 3 commits December 11, 2020 12:24

Remove printf.

c87a94b

There were some spots where there was a silent assumption that the cl…

1214893

…ass and the Numba integration were in the same file. I changed those to explicitly refer to the usmarray module in dpctl.

Found another spot where the current module was being used rather tha…

a2b2bb7

…n numpy_usm_shared in dpctl.dptensor. This fixes the ndindex issue.

etotmeni and others added 3 commits December 15, 2020 06:47

Fix numba path

0769a97

fix

eb53c26

Convert tests for USM array to unittest (#118)

f4333dd

* Split tests for usmarray in separate unittest test cases * Remove prints and use unittest assertions * Move functions to tests * Give names to functions * Add expectedFailure for failed tests * Clean code

PokhodenkoSA commented Dec 15, 2020

View reviewed changes

setup.py Outdated Show resolved Hide resolved

PokhodenkoSA and others added 2 commits December 15, 2020 07:56

Small code fixes

313727b

use include_path to find numba

ff877fc