Add get include and include headers #69

oleksandr-pavlyk · 2020-09-25T18:44:32Z

Headers are included in the layout of the package.
DPPL*Interface shared libraries are moved from environment/lib to the dpctl site-package location (Thanks @Alexander-Makaryev for pointing out $ORIGIN magic variable)
Introduced dpctl.get_include()
Added dpctl/__init__.pxd, which allows downstream users to do cimport dpctl.

1. installation script copies backends/include/* into dpctl/ 2. Use MANIFEST.in to include dpctl/include and dpctl/libDPPL*Inteface.* into installation layout 3. Added dpctl.get_include() function to provide location of includes needed to build extension linked to dpCtl 4. Provide dpctl/__init__.pxd allowing to do `cimport dptcl` to access relevant entities like dpctl.SyclQueue, etc.

Updated build scripts to account for move of DPPL*Interface dynamic libraries from environment lib to site-packages/dpctl/

diptorupd

Pointed out a few cosmetic changes. Will defer to Sergey on how this works out inside CI.

diptorupd · 2020-09-25T18:55:22Z

conda-recipe/bld.bat

+mkdir dpctl\include
+xcopy backends\include dpctl\include /E /Y
+
+

 REM required by dpglue


required by _opencl_core (dpctl.ocldrv)

Since we are touching the file, might as well fix the comment.

diptorupd · 2020-09-25T18:56:15Z

conda-recipe/bld.bat

-set "OpenCL_LIBDIR=%DPCPP_ROOT%/lib"
+set "DPPL_OPENCL_INTERFACE_LIBDIR=dpctl"
+set "DPPL_OPENCL_INTERFACE_INCLDIR=dpctl\include"
+set "OpenCL_LIBDIR=%DPCPP_ROOT%\lib"

 REM required by oneapi_interface


required by _sycl_core (dpct)

diptorupd · 2020-09-25T18:57:18Z

conda-recipe/build.sh

-export OpenCL_LIBDIR=${DPCPP_ROOT}/lib
+export DPPL_OPENCL_INTERFACE_LIBDIR=dpctl
+export DPPL_OPENCL_INTERFACE_INCLDIR=dpctl/include
+export OpenCL_LIBDIR=${DPCPP_ROOT}/libw


what is libw

diptorupd · 2020-09-25T18:58:24Z

MANIFEST.in

+recursive-include dpctl/include *.h *.hpp
+include dpctl/*.pxd
+include dpctl/*DPPL*Interface.*
+global-exclude *.cpp


diptorupd · 2020-09-26T20:59:55Z

@oleksandr-pavlyk I am getting an AssertFailure when I do conda build conda-recipe. The build_for_develop.sh script works fine.

Packaging dpctl
INFO:conda_build.build:Packaging dpctl
INFO conda_build.build:build(2222): Packaging dpctl
/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/environ.py:452: UserWarning: The environment variable 'ONEAPI_ROOT' is being passed through with value '/opt/intel/oneapi'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
UserWarning
Packaging dpctl-0.3.0-py36h6bb024c_5
INFO:conda_build.build:Packaging dpctl-0.3.0-py36h6bb024c_5
INFO conda_build.build:bundle_conda(1468): Packaging dpctl-0.3.0-py36h6bb024c_5
compiling .pyc files...
found egg dir: /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl-0.3.0+5.gbcf97e2-py3.6-linux-x86_64.egg
number of files: 52
WARNING :: get_rpaths_raw()=[] and patchelf=[['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib', '$ORIGIN']] disagree for /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl/_sycl_core.cpython-36m-x86_64-linux-gnu.so ::
WARNING :: get_rpaths_raw()=[] and patchelf=[['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib', '$ORIGIN']] disagree for /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl/_memory.cpython-36m-x86_64-linux-gnu.so ::
WARNING :: get_rpaths_raw()=['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib'] and patchelf=[['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib']] disagree for /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl/libDPPLOpenCLInterface.so ::
WARNING :: get_rpaths_raw()=['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib'] and patchelf=[['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib']] disagree for /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl/libDPPLSyclInterface.so ::
WARNING :: get_rpaths_raw()=['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib'] and patchelf=[['/home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib']] disagree for /home/diptorupd/.conda/envs/dpctl-devel2/conda-bld/dpctl_1601153736731/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/dpctl/_opencl_core.abi3.so ::
Traceback (most recent call last):
File "/home/diptorupd/.conda/envs/dpctl-devel2/bin/conda-build", line 11, in
sys.exit(main())
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 474, in main
execute(sys.argv[1:])
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 465, in execute
verify=args.verify, variants=args.variants)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/api.py", line 210, in build
notest=notest, variants=variants)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/build.py", line 3017, in build_tree
notest=notest,
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/build.py", line 2299, in build
newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/build.py", line 1598, in bundle_conda
files = post_process_files(metadata, initial_files)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/build.py", line 1444, in post_process_files
post_build(m, new_files, build_python=python)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 1315, in post_build
check_overlinking(m, files, host_prefix)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 1243, in check_overlinking
m.config.variant)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 1058, in check_overlinking_impl
for package in packages}
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 1058, in
for package in packages}
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 601, in library_nature
dsos, run_exports, _ = determine_package_nature(pkg, prefix, subdir, bldpkgs_dirs, output_folder, channel_urls)
File "/home/diptorupd/.conda/envs/dpctl-devel2/lib/python3.6/site-packages/conda_build/post.py", line 583, in determine_package_nature
assert isinstance(channeldata, dict) or isinstance(pkg, FakeDist)
AssertionError

PokhodenkoSA · 2020-09-28T09:37:24Z

@diptorupd it is known issue. Use conda-build=3.18. It does not work with newest conda-build=3.20.

oleksandr-pavlyk · 2020-09-28T13:43:00Z

@PokhodenkoSA Were you able to build conda package with this?

PokhodenkoSA · 2020-09-28T13:45:35Z

@oleksandr-pavlyk I did not try this concrete PR but switching to conda-build=3.18 will eliminate problem described by @diptorupd. I will try to build locally.

oleksandr-pavlyk · 2020-09-28T13:52:44Z

Trying conda build conda-recipe/ fails locally as {{ compiler('cxx')}} is being interpreted as GCC. How do you tell conda-build to build with dpcpp instead?

diptorupd · 2020-09-28T13:58:14Z

@PokhodenkoSA @oleksandr-pavlyk Any ideas why CI is failing at the build stage for this PR?

PokhodenkoSA · 2020-09-28T14:07:51Z

@PokhodenkoSA @oleksandr-pavlyk Any ideas why CI is failing at the build stage for this PR?

In Linux it is infrastructure problem. On Windows build and test passed - only validation checks False positive.
I think we need to restart CI.

diptorupd · 2020-09-28T17:32:30Z

LGTM. Let us merge after @PokhodenkoSA restarts the Linux CI and we get a clean build on Linux.

oleksandr-pavlyk · 2020-09-28T18:33:48Z

Was able to build locally on Linux, merging?

oleksandr-pavlyk added 3 commits September 25, 2020 13:11

DPPL shared objects are installed into dpctl/ not in lib/

2418236

Updated build scripts to account for move of DPPL*Interface dynamic libraries from environment lib to site-packages/dpctl/

fixed a blunder

bcf97e2

diptorupd requested a review from PokhodenkoSA September 25, 2020 18:53

diptorupd reviewed Sep 25, 2020

View reviewed changes

oleksandr-pavlyk added 2 commits September 28, 2020 09:55

addressed PR feedback

14f6dc5

added new line per PR feedback

34a78dd

make sure that _opencl_core can find Interface shared objects

aa71c98

oleksandr-pavlyk merged commit 304931b into master Sep 28, 2020

diptorupd mentioned this pull request Sep 30, 2020

dpctl needs to provide get_include() #64

Closed

diptorupd deleted the add-get-include-and-include-headers branch October 3, 2020 03:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add get include and include headers #69

Add get include and include headers #69

Uh oh!

oleksandr-pavlyk commented Sep 25, 2020

Uh oh!

diptorupd left a comment •

edited

Loading

Uh oh!

diptorupd Sep 25, 2020

Uh oh!

diptorupd Sep 25, 2020

Uh oh!

diptorupd Sep 25, 2020

Uh oh!

diptorupd Sep 25, 2020

Uh oh!

diptorupd Sep 25, 2020

Uh oh!

diptorupd commented Sep 26, 2020 •

edited

Loading

Uh oh!

PokhodenkoSA commented Sep 28, 2020

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

PokhodenkoSA commented Sep 28, 2020 •

edited

Loading

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

diptorupd commented Sep 28, 2020

Uh oh!

PokhodenkoSA commented Sep 28, 2020

Uh oh!

diptorupd commented Sep 28, 2020

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

Uh oh!

Add get include and include headers #69

Add get include and include headers #69

Uh oh!

Conversation

oleksandr-pavlyk commented Sep 25, 2020

Uh oh!

diptorupd left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

diptorupd Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

diptorupd Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

diptorupd Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

diptorupd Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

diptorupd Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

diptorupd commented Sep 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PokhodenkoSA commented Sep 28, 2020

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

PokhodenkoSA commented Sep 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

diptorupd commented Sep 28, 2020

Uh oh!

PokhodenkoSA commented Sep 28, 2020

Uh oh!

diptorupd commented Sep 28, 2020

Uh oh!

oleksandr-pavlyk commented Sep 28, 2020

Uh oh!

Uh oh!

diptorupd left a comment •

edited

Loading

diptorupd commented Sep 26, 2020 •

edited

Loading

PokhodenkoSA commented Sep 28, 2020 •

edited

Loading