Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: Build success but python cannot load SIRF #729

Open
paskino opened this issue Jun 22, 2022 · 20 comments
Open

Windows: Build success but python cannot load SIRF #729

paskino opened this issue Jun 22, 2022 · 20 comments

Comments

@paskino
Copy link
Contributor

paskino commented Jun 22, 2022

I built locally on Windows10 with VS community edition and NVIDIA SDK.

conda create --name sirf_b2 python numpy openblas matplotlib=3.4 docopt cmake deprecation -c conda-forge
conda activate sirf_b2
cmake ../SIRF-SuperBuild -DSIRF_TAG=origin/master -DSTIR_TAG=origin/master -DCIL_TAG=origin/master -DBUILD_Gadgetron=OFF -DUSE_SYSTEM_HDF5=OFF -DUSE_SYSTEM_Boost=ON -DBoost_ROOT=C:\Apps\boost_1_72_0 -DUSE_SYSTEM_FFTW=OFF

After sourcing the env_sirf.PS1 environment file on PowerShell, I can run SIRF C++ unit tests (excluding the MR tests as I haven't built Gadgetron)

cd builds\SIRF\build
ctest -v --output-on-failure -R CPLUS -E MR -C Release

which outputs:

Test project C:/Users/ofn77899/Dev/buildVC/builds/SIRF/build
    Start  6: REG_TEST_CPLUSPLUS
1/7 Test  #6: REG_TEST_CPLUSPLUS ......................   Passed    4.40 sec
    Start  9: PET_TESTS_CPLUSPLUS_1
2/7 Test  #9: PET_TESTS_CPLUSPLUS_1 ...................   Passed   66.29 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
3/7 Test #10: PET_TESTS_CPLUSPLUS_4 ...................   Passed   65.26 sec
    Start 14: SYN_TEST_STIR_TO_NIFTI_CPLUSPLUS
4/7 Test #14: SYN_TEST_STIR_TO_NIFTI_CPLUSPLUS ........   Passed    1.05 sec
    Start 15: SYN_TEST_GADGETRON_TO_NIFTI_CPLUSPLUS
5/7 Test #15: SYN_TEST_GADGETRON_TO_NIFTI_CPLUSPLUS ...   Passed    0.53 sec
    Start 18: SYN_TEST_CPLUSPLUS
6/7 Test #18: SYN_TEST_CPLUSPLUS ......................***Failed   12.42 sec
// ----------------------------------------------------------------------- //
//                  Starting STIRImageData->NiftiImageData test...
//------------------------------------------------------------------------ //

INFO: Reading C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/zenodo/dicom_as_nifti.nii via ITK non-DICOM IO

WARNING: Unable to determine patient position. Internally this will generally be handled by assuming HFS

Saving image to file (results/stir_to_nifti.nii)...done.


Image comparison: metadata do not match, doing resampling...

No transformations set, using identity.
// ----------------------------------------------------------------------- //
//                  Finished STIRImageData->NiftiImageData test.
//------------------------------------------------------------------------ //
// ----------------------------------------------------------------------- //
//                  Starting GadgetronImageData->NiftiImageData test...
//------------------------------------------------------------------------ //
group 2019-12-11 16:38:14
variable image_1
number of images: 31
image data type: 5
variable xml

GadgetronImagesVector::set_up_geom_info(). Warning, you set up geometry for slices whose width is not their distance.This setup does probably not account for overlaps or gaps between slices.

Image comparison: metadata do not match, doing resampling...

No transformations set, using identity.
// ----------------------------------------------------------------------- //
//                  Finished GadgetronImageData->NiftiImageData test.
//------------------------------------------------------------------------ //
// ----------------------------------------------------------------------- //
//                  Starting complex resampler test...
//------------------------------------------------------------------------ //
Started reading acquisitions from C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/grappa2_1rep.h5
0%..10%..20%..30%..40%..49%..59%..69%..79%..89%..98%..
Finished reading acquisitions from C:/Users/ofn77899/Dev/buildVC/sources/SIRF/data/examples/MR/grappa2_1rep.h5
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed, trying again...
connection to gadgetron server failed

Here's the error:
        Server running Gadgetron not accessible


    Start 20: SIRF_TEST_COMMON_CPLUSPLUS
7/7 Test #20: SIRF_TEST_COMMON_CPLUSPLUS ..............   Passed    0.10 sec

86% tests passed, 1 tests failed out of 7

Total Test time (real) = 150.22 sec

The following tests FAILED:
         18 - SYN_TEST_CPLUSPLUS (Failed)
Errors while running CTest

All unit tests involving python ctest -v --output-on-failure -R PYTHON -E MR -C Release fail with

File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

All output here

Test project C:/Users/ofn77899/Dev/buildVC/builds/SIRF/build
    Start  7: REG_TESTS_PYTHON
1/8 Test  #7: REG_TESTS_PYTHON .....................***Failed    0.09 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose

    Start  8: REG_PYTHON_ALGEBRA
2/8 Test  #8: REG_PYTHON_ALGEBRA ...................***Failed    1.55 sec
E
======================================================================
ERROR: test_algebra (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_algebra
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Registration\pReg\tests\test_algebra.py", line 22, in <module>
    import sirf.Reg as reg
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\Reg.py", line 28, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 11: PET_TESTS_PYTHON
3/8 Test #11: PET_TESTS_PYTHON .....................***Failed    0.10 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose

    Start 12: PET_PYTHON_ALGEBRA
4/8 Test #12: PET_PYTHON_ALGEBRA ...................***Failed    1.62 sec
E
======================================================================
ERROR: test_algebra (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_algebra
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\xSTIR\pSTIR\tests\test_algebra.py", line 22, in <module>
    import sirf.STIR as pet
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\STIR.py", line 39, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 13: PET_PYTHON_OBJFUN
5/8 Test #13: PET_PYTHON_OBJFUN ....................***Failed    1.48 sec
E
======================================================================
ERROR: test_ObjectiveFunction (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_ObjectiveFunction
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\sirf_b2\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\xSTIR\pSTIR\tests\test_ObjectiveFunction.py", line 22, in <module>
    import sirf.STIR as pet
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\STIR.py", line 39, in <module>
    from sirf import SIRF
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

    Start 16: SYN_TEST_GADGETRON_TO_NIFTI_PYTHON
6/8 Test #16: SYN_TEST_GADGETRON_TO_NIFTI_PYTHON ...***Failed    1.49 sec
Traceback (most recent call last):
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Synergistic\pSyn\tests\tests_syn.py", line 3, in <module>
    from sirf.SIRF import ImageData
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

    Start 17: SYN_TEST_STIR_TO_NIFTI_PYTHON
7/8 Test #17: SYN_TEST_STIR_TO_NIFTI_PYTHON ........***Failed    1.48 sec
Traceback (most recent call last):
  File "C:\Users\ofn77899\Dev\buildVC\sources\SIRF\src\Synergistic\pSyn\tests\tests_syn.py", line 3, in <module>
    from sirf.SIRF import ImageData
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\SIRF.py", line 36, in <module>
    import sirf.pysirf as pysirf
  File "C:\Users\ofn77899\Dev\buildVC\INSTALL\python\sirf\pysirf.py", line 13, in <module>
    from . import _pysirf
ImportError: DLL load failed while importing _pysirf: The specified module could not be found.

    Start 19: SYN_TEST_PYTHON
8/8 Test #19: SYN_TEST_PYTHON ......................***Failed    0.12 sec
C:\Apps\miniconda3\envs\sirf_b2\python.exe: No module named nose


0% tests passed, 8 tests failed out of 8

Total Test time (real) =   8.16 sec

The following tests FAILED:
          7 - REG_TESTS_PYTHON (Failed)
          8 - REG_PYTHON_ALGEBRA (Failed)
         11 - PET_TESTS_PYTHON (Failed)
         12 - PET_PYTHON_ALGEBRA (Failed)
         13 - PET_PYTHON_OBJFUN (Failed)
         16 - SYN_TEST_GADGETRON_TO_NIFTI_PYTHON (Failed)
         17 - SYN_TEST_STIR_TO_NIFTI_PYTHON (Failed)
         19 - SYN_TEST_PYTHON (Failed)
Errors while running CTest
@paskino
Copy link
Contributor Author

paskino commented Jun 22, 2022

The environment variables PATH and PYTHONPATH seem to be correct.

PATH

My $Env:Path is

C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;C:\Apps\miniconda3\envs\sirf_b2;C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin;C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin;C:\Apps\miniconda3\envs\sirf_b2\Library\bin;C:\Apps\miniconda3\envs\sirf_b2\Scripts;C:\Apps\miniconda3\envs\sirf_b2\bin;C:\Apps\miniconda3\condabin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\ProgramData\chocolatey\bin;C:\WINDOWS\System32\OpenSSH;C:\Program Files\Docker\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Apps\Vagrant\bin;C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps;C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\ofn77899\AppData\Local\Programs\Git\cmd;C:\Apps\GitHub CLI;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2;C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;C:\Apps\boost_1_72_0\lib64-msvc-14.2

PYTHONPATH

C:\Users\ofn77899\Dev\buildVC\INSTALL\python;C:\Users\ofn77899\Dev\buildVC\INSTALL\python;C:\Users\ofn77899\Dev\buildVC\INSTALL\python;

@evgueni-ovtchinnikov
Copy link
Contributor

@paskino You probably need to add C:\Users\ofn77899\Dev\buildVC\INSTALL\python to PATH to help Python find _pysirf.pyd and other SIRF Python dll's.

@evgueni-ovtchinnikov
Copy link
Contributor

@paskino regarding MR (and some SYN) tests: can you not run Gadgetron on SIRF VM?

@paskino
Copy link
Contributor Author

paskino commented Jun 22, 2022

Thanks @evgueni-ovtchinnikov , I tried to add to PATH with

$Env:Path="C:\Users\ofn77899\Dev\buildVC\INSTALL\python;$Env:Path"
(sirf_b2) PS C:\Users\ofn77899\Dev\buildVC\builds\SIRF\build> $Env:Path                                                 C:\Users\ofn77899\Dev\buildVC\INSTALL\python;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin;
C:\Apps\miniconda3\envs\sirf_b2;
C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin;
C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin;
C:\Apps\miniconda3\envs\sirf_b2\Library\bin;
C:\Apps\miniconda3\envs\sirf_b2\Scripts;
C:\Apps\miniconda3\envs\sirf_b2\bin;
C:\Apps\miniconda3\condabin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;
C:\WINDOWS\system32;C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\WINDOWS\System32\WindowsPowerShell\v1.0;
C:\ProgramData\chocolatey\bin;
C:\WINDOWS\System32\OpenSSH;
C:\Program Files\Docker\Docker\resources\bin;
C:\ProgramData\DockerDesktop\version-bin;
C:\Apps\Vagrant\bin;
C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps;
C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin;
C:\Users\ofn77899\AppData\Local\Programs\Git\cmd;
C:\Apps\GitHub CLI;C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2;
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2;
C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps;
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW;
C:\Apps\boost_1_72_0\lib64-msvc-14.2

but still get the same error.

@KrisThielemans
Copy link
Member

I think you have to compare your 2 paths from within python, as it'll change where it finds DLLs compare to where the executable finds them. I still think it's a conflict between the HDF5/FFTW/Boost library used by SIRF and the ones installed by conda and hence used by conda's python.

See also https://bugs.python.org/issue43173 for some info on where it finds DLLs

@paskino
Copy link
Contributor Author

paskino commented Jun 22, 2022

Just to be clear, I did not install HDF5, FFTW or Boost with conda

conda list
# packages in environment at C:\Apps\miniconda3\envs\sirf_b2:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.6.15            h5b45459_0    conda-forge
certifi                   2022.6.15       py310h5588dad_0    conda-forge
cmake                     3.23.2               h39d44d4_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
deprecation               2.1.0              pyh9f0ad1d_0    conda-forge
docopt                    0.6.2                      py_1    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
gettext                   0.19.8.1          ha2e2712_1008    conda-forge
glib                      2.70.2               h7755175_4    conda-forge
glib-tools                2.70.2               h7755175_4    conda-forge
gst-plugins-base          1.20.3               he07aa86_0    conda-forge
gstreamer                 1.20.3               hdff456e_0    conda-forge
icu                       70.1                 h0e60522_0    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
jpeg                      9e                   h8ffe710_1    conda-forge
kiwisolver                1.4.3           py310h476a331_0    conda-forge
krb5                      1.19.3               h1176d77_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      3.0                  h0e60522_0    conda-forge
libblas                   3.9.0              15_win64_mkl    conda-forge
libcblas                  3.9.0              15_win64_mkl    conda-forge
libclang                  14.0.5          default_h77d9078_0    conda-forge
libclang13                14.0.5          default_h77d9078_0    conda-forge
libdeflate                1.12                 h8ffe710_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libflang                  5.0.0           h6538335_20180525    conda-forge
libglib                   2.70.2               h3be07f2_4    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
liblapack                 3.9.0              15_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libopenblas               0.3.20          pthreads_hc469a61_0    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libtiff                   4.4.0                h2ed3b44_1    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.2.2                h57928b3_0    conda-forge
libwebp-base              1.2.2                h8ffe710_1    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.12               h8ffe710_1    conda-forge
llvm-meta                 5.0.0                         0    conda-forge
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
matplotlib                3.4.3           py310h5588dad_2    conda-forge
matplotlib-base           3.4.3           py310h79a7439_2    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
nose                      1.3.7                   py_1006    conda-forge
numpy                     1.22.4          py310hed7ac4c_0    conda-forge
openblas                  0.3.20          pthreads_h543f93c_0    conda-forge
openjpeg                  2.4.0                hb211442_1    conda-forge
openmp                    5.0.0                    vc14_1    conda-forge
openssl                   1.1.1p               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pcre                      8.45                 h0e60522_0    conda-forge
pillow                    9.1.1           py310h767b3fd_1    conda-forge
pip                       22.1.2             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.4          py310hbabf5d4_1    conda-forge
pyqt5-sip                 12.9.0          py310h8a704f9_1    conda-forge
python                    3.10.5          h9a09f29_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
qt-main                   5.15.4               h467ea89_2    conda-forge
setuptools                62.6.0          py310h5588dad_0    conda-forge
sip                       6.5.1           py310h8a704f9_2    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.38.5               h8ffe710_0    conda-forge
tbb                       2021.5.0             h2d74725_1    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1             py310he2412df_3    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
zlib                      1.2.12               h8ffe710_1    conda-forge
zstd                      1.5.2                h6255e5f_1    conda-forge

@paskino
Copy link
Contributor Author

paskino commented Jun 22, 2022

>>> import os
>>> for el in os.environ['PATH'].split(';'):
...     print (el)
...
C:\Users\ofn77899\Dev\buildVC\INSTALL\python
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Users\ofn77899\Dev\buildVC\INSTALL\bin
C:\Apps\miniconda3\envs\sirf_b2
C:\Apps\miniconda3\envs\sirf_b2\Library\mingw-w64\bin
C:\Apps\miniconda3\envs\sirf_b2\Library\usr\bin
C:\Apps\miniconda3\envs\sirf_b2\Library\bin
C:\Apps\miniconda3\envs\sirf_b2\Scripts
C:\Apps\miniconda3\envs\sirf_b2\bin
C:\Apps\miniconda3\condabin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp
C:\WINDOWS\system32
C:\WINDOWS
C:\WINDOWS\System32\Wbem
C:\WINDOWS\System32\WindowsPowerShell\v1.0
C:\ProgramData\chocolatey\bin
C:\WINDOWS\System32\OpenSSH
C:\Program Files\Docker\Docker\resources\bin
C:\ProgramData\DockerDesktop\version-bin
C:\Apps\Vagrant\bin
C:\Users\ofn77899\AppData\Local\Microsoft\WindowsApps
C:\Users\ofn77899\AppData\Local\Programs\Microsoft VS Code\bin
C:\Users\ofn77899\AppData\Local\Programs\Git\cmd
C:\Apps\GitHub CLI
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2
C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\Pscx\Apps
C:\Users\ofn77899\Dev\buildVC\INSTALL\FFTW
C:\Apps\boost_1_72_0\lib64-msvc-14.2

@paskino
Copy link
Contributor Author

paskino commented Jun 22, 2022

I made some progress, installing SIRF directly in the conda environment!

conda create --name sirf_b2 python numpy openblas matplotlib=3.4 docopt cmake deprecation nose -c conda-forge

cmake ../SIRF-SuperBuild -DSIRF_TAG=origin/master -DSTIR_TAG=origin/master \
  -DCIL_TAG=origin/master -DBUILD_Gadgetron=OFF -DUSE_SYSTEM_HDF5=OFF \
  -DUSE_SYSTEM_Boost=ON -DBoost_ROOT=C:\Apps\boost_1_72_0 \
  -DUSE_SYSTEM_FFTW=OFF \
  -DPYTHON_DEST_DIR=%CONDA_PREFIX%\lib\site-packages \
  -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%

Now the unittest fail trying to load a file, although the file in question exists in the appropriate directory.

sirf.Utilities.error: ??? '\'Error opening file "C:\\\\Apps\\\\miniconda3\\\\envs\\\\sirf_b3"\\\\share\\\\SIRF-3.3\\\\data\\\\examples\\\\PET\\\\thorax_single_slice\\\\emission.hv\\n\' exception caught at line 327 of C:\\Users\\ofn77899\\Dev\\buildVC3\\sources\\SIRF\\src\\xSTIR\\cSTIR\\cstir.cpp; the reconstruction engine output may provide more information'

@KrisThielemans
Copy link
Member

for el in os.environ['PATH'].split(';'):
... print (el)

and sys.path? (in your previous set-up)

@KrisThielemans
Copy link
Member

I made some progress, installing SIRF directly in the conda environment!

great. Are these lines the only difference?

-DPYTHON_DEST_DIR=%CONDA_PREFIX%\lib\site-packages \
  -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%

The file-not-found seems to be due to backslash issues (unless it's really not there). Likely the file-ath is the output of the following

os.path.join(examples_data_path('PET'),'thorax_single_slice','emission.hv')

What's the return value of examples_data_path('PET')? and of your SIRF_PATH environment variable? Content of my env_sirf.PS1 is

# need to be forward slashes, as otherwise cannot find files
$Env:SIRF_PATH = "C:/Users/krisf/Documents/devel/buildVC/SIRF-SuperBuild/sources/SIRF"

configured at

$Env:SIRF_PATH = "@SIRF_SOURCE_DIR@"

@paskino
Copy link
Contributor Author

paskino commented Jun 24, 2022

Looks like that installing in the conda environment works.

. C:\Apps\miniconda3\envs\sirf_b3\bin\env_sirf.PS1

ctest -C Release -R PET_ --output-on-failure             Test project C:/Users/ofn77899/Dev/buildVC3/builds/SIRF/build
    Start  9: PET_TESTS_CPLUSPLUS_1
1/5 Test  #9: PET_TESTS_CPLUSPLUS_1 ............   Passed   53.46 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
2/5 Test #10: PET_TESTS_CPLUSPLUS_4 ............   Passed   54.96 sec
    Start 11: PET_TESTS_PYTHON
3/5 Test #11: PET_TESTS_PYTHON .................   Passed  123.90 sec
    Start 12: PET_PYTHON_ALGEBRA
4/5 Test #12: PET_PYTHON_ALGEBRA ...............   Passed    8.65 sec
    Start 13: PET_PYTHON_OBJFUN
5/5 Test #13: PET_PYTHON_OBJFUN ................   Passed    1.67 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 242.76 sec

@KrisThielemans
Copy link
Member

ah, you forgot to source the PS1?

@paskino
Copy link
Contributor Author

paskino commented Jun 24, 2022

No, I did not use PowerShell, rather command prompt. So I had added the environment variables by hand.

@KrisThielemans
Copy link
Member

ok, but what were there values then? Is there a difference in what they should be between PS and CMD?

Also, is the same cause for your original problems (i.e. do yo really have to install in the conda dirs?)

@paskino
Copy link
Contributor Author

paskino commented Jun 24, 2022

Yes, outside of conda dirs neither PowerShell, nor CMD worked.

I had an issue with setting the path with " on CMD, but now, without double quotes, it seems to be working fine.

set SIRF_PATH=C:/Users/ofn77899/Dev/buildVC3/sources/SIRF
set SIRF_INSTALL_PATH=C:\Apps\miniconda3\envs\sirf_b3
ctest -C Release -R PET_ --output-on-failure
Test project C:/Users/ofn77899/Dev/buildVC3/builds/SIRF/build
    Start  9: PET_TESTS_CPLUSPLUS_1
1/5 Test  #9: PET_TESTS_CPLUSPLUS_1 ............   Passed   56.74 sec
    Start 10: PET_TESTS_CPLUSPLUS_4
2/5 Test #10: PET_TESTS_CPLUSPLUS_4 ............   Passed   63.69 sec
    Start 11: PET_TESTS_PYTHON
3/5 Test #11: PET_TESTS_PYTHON .................   Passed  105.89 sec
    Start 12: PET_PYTHON_ALGEBRA
4/5 Test #12: PET_PYTHON_ALGEBRA ...............   Passed    9.77 sec
    Start 13: PET_PYTHON_OBJFUN
5/5 Test #13: PET_PYTHON_OBJFUN ................   Passed    1.84 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) = 760.05 sec

@KrisThielemans
Copy link
Member

ok. could make a env_sirf.cmd.in then.

I think this indicates we do miss something in our env_sirf.PS1 though. Just not sure what it is. The only way I now to track it down is what I have suggested already.

@KrisThielemans
Copy link
Member

Aside from that, if you go the conda way, I would highly recommend using conda for fftw, boost and itk. Ideally HDF5 as well but that might fail.

@paskino
Copy link
Contributor Author

paskino commented Jul 4, 2022

I've just read here that

Note that on Windows, from Python 3.9 onwards, DLL paths must be added by calling os.add_dll_directory() within Python rather than by setting PATH.

Indeed, in my tests I'm using python 3.10. This would explain all my failed attempts in using environment variables.

More on this

https://docs.python.org/3/library/os.html#os.add_dll_directory
https://stackoverflow.com/questions/67805339/is-the-function-os-add-dll-directory-adding-directories-permanently

@paskino
Copy link
Contributor Author

paskino commented Aug 11, 2022

@KrisThielemans
Copy link
Member

KrisThielemans commented Aug 11, 2022

that cookie stuff referenced there is just ridiculously complicated. sigh. I guess you'd have to make a persistent cookie somehow (no idea how to do that in Python), as we surely don't want to tell people to call add_dll_directory explicitly like that whenever they want to import sirf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants