Test failures for CLI

[sebastic]

Followup for #734. The Debian package build for 3.0-rc1 still fails due to test failures, even with -m "not network", because the CLI related tests fail.

------------------------------ Captured log call -------------------------------
DEBUG    pyproj:crs.py:965 PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
DEBUG    pyproj:crs.py:965 PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
DEBUG    pyproj:transformer.py:81 PROJ_ERROR: proj_coordoperation_get_method_info: Object is not a DerivedCRS or BoundCRS
=============================== warnings summary ===============================
.pybuild/cpython3_3.8_pyproj/build/test/test_transformer.py::test_transformer_group__get_transform_crs
  /build/python-pyproj-3.0~rc1/.pybuild/cpython3_3.8_pyproj/build/pyproj/transformer.py:81: UserWarning: Best transformation is not available due to missing Grid(short_name=nl_nsgi_rdtrans2018.tif, full_name=, package_name=, url=https://cdn.proj.org/nl_nsgi_rdtrans2018.tif, direct_download=True, open_license=True, available=False)
    super().__init__(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
====== 18 failed, 292 passed, 24 deselected, 1 warnings in 111.28 seconds ======
E: pybuild pybuild:353: test: plugin distutils failed with: exit code=1: cd '/build/python-pyproj-3.0~rc1/.pybuild/cpython3_3.8_pyproj/build'; python3.8 -m pytest -v -m "not network"

Test output: pyproj.txt

[sebastic]

One problem seems to be that no pyproj executable is built:

# find . -name pyproj -print -exec ls -l {} \;   
./build/temp.linux-x86_64-3.9/pyproj
total 13104
-rw-r--r-- 1 root root  172656 Nov  3 05:18 _compat.o
-rw-r--r-- 1 root root 7106704 Nov  3 05:18 _crs.o
-rw-r--r-- 1 root root  785360 Nov  3 05:18 _datadir.o
-rw-r--r-- 1 root root  873760 Nov  3 05:18 _geod.o
-rw-r--r-- 1 root root  260032 Nov  3 05:18 _network.o
-rw-r--r-- 1 root root  181304 Nov  3 05:18 _sync.o
-rw-r--r-- 1 root root 2221728 Nov  3 05:18 _transformer.o
-rw-r--r-- 1 root root 1367256 Nov  3 05:18 database.o
-rw-r--r-- 1 root root  429944 Nov  3 05:18 list.o
./build/temp.linux-x86_64-3.8/pyproj
total 14816
-rw-r--r-- 1 root root  188296 Nov  3 05:19 _compat.o
-rw-r--r-- 1 root root 8208600 Nov  3 05:19 _crs.o
-rw-r--r-- 1 root root  880752 Nov  3 05:19 _datadir.o
-rw-r--r-- 1 root root  939896 Nov  3 05:19 _geod.o
-rw-r--r-- 1 root root  285944 Nov  3 05:20 _network.o
-rw-r--r-- 1 root root  199440 Nov  3 05:20 _sync.o
-rw-r--r-- 1 root root 2461976 Nov  3 05:19 _transformer.o
-rw-r--r-- 1 root root 1513096 Nov  3 05:19 database.o
-rw-r--r-- 1 root root  472024 Nov  3 05:19 list.o
./.pybuild/cpython3_3.9_pyproj/build/pyproj
total 5788
-rw-r--r-- 1 root root    2596 Nov  3 04:21 __init__.py
-rw-r--r-- 1 root root    6432 Nov  2 04:41 __main__.py
drwxr-xr-x 2 root root    4096 Nov  3 05:20 __pycache__
-rwxr-xr-x 1 root root   91688 Nov  3 05:18 _compat.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root 2936944 Nov  3 05:18 _crs.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    7467 Nov  2 04:41 _crs.pyi
-rwxr-xr-x 1 root root  350920 Nov  3 05:18 _datadir.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     257 Nov  2 04:41 _datadir.pyi
-rwxr-xr-x 1 root root  388072 Nov  3 05:18 _geod.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    1014 Mar 18  2020 _geod.pyi
-rwxr-xr-x 1 root root  137272 Nov  3 05:18 _network.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     193 Nov  2 04:41 _network.pyi
-rw-r--r-- 1 root root    2358 Nov  2 04:41 _show_versions.py
-rwxr-xr-x 1 root root   95264 Nov  3 05:18 _sync.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root      36 Nov  2 04:41 _sync.pyi
-rwxr-xr-x 1 root root  945888 Nov  3 05:18 _transformer.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    2657 Nov  2 04:41 _transformer.pyi
-rw-r--r-- 1 root root    2696 Nov  2 04:41 aoi.py
-rw-r--r-- 1 root root     534 May  2  2020 compat.py
drwxr-xr-x 3 root root    4096 Nov  3 05:20 crs
-rwxr-xr-x 1 root root  587024 Nov  3 05:18 database.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    1069 Nov  2 04:41 database.pyi
-rw-r--r-- 1 root root    3466 Nov  2 04:41 datadir.py
-rw-r--r-- 1 root root    3446 Nov  2 04:41 enums.py
-rw-r--r-- 1 root root     850 Nov  2 04:41 exceptions.py
-rw-r--r-- 1 root root   25099 Nov  2 04:41 geod.py
-rwxr-xr-x 1 root root  199904 Nov  3 05:18 list.cpython-39-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     187 Nov  2 04:41 list.pyi
-rw-r--r-- 1 root root    1864 Nov  2 04:41 network.py
-rw-r--r-- 1 root root   11082 Nov  2 04:41 proj.py
-rw-r--r-- 1 root root       0 Nov  2 04:41 py.typed
-rw-r--r-- 1 root root    8556 Nov  3 04:35 sync.py
-rw-r--r-- 1 root root   31559 Nov  2 04:41 transformer.py
-rw-r--r-- 1 root root    3308 Nov  2 04:41 utils.py
./.pybuild/cpython3_3.8_pyproj/build/pyproj
total 6372
-rw-r--r-- 1 root root    2596 Nov  3 04:21 __init__.py
-rw-r--r-- 1 root root    6432 Nov  2 04:41 __main__.py
drwxr-xr-x 2 root root    4096 Nov  3 05:20 __pycache__
-rwxr-xr-x 1 root root   97096 Nov  3 05:19 _compat.cpython-38-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root 3313312 Nov  3 05:19 _crs.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    7467 Nov  2 04:41 _crs.pyi
-rwxr-xr-x 1 root root  386216 Nov  3 05:19 _datadir.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     257 Nov  2 04:41 _datadir.pyi
-rwxr-xr-x 1 root root  408456 Nov  3 05:19 _geod.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    1014 Mar 18  2020 _geod.pyi
-rwxr-xr-x 1 root root  146224 Nov  3 05:20 _network.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     193 Nov  2 04:41 _network.pyi
-rw-r--r-- 1 root root    2358 Nov  2 04:41 _show_versions.py
-rwxr-xr-x 1 root root  101640 Nov  3 05:20 _sync.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root      36 Nov  2 04:41 _sync.pyi
-rwxr-xr-x 1 root root 1024632 Nov  3 05:19 _transformer.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    2657 Nov  2 04:41 _transformer.pyi
-rw-r--r-- 1 root root    2696 Nov  2 04:41 aoi.py
-rw-r--r-- 1 root root     534 May  2  2020 compat.py
drwxr-xr-x 3 root root    4096 Nov  3 05:20 crs
-rwxr-xr-x 1 root root  643008 Nov  3 05:19 database.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root    1069 Nov  2 04:41 database.pyi
-rw-r--r-- 1 root root    3466 Nov  2 04:41 datadir.py
-rw-r--r-- 1 root root    3446 Nov  2 04:41 enums.py
-rw-r--r-- 1 root root     850 Nov  2 04:41 exceptions.py
-rw-r--r-- 1 root root   25099 Nov  2 04:41 geod.py
-rwxr-xr-x 1 root root  214200 Nov  3 05:19 list.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root     187 Nov  2 04:41 list.pyi
-rw-r--r-- 1 root root    1864 Nov  2 04:41 network.py
-rw-r--r-- 1 root root   11082 Nov  2 04:41 proj.py
-rw-r--r-- 1 root root       0 Nov  2 04:41 py.typed
-rw-r--r-- 1 root root    8556 Nov  3 04:35 sync.py
-rw-r--r-- 1 root root   31559 Nov  2 04:41 transformer.py
-rw-r--r-- 1 root root    3308 Nov  2 04:41 utils.py
./pyproj
total 4340
-rw-r--r-- 1 1000 1000    2596 Nov  3 04:21 __init__.py
-rw-r--r-- 1 1000 1000    6432 Nov  2 04:41 __main__.py
-rw-r--r-- 1 root root  120426 Nov  3 05:17 _compat.c
-rw-r--r-- 1 1000 1000      78 Nov  2 04:41 _compat.pxd
-rw-r--r-- 1 1000 1000     157 Nov  2 04:41 _compat.pyx
-rw-r--r-- 1 root root 1799949 Nov  3 05:17 _crs.c
-rw-r--r-- 1 1000 1000    3617 Nov  2 04:41 _crs.pxd
-rw-r--r-- 1 1000 1000    7467 Nov  2 04:41 _crs.pyi
-rw-r--r-- 1 1000 1000   84757 Nov  3 04:35 _crs.pyx
-rw-r--r-- 1 root root  268233 Nov  3 05:17 _datadir.c
-rw-r--r-- 1 1000 1000     173 Nov  2 04:41 _datadir.pxd
-rw-r--r-- 1 1000 1000     257 Nov  2 04:41 _datadir.pyi
-rw-r--r-- 1 1000 1000    6462 Nov  2 04:41 _datadir.pyx
-rw-r--r-- 1 root root  329395 Nov  3 05:17 _geod.c
-rw-r--r-- 1 1000 1000    1405 May  2  2020 _geod.pxd
-rw-r--r-- 1 1000 1000    1014 Mar 18  2020 _geod.pyi
-rw-r--r-- 1 1000 1000   11065 Nov  2 04:41 _geod.pyx
-rw-r--r-- 1 root root  146869 Nov  3 05:17 _network.c
-rw-r--r-- 1 1000 1000     193 Nov  2 04:41 _network.pyi
-rw-r--r-- 1 1000 1000    1594 Nov  2 04:41 _network.pyx
-rw-r--r-- 1 1000 1000    2358 Nov  2 04:41 _show_versions.py
-rw-r--r-- 1 root root  124465 Nov  3 05:17 _sync.c
-rw-r--r-- 1 1000 1000      36 Nov  2 04:41 _sync.pyi
-rw-r--r-- 1 1000 1000     327 Nov  2 04:41 _sync.pyx
-rw-r--r-- 1 root root  707850 Nov  3 05:17 _transformer.c
-rw-r--r-- 1 1000 1000     701 Nov  2 04:41 _transformer.pxd
-rw-r--r-- 1 1000 1000    2657 Nov  2 04:41 _transformer.pyi
-rw-r--r-- 1 1000 1000   27040 Nov  2 04:41 _transformer.pyx
-rw-r--r-- 1 1000 1000    2696 Nov  2 04:41 aoi.py
-rw-r--r-- 1 1000 1000     999 Nov  2 04:41 base.pxi
-rw-r--r-- 1 1000 1000     534 May  2  2020 compat.py
drwxr-xr-x 2 1000 1000    4096 Nov  2 04:41 crs
-rw-r--r-- 1 root root  375006 Nov  3 05:17 database.c
-rw-r--r-- 1 1000 1000    1069 Nov  2 04:41 database.pyi
-rw-r--r-- 1 1000 1000   12968 Nov  2 04:41 database.pyx
-rw-r--r-- 1 1000 1000    3466 Nov  2 04:41 datadir.py
-rw-r--r-- 1 1000 1000    3446 Nov  2 04:41 enums.py
-rw-r--r-- 1 1000 1000     850 Nov  2 04:41 exceptions.py
-rw-r--r-- 1 1000 1000   25099 Nov  2 04:41 geod.py
-rw-r--r-- 1 root root  164908 Nov  3 05:17 list.c
-rw-r--r-- 1 1000 1000     187 Nov  2 04:41 list.pyi
-rw-r--r-- 1 1000 1000    1622 Nov  2 04:41 list.pyx
-rw-r--r-- 1 1000 1000    1864 Nov  2 04:41 network.py
-rw-r--r-- 1 1000 1000   20690 Nov  2 04:41 proj.pxi
-rw-r--r-- 1 1000 1000   11082 Nov  2 04:41 proj.py
-rw-r--r-- 1 1000 1000       0 Nov  2 04:41 py.typed
-rw-r--r-- 1 1000 1000    8556 Nov  3 04:35 sync.py
-rw-r--r-- 1 1000 1000   31559 Nov  2 04:41 transformer.py
-rw-r--r-- 1 1000 1000    3308 Nov  2 04:41 utils.py

The pyproj executable is only created when the install target is excuted, and this happens after the test target.

The procedure to build a Debian package comes down to:

python3 setup.py clean
python3 setup.py config
python3 setup.py build
python3 -m pytest -v -m "not network"
python3 setup.py install

This is similar to the autotools or cmake build systems with separate targets for the build steps.

[snowman2]

The procedure to build a Debian package comes down to

Is it possible to customize this process?

[sebastic]

Is it possible to customize this process?

Not really.

[snowman2]

Would it be helpful if I added a marker for the CLI tests so you can skip them?

[sebastic]

Yes, if we can't make the test pass easily, deselected them easily is a good alternative.

[snowman2]

@sebastic I noticed some failures are due to the old grids. It it possible to test without them?

[sebastic]

No, they are included in the proj-data package which provides the content of /usr/share/proj (proj.db among others), and is a strict dependency of libproj.