Modules in namespace packages are not imported correctly

[craig-reahl]

I have found that Nose 1.3.4(and 1.3.7) does not import tests,for installed packages, that are in namespace packages correctly.

Versions:
nose==1.3.4
python 3.4
Ubuntu 14.04.1 LTS

Consider this directory structure:

├── my_tests
│   ├── __init__.py
│   └── my_package
│        ├── __init__.py
│        └── some_test.py
│
├── my_tests_with_namespace
│   ├── __init__.py
│   └── my_package_in_ns
│        ├── __init__.py
│        └── some_test_ns.py
└── setup.py

The my_tests and my_tests_with_namespace directories(which are both python packages since they have __init__.py)
both contain tests inside them. The __init__.py of my_tests_with_namespace however declares a namespace.

-----TEST SNIPPET(some_test.py) BEGIN---------

class MyTestClass:
    pass

def other_test_no_ns():
        import my_tests.my_package.some_test
        assert MyTestClass is my_tests.my_package.some_test.MyTestClass, \
               '%s is not %s' % (MyTestClass, my_tests.my_package.some_test.MyTestClass)

-----TEST SNIPPET(some_test_ns.py) END---------

-----TEST SNIPPET(some_test_ns.py) BEGIN---------

class MyTestClassNS:
    pass

def other_test():
        import my_tests_with_namespace.my_package_in_ns.some_test_ns
        assert MyTestClassNS is my_tests_with_namespace.my_package_in_ns.some_test_ns.MyTestClassNS, \
               '%s is not %s' % (MyTestClassNS, my_tests_with_namespace.my_package_in_ns.some_test_ns.MyTestClassNS)

-----TEST SNIPPET(some_test_ns.py) END---------

Running these commands from the source directory works fine, as the tests are executed and pass:

:~/develop$ nosetests my_tests
:~/develop$ nosetests my_tests_with_namespace

Now I create an sdist(?egg?)and create and activate into a new virtualenv elsewhere.
Next, I install my project (myproject) using pip.

-----SAMPLE of commands BEGIN---------

python setup.py sdist --dist-dir=/tmp/
cd /tmp
virtualenv cs_virtualenv --python=/usr/bin/python3.4
source cs_virtualenv/bin/activate
pip install --no-index --find-links=/tmp/ myproject
pip install nose
nosetests my_tests
nosetests my_tests_with_namespace

-----SAMPLE of commands END---------

When running the same nosetest commands, but this time against the installed package, the test for the other_test() fails:

======================================================================
FAIL: some_test_ns.other_test
----------------------------------------------------------------------
AssertionError: <class 'some_test_ns.MyTestClassNS> is not <class 'my_tests_with_namespace.my_package_in_ns.some_test_ns.MyTestClassNS'>

The issue I am raising has to do with the fact that for installed tests, the same module gets imported twice with different names, which my test illustrate.

(Usecase: wanting to run test with devpi and tox for installed packages)

[craig-reahl]

Reply on the mailing list by: John Szakmeister

It could definitely be a Nose issue here. What strikes me here
is the name of the classes in the repr(). The first class, the
local MyTestClassNS, has an oddball name. It's completely
missing the "my_tests_with_namespace.my_package_in_ns" portion
of the name. It's possible that this is the result of something
that Nose is doing, but it's pretty odd. At the moment, it's
not clear to me what is at fault.

[F30]

We have a similar issue here, though it seems to occur in the exact opposite case: Not with the installed packages, but in the source directory.

Our structure roughly looks like that:

foo
├── src
│   ├── bar
│   │   ├── datalayer
│   │   │   ├── config
│   │   │   │   ├── __init__.py
│   │   │   │   ├── util.py
│   │   │   │   └── [...]
│   │   │   ├── __init__.py
│   │   │   ├── lib
│   │   │   │   ├── __init__.py
│   │   │   │   ├── sqlalchemy
│   │   │   │   │   ├── __init__.py
│   │   │   │   │   └── [...]
│   │   │   │   └── [...]
│   │   │   ├── model
│   │   │   │   ├── __init__.py
│   │   │   │   └── [...]
│   │   │   └── [...]
│   │   └── [...]
│   └── [...]
├── test
│   ├── __init__.py
│   └── [...]
└── [...]

bar's __init__.py uses traditional namespace packages, but that doesn't appear to influence the problem. Otherweise, there's no namespacing going on. The __init__.py files of config and model both import something from the (actually external) sqlalchemy package.

Now, when running nose from the top level (foo) or src directory, it fails with ImportErrors for sqlalchemy submodules. Everything works alright when starting from the test directory.

This happens because when running the test discovery for config and model, a module called sqlalchemy is already present – but it is the one from lib, completely omitting its namespacing! It obviously doesn't have the relevant submodules. However, a module with the complete path, namely bar.datalayer.lib.sqlalchemy happens to be in sys.modules as well.

[jszakmeister]

Do you think one of you can take a stab at fixing the issue? I imagine the problem is likely in nose.loader.

[lnshi]

hard to believe this is still open!!! ppl should really run away from this lib