Only installing namespaced packages, can't run system tests

[waprin]

I am trying to rebase my PR and get it all up to date.

I am having trouble running the system tests. Usually I do:

pip install -e . 

To install the package locally in 'editable' mode. However, that's only installing logging and pubsub (those marked as namespaced_packages in `setup.py) into my virtualenv. So when I run the system tests, it can't import bigquery.

Not sure if the repo needs to be fixed or my workflow do, still wrapping my head around how the namespaced pacakges are supposed to work. Is there a reason it's only logging and pubsub in there currently?

[tseaver]

@waprin is this in a "fresh" virtualenv?

[tseaver]

Since #2304 was merged, if you have tox installed on your path, you can now run just the BigQuery system tests via:

$ tox -e system-tests -- bigquery

[daspecster]

@tseaver, @dhermes I'm also running into the same issue now.

Earlier this week I had to change a test script I have to from google.cloud import vision.
This doesn't seem to work now, and I'm having the same issue as @waprin. I tried several new venvs and I haven't found the cause yet.

[daspecster]

I think I figured it out.

Reference: http://stackoverflow.com/questions/38918246/why-does-setup-py-sweeps-the-content-of-the-namespace-before-installing/38919020#38919020

I'm making a PR for this now.

[daspecster]

Actually nevermind, my idea didn't work to solve this.

@waprin I believe the pubsub and logging namespaces appear because of the gax and grpc dependencies get installed for them.

[waprin]

Ok the system test can be run that way so that works though I'm still a bit confused as to how it all gets installed, will figure it out, thanks!

[dhermes]

I'd be interested what the PyPA member has to say.

@jonparrott It seems impossible to install namespace packages in "development" / "editable" mode. Is this not the case?

---

@waprin I don't personally use "editable" mode, but did notice that system tests were becoming more difficult to run, hence #2304. Sorry for the problem and thanks for bringing it up. I think it's out of our hands (in the sense that setuptools / namespace packages / "editable" mode are the b0rken thing), but hope @jonparrott will contradict me on that.

[theacodes]

Correct - setuptools does not handle editable installs for namespace packages correctly. This is tracked here with a legendary issue on pip here (not surprised to see @tseaver there).

I've offered to do the work to fix this but I need some guidance from PyPA before I start hacking at it.

[tseaver]

@jonparrott it is arguable that pip is broken in general about installing distributions with namespace packages (coercing them into a single, shared tree). It has never worked to mix installations between pip and setuptools when packages installed by each share a namespace.

[theacodes]

@tseaver fair. :)