Setup.py not copying __init__.py into package.

[rh0dium]

Hi there,

This is an old issue but shows when you run -Wall. Perhaps you can fix it - It looks like setup.py is not copying __init__.py into the package when run pip install ndg_httpsclient

/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py:186: ImportWarning: Not importing directory '/Users/sklass/.virtualenvs/axis/lib/python2.7/site-packages/ndg': missing __init__.py file, filename, etc = imp.find_module(subname, path)

[philipkershaw]

Hello - I can't replicate this unfortunately. Can you provide some more info?

I've tried on Mac Yosemite with Python 2.7 and CentOS 6.6 with Python 2.6.

[rh0dium]

Mac Yosemite Python 2.7

mkvirtualenv -p /usr/local/bin/python2.7 ndg
pip install ndg-httpsclient
ls ~/.virtualenvs/ndg/lib/python2.7/site-packages/ndg/__init__.py
ls: /Users/sklass/.virtualenvs/ndg/lib/python2.7/site-packages/ndg/__init__.py: No such file or directory

Hope this helps

[philipkershaw]

Thanks - now I understand. I've been blissfully unaware as though the __init__.py file is not there the relevant packages and modules still load. e.g.

$ python
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ndg
>>> import ndg.httpsclient
>>> dir (ndg.httpsclient)
['__author__', '__builtins__', '__contact__', '__copyright__', '__date__', '__doc__', '__file__', '__license__', '__name__', '__package__', '__path__', '__revision__']
>>> from ndg.httpsclient.https import HTTPSConnection
>>> help(HTTPSConnection)

I'm assuming this is some effect of the namespace package magic. Are there explicit things you can't do because the __init__.py is missing?

[rh0dium]

Nope just a warning ;) I'm assuming something is tripping it but not sure what. It's an easy enough fix as the repo has it.

[rh0dium]

Any updates on this?

[philipkershaw]

Sorry for the slow response. I've been researching this further. There seem to be two main approaches for namespace packages - using pkg_resources and pkg_util. Looking at an equivalent case like the Paste package, it has a whole family of sub-packages under the paste namespace. The __init__.py uses a strategy if not pkg_resources use pkg_util:

https://bitbucket.org/ianb/pastedeploy/src/f30a7d518c6a79fcddfbe3f622337f81e41cb6a5/paste/__init__.py?at=default

When I install PasteDeploy, I get the same behaviour as ndg_httpsclient: the __init__.py is omitted. It's still not clear to me whether this is intended behaviour for namespace packages but clearly the code works.

[nelsonvarela]

Still getting this error. Code is working but the error warning needs to be fixed.

[lukas-gitl]

How is this still an issue? :/

[aaronrudkin]

Hi. I'm getting this in some code that includes a module from another package, but presumably something further up the line is including ndg. The warning message even comes up if I explicitly suppress all warnings. Makes things difficult to debug by clogging up logs. Python2.7 running on an old CentOS.

[philipkershaw]

I'll see if I can take a look at this again

[lukas-gitl]

How we work around this issue (really ugly):

# turn all warnings into errors
import warnings
warnings.simplefilter('error')

# -- suppress import warnings for ndg
with warnings.catch_warnings(record=True) as ws:
    warnings.simplefilter("always")
    # import everything that needs ndg
    # noinspection PyUnresolvedReferences
    import pytz
    # noinspection PyUnresolvedReferences
    import rollbar
    expected_error = (
        "Not importing directory '", "/ndg': missing __init__.py"
    )
    for w in ws:  # pragma: no cover
        assert issubclass(w.category, ImportWarning)
        assert str(w.message).startswith(expected_error[0]), str(w.message)
        assert str(w.message).endswith(expected_error[1]), str(w.message)

[mmautner]

Here is how to reproduce using Docker, using Python 2.7.6:

Dockerfile

FROM ubuntu:14.04
RUN apt-get -yqq update && apt-get -yqq install python-pip python-dev libffi-dev libxmlsec1-openssl libssl-dev
RUN pip install ndg-httpsclient==0.4.2

Build + run and you'll reproduce the warning we're all seeing:

$ docker build . -t ndgtest
$ docker run -it ndgtest python -W error -c "from pkg_resources import iter_entry_points"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2760, in <module>
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 738, in subscribe
    callback(dist)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2760, in <lambda>
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2316, in activate
    declare_namespace(pkg)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1922, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1874, in _handle_ns
    loader = importer.find_module(packageName)
  File "/usr/lib/python2.7/pkgutil.py", line 186, in find_module
    file, filename, etc = imp.find_module(subname, path)
ImportWarning: Not importing directory '/usr/local/lib/python2.7/dist-packages/ndg': missing __init__.py

Please re-open this issue @philipkershaw

Looks like this issue is attributable to the contents of ndg/__init__.py deviating from the prescribed contents