How should I correctly lint an implicit namespace package?

[dalai4git]

I am trying to lint a namespace package (PEP420), but I can't get it to work without getting import-error. I am not sure my approach is correct, so I'd appreciate some pointers. In the example below I just want to lint org.company.package2.

Steps to reproduce

# mkdir -p org/company/package1
# mkdir -p org/company/package2
# touch org/company/package1/__init__.py
# echo "class Module:\n    pass\n" >> org/company/package1/module.py
# touch org/company/package2/__init__.py
# echo "from org.company.package1.module import Module\n" >> org/company/package2/module.py
# python
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from org.company.package1.module import Module
>>> 

Current behavior

Specifying the package does not work at all

# pylint org.company.package2
No config file found, using default configuration
************* Module org.company.package2
F:  1, 0: No module named org.company.package2 (fatal)

Specifying the path gives me an import-error

# pylint -r -n org/company/package2 
No config file found, using default configuration
************* Module package2.module
C:  2, 0: Trailing newlines (trailing-newlines)
C:  1, 0: Missing module docstring (missing-docstring)
E:  1, 0: Unable to import 'org.company.package1.module' (import-error)
W:  1, 0: Unused Module imported from org.company.package1.module (unused-import)

pylint --version output

pylint 1.6.5, 
astroid 1.4.9
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609]

With master version of pylint and astroid

This seems to work correctly but is not what I want

# ~/workspace/venv/bin/pylint org       
No config file found, using default configuration
************* Module company.package1.module
C:  3, 0: Trailing newlines (trailing-newlines)
C:  1, 0: Missing module docstring (missing-docstring)
C:  1, 0: Missing class docstring (missing-docstring)
R:  1, 0: Too few public methods (0/2) (too-few-public-methods)
************* Module company.package2.module
C:  2, 0: Trailing newlines (trailing-newlines)
C:  1, 0: Missing module docstring (missing-docstring)
W:  1, 0: Unused Module imported from org.company.package1.module (unused-import)

This doesn't seem to be doing anything

# ~/workspace/venv/bin/pylint org.company       
No config file found, using default configuration

and this apparently crashes

# ~/workspace/venv/bin/pylint org.company.package       
No config file found, using default configuration
Traceback (most recent call last):
  File "/home/dalai/workspace/venv/bin/pylint", line 9, in <module>
    load_entry_point('pylint==1.7.0', 'console_scripts', 'pylint')()
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/__init__.py", line 13, in run_pylint
    Run(sys.argv[1:])
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/lint.py", line 1300, in __init__
    linter.check(args)
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/lint.py", line 726, in check
    self._do_check(files_or_modules)
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/lint.py", line 837, in _do_check
    for descr in self.expand_files(files_or_modules):
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/lint.py", line 869, in expand_files
    self.config.black_list_re)
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/utils.py", line 894, in expand_modules
    modpath = _modpath_from_file(subfilepath, is_namespace)
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg/pylint/utils.py", line 833, in _modpath_from_file
    return modutils.modpath_from_file_with_callback(filename, is_package_cb=_is_package_cb)
  File "/home/dalai/workspace/venv/lib/python3.5/site-packages/astroid-1.5.0-py3.5.egg/astroid/modutils.py", line 310, in modpath_from_file_with_callback
    filename, ', \n'.join(sys.path)))
ImportError: Unable to find module for /home/dalai/workspace/org/company/package2/module.py in /home/dalai/workspace, 
., 
/home/dalai/workspace/venv/bin, 
/home/dalai/workspace/venv/lib/python3.5/site-packages/pylint-1.7.0-py3.5.egg, 
/home/dalai/workspace/venv/lib/python3.5/site-packages/editdistance-0.3.1-py3.5-linux-x86_64.egg, 
/home/dalai/workspace/venv/lib/python3.5/site-packages/astroid-1.5.0-py3.5.egg, 
/usr/lib/python35.zip, 
/usr/lib/python3.5, 
/usr/lib/python3.5/plat-x86_64-linux-gnu, 
/usr/lib/python3.5/lib-dynload, 
/home/dalai/workspace/venv/lib/python3.5/site-packages, 
/home/dalai/workspace/venv/lib/python3.5/site-packages/astroid-1.5.0-py3.5.egg/astroid/brain

[PCManticore]

Interesting, thank you for reporting this issue! I thought it was solved in the master branch. Why it is not what you want, can you clarify a bit?

[dalai4git]

I want to only lint the org.company.package2 package and not both. Pylint is being run in jenkins and ideally the report should be clean of warnings/errors from other (possibly external) packages. Currently I just use pylint 1.6.5, add __init__.py files automatically before running and ignore the import-errors. I was just wondering if I could simplify this with a newer version of pylint.

So if I understand correctly, implicit namespace packages are only supported by the version in the master branch and not the currently released one?

[brycepg]

I'm getting an error for implicit namespace packages on master. The workaround I currently have is making implicit namespace packages explicit.

I'll try to work on this issue

[tgpfeiffer]

With

pylint 2.5.3
astroid 2.4.2
Python 3.8.3 (default, May 17 2020, 14:48:56) [GCC]

I still see the problem. The minimal example from @dalai4git shows

$ pylint org 
************* Module org.company.package1.module
org/company/package1/module.py:3:0: C0305: Trailing newlines (trailing-newlines)
org/company/package1/module.py:1:0: C0114: Missing module docstring (missing-module-docstring)
org/company/package1/module.py:1:0: C0115: Missing class docstring (missing-class-docstring)
org/company/package1/module.py:1:0: R0903: Too few public methods (0/2) (too-few-public-methods)
************* Module org.company.package2.module
org/company/package2/module.py:2:0: C0305: Trailing newlines (trailing-newlines)
org/company/package2/module.py:1:0: C0114: Missing module docstring (missing-module-docstring)
org/company/package2/module.py:1:0: W0611: Unused Module imported from org.company.package1.module (unused-import)

$ pylint org/company 
************* Module company
org/company/__init__.py:1:0: F0010: error while code parsing: Unable to load file org/company/__init__.py:
[Errno 2] No such file or directory: 'org/company/__init__.py' (parse-error)

$ pylint org/company/package2
************* Module package2.module
org/company/package2/module.py:2:0: C0305: Trailing newlines (trailing-newlines)
org/company/package2/module.py:1:0: C0114: Missing module docstring (missing-module-docstring)
org/company/package2/module.py:1:0: E0401: Unable to import 'org.company.package1.module' (import-error)
org/company/package2/module.py:1:0: W0611: Unused Module imported from org.company.package1.module (unused-import)

Is this expected to be working in the latest release?