Python 3.9-dev build failure

[hugovk]

Between these two builds the 3.9-dev build started failing on Travis CI:

• Yesterday: https://travis-ci.org/github/python-pillow/Pillow/jobs/706015838
• Today: https://travis-ci.org/github/hugovk/Pillow/jobs/706476038

Finished processing dependencies for Pillow==7.3.0.dev0
python3 selftest.py
Traceback (most recent call last):
  File "/home/travis/build/hugovk/Pillow/selftest.py", line 6, in <module>
    from PIL import Image, features
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/Image.py", line 94, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/_imaging.py", line 8, in <module>
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/_imaging.py", line 7, in __bootstrap__
TypeError: exec_module() missing 1 required positional argument: 'module'
Makefile:59: recipe for target 'install-coverage' failed
make: *** [install-coverage] Error 1
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 495, in _importconftest
    return self._conftestpath2mod[key]
KeyError: PosixPath('/home/travis/build/hugovk/Pillow/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/python/3.9-dev/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/python/3.9-dev/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pytest/__main__.py", line 7, in <module>
    raise SystemExit(pytest.main())
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 105, in main
    config = _prepareconfig(args, plugins)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 257, in _prepareconfig
    return pluginmanager.hook.pytest_cmdline_parse(
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/helpconfig.py", line 90, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 836, in pytest_cmdline_parse
    self.parse(args)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1044, in parse
    self._preparse(args, addopts=addopts)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1001, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 899, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(early_config.known_args_namespace)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 441, in _set_initial_conftests
    self._try_load_conftest(anchor)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 447, in _try_load_conftest
    self._getconftestmodules(anchor)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 473, in _getconftestmodules
    mod = self._importconftest(conftestpath)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 520, in _importconftest
    self.consider_conftest(mod)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 575, in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 386, in register
    self.consider_module(plugin)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 581, in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 586, in _import_plugin_specs
    self.import_plugin(import_spec)
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/config/__init__.py", line 613, in import_plugin
    __import__(importspec)
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/_pytest/assertion/rewrite.py", line 152, in exec_module
    exec(co, module.__dict__)
  File "/home/travis/build/hugovk/Pillow/Tests/helper.py", line 14, in <module>
    from PIL import Image, ImageMath, features
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/Image.py", line 94, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/_imaging.py", line 8, in <module>
  File "/home/travis/virtualenv/python3.9-dev/lib/python3.9/site-packages/Pillow-7.3.0.dev0-py3.9-linux-x86_64.egg/PIL/_imaging.py", line 7, in __bootstrap__
TypeError: exec_module() missing 1 required positional argument: 'module'

3.9-dev is rebuilt from CPython daily. Diffing the logs, here's the CPython commits:

-platform     linux 3.9.0b4+ (heads/3.9:edeaf61, Jul  7 2020, 06:29:52)
+platform     linux 3.9.0b4+ (heads/3.9:1d1c574, Jul  8 2020, 07:05:21)

And the changes:

• https://github.com/python/cpython/compare/edeaf61b6827ab3a8673aff1fb7717917f08f003..1d1c5743400bdf384ec83eb6ba5b39a355d121e3

Nothing obvious to me, let's see if tomorrow's today's build fixes it.

[hugovk]

-platform     linux 3.9.0b4+ (heads/3.9:edeaf61, Jul  7 2020, 06:29:52)
+platform     linux 3.9.0b4+ (heads/3.9:1d1c574, Jul  8 2020, 07:05:21)

Still happening:

platform     linux 3.9.0b4+ (heads/3.9:e689789, Jul  9 2020, 07:57:24)

https://travis-ci.org/github/python-pillow/Pillow/jobs/706644607#L2279

[hugovk]

Reported to CPython: https://bugs.python.org/issue41268

[radarhere]

I find that installing setuptools 47.3.1 causes Python 3.9 to pass - see #4779

The 47.3.2 introduces https://github.com/pypa/setuptools/pull/2071/files - calling exec_module() with no arguments, just like in the error reported. So I believe that this is actually a problem between setuptools and Python 3.9.

[hugovk]

Good find! Please could you report this to setuptools?

[radarhere]

Okay, done - pypa/setuptools#2246

[hugovk]

I find that installing setuptools 47.3.1 causes Python 3.9 to pass - see #4779

The 47.3.2 introduces pypa/setuptools#2071 (files) - calling exec_module() with no arguments, just like in the error reported. So I believe that this is actually a problem between setuptools and Python 3.9.

Also confirmed the last passing log had setuptools 47.3.1 and the first failure had 49.1.0.

[radarhere]

I've concluded that the reason this is happening for the Python 3.9 build has nothing to do with Python 3.9 - it's that the Travis job has the latest version of setuptools from the time of creation baked in, and creating the job image for a beta version of Python 3.9 has a later version than the other Python jobs.

[radarhere]

Should we close this then?

[hugovk]

Yes!