`pytest --help` fails in a partially configured app

[asottile-sentry]

having a difficult time narrowing down a minimal example -- the repo involved is https://github.com/getsentry/sentry

I have figured out why it is happening and the stacktrace for it:

full stacktrace with error

$ pytest --help
/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/trio/_core/_multierror.py:511: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
  warnings.warn(
Traceback (most recent call last):
  File "/Users/asottile/workspace/sentry/.venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 167, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/_pytest/helpconfig.py", line 152, in pytest_cmdline_main
    config._do_configure()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1037, in _do_configure
    self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/hooks.py", line 308, in call_historic
    res = self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pytest_django/plugin.py", line 367, in pytest_configure
    _setup_django(config)
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/pytest_django/plugin.py", line 238, in _setup_django
    blocking_manager = config.stash[blocking_manager_key]
  File "/Users/asottile/workspace/sentry/.venv/lib/python3.10/site-packages/_pytest/stash.py", line 80, in __getitem__
    return cast(T, self._storage[key])
KeyError: <_pytest.stash.StashKey object at 0x1066ab520>

basically what's happening is the setup is skipped here:

pytest-django/pytest_django/plugin.py

Lines 300 to 301 in 6cf63b6

	if options.version or options.help:
	return

normally it sets the thing that's being looked up here:

pytest-django/pytest_django/plugin.py

Line 358 in 6cf63b6

early_config.stash[blocking_manager_key] = DjangoDbBlocker(_ispytest=True)

which then fails to lookup here:

pytest-django/pytest_django/plugin.py

Line 238 in 6cf63b6

blocking_manager = config.stash[blocking_manager_key]

something about sentry's tests/conftest.py initializes enough of django that pytest-django takes over. but since the setup has been skipped it fails to set up properly. I suspect that #238 is playing poorly with something.

of note this worked before I upgraded pytest-django (I was previously on 4.4.0 and upgraded to 4.7.0 to get django 4.x support)

will try and narrow down a smaller reproduction...

[adamchainz]

That’s a fun one! Hopefully using config.stash.get() calls and acting only on non-None values will be enough to fix the issue...

[asottile-sentry]

here's a minimal case:

==> t.py <==
WAT = 1

==> tests/__init__.py <==

==> tests/conftest.py <==
import os

def pytest_configure():
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 't')
    from django.conf import settings
    settings.WAT