hook specs are not set for other plugin hooks

[bubenkoff]

In pytest-bdd we have custom hooks, and they are regstered via:

def pytest_addhooks(pluginmanager):
    """Register plugin hooks."""
    pluginmanager.add_hookspecs(hooks)

this works fine for non-xdist runs
but with xdist we get this:

pytest_bdd/scenario.py:151: in _execute_scenario
    scenario=scenario,
.env/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:727: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
.env/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

hook = <_HookCaller 'pytest_bdd_before_scenario'>
methods = [<_pytest.vendored_packages.pluggy.HookImpl instance at 0x7efccaa97f38>]
kwargs = {'feature': <pytest_bdd.feature.Feature object at 0x7efcca995c50>, 'request': <FixtureRequest for <Function 'test_steps'>>, 'scenario': <pytest_bdd.feature.Scenario object at 0x7efcca927310>}

    self._inner_hookexec = lambda hook, methods, kwargs: \
>       _MultiCall(methods, kwargs, hook.spec_opts).execute()
E   AttributeError: '_HookCaller' object has no attribute 'spec_opts'

hook       = <_HookCaller 'pytest_bdd_before_scenario'>
kwargs     = {'feature': <pytest_bdd.feature.Feature object at 0x7efcca995c50>, 'request': <FixtureRequest for <Function 'test_steps'>>, 'scenario': <pytest_bdd.feature.Scenario object at 0x7efcca927310>}
methods    = [<_pytest.vendored_packages.pluggy.HookImpl instance at 0x7efccaa97f38>]

so looks like hook specs are not parsed and set when xdist's remote run happens

[RonnyPfannschmidt]

as far as i can tell we need a way to propperly install dependencies into pytest on remotes
it might be easier to provide a remote python with all needed plugins installed for the short term

[bubenkoff]

how exactly?

[RonnyPfannschmidt]

for example creating a virtualenv with the correct python + dependencies and then providing that as the python argument to the xspec

propper remote virtualenv orchestration is a hard topic that we dont want to solve prematurely/broken if possible

[bubenkoff]

im about running pytest with xdist with just -n (so popen, no need of the dependencies)
this is a bug on a different level

[RonnyPfannschmidt]

that means we changed the pluginmanager construction sequence and it leaves out setuptools, thats a hard one , will take a look this afternoon

[bubenkoff]

@RonnyPfannschmidt any chance for you to look at it?

[RonnyPfannschmidt]

i think i can manage saturday/sunday

[bh]

Any news?

[RonnyPfannschmidt]

unfortunately i couldnt work on it, my own computer broke down

[dubglan]

So currently xdist does not work with pytest 2.8+ if environment contains plugins with custom hooks and there's no workaround, is that correct?

[bubenkoff]

im affraid you're correct

On 14 November 2015 at 07:03, Gleb Dubovik notifications@github.com wrote:

So currently xdist does not work with pytest 2.8+ if environment contains
plugins with custom hooks and there's no workaround, is that correct?

—
Reply to this email directly or view it on GitHub
#1062 (comment).

Anatoly Bubenkov

[RonnyPfannschmidt]

We got something merged to feature which Contains a potential fix

Can someone try

[dubglan]

@RonnyPfannschmidt No, looks like it's still broken with latest pytest (features branch) and latest xdist (master):
E ============================= test session starts ==============================
E platform linux2 -- Python 2.7.6, pytest-2.8.3.dev1, py-1.4.30, pluggy-0.3.1
E rootdir: /nsx/nsx-dev-testing/pytest-traditions/.tox/py27/tmp/popen-gw2/test_outcomes_1__def_test_fail1/tests, inifile: pytest.ini
E plugins: xdist-v1.13.2.dev17-ng14f39a7, cov-2.2.0
...
E INTERNALERROR> File "/nsx/nsx-dev-testing/pytest-traditions/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/plu
ggy.py", line 333, in
E INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
E INTERNALERROR> AttributeError: '_HookCaller' object has no attribute 'spec_opts'

[RonnyPfannschmidt]

I'll look into details after my replacement PC is finished

[dubglan]

@RonnyPfannschmidt sorry for confusion, started debugging when understood that I might have installed pytest master branch instead of features. Either this or something else got fixed, now it works for me. Thank you! Do you know when can we expect the release?

[nicoddemus]

If you tried features, that might have went into 2.8.3 already. Could you try that version?

[dubglan]

@nicoddemus tried, doesn't work, master doesn't work either

[nicoddemus]

Duh sorry, my bad, late night here... if it's in features it will only be available in 2.9 unfortunately, which will still take awhile I think.

[dubglan]

That's a pity. We have a set of conventions around the use of the test runner as well as some integration code with internal systems (bugtracker, result reporting, style checking, etc.) - all of that wrapped in shared pytest plugins. Parallel test execution was one of the features why we chose pytest in the first place. Loosing either is a major regression for us. I guess we'll probably have to stick with 2.7.

Is there any chance of backporting the fix to 2.8 series? After all, regression was introduced in 2.8.

[nicoddemus]

Is there any chance of backporting the fix to 2.8 series?

I'm actually not sure what the fix was. Perhaps @RonnyPfannschmidt can answer that.

[RonnyPfannschmidt]

Not sure if that's a good idea over starting a timely 2.9 instead, we do want to try and stick to semver after all

[nicoddemus]

Could you point me to the PR for this change? I have missed it somehow.

Em qui, 26 de nov de 2015 03:48, Ronny Pfannschmidt <
notifications@github.com> escreveu:

Not sure if that's a good idea over starting a timely 2.9 instead, we do
want to try and stick to semver after all

—
Reply to this email directly or view it on GitHub
#1062 (comment).

[dubglan]

Well, I would suggest that this is rather a bug fix vs a new feature. But
that's entirely up to you.

I don't know what feature we're talking about here, but is it possible to
separate the bug fix part from new functionality - or is it more of a
complete rewrite which unintentionally fixed the issue? If given enough
coaching, we can help out and contribute.

Finally, most important question, to the best of your knowledge, what would
be the expected release timeframe for 2.9?

On Wednesday, November 25, 2015, Ronny Pfannschmidt <
notifications@github.com> wrote:

Not sure if that's a good idea over starting a timely 2.9 instead, we do
want to try and stick to semver after all

—
Reply to this email directly or view it on GitHub
#1062 (comment).

[RonnyPfannschmidt]

its includes a necessary api change thats required to propperly handle the case, so its not something easily backported

[RonnyPfannschmidt]

#1124

[nicoddemus]

Oh that change was related to this? Thanks @RonnyPfannschmidt

Well I completely agree that we shouldn't break semver and that should be released into 2.9. Should we start planning a 2.9 release then? Our changelog for 2.9.0.dev is a little slim at the moment, personally I would like to have some new features in place before 2.9.0, for example:

• Move py.code to pytest.code (Move py.code to pytest.code #103): this will make it easier to fix a number of lingering bugs (which can then be done in later big-fix releases like 2.9.1, 2.9.2, etc);
• re-enable all warnings by default (re-enable all warnings by default  #1191)
• Option to treat warnings as errors (Option to treat warnings as errors #1173)

Those are all doable in a short time frame, we would just have to agree on them for a 2.9 milestone and start working.

(I just realized this is getting out of topic, should we move this to a separate issue?)

[RonnyPfannschmidt]

im considering being less conservative about feature releases, but a discussion is needed in the end - a normal issue tracker is unsuitable for that

[nicoddemus]

im considering being less conservative about feature releases

What do you mean? That you don't have a problem to release them more often?

[RonnyPfannschmidt]

yes - but but that also has downsides, @hpk42 and me weight those different and i#d like to arrive at a shared conclusion before ending up doing something rash

[nicoddemus]

I see.

Well, created an issue at #1195, let's move the discussion there for now until we have a better place for it.

[nicoddemus]

By the way, I think this can be closed since it is already fixed in features, right?

[nicoddemus]

Hey,

A quick announcement of a small plugin I just published to PyPI,
pytest-drop-dup-tests.

Pytest by default will collect all tests from directories or files given
in the command-line. For example, if you execute:

py.test tests/unit tests/

Tests from tests/unit will appear twice, because they will be collected
again when pytest sees the tests directory in the command-line.

This plugin is intended for the cases where the user wants to run all tests
from tests/unit first and then the rest of the tests under tests,
without duplicates.

This plugin was born from the discussion taken in #1187[1].

It was also a chance to try the amazing cookicutter[2] and
cookiecutter-pytest-plugin[3] projects, got a working plugin with tests,
CI, README, license, tox, etc in under 10 minutes. :)

[1] #1187
[2] https://github.com/audreyr/cookiecutter
[3] https://github.com/pytest-dev/cookiecutter-pytest-plugin

Cheers,
Bruno.

[saulshanabrook]

Did #1124 fix this? I am running version 2.8.5 and just got:

============================= test session starts ==============================
platform linux -- Python 3.5.0, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
django settings: configs.settings (from environment variable)
rootdir: /app, inifile: pytest.ini
plugins: env-0.6.0, django-2.9.1, xdist-1.13.1, flake8-0.1, cov-2.2.0
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/main.py", line 88, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/xdist/dsession.py", line 494, in pytest_sessionstart
INTERNALERROR>     nodes = self.nodemanager.setup_nodes(putevent=self.queue.put)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/xdist/slavemanage.py", line 41, in setup_nodes
INTERNALERROR>     specs=self.specs)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/app/.heroku/python/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> AttributeError: '_HookCaller' object has no attribute 'spec_opts'

[saulshanabrook]

Looks like something fixed it, cause if I install the latest commit from features (6c17020 git+https://github.com/pytest-dev/pytest.git@6c170201d65049791271bda2f1cb0a889e256ff4), it does work.

[RonnyPfannschmidt]

@saulshanabrook the pr was indeed against the features branch - a fix will be released in 2.9

[nicoddemus]

@RonnyPfannschmidt shouldn't we close this? It is fixed in features after all.

[RonnyPfannschmidt]

Yes, we should also aim for a 2.9 before February