Giving `file_or_dir` twice collects tests twice

[htgoebel]

When passing the same directory twice to py.test, all tests are run twice. The same is true if passing a sub-directory and a parent-directory:

$ python3 -m py.test --collect-only tests/ | grep collected
collecting ... collected 338 items
$ python3 -m py.test --collect-only tests/ tests/ | grep collected
collecting ... collected 676 items
$ python3 -m py.test --collect-only tests/unit/ tests/ | grep collected
collecting ... collected 418 items

platform linux -- Python 3.4.3, pytest-2.8.3, py-1.4.30, pluggy-0.3.1 -- /usr/bin/python3

Rationale: I want to run the unit-test (which are in a certain directory) prior to he other ones. Thus I put the tests/unit directory in front, but this gave me duplicate tests.

[nicoddemus]

Thanks for the report @htgoebel!

I had used this behavior of collecting tests more than once from the command-line in order to run the same test multiple times to track down some flaky behavior, but I think my use-case is a little contrived and yours makes more sense IMO.

Any more thoughts, @hpk42 @RonnyPfannschmidt @The-Compiler?

[The-Compiler]

I usually do something like while (($? == 0)); do py.test -k 'test_foo'; done in that case... I guess I should write a pytest-repeat plugin 😆

I don't really have an opinion on this, I can find arguments for and against both behaviours, and I'd be fine with both.

[hpk42]

don't have a strong opinion either but if in doubt i'd rather not change the default and better introduce a --drop-duplicate option or so. Maybe it's even better to write a pytest-drop-duplicates plugin which, when installed, will drop duplicates? It's easy to implement through the pytest_collect_modifyitems hook, basically a plugin with 20 lines of code or so.

[RonnyPfannschmidt]

I'm for the plugin solution

[htgoebel]

Plugin sounds complicated. So I'm for a plugin keep-duplicates.

[RonnyPfannschmidt]

a propperly separate keep-duplicates plugin would be impossible to do cleanly since the duplicates would be gone by the time it gets to the plugin

[nicoddemus]

Making a plugin is actually really easy using the cookiecutter-pytest-plugin, in fact just made one myself to quickly see it in action.

@htgoebel, could you try it out? If all is well I will publish it to PyPI, it still needs more tests and README. To install it use:

pip install git+https://github.com/nicoddemus/pytest-drop-dup-tests.git

No other configuration is necessary.

[RonnyPfannschmidt]

i'm closing this as working as intended given that @nicoddemus already finished a plugin

[nicoddemus]

Btw kudos to @hackebrot, it took me 10 minutes to have that plugin working, well done! cookiecutter-pytest-plugin makes it really simple to create new plugins, covering a lot of the boiler plate of setting up a new repository, setup.py, C.I., etc etc. Highly recommended. 👍

[htgoebel]

@nicoddemus The plugin does not show any effect here. I still get duplicates.

[nicoddemus]

Can you show me the output of a sample session?

[htgoebel]

It is the same as in my very fist comment here. I used --traceconfig to verified the plugin is loaded:

$ rm -rf /tmp/pytest-* ; python3 -m py.test --collect-only --traceconfig tests/unit/ tests/ | grep -P 'collected|-dup-'
  pytest-drop-dup-tests-0.1.0 at /home/xxx/.local/lib/python3.4/site-packages/pytest_drop_dup_tests.py
    drop-dup-tests      : /home/xxx/.local/lib/python3.4/site-packages/pytest_drop_dup_tests.py
plugins: drop-dup-tests-0.1.0
collected 418 items