pytest explodes if getcwd fails

[bukzor]

Demonstration below.

While in my toy example it's quite clear what the issue is, the fact that it doesn't show the original error, or even have the erroneous code anywhere on the stack means that it's extremely hard to find the source of this issue in a real project.

$ cat test.py
def test_foo(tmpdir):
    tmpdir.chdir()
    tmpdir.remove()

    assert False, 'naw'

$ py.test test.py
=========================================================================================== test session starts ===========================================================================================
platform linux2 -- Python 2.7.6, pytest-2.8.4, py-1.4.30, pluggy-0.3.1 -- /nail/home/buck/trees/yelp/pip-faster/venv-venv_update/bin/python2.7
cachedir: ../.cache
rootdir: /nail/home/buck/trees/yelp/pip-faster, inifile: pytest.ini
plugins: xdist-1.13.1, timeout-1.0.0
collected 1 items

test.py::test_foo
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/main.py", line 90, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/main.py", line 121, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/main.py", line 146, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 253, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 253, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/runner.py", line 65, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/runner.py", line 75, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/runner.py", line 121, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/skipping.py", line 170, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/runner.py", line 224, in pytest_runtest_makereport
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/python.py", line 751, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/python.py", line 744, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/_pytest/main.py", line 404, in _repr_failure_py
INTERNALERROR>     style=style, tbfilter=tbfilter)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_code/code.py", line 412, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_code/code.py", line 590, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_code/code.py", line 582, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_code/code.py", line 549, in repr_traceback_entry
INTERNALERROR>     path = self._makepath(entry.path)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_code/code.py", line 562, in _makepath
INTERNALERROR>     np = py.path.local().bestrelpath(path)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_path/local.py", line 149, in __init__
INTERNALERROR>     self.strpath = py.error.checked_call(os.getcwd)
INTERNALERROR>   File "/nail/home/buck/trees/yelp/pip-faster/venv-venv_update/lib/python2.7/site-packages/py/_error.py", line 84, in checked_call
INTERNALERROR>     raise cls("%s%r" % (func.__name__, args))
INTERNALERROR> ENOENT: [No such file or directory]: getcwd()

====================================================================================== no tests ran in 0.18 seconds =======================================================================================

[hpk42]

agred, that's not robust behaviour. Also it's wrong for traceback representation to use py.path.local(). Instead we should use the "startdir" as it is stored on the terminal writer plugin (we can also put it to the config object in addition or instead) for computing relative paths. If we can merge Bruno's "move py.code to pytest" we could probably more easily fix this properly.

It's probably also possible to fix this in the pytest-2.8 series maybe by e.g. changing to "startdir" before we call the pytest_runtest_makereport hook. @bukzor if you feel like trying that in a PR you are welcome (with a test which you basically already have) .

[bukzor]

Full disclosure: I don't plan to work on this.

I was able to find the rogue path.chdir() and changed it to with path.as_cwd() which enabled a good traceback.

On Tue, Dec 8, 2015 at 12:54 PM holger krekel notifications@github.com
wrote:

agred, that's not robust behaviour. Also it's wrong for traceback
representation to use py.path.local(). Instead we should use the
"startdir" as it is stored on the terminal writer plugin (we can also put
it to the config object in addition or instead) for computing relative
paths. If we can merge Bruno's "move py.code to pytest" we could probably
more easily fix this properly.

It's probably also possible to fix this in the pytest-2.8 series maybe by
e.g. changing to "startdir" before we call the pytest_runtest_makereport
hook. @bukzor https://github.com/bukzor if you feel like trying that in
a PR you are welcome (with a test which you basically already have) .

—
Reply to this email directly or view it on GitHub
#1235 (comment).

[nicoddemus]

@bukzor no problem, thanks for reporting anyway!

[MagiMaster]

Ran in to this bug recently.

[floer32]

Seems like in every case someone should just create a subdirectory OF tmpdir (easy using the sub() method) and then they can freely do whatever they want, with that subdirectory ... and no issues when it gets deleted.

[floer32]

For example:

@pytest.fixture
def testing_workdir(tmpdir, request):
    """ Create a workdir in a safe temporary folder; cd into dir above before test, cd out after

    :param tmpdir: py.test fixture, will be injected
    :param request: py.test fixture-related, will be injected (see pytest docs)
    """

    saved_path = os.getcwd()

    tmpdir.chdir()
    workdir = tmpdir.mkdir('mysubdir')

    def return_to_saved_path():
        os.chdir(saved_path)

    request.addfinalizer(return_to_saved_path)

    return str(workdir)

any test function using this fixture, testing_workdir will just get a subdirectory that is (a) temporary and (b) safe to delete :)

Oh, I'm not suggesting that py.test should do this, I'm just suggesting that OPs or commentors on this thread and #470, can take care of this for themselves with a fixture like this.

[MagiMaster]

I agree that it's due to incorrectly written tests, but the problem is that it's hard to figure out which test caused the issue (especially when you didn't write all of them yourself).

[floer32]

ah, I hear you on that @MagiMaster