pathlib.Path change breaks on Mac OS X with OSError: AF_UNIX path too long

[bartleyg]

• Version: 914bc79
• Python: 3.6
• OS: osx

What was wrong?

The recently merged PR #921 adding pathlib.Path to IPCProvider breaks on Mac OS X, but not Ubuntu Linux.

$ tox -e py36-core hangs indefinitely:
tests/core/providers/test_ipc_provider.py::test_sync_waits_for_full_result FAILED [ 56%]

The specific test output is:

$ pytest tests/core/providers/test_ipc_provider.py 
================================================================ test session starts ================================================================
platform darwin -- Python 3.6.5, pytest-3.6.2, py-1.5.3, pluggy-0.6.0 -- /Users/bartley/GoogleDrive/python/test-fresh-clone-web3.py/web3.py/venv/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/bartley/GoogleDrive/python/test-fresh-clone-web3.py/web3.py, inifile: pytest.ini
plugins: xdist-1.22.2, pythonpath-0.7.2, mock-1.10.0, forked-0.2, hypothesis-3.60.1, flaky-3.4.0
collected 3 items                                                                                                                                   

tests/core/providers/test_ipc_provider.py::test_ipc_no_path PASSED                                                                            [ 33%]
tests/core/providers/test_ipc_provider.py::test_sync_waits_for_full_result FAILED                                                             [ 66%]^C

===================================================================== FAILURES ======================================================================
__________________________________________________________ test_sync_waits_for_full_result __________________________________________________________

jsonrpc_ipc_pipe_path = '/var/folders/cg/d63t_4qx3hd4qp_xr43wxjr40000gn/T/tmpzqn39p25/e3acda31-3b21-4eaa-a928-9ac167b2d1d4.ipc'
serve_empty_result = None

    def test_sync_waits_for_full_result(jsonrpc_ipc_pipe_path, serve_empty_result):
        provider = IPCProvider(pathlib.Path(jsonrpc_ipc_pipe_path), timeout=3)
>       result = provider.make_request("method", [])

jsonrpc_ipc_pipe_path = '/var/folders/cg/d63t_4qx3hd4qp_xr43wxjr40000gn/T/tmpzqn39p25/e3acda31-3b21-4eaa-a928-9ac167b2d1d4.ipc'
provider   = <web3.providers.ipc.IPCProvider object at 0x106a0d860>
serve_empty_result = None

tests/core/providers/test_ipc_provider.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
web3/providers/ipc.py:206: in make_request
    with self._lock, self._socket as sock:
web3/providers/ipc.py:46: in __enter__
    self.sock = self._open()
web3/providers/ipc.py:59: in _open
    return get_ipc_socket(self.ipc_path)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

ipc_path = '/private/var/folders/cg/d63t_4qx3hd4qp_xr43wxjr40000gn/T/tmpzqn39p25/e3acda31-3b21-4eaa-a928-9ac167b2d1d4.ipc', timeout = 0.1

    def get_ipc_socket(ipc_path, timeout=0.1):
        if sys.platform == 'win32':
            # On Windows named pipe is used. Simulate socket with it.
            from web3.utils.windows import NamedPipe
    
            return NamedPipe(ipc_path)
        else:
            sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>           sock.connect(ipc_path)
E           OSError: AF_UNIX path too long

ipc_path   = '/private/var/folders/cg/d63t_4qx3hd4qp_xr43wxjr40000gn/T/tmpzqn39p25/e3acda31-3b21-4eaa-a928-9ac167b2d1d4.ipc'
sock       = <socket.socket fd=10, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
timeout    = 0.1

web3/providers/ipc.py:30: OSError
===Flaky Test Report===


===End Flaky Test Report===
============================================================= slowest 10 test durations =============================================================
0.00s setup    tests/core/providers/test_ipc_provider.py::test_sync_waits_for_full_result
0.00s call     tests/core/providers/test_ipc_provider.py::test_sync_waits_for_full_result
0.00s setup    tests/core/providers/test_ipc_provider.py::test_ipc_no_path
0.00s call     tests/core/providers/test_ipc_provider.py::test_ipc_no_path
0.00s teardown tests/core/providers/test_ipc_provider.py::test_ipc_no_path
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

As can be seen above, the AF_UNIX path too long failure happens at web3/providers/ipc.py:30 sock.connect(ipc_path).

How can it be fixed?

Rollback the pathlib.Path change for now, or don't run test_sync_waits_for_full_result() on Mac OS X with a pathlib.Path as the argument to the IPCProvider() constructor.

[voith]

@bartleyg A simple fix would be to change the uuid4 to a short hardcoded name in the following code:

web3.py/tests/core/providers/test_ipc_provider.py

Line 26 in 914bc79

ipc_path = os.path.join(temp_dir, '{0}.ipc'.format(uuid.uuid4()))

Also, worth having a look at this commit.

[miohtama]

As a stop gap fix to the issue you can run the test suite on OSX with some skips:

py.test -k "(not test_sync_waits_for_full_result) and (not parity) and (not goethereum)"

[carver]

Yeesh OSX, that long temporary directory and short file path limit doesn't leave much room for the file name. Detecting a too-long path and then choosing a shorter one is probably the best solution here. (roughly like the code @voith pointed to)