Skip to content

Commit

Permalink
Add documentation around read buffer; Raise clear exception:
Browse files Browse the repository at this point in the history
- Document ``read_buffer_limit`` on the ``AsyncIPCProvider`` class.

- Add ``ReadBufferLimitReached`` exception to be raised when the read buffer
  limit is reached, prompting the user to increase the limit.

- Add a base ``PersistentConnectionError`` exception class for persistent
  connection errors to inherit from.
  • Loading branch information
fselmo committed Sep 20, 2024
1 parent 517399c commit 7490fe3
Show file tree
Hide file tree
Showing 5 changed files with 41 additions and 4 deletions.
4 changes: 4 additions & 0 deletions docs/providers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,10 @@ AsyncIPCProvider
JSON-RPC server.

* ``ipc_path`` is the filesystem path to the IPC socket:
* ``read_buffer_limit`` is the maximum size of data, in bytes, that can be read
from the socket at one time. Defaults to 20MB (20 * 1024 * 1024). Raises
``ReadBufferLimitReached`` if the limit is reached, suggesting that the buffer
limit be increased.

This provider inherits from the
:class:`~web3.providers.persistent.PersistentConnectionProvider` class. Refer to
Expand Down
2 changes: 1 addition & 1 deletion newsfragments/3492.feature.rst
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Add a configuration option for the ``read_buffer_limit`` for ``AsyncIPCProvider`` in order to control the expected message size limit. Set this default value to 20MB.
Add a configuration option for the ``read_buffer_limit`` for ``AsyncIPCProvider`` in order to control the expected message size limit (defaults to 20MB). Add ``ReadBufferLimitReached`` for when the read limit is reached, extend from ``PersistentConnectionError``.
11 changes: 10 additions & 1 deletion tests/core/providers/test_async_ipc_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
from web3.datastructures import (
AttributeDict,
)
from web3.exceptions import (
ReadBufferLimitReached,
)
from web3.providers import (
AsyncIPCProvider,
)
Expand Down Expand Up @@ -312,7 +315,13 @@ async def test_async_ipc_reader_can_read_20mb_message(
async def test_async_ipc_reader_raises_on_msg_over_20mb(
jsonrpc_ipc_pipe_path, serve_larger_than_20mb_response
):
with pytest.raises(ValueError):
with pytest.raises(
ReadBufferLimitReached,
match=(
rf"Read buffer limit of `{TWENTY_MB}` bytes was reached. "
"Consider increasing the ``read_buffer_limit`` on the AsyncIPCProvider."
),
):
async with AsyncWeb3(
AsyncIPCProvider(pathlib.Path(jsonrpc_ipc_pipe_path))
) as w3:
Expand Down
15 changes: 14 additions & 1 deletion web3/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,20 @@ def __init__(
super().__init__(message)


class PersistentConnectionClosedOK(Web3Exception):
class PersistentConnectionError(Web3Exception):
"""
Raised when a persistent connection encounters an error.
"""


class ReadBufferLimitReached(PersistentConnectionError):
"""
Raised when the read buffer limit is reached while reading data from a persistent
connection.
"""


class PersistentConnectionClosedOK(PersistentConnectionError):
"""
Raised when a persistent connection is closed gracefully by the server.
"""
Expand Down
13 changes: 12 additions & 1 deletion web3/providers/persistent/async_ipc.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from ...exceptions import (
PersistentConnectionClosedOK,
ProviderConnectionError,
ReadBufferLimitReached,
Web3TypeError,
)
from ..ipc import (
Expand Down Expand Up @@ -96,7 +97,17 @@ async def socket_send(self, request_data: bytes) -> None:
)

async def socket_recv(self) -> RPCResponse:
data = await self._reader.readline()
try:
data = await self._reader.readline()
except ValueError as e:
if all(kw in str(e) for kw in ("limit", "chunk")):
raise ReadBufferLimitReached(
f"Read buffer limit of `{self.read_buffer_limit}` bytes was "
"reached. Consider increasing the ``read_buffer_limit`` on the "
"AsyncIPCProvider."
) from e
raise

if not data:
raise PersistentConnectionClosedOK("Socket reader received end of stream.")
return self.decode_rpc_response(data)
Expand Down

0 comments on commit 7490fe3

Please sign in to comment.