Skip to content

bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv #21439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

tontinton
Copy link
Contributor

@tontinton tontinton commented Jul 10, 2020

https://bugs.python.org/issue41270

I got about 120% better performance on await reader.read() using this branch.

The way I tested was writing a server / client:
server.py:

import asyncio
import contextlib
import datetime


async def client_connected(reader, writer):
    start = datetime.datetime.now()

    with contextlib.closing(writer):
        for i in range(100000):
            await reader.read(1024 * 64)  # tweak this parameter as much as you like

    diff = datetime.datetime.now() - start
    print(f'{diff.seconds}.{diff.microseconds}')


async def main():
    server = await asyncio.start_server(client_connected, '127.0.0.1', 8888)

    addr = server.sockets[0].getsockname()
    print(f'Serving on {addr}')

    async with server:
        await server.serve_forever()


if __name__ == "__main__":
    asyncio.run(main())

client.py:

import asyncio
import contextlib


async def flood(ip, port):
    message = b'A' * 1024 * 128  # tweak this parameter as much as you like
    reader, writer = await asyncio.open_connection(ip, port)
    with contextlib.closing(writer):
        while True:
            writer.write(message)
            await writer.drain()


if __name__ == "__main__":
    asyncio.run(flood('127.0.0.1', 8888))

@tontinton tontinton requested review from 1st1 and asvetlov as code owners July 10, 2020 21:32
@tontinton tontinton changed the title Fix issue 41273 bpo-41270 asyncio: proactor read transport: better performance by using recv_into instead of recv Jul 10, 2020
@tontinton tontinton changed the title bpo-41270 asyncio: proactor read transport: better performance by using recv_into instead of recv bpo-41270: asyncio's proactor read transport's better performance by using recv_into instead of recv Jul 10, 2020
@tontinton tontinton force-pushed the fix-issue-41273 branch 2 times, most recently from 96f0f5a to 9553dc1 Compare July 10, 2020 22:37
By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.

This betters performance for any stream using proactor (basically any
asyncio stream on windows).
By doubling the read buffer size we get better performance.
@methane methane changed the title bpo-41270: asyncio's proactor read transport's better performance by using recv_into instead of recv bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv Jul 11, 2020
@tontinton tontinton closed this Jul 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants