Skip to content

Prefetch txs in a background thread #11

@fungibit

Description

@fungibit

In "tight" iter_txs() loops, much of the time is spent deserializing the transactions.

In cases like this ...

for tx in iter_txs():
    some_io_bound_task(tx)

... it is useful to deserialize the txs of the next block in the bg, while the IO task runs.

Even in cases like this ...

for tx in iter_txs():
    some_cpu_bound_task(tx)

... it can also help, because some of the deserialization code runs with nogil, so this can utilize more than one core concurrently. (It is hard to estimate if the benefit is significant in this case, though. My intuition is it can give up to 20-30% speedup, with little effort).

This option should be enabled by default in iter_txs(), and disabled by default in iter_blocks().

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions