Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thread-safe python readers #2261

Merged
merged 7 commits into from
Jun 20, 2023
Merged

Thread-safe python readers #2261

merged 7 commits into from
Jun 20, 2023

Conversation

texodus
Copy link
Member

@texodus texodus commented Jun 19, 2023

This PR introduces a rewrite of perspective-python's concurrency model, enabling safe GIL-less+concurrent calls to Perspective's API, as well as additional assorted performance fixes (including a significant improvement to to_arrow() for both Python and Wasm). In addition to Perspective's internal parallelism, PerspectiveManager can now process requests concurrently by providing a thread pool dispatch to PerspectiveManager.set_loop_callback():

with concurrent.futures.ThreadPoolExecutor() as executor:
    manager.set_loop_callback(psp_loop.run_in_executor, executor)
  • Fixes a regression in 2.2.1 (only this version) which disabled PSP_PARALLEL_FOR for all python builds.
  • The semantics of async mode have changed. Before this PR, set_loop_callback() causes GIL unlock behavior, and calls to Perspective APIs from different threads will throw an exception as a protection against the engine's internal thread-unsafeness. With this PR, the GIL is released always, and Perspective's API implements its own internal locking. As a result, a PerspectiveManager can now be called without the GIL via multiple threads from Python, e.g. from a concurrent.futures.ThreadPoolExecutor, substantially increasing thread utilization on servers with many concurrent client requests.
  • Apache Arrow reading/writing have been ported to utilize PSP_PARALLEL_FOR across columns.
  • Removes numerous superfluous heap allocations from to_arrow(), improving serialization performance by ~50%:
  • Fixes a bug in Python PerspectiveManager which caused some callbacks to become crossed when multiple clients are connected.

Historic to_arrow() wasm performance

Screenshot 2023-06-19 at 3 17 47 PM

Historic Python performance across a range of benchmarks (2.2.1 has a specific regression in PSP_PARALLEL_FOR):

Screenshot 2023-06-20 at 1 54 43 AM

@texodus texodus added enhancement Feature requests or improvements Python breaking labels Jun 19, 2023
@texodus texodus mentioned this pull request Jun 19, 2023
3 tasks
@texodus texodus force-pushed the rw-lock-v2 branch 2 times, most recently from bf24ad6 to 378fa68 Compare June 19, 2023 05:18
@texodus texodus merged commit e76e396 into master Jun 20, 2023
@texodus texodus deleted the rw-lock-v2 branch June 20, 2023 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking enhancement Feature requests or improvements Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant