Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a rewrite of
perspective-python
's concurrency model, enabling safe GIL-less+concurrent calls to Perspective's API, as well as additional assorted performance fixes (including a significant improvement toto_arrow()
for both Python and Wasm). In addition to Perspective's internal parallelism,PerspectiveManager
can now process requests concurrently by providing a thread pool dispatch toPerspectiveManager.set_loop_callback()
:PSP_PARALLEL_FOR
for all python builds.set_loop_callback()
causes GIL unlock behavior, and calls to Perspective APIs from different threads will throw an exception as a protection against the engine's internal thread-unsafeness. With this PR, the GIL is released always, and Perspective's API implements its own internal locking. As a result, aPerspectiveManager
can now be called without the GIL via multiple threads from Python, e.g. from aconcurrent.futures.ThreadPoolExecutor
, substantially increasing thread utilization on servers with many concurrent client requests.PSP_PARALLEL_FOR
across columns.to_arrow()
, improving serialization performance by ~50%:PerspectiveManager
which caused some callbacks to become crossed when multiple clients are connected.Historic
to_arrow()
wasm performanceHistoric Python performance across a range of benchmarks (2.2.1 has a specific regression in
PSP_PARALLEL_FOR
):