Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an optimized token router filter #948

Merged
merged 4 commits into from
May 14, 2024
Merged

Add an optimized token router filter #948

merged 4 commits into from
May 14, 2024

Conversation

Jake-Shadle
Copy link
Collaborator

@Jake-Shadle Jake-Shadle commented May 14, 2024

This PR introduces a HashedTokenRouter filter in parallel with the current TokenRouter that is significantly faster.

  1. The current token router linearly searches through every token in every endpoint and does a full byte comparison. In contrast, the new token router instead hashes every token when a cluster's endpoints are updated and buckets the addresses by the hash, so that the lookup is a single operation.
  2. Once 1 or more addresses are found by token in a locality, we abort the search, rather than keep searching, as this doesn't really make sense.
token_router                      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ token_router                                 │               │               │               │         │
   ├─ HashedTokenRouter                         │               │               │               │         │
   │  ├─ multi:2..128:duplicates  1.882 µs      │ 32.4 µs       │ 13.71 µs      │ 14.28 µs      │ 100     │ 100
   │  │                           56.27 TB/s    │ 3.27 TB/s     │ 7.724 TB/s    │ 7.417 TB/s    │         │
   │  ├─ multi:2..128:unique      1.332 µs      │ 41.31 µs      │ 13.8 µs       │ 15.21 µs      │ 100     │ 100
   │  │                           79.46 TB/s    │ 2.563 TB/s    │ 7.67 TB/s     │ 6.959 TB/s    │         │
   │  ├─ single:duplicates        791.7 ns      │ 20.56 µs      │ 6.602 µs      │ 6.8 µs        │ 100     │ 100
   │  │                           2.075 TB/s    │ 79.88 GB/s    │ 248.8 GB/s    │ 241.6 GB/s    │         │
   │  ╰─ single:unique            1.271 µs      │ 22.73 µs      │ 6.742 µs      │ 7.238 µs      │ 100     │ 100
   │                              1.292 TB/s    │ 72.29 GB/s    │ 243.7 GB/s    │ 227 GB/s      │         │
   ╰─ TokenRouter                               │               │               │               │         │
      ├─ multi:2..128:duplicates  31.77 ms      │ 65.87 ms      │ 61.48 ms      │ 59.32 ms      │ 100     │ 100
      │                           3.334 GB/s    │ 1.608 GB/s    │ 1.723 GB/s    │ 1.786 GB/s    │         │
      ├─ multi:2..128:unique      23.23 ms      │ 71.5 ms       │ 59.99 ms      │ 56.36 ms      │ 100     │ 100
      │                           4.557 GB/s    │ 1.481 GB/s    │ 1.765 GB/s    │ 1.878 GB/s    │         │
      ├─ single:duplicates        8.828 ms      │ 9.692 ms      │ 8.981 ms      │ 8.999 ms      │ 100     │ 100
      │                           186.1 MB/s    │ 169.5 MB/s    │ 182.9 MB/s    │ 182.5 MB/s    │         │
      ╰─ single:unique            8.824 ms      │ 9.566 ms      │ 9.027 ms      │ 9.051 ms      │ 100     │ 100
                                  186.2 MB/s    │ 171.8 MB/s    │ 182 MB/s      │ 181.5 MB/s    │         │

Both serialization and deserialization were doing unnecessary temporary heap allocations
src/filters/read.rs Outdated Show resolved Hide resolved
@quilkin-bot
Copy link
Collaborator

Build Succeeded 🥳

Build Id: 9d024cad-0864-4905-8941-86a584f78242

The following development images have been built, and will exist for the next 30 days:

To build this version:

git fetch git@github.com:googleforgames/quilkin.git pull/948/head:pr_948 && git checkout pr_948
cargo build

@Jake-Shadle Jake-Shadle merged commit 1556b19 into main May 14, 2024
11 checks passed
@Jake-Shadle Jake-Shadle deleted the opt-token-router branch May 14, 2024 11:40
@markmandel markmandel added kind/feature New feature or request area/performance Anything to do with Quilkin being slow, or making it go faster. labels Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/performance Anything to do with Quilkin being slow, or making it go faster. kind/feature New feature or request size/l
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants