Skip to content

docs: update benchmark results from GCP c2-standard-8#160

Merged
kacy merged 1 commit intomainfrom
docs/benchmark-results-final
Feb 16, 2026
Merged

docs: update benchmark results from GCP c2-standard-8#160
kacy merged 1 commit intomainfrom
docs/benchmark-results-final

Conversation

@kacy
Copy link
Owner

@kacy kacy commented Feb 16, 2026

summary

populates all previously-pending benchmark tables in bench/README.md and updates README.md highlights with data collected on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz, Ubuntu 22.04).

tables updated:

  • encryption: SET 47% overhead at P=16, GET 0% at P=1
  • vector similarity: added qdrant and gRPC columns. ember gRPC leads at 1,462 q/s with 36 MB (3-5x less memory than alternatives)
  • gRPC vs RESP3: pipelined RESP3 at 84-109k ops/s, gRPC unary at 6.3k
  • pub/sub: 10.9k msg/s single subscriber, 27k fanout at 100 subscribers
  • protobuf: ~0% schema validation overhead
  • memory by data type: concurrent 128 B/key (less than redis 173 B/key), sharded 208 B/key, hash 555 B/key, sorted set 179 B/member, vector 853 B/vec
  • memory summary: updated from old single-mode numbers to concurrent/sharded split
  • redis-benchmark throughput: updated to latest run numbers

what was tested

data collected from two benchmark runs on GCP c2-standard-8 VMs. all scripts completed successfully after fixing gRPC port conflicts (#159), memory benchmark DBSIZE parsing (#157, #158), and protobuf key collision (#157).

populates all previously-pending tables in bench/README.md with data
collected on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz):

- encryption: SET 47% overhead (P=16), GET 0% (P=1)
- vector: ember 1,264 q/s (RESP), 1,462 q/s (gRPC), 36 MB
  vs chromadb 390 q/s (122 MB), pgvector 831 q/s (179 MB),
  qdrant 596 q/s (121 MB)
- grpc vs resp3: pipelined RESP 84-109k ops/s, grpc unary 6.3k
- pub/sub: 10.9k msg/s (1 sub), 27k fanout msg/s (100 sub)
- protobuf: ~0% overhead for schema validation
- memory: concurrent 128 B/key, sharded 208 B/key, redis 173 B/key

also updates README.md highlights with latest numbers.
@kacy kacy merged commit 6775aed into main Feb 16, 2026
7 checks passed
@kacy kacy deleted the docs/benchmark-results-final branch February 16, 2026 03:44
kacy added a commit that referenced this pull request Feb 19, 2026
populates all previously-pending tables in bench/README.md with data
collected on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz):

- encryption: SET 47% overhead (P=16), GET 0% (P=1)
- vector: ember 1,264 q/s (RESP), 1,462 q/s (gRPC), 36 MB
  vs chromadb 390 q/s (122 MB), pgvector 831 q/s (179 MB),
  qdrant 596 q/s (121 MB)
- grpc vs resp3: pipelined RESP 84-109k ops/s, grpc unary 6.3k
- pub/sub: 10.9k msg/s (1 sub), 27k fanout msg/s (100 sub)
- protobuf: ~0% overhead for schema validation
- memory: concurrent 128 B/key, sharded 208 B/key, redis 173 B/key

also updates README.md highlights with latest numbers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant