Skip to content

docs: populate benchmark tables with GCP results#155

Closed
kacy wants to merge 1 commit intomainfrom
docs/benchmark-results-update
Closed

docs: populate benchmark tables with GCP results#155
kacy wants to merge 1 commit intomainfrom
docs/benchmark-results-update

Conversation

@kacy
Copy link
Owner

@kacy kacy commented Feb 15, 2026

summary

  • fill in benchmark tables with real data from GCP c2-standard-8 run (2026-02-15)
  • encryption overhead: 44% SET (P=16), 16% GET (P=16), 0% GET (P=1)
  • vector: add qdrant column and gRPC results — ember 1,372 q/s via gRPC at 36MB RSS
  • gRPC vs RESP3: RESP3 pipelined dominates (83-109k ops/s), gRPC unary at 6.3k
  • pub/sub: 11k msg/s single subscriber, 27k msg/s fan-out at 100 subscribers
  • update README highlights with latest numbers

what was tested

all benchmarks run on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz, 32GB RAM):

  • bench/compare-redis.sh — throughput comparison
  • bench/bench-memtier.sh — latency + throughput
  • bench/bench-vector.sh --ember-grpc — vector with qdrant + gRPC
  • bench/bench-encryption.sh — AES-256-GCM overhead
  • bench/bench-grpc.sh — gRPC vs RESP3 standard ops
  • bench/bench-pubsub.sh — pub/sub throughput

still pending

  • memory by data type (bench-memory.sh has a grep bug — fix tracked separately)
  • protobuf storage overhead (feature detection issue at runtime)
  • SIFT1M recall accuracy (dataset download + longer run needed)

populate tables with real data from benchmark run on 2026-02-15:
- encryption: SET 44% overhead (P=16), GET near-zero (P=1)
- vector: add qdrant + gRPC columns, ember 1.4k q/s at 36MB
- gRPC vs RESP3: RESP3 pipelined 83-109k ops/s, gRPC unary 6.3k
- pub/sub: 11k msg/s (1 sub), 27k fanout (100 sub)
- README highlights updated with latest numbers
@kacy
Copy link
Owner Author

kacy commented Feb 16, 2026

superseded by #160 which has the complete benchmark data from the final GCP run.

@kacy kacy closed this Feb 16, 2026
@kacy kacy deleted the docs/benchmark-results-update branch February 25, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant