Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add backfill benchmarks #412

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Add backfill benchmarks #412

wants to merge 14 commits into from

Conversation

ryanslade
Copy link
Contributor

@ryanslade ryanslade commented Oct 17, 2024

This change adds a benchmark that runs a simple backfill against 10k, 100k and 1 million rows.

This should give us a baseline metric that we can use to compare performance over time.

Example output:

make bench 
go test ./internal/benchmarks -v -benchtime=1x -bench .
2024/10/18 14:55:15 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 27.2.0
  API Version: 1.46
  Operating System: Docker Desktop
  Total Memory: 7838 MB
  Testcontainers for Go Version: v0.33.0
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: 3bb08eb17b97b8f2c377c270bed485424b032d695f7a01486bab7cd6e19f7d16
  Test ProcessID: 07bacf41-6b37-4abf-83ee-504fd339ec78
2024/10/18 14:55:16 🐳 Creating container for image testcontainers/ryuk:0.8.1
2024/10/18 14:55:16 ✅ Container created: 2e899fc51d0f
2024/10/18 14:55:16 🐳 Starting container: 2e899fc51d0f
2024/10/18 14:55:16 ✅ Container started: 2e899fc51d0f
2024/10/18 14:55:16 ⏳ Waiting for container id 2e899fc51d0f image: testcontainers/ryuk:0.8.1. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms skipInternalCheck:false}
2024/10/18 14:55:17 🔔 Container is ready: 2e899fc51d0f
2024/10/18 14:55:17 🐳 Creating container for image postgres:15.3
2024/10/18 14:55:17 ✅ Container created: b46125d9f39e
2024/10/18 14:55:17 🐳 Starting container: b46125d9f39e
2024/10/18 14:55:17 ✅ Container started: b46125d9f39e
2024/10/18 14:55:17 ⏳ Waiting for container id b46125d9f39e image: postgres:15.3. Waiting for: &{timeout:<nil> deadline:0x1400044ef70 Strategies:[0x1400047c360]}
2024/10/18 14:55:18 🔔 Container is ready: b46125d9f39e
goos: darwin
goarch: arm64
pkg: github.com/xataio/pgroll/internal/benchmarks
cpu: Apple M2 Pro
BenchmarkBackfill
BenchmarkBackfill/10000
    benchmarks_test.go:76: Seeded 10000 rows in 20.667541ms (483850 rows/s)
    benchmarks_test.go:110: Backfilled 10000 rows in 105.647ms
BenchmarkBackfill/10000-10         	      1	105647000 ns/op	    94655 rows/s
BenchmarkBackfill/100000
    benchmarks_test.go:76: Seeded 100000 rows in 84.946292ms (1177214 rows/s)
    benchmarks_test.go:110: Backfilled 100000 rows in 2.040057666s
BenchmarkBackfill/100000-10        	      1	2040057666 ns/op	    49018 rows/s
BenchmarkBackfill/1000000
    benchmarks_test.go:76: Seeded 1000000 rows in 662.547542ms (1509326 rows/s)
    benchmarks_test.go:110: Backfilled 1000000 rows in 56.917213s
BenchmarkBackfill/1000000-10       	      1	56917213000 ns/op	    17569 rows/s
PASS
2024/10/18 14:56:18 🐳 Terminating container: b46125d9f39e
2024/10/18 14:56:18 🚫 Container terminated: b46125d9f39e
ok  	github.com/xataio/pgroll/internal/benchmarks	63.795s

Part of #408

@ryanslade ryanslade marked this pull request as ready for review October 18, 2024 13:12
@ryanslade
Copy link
Contributor Author

@andrew-farries I think is is fine for now.

What I'd like to do is add the other benchmarks in separate PR's and then once we have them all work on the job of persisting to S3.

For now, we can always check out branches locally and run the benchmarks to spot check performance improvements.

@ryanslade
Copy link
Contributor Author

Also, as we add more benchmarks some pattern will probably emerge which will help pull out some duplicated code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant