Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Spanner benchmarking #562

Closed
wants to merge 3 commits into from

Conversation

FirePing32
Copy link

fixes: #438

@FirePing32 FirePing32 requested a review from a team as a code owner December 1, 2020 18:15
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner-django API. label Dec 1, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 1, 2020
@c24t
Copy link
Contributor

c24t commented Dec 1, 2020

Looks like benchmarking.py is taken from @odeke-em's gist here: https://gist.github.com/odeke-em/929d308f1fb80a1661d2e3ccb6a7b58b.

As @odeke-em mentions in #438 (comment), what we're really interested in here is the extra latency that spanner-django adds on top of python-spanner. The python-spanner repo includes some YCSB benchmarks (added in googleapis/google-cloud-python#4539), but it's not clear that they're measuring the right thing, or that we can get a meaningful comparison in django.

Ideally we'd run these benchmarks as a periodic CI task and flag regressions automatically instead of checking the results into the repo.

@c24t
Copy link
Contributor

c24t commented Dec 1, 2020

I suspect we'll want the benchmarking script to run against the emulator since we're only interested in the performance of the ORM library here, and not the spanner service itself.

@c24t
Copy link
Contributor

c24t commented Jan 29, 2021

Closing as inactive, @FirePing32 feel free to reopen if you change this to address the comments above (run benchmarks in CI, against emulator, etc.).

@c24t c24t closed this Jan 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner-django API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

performance testing harness
2 participants