Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reusable shared buffer for transactions #9

Merged
merged 1 commit into from
Jul 1, 2024

Conversation

aroradaman
Copy link
Member

@aroradaman aroradaman commented May 20, 2024

This PR introduces a shared reusable buffer for transactions within a realNFTables instance.
This improves performance by reducing the memory allocations for running nft transactions.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2024
@k8s-ci-robot k8s-ci-robot requested review from aojea and danwinship May 20, 2024 16:56
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 20, 2024
Copy link

linux-foundation-easycla bot commented May 20, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: aroradaman / name: Daman Arora (841e0a5)

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 20, 2024
@aroradaman aroradaman force-pushed the reusable-buffer branch 2 times, most recently from 8b00533 to 83d2b7d Compare May 20, 2024 17:18
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels May 20, 2024
@danwinship
Copy link
Collaborator

_ No description provided. _

It's always good to provide a PR description. For the benefit of people in the future trying to figure out why stuff was done

go.mod Outdated
@@ -1,4 +1,4 @@
module sigs.k8s.io/knftables
module github.com/aroradaman/knftables
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you shouldn't need to do this... make test should work anyway... what problem where you having?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was planning to point to my fork in k/k and then run pull-kubernetes-e2e-gci-gce-nftables for testing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will drop the dummy commit after testing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, you probably just need to use the replace section of go.mod in the k/k PR. (The existing replacements are all just for the staging repos, but you can add a line like:

sigs.k8s.io/knftables => github.com/aroradaman/knftables TAG-OR-HASH

(and then run go mod tidy if you specified a hash, to make it convert it to a fake tag)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This + ./hack/update-vendor.sh did the trick.

nftables.go Outdated Show resolved Hide resolved
@aroradaman
Copy link
Member Author

k/k MR for testing the change by trigerringpull-kubernetes-e2e-gci-gce-nftables - kubernetes/kubernetes#125003

Signed-off-by: Daman Arora <aroradaman@gmail.com>
@aroradaman aroradaman force-pushed the reusable-buffer branch 2 times, most recently from 93d0c09 to 841e0a5 Compare May 20, 2024 17:58
@aroradaman
Copy link
Member Author

k/k MR for testing the change by trigerringpull-kubernetes-e2e-gci-gce-nftables - kubernetes/kubernetes#125003

pull-kubernetes-e2e-gci-gce-nftables is not stable right now - kubernetes/test-infra#32485 skipping this test

@aroradaman aroradaman changed the title [WIP] add reusable buffer for transactions Add reusable shared buffer for transactions May 21, 2024
@aroradaman aroradaman marked this pull request as ready for review May 21, 2024 07:43
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 21, 2024
@k8s-ci-robot k8s-ci-robot requested a review from danwinship May 21, 2024 07:43
@danwinship
Copy link
Collaborator

There's a kind-based nftables job too... um... it's probably /test pull-kubernetes-e2e-kind-nftables but if that's wrong the error message will give you the whole list of tests and you can find it there. (If you had modified pkg/proxy/nftables it would run that test automatically, but it won't do it if you only modified knftables.)

There's an nftables scalability job too, and it would be cool to see if this ends up visible in the results... https://gist.github.com/aojea/276d1f8061403b6687ee12b2f2c7522e explains (part of) how to analyze the prometheus dumps...

@aroradaman
Copy link
Member Author

There's a kind-based nftables job too... um... it's probably /test pull-kubernetes-e2e-kind-nftables but if that's wrong the error message will give you the whole list of tests and you can find it there. (If you had modified pkg/proxy/nftables it would run that test automatically, but it won't do it if you only modified knftables.)

There's an nftables scalability job too, and it would be cool to see if this ends up visible in the results... https://gist.github.com/aojea/276d1f8061403b6687ee12b2f2c7522e explains (part of) how to analyze the prometheus dumps...

pull-kubernetes-e2e-kind-nftables passed for k/k.
Will keep any eye on the scalability job.

@aroradaman
Copy link
Member Author

@danwinship do you want to ship this with k/k 1.31?

@danwinship
Copy link
Collaborator

yes, let's get this in
/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 1, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aroradaman, danwinship

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 1, 2024
@k8s-ci-robot k8s-ci-robot merged commit ba21e1b into kubernetes-sigs:master Jul 1, 2024
5 checks passed
@aroradaman
Copy link
Member Author

Should I point the knftables dependency in k/k to this commit, or are we cutting a release?

@danwinship
Copy link
Collaborator

Should wait for a release.
Bumping the dep in k/k is slightly annoying (need a deps approver), so we don't do it for every commit. Should wait to see if we're going to need any more fixes here first...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants