feat: support the combination of Nginx variables for annotation upstream-hash-by. #5571

agile6v · 2020-05-19T03:56:25Z

What this PR does / why we need it:

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Which issue/s this PR fixes

fixes #4846

Checklist:

My change requires a change to the documentation.
I have updated the documentation accordingly.
I've read the CONTRIBUTION guide
I have added tests to cover my changes.
All new and existing tests passed.

k8s-ci-robot · 2020-05-19T03:56:32Z

Hi @agile6v. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aledbf · 2020-05-19T09:03:37Z

/ok-to-test

aledbf · 2020-05-19T09:03:47Z

/assign @ElvinEfendi

aledbf · 2020-05-19T13:05:15Z

/retest

codecov-commenter · 2020-05-19T13:13:32Z

Codecov Report

Merging #5571 into master will decrease coverage by 0.31%.
The diff coverage is 46.49%.

@@            Coverage Diff             @@
##           master    #5571      +/-   ##
==========================================
- Coverage   58.64%   58.33%   -0.32%     
==========================================
  Files          88       88              
  Lines        6913     7195     +282     
==========================================
+ Hits         4054     4197     +143     
- Misses       2414     2534     +120     
- Partials      445      464      +19

Impacted Files	Coverage Δ
cmd/nginx/main.go	`5.61% <0.00%> (-0.80%)`	⬇️
...ternal/ingress/annotations/sessionaffinity/main.go	`60.41% <ø> (+1.59%)`	⬆️
internal/ingress/controller/template/template.go	`79.67% <0.00%> (-0.43%)`	⬇️
internal/ingress/types.go	`0.00% <ø> (ø)`
internal/k8s/main.go	`58.06% <0.00%> (-11.17%)`	⬇️
internal/ingress/controller/store/store.go	`58.77% <40.00%> (-1.32%)`	⬇️
internal/ingress/annotations/class/main.go	`54.54% <50.00%> (-34.35%)`	⬇️
internal/ingress/controller/controller.go	`49.59% <83.33%> (+0.18%)`	⬆️
internal/admission/controller/main.go	`90.47% <96.77%> (+4.36%)`	⬆️
internal/admission/controller/server.go	`100.00% <100.00%> (ø)`
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b18ea26...c035a14. Read the comment docs.

aledbf · 2020-05-19T13:46:56Z

/retest

aledbf · 2020-05-19T14:46:32Z

/retest

ElvinEfendi · 2020-05-20T11:20:44Z

@agile6v thanks for the PR. I think we should stick to how Nginx does this and support nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri$host" instead of nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri,$host".

Also I think we should only be changing lua_ngx_var function to support compound arguments like "$request_uri$host". Everything else will then just work.

agile6v · 2020-05-21T06:40:45Z

Hi @ElvinEfendi

@agile6v thanks for the PR. I think we should stick to how Nginx does this and support nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri$host" instead of nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri,$host".

Yes. I agree with your suggestion.

Also I think we should only be changing lua_ngx_var function to support compound arguments like "$request_uri$host". Everything else will then just work.

lua_ngx_var function will be called when every request comes. So in order to improve performance parse operation should be completed early.

Thanks.

aledbf · 2020-05-21T19:44:10Z

/retest

aledbf · 2020-05-21T19:52:29Z

@agile6v please squash the commits

k8s-ci-robot · 2020-05-22T02:32:36Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

agile6v · 2020-05-24T06:24:39Z

ping @ElvinEfendi

ElvinEfendi · 2020-05-28T15:13:01Z

rootfs/etc/nginx/lua/util.lua

+
+-- given the compound Nginx variables i.e {"$request_uri"} or {"$request_uri$host$scheme"}
+-- or {"${request_uri}-text-value"}, it returns the parsed value.
+function _M.lua_ngx_var(data)


Can we delete this now? Does not seem to be used anymore.

OK. I move function lua_ngx_var to util_test.lua file.

What I meant this PR replaced lua_ngx_var with something else. Don't you think we should remove all references of lua_ngx_var (including tests), and add tests for the two new utility functions you introduced?

Hi @ElvinEfendi The name of lua_ngx_var function is preserved but it contains two utility functions introduced this time. Function parse_complex_value and generate_var_value need to be used together (split into two functions because of performance considerations), So they are not individually written unit tests.

split into two functions because of performance considerations

This indicates me they should be individually unit tested. But I don't feel strongly if you don't add individual unit tests for them in this PR. But IMHO we should definitely delete all references to lua_ngx_var as it is promoting less performant solution.

--

the integration test for them is already being done as part of chash balancer test.

Hi @ElvinEfendi What do you think of the latest commit?

ElvinEfendi · 2020-05-28T15:15:11Z

@agile6v sorry for delay here. The PR looks great! Only lua_ngx_var seems to be redundant now.

agile6v · 2020-05-29T05:01:19Z

/retest

ElvinEfendi · 2020-05-30T13:14:57Z

/retest

agile6v · 2020-05-31T15:13:57Z

/retest

aledbf · 2020-05-31T16:35:50Z

/retest

aledbf · 2020-05-31T16:37:21Z

The error in the lua job seems valid

ERROR: module 'ngx.process' not found:

aledbf · 2020-05-31T16:40:28Z

@agile6v please rebase and squash the commits

aledbf · 2020-05-31T16:43:59Z

Running the patch against master works

wget https://github.com/kubernetes/ingress-nginx/pull/5571.patch
--2020-05-31 12:40:43--  https://github.com/kubernetes/ingress-nginx/pull/5571.patch
Resolving github.com (github.com)... 140.82.118.3
Connecting to github.com (github.com)|140.82.118.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://patch-diff.githubusercontent.com/raw/kubernetes/ingress-nginx/pull/5571.patch [following]
--2020-05-31 12:40:44--  https://patch-diff.githubusercontent.com/raw/kubernetes/ingress-nginx/pull/5571.patch
Resolving patch-diff.githubusercontent.com (patch-diff.githubusercontent.com)... 140.82.118.3
Connecting to patch-diff.githubusercontent.com (patch-diff.githubusercontent.com)|140.82.118.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Cookie coming from patch-diff.githubusercontent.com attempted to set domain to github.com
Length: unspecified [text/plain]
Saving to: ‘5571.patch’

5571.patch                                                                                      [  <=>                                                                                                                                                                                                                                    ]  15,39K  69,2KB/s    in 0,2s    

2020-05-31 12:40:45 (69,2 KB/s) - ‘5571.patch’ saved [15759]

 ⚙ aledbf@me  ~/.../k8s.io/ingress-nginx   ref ●  patch -p1 < 5571.patch 
patching file docs/user-guide/nginx-configuration/annotations.md
Hunk #1 succeeded at 213 (offset 1 line).
patching file internal/ingress/annotations/upstreamhashby/main_test.go
patching file rootfs/etc/nginx/lua/balancer/chash.lua
patching file rootfs/etc/nginx/lua/balancer/chashsubset.lua
patching file rootfs/etc/nginx/lua/test/util_test.lua
patching file rootfs/etc/nginx/lua/util.lua
patching file rootfs/etc/nginx/lua/test/util_test.lua
patching file rootfs/etc/nginx/lua/util.lua
patching file rootfs/etc/nginx/lua/test/util_test.lua
 ⚙ aledbf@me  ~/.../k8s.io/ingress-nginx   master ●  make lua-test
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
143 successes / 0 failures / 0 errors / 0 pending : 0.090901 seconds

…ion upstream-hash-by.

aledbf · 2020-06-01T17:44:25Z

/retest

ElvinEfendi · 2020-06-01T20:52:56Z

/lgtm
/approve

thanks @agile6v !

k8s-ci-robot · 2020-06-01T20:53:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: agile6v, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ElvinEfendi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aledbf · 2020-06-01T21:48:22Z

/retest

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 19, 2020

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 19, 2020

k8s-ci-robot requested review from aledbf and ElvinEfendi May 19, 2020 03:56

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 19, 2020

k8s-ci-robot assigned ElvinEfendi May 19, 2020

ElvinEfendi mentioned this pull request May 20, 2020

not support variables concatanation for upstream-hash-by annotation and not support endpoints set weight #4846

Closed

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 21, 2020

k8s-ci-robot removed the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 22, 2020

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label May 22, 2020

agile6v force-pushed the dev branch from 681f907 to a22ec53 Compare May 22, 2020 03:26

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels May 22, 2020

agile6v force-pushed the dev branch from a22ec53 to bebd635 Compare May 22, 2020 07:08

ElvinEfendi reviewed May 28, 2020

View reviewed changes

Support the combination of nginx variables and text value for annotat…

c035a14

…ion upstream-hash-by.

agile6v force-pushed the dev branch from 4ed484a to c035a14 Compare May 31, 2020 22:38

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 1, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 1, 2020

k8s-ci-robot merged commit d061375 into kubernetes:master Jun 1, 2020

agile6v deleted the dev branch June 4, 2020 09:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support the combination of Nginx variables for annotation upstream-hash-by. #5571

feat: support the combination of Nginx variables for annotation upstream-hash-by. #5571

agile6v commented May 19, 2020

k8s-ci-robot commented May 19, 2020

aledbf commented May 19, 2020

aledbf commented May 19, 2020

aledbf commented May 19, 2020

codecov-commenter commented May 19, 2020 •

edited

Loading

aledbf commented May 19, 2020

aledbf commented May 19, 2020

ElvinEfendi commented May 20, 2020 •

edited

Loading

agile6v commented May 21, 2020 •

edited

Loading

aledbf commented May 21, 2020

aledbf commented May 21, 2020

k8s-ci-robot commented May 22, 2020

agile6v commented May 24, 2020

ElvinEfendi May 28, 2020

agile6v May 29, 2020

ElvinEfendi May 29, 2020

agile6v May 29, 2020

ElvinEfendi May 29, 2020

agile6v May 30, 2020

ElvinEfendi commented May 28, 2020

agile6v commented May 29, 2020

ElvinEfendi commented May 30, 2020

agile6v commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented Jun 1, 2020

ElvinEfendi commented Jun 1, 2020

k8s-ci-robot commented Jun 1, 2020

aledbf commented Jun 1, 2020

feat: support the combination of Nginx variables for annotation upstream-hash-by. #5571

feat: support the combination of Nginx variables for annotation upstream-hash-by. #5571

Conversation

agile6v commented May 19, 2020

What this PR does / why we need it:

Types of changes

Which issue/s this PR fixes

Checklist:

k8s-ci-robot commented May 19, 2020

aledbf commented May 19, 2020

aledbf commented May 19, 2020

aledbf commented May 19, 2020

codecov-commenter commented May 19, 2020 • edited Loading

Codecov Report

aledbf commented May 19, 2020

aledbf commented May 19, 2020

ElvinEfendi commented May 20, 2020 • edited Loading

agile6v commented May 21, 2020 • edited Loading

aledbf commented May 21, 2020

aledbf commented May 21, 2020

k8s-ci-robot commented May 22, 2020

agile6v commented May 24, 2020

ElvinEfendi May 28, 2020

Choose a reason for hiding this comment

agile6v May 29, 2020

Choose a reason for hiding this comment

ElvinEfendi May 29, 2020

Choose a reason for hiding this comment

agile6v May 29, 2020

Choose a reason for hiding this comment

ElvinEfendi May 29, 2020

Choose a reason for hiding this comment

agile6v May 30, 2020

Choose a reason for hiding this comment

ElvinEfendi commented May 28, 2020

agile6v commented May 29, 2020

ElvinEfendi commented May 30, 2020

agile6v commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented May 31, 2020

aledbf commented Jun 1, 2020

ElvinEfendi commented Jun 1, 2020

k8s-ci-robot commented Jun 1, 2020

aledbf commented Jun 1, 2020

codecov-commenter commented May 19, 2020 •

edited

Loading

ElvinEfendi commented May 20, 2020 •

edited

Loading

agile6v commented May 21, 2020 •

edited

Loading