Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Fix object overwrite on patch for fake client #1651

Merged

Conversation

FillZpp
Copy link
Contributor

@FillZpp FillZpp commented Sep 3, 2021

Signed-off-by: FillZpp FillZpp.pub@gmail.com

Previously the fake client tried to decode the new object after patching to the object outside, which will cause some fields that should be removed in this patch remain in the object.

It should overwrite the object using reflect set, just like the way in cache get.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 3, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @FillZpp. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 3, 2021
@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 3, 2021
@FillZpp FillZpp changed the title Fix object overwrite on patch for fake client 🐛 Fix object overwrite on patch for fake client Sep 3, 2021
@FillZpp
Copy link
Contributor Author

FillZpp commented Sep 3, 2021

/cc @alvaroaleman @joelanford @vincepri

Copy link
Member

@alvaroaleman alvaroaleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is essentially the same issue as #1639 which was fixed by zeroing the target object in the decoder (but your approach is likely more efficient here).

Could you also do this for Get and List? They have the same issue

/ok-to-test

j, err := json.Marshal(o)
if err != nil {
return err
objVal := reflect.ValueOf(obj)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use reflect to assign the value, we need to deepcopy first

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 3, 2021
Signed-off-by: FillZpp <FillZpp.pub@gmail.com>
@FillZpp FillZpp force-pushed the fix-fake-client-patch-overwrite branch from 5a813b3 to af7484e Compare September 6, 2021 08:50
@FillZpp
Copy link
Contributor Author

FillZpp commented Sep 6, 2021

This is essentially the same issue as #1639 which was fixed by zeroing the target object in the decoder (but your approach is likely more efficient here).

Could you also do this for Get and List? They have the same issue

Sure. But I find that we can not simply use reflect.Set to overwrite the object, for it fails when use an unstructured object to get a typed object in client, which causes AssignableTo check error like this:

client had type *v1.DeploymentList, but *unstructured.UnstructuredList was asked for

So I have to zero each object before decode into it, just like the thing you do in targetZeroingDecoder.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 6, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alvaroaleman, FillZpp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 6, 2021
@k8s-ci-robot k8s-ci-robot merged commit 386c2b5 into kubernetes-sigs:master Sep 6, 2021
@k8s-ci-robot k8s-ci-robot added this to the v0.10.x milestone Sep 6, 2021
timebertt added a commit to timebertt/gardener that referenced this pull request Oct 14, 2021
These tests were not preparing the test objects correctly: they only updated
them in memory but not on the fake client. This wasn't caught until now
because the fake client mimicked the real json decoder, which didn't unset
fields not present on the server. Now that the fake client zeroes fields,
the tests started failing (which is correct). So fix the tests.

ref kubernetes-sigs/controller-runtime#1651
timebertt added a commit to gardener/gardener that referenced this pull request Oct 14, 2021
* Upgrade to k8s.io/*@v0.22.2 in go.mod

* [automated] make revendor

* [automated] make generate

* [automated] make revendor

github.com/go-openapi/spec seems to be orphaned after previous make generate

* Upgrade to c-r@v0.10.2 in go.mod

Also, upgrade setup-envtest (doesn't have a tagged release yet, so use
release commit instead)

* [automated] make revendor

* Upgrade to controller-tools@v0.7.0 in go.mod

* [automated] make revendor

* Add missing WarningsOn{Create,Update} to rest strategies

* Replace dot imports for github.com/onsi/gomega/types

Fix linting errors: `Assertion` redeclared in this block (typecheck)

* Switch to typed values for WebhookInstallOptions.*Webhooks

ref kubernetes-sigs/controller-runtime#1626

* RequestCertificate now takes an optional requestedDuration

ref kubernetes/kubernetes#99494

* Switch to matchers.DeepEqual to test semantic equality

Maps (e.g. labels, selectors, resource requirements) might be sorted differently
than expected. Hence, use semantic equality instead of strict equality, as this
is what matters to us.
Also, DeepEqual outputs yaml and adds a nice diff indicator instead of printing
some large confusing go struct representation.

* Add new memorySwap field to expected kubelet config

ref kubernetes/kubernetes#102823

* Round condition.lastUpdateTime to seconds in test

There were several changes in the fake clients that might cause the failure
to happen just now.

* Correct unit tests falsely succeeding

These tests were not preparing the test objects correctly: they only updated
them in memory but not on the fake client. This wasn't caught until now
because the fake client mimicked the real json decoder, which didn't unset
fields not present on the server. Now that the fake client zeroes fields,
the tests started failing (which is correct). So fix the tests.

ref kubernetes-sigs/controller-runtime#1651

* Remove workarounds for missing zeroing in json decoder

Now that the c-r client zeroes fields before decoding into the object,
we can drop our workarounds for this, so basically drop
kutil.CreateResetObjectFunc and its usages.

ref kubernetes-sigs/controller-runtime#1640

* Drop setting webhook gvk explicitly in envtest

webhookConfig.SetGroupVersionKind is not needed anymore with
kubernetes-sigs/controller-runtime#1665

* Add some follow-up TODO comments

* [automated] make generate

but with go 1.16.9

* Address review comments
krgostev pushed a commit to krgostev/gardener that referenced this pull request Apr 21, 2022
* Upgrade to k8s.io/*@v0.22.2 in go.mod

* [automated] make revendor

* [automated] make generate

* [automated] make revendor

github.com/go-openapi/spec seems to be orphaned after previous make generate

* Upgrade to c-r@v0.10.2 in go.mod

Also, upgrade setup-envtest (doesn't have a tagged release yet, so use
release commit instead)

* [automated] make revendor

* Upgrade to controller-tools@v0.7.0 in go.mod

* [automated] make revendor

* Add missing WarningsOn{Create,Update} to rest strategies

* Replace dot imports for github.com/onsi/gomega/types

Fix linting errors: `Assertion` redeclared in this block (typecheck)

* Switch to typed values for WebhookInstallOptions.*Webhooks

ref kubernetes-sigs/controller-runtime#1626

* RequestCertificate now takes an optional requestedDuration

ref kubernetes/kubernetes#99494

* Switch to matchers.DeepEqual to test semantic equality

Maps (e.g. labels, selectors, resource requirements) might be sorted differently
than expected. Hence, use semantic equality instead of strict equality, as this
is what matters to us.
Also, DeepEqual outputs yaml and adds a nice diff indicator instead of printing
some large confusing go struct representation.

* Add new memorySwap field to expected kubelet config

ref kubernetes/kubernetes#102823

* Round condition.lastUpdateTime to seconds in test

There were several changes in the fake clients that might cause the failure
to happen just now.

* Correct unit tests falsely succeeding

These tests were not preparing the test objects correctly: they only updated
them in memory but not on the fake client. This wasn't caught until now
because the fake client mimicked the real json decoder, which didn't unset
fields not present on the server. Now that the fake client zeroes fields,
the tests started failing (which is correct). So fix the tests.

ref kubernetes-sigs/controller-runtime#1651

* Remove workarounds for missing zeroing in json decoder

Now that the c-r client zeroes fields before decoding into the object,
we can drop our workarounds for this, so basically drop
kutil.CreateResetObjectFunc and its usages.

ref kubernetes-sigs/controller-runtime#1640

* Drop setting webhook gvk explicitly in envtest

webhookConfig.SetGroupVersionKind is not needed anymore with
kubernetes-sigs/controller-runtime#1665

* Add some follow-up TODO comments

* [automated] make generate

but with go 1.16.9

* Address review comments
krgostev pushed a commit to krgostev/gardener that referenced this pull request Jul 5, 2022
* Upgrade to k8s.io/*@v0.22.2 in go.mod

* [automated] make revendor

* [automated] make generate

* [automated] make revendor

github.com/go-openapi/spec seems to be orphaned after previous make generate

* Upgrade to c-r@v0.10.2 in go.mod

Also, upgrade setup-envtest (doesn't have a tagged release yet, so use
release commit instead)

* [automated] make revendor

* Upgrade to controller-tools@v0.7.0 in go.mod

* [automated] make revendor

* Add missing WarningsOn{Create,Update} to rest strategies

* Replace dot imports for github.com/onsi/gomega/types

Fix linting errors: `Assertion` redeclared in this block (typecheck)

* Switch to typed values for WebhookInstallOptions.*Webhooks

ref kubernetes-sigs/controller-runtime#1626

* RequestCertificate now takes an optional requestedDuration

ref kubernetes/kubernetes#99494

* Switch to matchers.DeepEqual to test semantic equality

Maps (e.g. labels, selectors, resource requirements) might be sorted differently
than expected. Hence, use semantic equality instead of strict equality, as this
is what matters to us.
Also, DeepEqual outputs yaml and adds a nice diff indicator instead of printing
some large confusing go struct representation.

* Add new memorySwap field to expected kubelet config

ref kubernetes/kubernetes#102823

* Round condition.lastUpdateTime to seconds in test

There were several changes in the fake clients that might cause the failure
to happen just now.

* Correct unit tests falsely succeeding

These tests were not preparing the test objects correctly: they only updated
them in memory but not on the fake client. This wasn't caught until now
because the fake client mimicked the real json decoder, which didn't unset
fields not present on the server. Now that the fake client zeroes fields,
the tests started failing (which is correct). So fix the tests.

ref kubernetes-sigs/controller-runtime#1651

* Remove workarounds for missing zeroing in json decoder

Now that the c-r client zeroes fields before decoding into the object,
we can drop our workarounds for this, so basically drop
kutil.CreateResetObjectFunc and its usages.

ref kubernetes-sigs/controller-runtime#1640

* Drop setting webhook gvk explicitly in envtest

webhookConfig.SetGroupVersionKind is not needed anymore with
kubernetes-sigs/controller-runtime#1665

* Add some follow-up TODO comments

* [automated] make generate

but with go 1.16.9

* Address review comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants