-
Notifications
You must be signed in to change notification settings - Fork 256
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update controller runtime #1800
Conversation
@kannon92: The label(s) In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: kannon92 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for kubernetes-sigs-kueue canceled.
|
Opened this up to demonstrate the failures. Discussion here: kubernetes-sigs/controller-runtime#2681 |
@alculquicondor @tenzen-y any ideas on this one? Trying to add the patch interceptor but it is still saying patch is not allowed. Do I need to use a different patch option? |
Where? Can you expand? |
if patch.Type() != types.ApplyPatchType { | ||
return clnt.Patch(ctx, obj, patch, opts...) | ||
} | ||
check, ok := obj.DeepCopyObject().(client.Object) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would expect you to override the type to types.StrategicMergePatchType
.
Also, maybe we should have NewFakeClientWithWeakSSA
, that we explicitly use when we are sure it's safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only issue is that we hit loads of these failures throughout Kueue.
Its whatever UpdateStatus is called so there are quite a few tests that fail with this.
pkg/util/testing/client.go
Outdated
|
||
kueue "sigs.k8s.io/kueue/apis/kueue/v1beta1" | ||
"sigs.k8s.io/kueue/pkg/controller/core/indexer" | ||
) | ||
|
||
func NewFakeClient(objs ...client.Object) client.Client { | ||
return NewClientBuilder().WithObjects(objs...).WithStatusSubresource(objs...).Build() | ||
return NewClientBuilder().WithObjects(objs...).WithStatusSubresource(objs...). | ||
WithInterceptorFuncs(interceptor.Funcs{Patch: func(ctx context.Context, clnt client.WithWatch, obj client.Object, patch client.Patch, opts ...client.PatchOption) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kannon92 we are mostly using SSA to update the status try to do the same
SubResourcePatch func(ctx context.Context, client client.Client, subResourceName string, obj client.Object, patch client.Patch, opts ...client.SubResourcePatchOption) error
also I don't think we can hit the object not found, so maybe try without that part
I just mean that I am still hitting the same failures that the unit tests are getting. I can reproduce the failures with just the workload_test.go so I use that to do a quick test. |
WithInterceptorFuncs(interceptor.Funcs{SubResourcePatch: func(ctx context.Context, clnt client.Client, SubResourceName string, obj client.Object, patch client.Patch, opts ...client.SubResourcePatchOption) error { | ||
// Apply patches are supposed to upsert, but fake client fails if the object doesn't exist, | ||
// if an apply patch occurs for an object that doesn't yet exist, create it. | ||
if patch.Type() != types.ApplyPatchType { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using a debugger, I can confirm that we are hitting this function now.
I don't really know what behavior we actually want in this function though. I took this code from an example about how to work around SSA patch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The existing logic is just trying to use a Create instead of a Patch when the object doesn't exist. That's one part of the problem, although we never use SSA for objects that don't exist in Kueue. The next one is the patch type. We need to make it a StrategicMergePatch.
@kannon92: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
||
kueue "sigs.k8s.io/kueue/apis/kueue/v1beta1" | ||
"sigs.k8s.io/kueue/pkg/controller/core/indexer" | ||
) | ||
|
||
func NewFakeClient(objs ...client.Object) client.Client { | ||
return NewClientBuilder().WithObjects(objs...).WithStatusSubresource(objs...).Build() | ||
return NewClientBuilder().WithObjects(objs...).WithStatusSubresource(objs...). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh, this function already calls NewclientBuilder, so no need to add the interceptor again.
"k8s.io/apimachinery/pkg/runtime" | ||
"k8s.io/apimachinery/pkg/types" | ||
clientgoscheme "k8s.io/client-go/kubernetes/scheme" | ||
"sigs.k8s.io/controller-runtime/pkg/client" | ||
"sigs.k8s.io/controller-runtime/pkg/client/fake" | ||
"sigs.k8s.io/controller-runtime/pkg/client/interceptor" | ||
|
||
kueue "sigs.k8s.io/kueue/apis/kueue/v1beta1" | ||
"sigs.k8s.io/kueue/pkg/controller/core/indexer" | ||
) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wold just go with something like:
type smPatchWrapper struct {
client.Patch
}
func (*smPatchWrapper) Type() types.PatchType {
return types.StrategicMergePatchType
}
func wrapSSA(patch client.Patch) client.Patch {
if patch.Type() == types.ApplyPatchType {
return &smPatchWrapper{Patch: patch}
}
return patch
}
func SSAToSM(ctx context.Context, clnt client.Client, SubResourceName string, obj client.Object, patch client.Patch, opts ...client.SubResourcePatchOption) error {
return clnt.SubResource(SubResourceName).Patch(ctx, obj, wrapSSA(patch), opts...)
}
Then in the tests that require this , use this function as
manageBuilder = manageBuilder.WithInterceptorFuncs(interceptor.Funcs{SubResourcePatch: utiltesting.SSAToSM})
Note: It cannot be just added in 'NewClientBuilder' since:
- We don't need it in all the tests.
WithInterceptorFuncs
that might be needed afterNewClientBuilder
return will replace it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't declare manageBuilder anywhere other than in multikueue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is just an usage example
@trasc @alculquicondor I'm not too familar on SSA so @alculquicondor said that they will take this over. /close |
@kannon92: Closed this PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
What type of PR is this?
/kind dependency
What this PR does / why we need it:
Update controller runtime.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
This is failing due to
Does this PR introduce a user-facing change?