Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clusterroles.rbac.authorization.k8s.io "rook-operator" is forbidden: attempt to grant extra privileges: #14168

Closed
debianmaster opened this issue May 12, 2017 · 5 comments · Fixed by #15021

Comments

@debianmaster
Copy link

debianmaster commented May 12, 2017

I'm trying to create a rook.io storage cluster on openshift master branch version as of today.
And seeing following error

Version
oc v3.6.0-alpha.1+f24a57f-496
kubernetes v1.6.1+5115d708d7
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://34.209.8.118.nip.io:8443
openshift v3.6.0-alpha.1+f24a57f-496
kubernetes v1.6.1+5115d708d7
Steps To Reproduce
git clone https://github.com/rook/rook.git
cd rook
oc new-project rook
oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:rook:rook-api
oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:default:rook-operator
oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:rook:default
oc adm policy add-scc-to-user anyuid -z default -n rook
oc adm policy add-scc-to-user privileged -z default -n rook
oc create -f demo/kubernetes/rook-operator.yaml
Current Result
Error from server (Forbidden): error when creating "demo/kubernetes/rook-operator.yaml": clusterroles.rbac.authorization.k8s.io "rook-operator" is forbidden: attempt to grant extra privileges: [{[get] [] [namespaces] [] []} {[list] [] [namespaces] [] []} {[watch] [] [namespaces] [] []} {[create] [] [namespaces] [] []} {[update] [] [namespaces] [] []} {[get] [] [serviceaccounts] [] []} {[list] [] [serviceaccounts] [] []} {[watch] [] [serviceaccounts] [] []} {[create] [] [serviceaccounts] [] []} {[update] [] [serviceaccounts] [] []} {[get] [] [secrets] [] []} {[list] [] [secrets] [] []} {[watch] [] [secrets] [] []} {[create] [] [secrets] [] []} {[update] [] [secrets] [] []} {[get] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []} {[create] [] [pods] [] []} {[update] [] [pods] [] []} {[get] [] [services] [] []} {[list] [] [services] [] []} {[watch] [] [services] [] []} {[create] [] [services] [] []} {[update] [] [services] [] []} {[get] [] [nodes] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[create] [] [nodes] [] []} {[update] [] [nodes] [] []} {[get] [] [configmaps] [] []} {[list] [] [configmaps] [] []} {[watch] [] [configmaps] [] []} {[create] [] [configmaps] [] []} {[update] [] [configmaps] [] []} {[get] [] [events] [] []} {[list] [] [events] [] []} {[watch] [] [events] [] []} {[create] [] [events] [] []} {[update] [] [events] [] []} {[get] [] [persistentvolumes] [] []} {[list] [] [persistentvolumes] [] []} {[watch] [] [persistentvolumes] [] []} {[create] [] [persistentvolumes] [] []} {[update] [] [persistentvolumes] [] []} {[get] [] [persistentvolumeclaims] [] []} {[list] [] [persistentvolumeclaims] [] []} {[watch] [] [persistentvolumeclaims] [] []} {[create] [] [persistentvolumeclaims] [] []} {[update] [] [persistentvolumeclaims] [] []} {[get] [extensions] [thirdpartyresources] [] []} {[list] [extensions] [thirdpartyresources] [] []} {[watch] [extensions] [thirdpartyresources] [] []} {[create] [extensions] [thirdpartyresources] [] []} {[get] [extensions] [deployments] [] []} {[list] [extensions] [deployments] [] []} {[watch] [extensions] [deployments] [] []} {[create] [extensions] [deployments] [] []} {[get] [extensions] [daemonsets] [] []} {[list] [extensions] [daemonsets] [] []} {[watch] [extensions] [daemonsets] [] []} {[create] [extensions] [daemonsets] [] []} {[get] [extensions] [replicasets] [] []} {[list] [extensions] [replicasets] [] []} {[watch] [extensions] [replicasets] [] []} {[create] [extensions] [replicasets] [] []} {[get] [rbac.authorization.k8s.io] [clusterroles] [] []} {[list] [rbac.authorization.k8s.io] [clusterroles] [] []} {[watch] [rbac.authorization.k8s.io] [clusterroles] [] []} {[create] [rbac.authorization.k8s.io] [clusterroles] [] []} {[update] [rbac.authorization.k8s.io] [clusterroles] [] []} {[get] [rbac.authorization.k8s.io] [clusterrolebindings] [] []} {[list] [rbac.authorization.k8s.io] [clusterrolebindings] [] []} {[watch] [rbac.authorization.k8s.io] [clusterrolebindings] [] []} {[create] [rbac.authorization.k8s.io] [clusterrolebindings] [] []} {[update] [rbac.authorization.k8s.io] [clusterrolebindings] [] []} {[get] [storage.k8s.io] [storageclasses] [] []} {[list] [storage.k8s.io] [storageclasses] [] []} {[watch] [storage.k8s.io] [storageclasses] [] []} {[*] [rook.io] [*] [] []}] user=&{system:admin  [system:cluster-admins system:authenticated] map[]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /swaggerapi /swaggerapi/* /version]}] ruleResolutionErrors=[]
Expected Result

operation should succeed and a pod should be created.

@debianmaster debianmaster changed the title Error from server (Forbidden): error when creating "demo/kubernetes/rook-operator.yaml": clusterroles.rbac.authorization.k8s.io "rook-operator" is forbidden May 12, 2017
@debianmaster debianmaster changed the title clusterroles.rbac.authorization.k8s.io "rook-operator" is forbidden clusterroles.rbac.authorization.k8s.io "rook-operator" is forbidden: attempt to grant extra privileges: May 12, 2017
@enj
Copy link
Contributor

enj commented May 12, 2017

@deads2k would #14064 fix this? The rule coverage is failing since kube RBAC is not aware of the * * * that applies to cluster admin.

@debianmaster I suggest converting those resources into the origin equivalents, either by hand or something like this.

@deads2k
Copy link
Contributor

deads2k commented May 12, 2017

kube RBAC rules aren't respected and will be deleted/stomped to match openshift RBAC resources.

@enj
Copy link
Contributor

enj commented May 12, 2017

@debianmaster based on @deads2k's comment you have no choice but to convert those resources to openshift RBAC.

@debianmaster
Copy link
Author

:( is there a plan to change this to match w/ k8s ?

Trying to understand more. is k8s RBAC not same as Openshift RBAC ?

@enj
Copy link
Contributor

enj commented May 12, 2017

@debianmaster

:( is there a plan to change this to match w/ k8s ?

I think that is at least a release away.

Trying to understand more. is k8s RBAC not same as Openshift RBAC ?

There are a lot of subtle differences.

@enj enj added this to the 3.7.0 milestone May 12, 2017
@simo5 simo5 mentioned this issue Jul 19, 2017
67 tasks
openshift-merge-robot added a commit that referenced this issue Aug 17, 2017
Automatic merge from submit-queue

Migrate to Kubernetes RBAC

Trello xref: https://trello.com/c/n3bR3Ys9

Fixes #12303
Fixes #13549
Fixes #13432
Fixes #15338
Fixes #14168
Fixes #10056

Need to investigate:

- [x] ...

Dependencies:

- [x] Prerequisite #15342
- [x] Requires openshift/openshift-ansible/pull/4933 @sdodson 
- [x] Blocked on openshift/openshift-ansible/issues/4967
- [x] Prerequisite kubernetes/kubernetes#50639

Followups:
- [ ] #15412
- [ ] #13316
- [ ] #13156
- [ ] #13430
- [ ] Should delete with proxy return details?
- [ ] Make project creation use RBAC instead of proxy endpoints?
- [ ] Remove policy objects from bootstrap roles
- [ ] Check if delegated_test.go can be revived
- [ ] Check to see if the deleted unit tests are reflected upstream and fix gaps
- [ ] Open issue to remove `openshiftSubjectLocator`
- [ ] Open issue to revisit forbidden message maker
- [ ] Update upstream `subject_locator_test` with origin's extensive testing
- [ ] Fix proxied create: ` _ bool is includeUnintialized, which we should really be passing through to the underlying API... it's odd there's not a CreateOptions parameter to Create`
- [ ] Fix proxied update: `if initializers use Update() to initialize objects (which I think they do), we may need to pass GetOptions{IncludeUninitialized: true} here...`
- [ ] Fix panics() in Convert...OrDie() functions
- [ ] glog.Fatal on post stark hook error
- [ ] Remove `TestPolicyCache`?
- [ ] Use discovery API based gating?
- [ ] upstream rules have always required a group. followup issue to remove getAPIGroupLegacy from `pkg/authorization/authorizer/scope/converter.go`
- [ ] issue to remove "normalizeResources" from `pkg/cmd/server/bootstrappolicy/policy.go`
- [ ] issue to  find callers of `clusterpolicyregistry "github.com/openshift/origin/pkg/authorization/registry/clusterpolicy"` and move to point of use
- [ ] issue to switch our encoding to rbac in `pkg/cmd/server/admin/create_bootstrappolicy_file.go`
- [ ] Exercise proxied endpoints
- [ ] hack/test-cmd.sh of gated overwrite bootstrap policy
- [ ] Delete unused legacy policy registry code
- [ ] Make RBAC discovery rule authoritative `pkg/authorization/apis/authorization/types.go`
- [ ] Fix `ignoreError` in `pkg/oc/admin/router/router.go`
- [ ] Confirm changes to `TestAuthorizationResolution` and `TestAuthorizationResourceAccessReview` in `test/integration/authorization_test.go`

Done:

- Store ClusterRoles as native RBAC Objects via Kubernetes.
- Provides backwards compatible API for the old policy based roles.
- Use Kubernetes authorizer

TODO:

- [x] Delete policy end points
- [x] Decide what to do with overwrite policy
- [x] Remove or gate `oc create policybinding`
- [x] Move new impersonation code to `pkg/auth/client/impersonate.go`
- [x] Remove any unnecessary conversions
- [x] Review new `proxy.go` files
- [x] Remove reason logic `allowed by rule in ...`
- [x] Add interface assertion to proxy files
- [x] Confirm we need `pkg/authorization/util/convert/convert.go`
- [x] Confrim we need to expose some of the private conversion functions
- [x] Add protect/autoupdate annotation conversion to general conversion functions
- [x] ~~Support watch on proxied endpoints~~
- [x] Cherry pick kubernetes/kubernetes#49868 -> #15721
- [x] Fix upstream commits
- [x] Restore and version gate `NewCmdMigrateAuthorization`
- [x] ~~Wrap other errors in proxy files?~~ Remove all error wrapping
- [x] Make `NewImpersonatingRBACFromContext` more generic
- [x] Kube authorizer's reason on deny contains evaluation errors - do we want to preserve those?
- [x] Review `ImpersonatingRESTClient` in `pkg/auth/client/impersonate.go`
- [ ] Review `pkg/project/auth/cache.go` and ` pkg/project/auth/cache_test.go`
- [ ] Review ` pkg/authorization/authorizer/scope/converter_test.go`
- [ ] Review `k8s.io/kubernetes/staging/src/k8s.io/client-go/rest/request.go`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants