Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rbd: consider ErrPermissionDenied for vol #4883

Merged
merged 1 commit into from
Oct 3, 2024
Merged

rbd: consider ErrPermissionDenied for vol #4883

merged 1 commit into from
Oct 3, 2024

Conversation

Madhu-1
Copy link
Collaborator

@Madhu-1 Madhu-1 commented Oct 1, 2024

In the case of RDR with restricted access the ceph user will not have access to all the objects or all the pools where mapping exists

This commit adds a check to continue to get the volume if there is a permission error

@mergify mergify bot added the component/rbd Issues related to RBD label Oct 1, 2024
@Madhu-1
Copy link
Collaborator Author

Madhu-1 commented Oct 1, 2024

@rewantsoni tested this PR in RDR cluster for rados namespace, @rewantsoni can you please confirm the changes works/provide logs.

nixpanic
nixpanic reviewed Oct 1, 2024
View reviewed changes
internal/rbd/rbd_util.go Outdated Show resolved Hide resolved
nixpanic
nixpanic reviewed Oct 1, 2024
View reviewed changes
internal/rbd/rbd_util.go Outdated Show resolved Hide resolved
@rewantsoni
Copy link

Without the change, we were getting the following error:

│ I1001 14:31:23.874736       1 rbd_util.go:1272] ID: 24 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd found new poolID mapping 3 for existing pooID 1 │
│ I1001 14:31:23.875896       1 omap.go:89] ID: 24 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd got omap values: (pool="ocs-storagecluster-cephblockp │
│ ool", namespace="cephradosnamespace-e4b341af7c3c8c27d840cca49a8cbefb", name="csi.volume.933c7538-9dec-4842-8cc5-48cf584af2fd"): map[csi.imageid:b47c9f79ee60]                                                   │
│ E1001 14:31:23.904043       1 utils.go:271] ID: 24 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd GRPC error: rpc error: code = Internal desc = faile │
│ d to get volume from id "0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd": rados: ret=-1, Operation not permitted  

VRC:
  spec:
    autoResync: false
    dataSource:
      apiGroup: ""
      kind: PersistentVolumeClaim
      name: my-pvc-rbd1
    replicationHandle: ""
    replicationState: primary
    volumeReplicationClass: vrc-test
  status:
    conditions:
    - lastTransitionTime: "2024-10-01T11:32:38Z"
      message: ""
      observedGeneration: 4
      reason: FailedToPromote
      status: "False"
      type: Completed
    - lastTransitionTime: "2024-10-01T11:32:38Z"
      message: ""
      observedGeneration: 4
      reason: Error
      status: "True"
      type: Degraded
    - lastTransitionTime: "2024-10-01T11:32:38Z"
      message: ""
      observedGeneration: 4
      reason: NotResyncing
      status: "False"
      type: Resyncing
    - lastTransitionTime: "2024-10-01T11:33:55Z"
      message: ""
      observedGeneration: 4
      reason: PrerequisiteMet
      status: "True"
      type: Validated
    message: 'failed to get volume from id "0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd":
      rados: ret=-1, Operation not permitted'
    observedGeneration: 4
    state: Unknown

With the fix, it is working as expected

E1001 15:24:57.953560       1 rbd_util.go:1232] ID: 30 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd [Madhu-2] failed to generate volume from volume ID (0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd) (rados: ret=-1, Operation not permitted)
I1001 15:24:57.980634       1 mirror.go:283] ID: 30 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd Site status of MirrorUUID: , state: stopped, description: local image is primary, lastUpdate: 1727796279, up: true
I1001 15:24:57.980655       1 mirror.go:283] ID: 30 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd Site status of MirrorUUID: 897ec90e-bfcb-47fe-a25f-8d8e8a508758, state: replaying, description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1727796237,"remote_snapshot_timestamp":1727796237,"replay_state":"idle"}, lastUpdate: 1727796278, up: true
I1001 15:24:57.980667       1 replication.go:917] ID: 30 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1727796237,"remote_snapshot_timestamp":1727796237,"replay_state":"idle"}
I1001 15:24:57.980892       1 utils.go:273] ID: 30 Req-ID: 0001-0020-81fd58def37a35d1b4d4bee7ba35c3c3-0000000000000001-933c7538-9dec-4842-8cc5-48cf584af2fd GRPC response: {"last_sync_duration":{},"last_sync_time":{"seconds":1727796237}}


VRC:
  spec:
    autoResync: false
    dataSource:
      apiGroup: ""
      kind: PersistentVolumeClaim
      name: my-pvc-rbd1
    replicationHandle: ""
    replicationState: primary
    volumeReplicationClass: vrc-test
  status:
    conditions:
    - lastTransitionTime: "2024-10-01T15:23:42Z"
      message: ""
      observedGeneration: 10
      reason: Promoted
      status: "True"
      type: Completed
    - lastTransitionTime: "2024-10-01T15:23:52Z"
      message: ""
      observedGeneration: 10
      reason: Healthy
      status: "False"
      type: Degraded
    - lastTransitionTime: "2024-10-01T15:23:52Z"
      message: ""
      observedGeneration: 10
      reason: NotResyncing
      status: "False"
      type: Resyncing
    - lastTransitionTime: "2024-10-01T11:33:55Z"
      message: ""
      observedGeneration: 8
      reason: PrerequisiteMet
      status: "True"
      type: Validated
    lastCompletionTime: "2024-10-01T15:49:59Z"
    lastSyncDuration: 0s
    lastSyncTime: "2024-10-01T15:49:00Z"
    message: volume is marked primary
    observedGeneration: 10
    state: Primary

@Madhu-1 Madhu-1 force-pushed the fix-rdr branch 2 times, most recently from cc22912 to 6ec3765 Compare October 2, 2024 09:10
@Madhu-1 Madhu-1 requested a review from nixpanic October 2, 2024 09:10
@nixpanic nixpanic requested a review from a team October 2, 2024 16:07
@Rakshith-R
Copy link
Contributor

@Mergifyio queue

@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 3, 2024 •
edited
Loading

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 88b964f

@Madhu-1
rbd: consider ErrPermissionDenied for vol
abdc933 
Incase of RDR with restricted access the
ceph user will not have access to all the objects
or all the pools where mapping exists

This commits add a check to continue to get
the volume if there is a permission error

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
@mergify mergify bot added the ok-to-test Label to trigger E2E tests label Oct 3, 2024
@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/upgrade-tests-cephfs

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/upgrade-tests-rbd

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/k8s-e2e-external-storage/1.31

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/k8s-e2e-external-storage/1.30

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e-helm/k8s-1.31

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e-helm/k8s-1.30

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e/k8s-1.31

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e/k8s-1.30

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/k8s-e2e-external-storage/1.29

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e-helm/k8s-1.29

@ceph-csi-bot
Copy link
Collaborator

/test ci/centos/mini-e2e/k8s-1.29

@ceph-csi-bot ceph-csi-bot removed the ok-to-test Label to trigger E2E tests label Oct 3, 2024
@mergify mergify bot merged commit 88b964f into ceph:devel Oct 3, 2024
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rakshith-R Rakshith-R Rakshith-R approved these changes

@nixpanic nixpanic nixpanic approved these changes

Assignees
No one assigned
Labels
component/rbd Issues related to RBD
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Madhu-1 @rewantsoni @Rakshith-R @ceph-csi-bot @nixpanic