Skip to content

nvmeof: node plugin refuses DH-CHAP volumes based on the kernel #6647

Open

Description

@gadididi

Describe the bug

The NVMe-oF node plugin refuses DH-CHAP volumes based on the kernel version string (uname -r < 6.0.0), introduced in PR #6301. Distro kernels that backport NVMe in-band authentication (e.g. RHEL 9.8, 5.14.0-687.x) report a version older than 6.0.0 but fully support DH-CHAP. On such nodes NodeStageVolume fails with DH-CHAP requires kernel v6.0.0 or newer even though DH-CHAP works.

Environment details

  • Image/version of Ceph CSI driver : ODF Ceph CSI 4.23 (nvmeof node plugin, includes PR nvmeof: gate DH-CHAP on kernel support #6301)
  • Helm chart version : N/A (deployed via Rook / ODF)
  • Kernel version : 5.14.0-687.42.1.el9_8.x86_64 (RHEL 9.8, worker node m-3)
  • Mounter used for mounting PVC (for cephFS its fuse or kernel. for rbd its
    krbd or rbd-nbd) : N/A (NVMe-oF)
  • Kubernetes cluster version : v1.36.3 (OpenShift)
  • Ceph cluster version : N/A

Steps to reproduce

Steps to reproduce the behavior:

  1. Setup details: OpenShift nodes running RHEL 9.8 (kernel 5.14.0-687.x), NVMe-oF gateway, StorageClass with dhchapMode: "unidirectional".
  2. Deployment to trigger the issue: create a PVC (volumeMode: Block) from that StorageClass and a pod using it.
  3. See error: pod stays in ContainerCreating, NodeStageVolume fails.

Actual results

NodeStageVolume returns:

rpc error: code = InvalidArgument desc = DH-CHAP requires kernel v6.0.0 or newer, running on 5.14.0-687.42.1.el9_8.x86_64

The pod was stuck for 2 days.

The node does support DH-CHAP:

  • nvme_auth is loaded and used by nvme_core (lsmod)
  • manual nvme connect --dhchap-secret ... from the node plugin container succeeds (connecting to device: nvme1)
  • the gateway shows Uses DHCHAP: Yes for the host NQN

Expected behavior

The driver should not gate DH-CHAP on the kernel version number. It should attempt the connect and return the real nvme connect error if the kernel does not support DH-CHAP.

Logs

  • csi-nvmeofplugin and driver-registrar logs from the node plugin pod on m-3 (openshift-storage.nvmeof.csi.ceph.com-nodeplugin-w62tg).

Additional context

Caused by PR #6301 (nvmeof: gate DH-CHAP on kernel support), which checks only the kernel version, based on the upstream in-band authentication commit (torvalds/linux@88b140fe). Vendor kernels that backport features break this check.

Activity

  1. self-assigned this
    on Oct 14, 2026
  2. added theissue type on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

component/nvme-ofIssues and PRs related to NVMe-oF.

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions