Skip to content

helm: cephfs: fix nodeplugin secrets RBAC for metadata KMS - #6464

Open
donch wants to merge 1 commit into
ceph:develfrom
donch:fix-nodeplugin-secrets-rbac
Open

helm: cephfs: fix nodeplugin secrets RBAC for metadata KMS#6464
donch wants to merge 1 commit into
ceph:develfrom
donch:fix-nodeplugin-secrets-rbac

Conversation

@donch

@donch donch commented Aug 6, 2026

Copy link
Copy Markdown

Summary

Fixes a case where the nodeplugin never gets secrets RBAC for
metadata-type encryption KMS, when encryptionKMSConfig.secretNamespace
is intentionally left unset to enable per-tenant secret resolution (the
Secret is resolved from the PVC's own namespace at runtime, per existing
documented behavior). This currently results in NodeStageVolume panicking
with a nil pointer dereference instead of returning a clean error.

Problem

The chart README documents two things separately, but the code doesn't
actually let you have both at once:

  1. Omitting encryptionKMSConfig.secretNamespace is the documented way to
    resolve the encryption Secret from each tenant's own namespace at mount
    time (per-tenant encryption keys).
  2. rbac.leastPrivileges: false is documented to grant "global secret
    access to the cluster... via a ClusterRole" — i.e. exactly the kind of
    cross-namespace access needed for (1) to actually work.

But both RBAC templates for the nodeplugin (nodeplugin-clusterrole.yaml
for rbac.leastPrivileges: false, and the namespaced Role template for
rbac.leastPrivileges: true) are additionally gated on
.Values.encryptionKMSConfig.secretNamespace being set:

{{- if and .Values.encryptionKMSConfig .Values.encryptionKMSConfig.secretNamespace (not .Values.rbac.leastPrivileges) }}
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch"]
{{- end -}}

So in practice, following the documented per-tenant pattern (1) means this
condition is never true, and the promised global ClusterRole (2) is never
generated — contradicting the README. The nodeplugin ends up with zero
secrets permissions, and volumes using this setup fail to mount.

Fix

This PR drops the secretNamespace requirement from the
rbac.leastPrivileges: false ClusterRole condition, so the chart actually
delivers what the README already promises for this mode:

{{- if and .Values.encryptionKMSConfig (not .Values.rbac.leastPrivileges) }}
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch"]
{{- end -}}

This scope was already global (no resourceNames) prior to this change, so
this doesn't widen the permission set granted when the rule does apply —
it only widens when the rule is generated, to also cover the
secretNamespace-omitted case.

The rbac.leastPrivileges: true namespaced Role template is intentionally
left untouched: it relies on a single, known secretNamespace +
secretName to scope a Role via resourceNames, which is fundamentally
incompatible with "any tenant namespace, any secret name" resolution. Making
least-privilege RBAC work for the per-tenant case would need a different
mechanism (e.g. a values-provided list of tenant namespaces to generate
scoped Roles for), which felt like a bigger design discussion than this
PR should carry. Happy to follow up separately once there's agreement on
the right shape for that.

Testing

  • Reproduced the panic on cephcsi v3.16.2 and v3.17.0 with
    rbac.leastPrivileges: false, encryptionKMSType: metadata,
    secretNamespace omitted, and a per-tenant-namespace Secret.
  • Applied this patch locally (helm upgrade from a local chart checkout,
    keeping the existing stable cephcsi image), confirmed the generated
    ClusterRole now includes the secrets rule, and confirmed a fresh PVC
    mounts and encrypts correctly end-to-end (verified content and filenames
    are encrypted at rest via a separate unprivileged mount without the
    fscrypt key unlocked).

Both nodeplugin RBAC templates were gated on
encryptionKMSConfig.secretNamespace being set, but omitting it is
the documented way to resolve the encryption secret per-tenant
namespace. This meant the nodeplugin never got secrets RBAC in that
case, contradicting the leastPrivileges: false documentation and
causing NodeStageVolume to fail for per-tenant metadata KMS setups.

Signed-off-by: David Donchez <david.donchez@enix.fr>
@donch
donch requested review from a team as code owners August 6, 2026 16:01
@mergify mergify Bot added component/cephfs Issues related to CephFS component/deployment Helm chart, kubernetes templates and configuration Issues/PRs bug Something isn't working labels Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working component/cephfs Issues related to CephFS component/deployment Helm chart, kubernetes templates and configuration Issues/PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant