helm: cephfs: fix nodeplugin secrets RBAC for metadata KMS - #6464
Open
donch wants to merge 1 commit into
Open
Conversation
Both nodeplugin RBAC templates were gated on encryptionKMSConfig.secretNamespace being set, but omitting it is the documented way to resolve the encryption secret per-tenant namespace. This meant the nodeplugin never got secrets RBAC in that case, contradicting the leastPrivileges: false documentation and causing NodeStageVolume to fail for per-tenant metadata KMS setups. Signed-off-by: David Donchez <david.donchez@enix.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a case where the nodeplugin never gets
secretsRBAC formetadata-type encryption KMS, whenencryptionKMSConfig.secretNamespaceis intentionally left unset to enable per-tenant secret resolution (the
Secret is resolved from the PVC's own namespace at runtime, per existing
documented behavior). This currently results in
NodeStageVolumepanickingwith a nil pointer dereference instead of returning a clean error.
Problem
The chart README documents two things separately, but the code doesn't
actually let you have both at once:
encryptionKMSConfig.secretNamespaceis the documented way toresolve the encryption Secret from each tenant's own namespace at mount
time (per-tenant encryption keys).
rbac.leastPrivileges: falseis documented to grant "global secretaccess to the cluster... via a ClusterRole" — i.e. exactly the kind of
cross-namespace access needed for (1) to actually work.
But both RBAC templates for the nodeplugin (
nodeplugin-clusterrole.yamlfor
rbac.leastPrivileges: false, and the namespacedRoletemplate forrbac.leastPrivileges: true) are additionally gated on.Values.encryptionKMSConfig.secretNamespacebeing set:So in practice, following the documented per-tenant pattern (1) means this
condition is never true, and the promised global ClusterRole (2) is never
generated — contradicting the README. The nodeplugin ends up with zero
secretspermissions, and volumes using this setup fail to mount.Fix
This PR drops the
secretNamespacerequirement from therbac.leastPrivileges: falseClusterRole condition, so the chart actuallydelivers what the README already promises for this mode:
This scope was already global (no
resourceNames) prior to this change, sothis doesn't widen the permission set granted when the rule does apply —
it only widens when the rule is generated, to also cover the
secretNamespace-omitted case.The
rbac.leastPrivileges: truenamespacedRoletemplate is intentionallyleft untouched: it relies on a single, known
secretNamespace+secretNameto scope aRoleviaresourceNames, which is fundamentallyincompatible with "any tenant namespace, any secret name" resolution. Making
least-privilege RBAC work for the per-tenant case would need a different
mechanism (e.g. a values-provided list of tenant namespaces to generate
scoped
Roles for), which felt like a bigger design discussion than thisPR should carry. Happy to follow up separately once there's agreement on
the right shape for that.
Testing
cephcsiv3.16.2andv3.17.0withrbac.leastPrivileges: false,encryptionKMSType: metadata,secretNamespaceomitted, and a per-tenant-namespace Secret.helm upgradefrom a local chart checkout,keeping the existing stable
cephcsiimage), confirmed the generatedClusterRolenow includes thesecretsrule, and confirmed a fresh PVCmounts and encrypts correctly end-to-end (verified content and filenames
are encrypted at rest via a separate unprivileged mount without the
fscrypt key unlocked).