feat: add a flag to remove k0rdent CRDs on management removal#2539
Open
kylewuolle wants to merge 3 commits intok0rdent:mainfrom
Open
feat: add a flag to remove k0rdent CRDs on management removal#2539kylewuolle wants to merge 3 commits intok0rdent:mainfrom
kylewuolle wants to merge 3 commits intok0rdent:mainfrom
Conversation
kylewuolle
requested review from
BROngineer,
Kshatrix,
a13x5,
eromanova,
wahabmk and
zerospiel
as code owners
Kshatrix
requested changes
Mar 17, 2026
…ield. Also, fetch installed helm charts to find additional CRDs that were created
zerospiel
requested changes
Mar 18, 2026
| for _, f := range rel.Chart.Files { | ||
| if strings.HasPrefix(f.Name, "crds/") { | ||
| var obj apiextv1.CustomResourceDefinition | ||
| _ = yaml.Unmarshal(f.Data, &obj) |
Collaborator
There was a problem hiding this comment.
do i undestand it correctly, that we perform a whole CRD object deserialization (which is normally huge) just to extract its name and along the way skipping the error checking? what if it is not a CRD? it is not a big deal in this case, but one redundant delete API call will be made
i'd suggest something more lightweight, maybe extracting the raw data could help, if it can avoid object's deserialization
zerospiel
reviewed
Mar 18, 2026
…leases don't always correlate to CRDs. Made the cleanupCRDs flag plural as well.
eromanova
reviewed
Mar 26, 2026
| "network.azure.com", | ||
| "openstack.k-orc.cloud", | ||
| "resources.azure.com", | ||
| "source.toolkit.fluxcd.io", |
Collaborator
There was a problem hiding this comment.
We have a scenario where k0rdent is deployed with Flux disabled, since flux can be already installed on the cluster beforehand. The current logic removes all Flux CRDs (even unmanaged) during k0rdent deinstallation, which is likely not the desired behavior. The same issue applies to other components such as velero, capi operator, etc.
I think we need to check labels on CRDs instead (for example, all kcm-managed management and regional components have meta.helm.sh/release-name: kcm label) or think about a better option for identifying k0rdent-related CRDs for dependent components.
Contributor
Author
There was a problem hiding this comment.
The problem is a lot of CRDs don't have the kcm label set. We could restrict deletion to just CRDs with that label but it'd leave many others in place.
Here's the list of CRDs without the kcm label currently:
awsclustercontrolleridentities.infrastructure.cluster.x-k8s.io
awsclusterroleidentities.infrastructure.cluster.x-k8s.io
awsclusters.infrastructure.cluster.x-k8s.io
awsclusterstaticidentities.infrastructure.cluster.x-k8s.io
awsclustertemplates.infrastructure.cluster.x-k8s.io
awsfargateprofiles.infrastructure.cluster.x-k8s.io
awsmachinepools.infrastructure.cluster.x-k8s.io
awsmachines.infrastructure.cluster.x-k8s.io
awsmachinetemplates.infrastructure.cluster.x-k8s.io
awsmanagedclusters.infrastructure.cluster.x-k8s.io
awsmanagedclustertemplates.infrastructure.cluster.x-k8s.io
awsmanagedcontrolplanes.controlplane.cluster.x-k8s.io
awsmanagedcontrolplanetemplates.controlplane.cluster.x-k8s.io
awsmanagedmachinepools.infrastructure.cluster.x-k8s.io
azureasomanagedclusters.infrastructure.cluster.x-k8s.io
azureasomanagedclustertemplates.infrastructure.cluster.x-k8s.io
azureasomanagedcontrolplanes.infrastructure.cluster.x-k8s.io
azureasomanagedcontrolplanetemplates.infrastructure.cluster.x-k8s.io
azureasomanagedmachinepools.infrastructure.cluster.x-k8s.io
azureasomanagedmachinepooltemplates.infrastructure.cluster.x-k8s.io
azureclusteridentities.infrastructure.cluster.x-k8s.io
azureclusters.infrastructure.cluster.x-k8s.io
azureclustertemplates.infrastructure.cluster.x-k8s.io
azuremachinepoolmachines.infrastructure.cluster.x-k8s.io
azuremachinepools.infrastructure.cluster.x-k8s.io
azuremachines.infrastructure.cluster.x-k8s.io
azuremachinetemplates.infrastructure.cluster.x-k8s.io
azuremanagedclusters.infrastructure.cluster.x-k8s.io
azuremanagedclustertemplates.infrastructure.cluster.x-k8s.io
azuremanagedcontrolplanes.infrastructure.cluster.x-k8s.io
azuremanagedcontrolplanetemplates.infrastructure.cluster.x-k8s.io
azuremanagedmachinepools.infrastructure.cluster.x-k8s.io
azuremanagedmachinepooltemplates.infrastructure.cluster.x-k8s.io
backuprepositories.velero.io
backups.velero.io
backupstoragelocations.velero.io
bastionhosts.network.azure.com
clusterclasses.cluster.x-k8s.io
clusterresourcesetbindings.addons.cluster.x-k8s.io
clusterresourcesets.addons.cluster.x-k8s.io
clusters.cluster.x-k8s.io
clusters.k0smotron.io
datadownloads.velero.io
datauploads.velero.io
deletebackuprequests.velero.io
devclusters.infrastructure.cluster.x-k8s.io
devclustertemplates.infrastructure.cluster.x-k8s.io
devmachines.infrastructure.cluster.x-k8s.io
devmachinetemplates.infrastructure.cluster.x-k8s.io
dockerclusters.infrastructure.cluster.x-k8s.io
dockerclustertemplates.infrastructure.cluster.x-k8s.io
dockermachinepools.infrastructure.cluster.x-k8s.io
dockermachinepooltemplates.infrastructure.cluster.x-k8s.io
dockermachines.infrastructure.cluster.x-k8s.io
dockermachinetemplates.infrastructure.cluster.x-k8s.io
downloadrequests.velero.io
eksconfigs.bootstrap.cluster.x-k8s.io
eksconfigtemplates.bootstrap.cluster.x-k8s.io
extensionconfigs.runtime.cluster.x-k8s.io
extensions.kubernetesconfiguration.azure.com
fleetsmembers.containerservice.azure.com
gcpclusters.infrastructure.cluster.x-k8s.io
gcpclustertemplates.infrastructure.cluster.x-k8s.io
gcpmachines.infrastructure.cluster.x-k8s.io
gcpmachinetemplates.infrastructure.cluster.x-k8s.io
gcpmanagedclusters.infrastructure.cluster.x-k8s.io
gcpmanagedclustertemplates.infrastructure.cluster.x-k8s.io
gcpmanagedcontrolplanes.infrastructure.cluster.x-k8s.io
gcpmanagedcontrolplanetemplates.infrastructure.cluster.x-k8s.io
gcpmanagedmachinepools.infrastructure.cluster.x-k8s.io
gcpmanagedmachinepooltemplates.infrastructure.cluster.x-k8s.io
gkeconfigs.bootstrap.cluster.x-k8s.io
gkeconfigtemplates.bootstrap.cluster.x-k8s.io
globalinclusterippools.ipam.cluster.x-k8s.io
inclusterippools.ipam.cluster.x-k8s.io
infobloxinstances.ipam.cluster.x-k8s.io
infobloxippools.ipam.cluster.x-k8s.io
ipaddressclaims.ipam.cluster.x-k8s.io
ipaddresses.ipam.cluster.x-k8s.io
jointokenrequests.k0smotron.io
k0scontrollerconfigs.bootstrap.cluster.x-k8s.io
k0scontrolplanes.controlplane.cluster.x-k8s.io
k0scontrolplanetemplates.controlplane.cluster.x-k8s.io
k0smotroncontrolplanes.controlplane.cluster.x-k8s.io
k0smotroncontrolplanetemplates.controlplane.cluster.x-k8s.io
k0sworkerconfigs.bootstrap.cluster.x-k8s.io
k0sworkerconfigtemplates.bootstrap.cluster.x-k8s.io
kubevirtclusters.infrastructure.cluster.x-k8s.io
kubevirtclustertemplates.infrastructure.cluster.x-k8s.io
kubevirtmachines.infrastructure.cluster.x-k8s.io
kubevirtmachinetemplates.infrastructure.cluster.x-k8s.io
machinedeployments.cluster.x-k8s.io
machinedrainrules.cluster.x-k8s.io
machinehealthchecks.cluster.x-k8s.io
machinepools.cluster.x-k8s.io
machines.cluster.x-k8s.io
machinesets.cluster.x-k8s.io
managedclusters.containerservice.azure.com
managedclustersagentpools.containerservice.azure.com
natgateways.network.azure.com
openstackclusteridentities.infrastructure.cluster.x-k8s.io
openstackclusters.infrastructure.cluster.x-k8s.io
openstackclustertemplates.infrastructure.cluster.x-k8s.io
openstackfloatingippools.infrastructure.cluster.x-k8s.io
openstackmachines.infrastructure.cluster.x-k8s.io
openstackmachinetemplates.infrastructure.cluster.x-k8s.io
openstackservers.infrastructure.cluster.x-k8s.io
podvolumebackups.velero.io
podvolumerestores.velero.io
pooledremotemachines.infrastructure.cluster.x-k8s.io
privateendpoints.network.azure.com
remoteclusters.infrastructure.cluster.x-k8s.io
remoteclustertemplates.infrastructure.cluster.x-k8s.io
remotemachines.infrastructure.cluster.x-k8s.io
remotemachinetemplates.infrastructure.cluster.x-k8s.io
resourcegroups.resources.azure.com
restores.velero.io
rosaclusters.infrastructure.cluster.x-k8s.io
rosacontrolplanes.controlplane.cluster.x-k8s.io
rosamachinepools.infrastructure.cluster.x-k8s.io
rosanetworks.infrastructure.cluster.x-k8s.io
rosaroleconfigs.infrastructure.cluster.x-k8s.io
schedules.velero.io
serverstatusrequests.velero.io
virtualnetworks.network.azure.com
virtualnetworkssubnets.network.azure.com
volumesnapshotlocations.velero.io
vsphereclusteridentities.infrastructure.cluster.x-k8s.io
vsphereclusters.infrastructure.cluster.x-k8s.io
vsphereclustertemplates.infrastructure.cluster.x-k8s.io
vspheredeploymentzones.infrastructure.cluster.x-k8s.io
vspherefailuredomains.infrastructure.cluster.x-k8s.io
vspheremachines.infrastructure.cluster.x-k8s.io
vspheremachinetemplates.infrastructure.cluster.x-k8s.io
vspherevms.infrastructure.cluster.x-k8s.io
| "velero.io", | ||
| } | ||
|
|
||
| var crdList apiextv1.CustomResourceDefinitionList |
Collaborator
There was a problem hiding this comment.
Potential improvement: This cleanup logic should likely be extended to Regions as well. When a Region is deleted, all k0rdent-related CRDs within that Region should also be cleaned up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR adds a flag which defaults to false that will cleanup k0rdernt CRDs when the management object is deleted.
Which issue(s) this PR fixes (optional,
Fixes #123):Fixes #2374