feat: add OpenMetadataEntityTag CRD#5
Open
berimbolo13 wants to merge 5 commits intomasterfrom
Open
Conversation
Introduces a new CRD for declaratively tagging OpenMetadata entities discovered by ingestion (tables, topics, schemas, etc.). The reconciler queries OM's search endpoint for entities matching FQN patterns and applies a tag via the bulk tag-asset endpoints, recording assignments in status to drive drift detection and rename cleanup.
There was a problem hiding this comment.
Pull request overview
Adds a new OpenMetadataEntityTag custom resource + controller path to declaratively apply/remove OpenMetadata tags on entities discovered via ingestion (i.e., not created by this operator), using OpenMetadata search + bulk tag-asset endpoints.
Changes:
- Introduces the
OpenMetadataEntityTagCRD/API types (match entity type + FQN patterns, tag FQN, connection ref) and wires the controller intocmd/main.go. - Implements reconciler/handler logic to search matched entities, diff vs.
status.tagAssignments, and bulk add/remove tags (including rename + finalizer cleanup paths). - Extends the OpenMetadata client package with search + bulk tag-asset operations and adds unit/controller tests plus RBAC/CRD manifests.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/omclient/types.go | Adds search/bulk-tag request/response types used by entity-tagging. |
| internal/omclient/interface.go | Introduces EntityTagClient interface for tagging/search operations. |
| internal/omclient/entitytag.go | Implements search pagination/query building and bulk add/remove tag-asset calls. |
| internal/handler/entitytag_index.go | Maps supported entity types to OpenMetadata search index names. |
| internal/handler/entitytag_handler.go | Core Observe→Compare→Converge logic, rename handling, and deletion finalizer cleanup. |
| internal/handler/entitytag_handler_test.go | Unit tests for diffing, recorded tag lookup, and index resolution. |
| internal/controller/openmetadataentitytag_controller.go | New controller wiring finalizer + delegation to handler. |
| internal/controller/openmetadataentitytag_controller_test.go | Envtest coverage for controller flow (finalizer, apply, delete cleanup). |
| config/rbac/role.yaml | Grants manager-role access to the new CR and its status/finalizers. |
| config/rbac/openmetadataentitytag_viewer_role.yaml | Adds read-only ClusterRole for the new CR. |
| config/rbac/openmetadataentitytag_editor_role.yaml | Adds editor ClusterRole for the new CR. |
| config/rbac/openmetadataentitytag_admin_role.yaml | Adds admin ClusterRole for the new CR. |
| config/rbac/kustomization.yaml | Includes the new RBAC role manifests. |
| config/crd/kustomization.yaml | Includes the new CRD base. |
| config/crd/bases/openmetadata.vortexa.com_openmetadataentitytags.yaml | New generated CRD manifest for OpenMetadataEntityTag. |
| cmd/main.go | Registers the new controller with the manager. |
| api/v1alpha1/zz_generated.deepcopy.go | Generated deepcopy updates for new API types. |
| api/v1alpha1/tag_types.go | Adds TagRef API type. |
| api/v1alpha1/openmetadataentitytag_types.go | Adds OpenMetadataEntityTag API types/spec/status. |
| api/v1alpha1/conditions.go | Adds new condition reasons for entity-tagging reconcile outcomes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+45
to
+55
| // SearchEntities returns every entity in the named search index whose | ||
| // fullyQualifiedName matches any include pattern and no exclude pattern. | ||
| // Wildcards '*' (zero or more chars) and '?' (one char) in patterns are | ||
| // passed through to OpenMetadata's search endpoint unchanged. Pagination is | ||
| // handled internally. | ||
| func (c *Client) SearchEntities(ctx context.Context, searchIndex string, includes, excludes []string) ([]EntitySummary, error) { | ||
| if len(includes) == 0 { | ||
| return nil, nil | ||
| } | ||
| q := buildSearchQuery(includes, excludes) | ||
|
|
Comment on lines
+123
to
+126
| msg := fmt.Sprintf("Applied %s to %d %s entities", tagFQN, len(matched), et.Spec.Match.EntityType) | ||
| h.setConditionAndPersist(ctx, et, metav1.ConditionTrue, omv1alpha1.ReasonInSync, msg) | ||
| h.emitEvent(et, corev1.EventTypeNormal, omv1alpha1.ReasonInSync, msg) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new
OpenMetadataEntityTagCRD for declaratively tagging OpenMetadata entities the operator does not create directly (tables, topics, databaseSchemas, databases, dashboards, mlmodels, pipelines, containers, searchIndexes), i.e. entities discovered by OM's ingestion pipelines.Each CR specifies:
spec.match.entityType: which OM entity type to targetspec.match.includes/excludes: Lucene-style FQN patterns (*and?wildcards) selecting the assets in scopespec.tag.tagFQN: the tag to apply (e.g.Tier.Tier3)spec.openMetadataConnectionRef: the cluster-scoped connectionThe reconciler:
/v1/search/queryendpoint with the include/exclude patterns to find currently-matching entitiesstatus.TagAssignments(what we applied last reconcile) and either:status.TagAssignments, sorted by FQNWhy
OM's ingestion pipelines write entities directly to the OM API, bypassing Kubernetes, so we have no native way to declaratively tag them.
Design notes
PUT /v1/tags/{id}/assets/addand.../remove, which take a tag UUID + list of asset references. One HTTP call per CR per reconcile, regardless of how many entities match.status.TagAssignmentsshares the sametagFQN. Simplifies rename detection (one comparison) and deletion (one bulk-remove).