Add PathPattern (directory discovery) to ArtifactGenerator

[vocarista]

Part of controlplaneio-fluxcd/flux-operator#421

This PR introduces the pathPattern feature to the ArtifactGenerator CRD to support dynamic, directory-based ExternalArtifact generation. This resolves the need for users to manually define an OutputArtifact for every single directory (e.g., every app/environment in a monorepo).

Example

Given a monorepo with this directory structure:

apps/
├── auth/
│   └── envs/
│       ├── dev/
│       │   └── app.yaml
│       └── prod/
│           └── app.yaml
└── payments/
    └── envs/
        ├── dev/
        │   └── app.yaml
        └── staging/
            └── app.yaml

Before (static)

Each artifact must be explicitly listed. Adding a new app or environment requires updating this spec:

apiVersion: source.extensions.fluxcd.io/v1beta1
kind: ArtifactGenerator
metadata:
  name: platform-apps
  namespace: flux-system
spec:
  sources:
    - alias: monorepo
      kind: GitRepository
      name: my-monorepo
  artifacts:
    - name: auth-dev
      copy:
        - from: "@monorepo/apps/auth/envs/dev/**"
          to: "@artifact/"
    - name: auth-prod
      copy:
        - from: "@monorepo/apps/auth/envs/prod/**"
          to: "@artifact/"
    - name: payments-dev
      copy:
        - from: "@monorepo/apps/payments/envs/dev/**"
          to: "@artifact/"
    - name: payments-staging
      copy:
        - from: "@monorepo/apps/payments/envs/staging/**"
          to: "@artifact/"

After (with pathPattern)

A single pathPattern replaces all explicit entries. The artifacts block uses {capture} placeholders that are expanded once per matched directory:

apiVersion: source.extensions.fluxcd.io/v1beta1
kind: ArtifactGenerator
metadata:
  name: platform-apps
  namespace: flux-system
spec:
  sources:
    - alias: monorepo
      kind: GitRepository
      name: my-monorepo
  pathPattern: "@monorepo/apps/{app}/envs/{env}"
  artifacts:
    - name: "{app}-{env}"
      copy:
        - from: "@monorepo/apps/{app}/envs/{env}/**"
          to: "@artifact/"

Both specs produce the same 4 ExternalArtifact objects in the cluster. With pathPattern, captured variables can be used as placeholders and are injected as labels on each generated ExternalArtifact (e.g., app: auth, env: dev).

When a new directory (e.g., apps/payments/envs/prod/) is added to the repo, the controller automatically creates the corresponding ExternalArtifact — no spec update required. Conversely, when a directory is removed, its ExternalArtifact is garbage collected.

Key Features & Implementation Details:

1. API Extension: Added spec.pathPattern to ArtifactGeneratorSpec and support for capture placeholders in dynamic artifact fields.
2. Dynamic Discovery: Implemented directory traversal that dynamically converts patterns like @monorepo/apps/{app}/envs/{env} into regex with named capture groups to match target directories.
3. Placeholder Rendering: When pathPattern is defined, spec.outputArtifacts acts as a template. The Name, Copy.From, and Copy.To fields are rendered by substituting capture placeholders (e.g., {app}-{env}).
4. Label Injection: Captured path variables are automatically injected as standard Kubernetes labels on the generated ExternalArtifact objects, facilitating easy discovery by downstream controllers (like Kustomize or Helm).
5. Selective Reconciliation: Ensured the pipeline isolates revisions correctly, meaning if only apps/auth/envs/dev changes, only the auth-dev artifact gets a new revision hash, leaving other untouched artifacts intact.
6. Documentation: Added the "Path Pattern (Directory Discovery)" section to the V1Beta1 spec docs.

Testing:

1. Unit Tests (artifactgenerator_pathpattern_test.go): Covered static paths, single capture patterns, invalid formats, exact-depth boundary files, and Exclude/Strategy field preservation.
2. Integration Tests (artifactgenerator_controller_test.go): Covered end-to-end artifact generation, strict label assertions, unpacking and asserting against physical copy paths, garbage collection on directory deletion, and selective revision updates.

[matheuscscp]

@vocarista Please edit the PR description with YAML examples explaining the new feature's behavior.

[matheuscscp]

LGTM! 🚀

Thanks very much @vocarista! I had to fast-forward things here a little bit since we're very close to the Flux 2.9 release (Mon/Tue next week), so I wanted to get this done and release source-watcher. You can continue from the flux-operator side, we don't need to rush there.

[stefanprodan]

LGTM