From 48beb386e6a0dbfea929af7656db159ca427c198 Mon Sep 17 00:00:00 2001 From: Mevan Date: Fri, 24 Jul 2026 00:02:24 +0530 Subject: [PATCH] docs: add v1.2.0-rc.2 pre-release docs Snapshot current docs as opt-in v1.2.0-rc.2 version (non-default, noIndex, unreleased banner) alongside v1.2.0-rc.1 and v1.2.0-m.1; keep v1.1.x as lastVersion. Point githubRef at the release-v1.2.0-rc.2 branch since it already exists, and add the v1.2.0-rc.2 row to the changelog, comparing against v1.2.0-rc.1. Signed-off-by: Mevan --- docs/changelog.md | 1 + docusaurus.config.ts | 5 + .../version-v1.2.0-rc.2/_constants.mdx | 13 + .../version-v1.2.0-rc.2/ai/finops-agent.mdx | 195 + .../ai/mcp-prompt-scenarios.mdx | 70 + .../version-v1.2.0-rc.2/ai/mcp-servers.mdx | 350 ++ .../version-v1.2.0-rc.2/ai/overview.mdx | 38 + .../ai/portal-assistant.mdx | 262 ++ .../version-v1.2.0-rc.2/ai/sre-agent.mdx | 196 + .../version-v1.2.0-rc.2/changelog.md | 48 + .../concepts/developer-abstractions.md | 160 + .../concepts/platform-abstractions.md | 315 ++ .../concepts/resource-relationships.md | 211 ++ .../concepts/runtime-model.md | 122 + .../developer-guide/dependencies/endpoints.md | 225 ++ .../developer-guide/dependencies/resources.md | 235 ++ .../deploy-and-promote.md | 79 + .../environment-overrides.md | 105 + .../logs-and-troubleshooting.md | 108 + .../deploying-applications/overview.md | 49 + .../creating-a-component.md | 246 ++ .../creating-a-project.md | 100 + .../projects-and-components/overview.md | 101 + .../workflows/ci/auto-build.md | 82 + .../workflows/ci/images/create-secret.png | Bin 0 -> 499580 bytes .../workflows/ci/images/secret-management.png | Bin 0 -> 316974 bytes .../ci/images/workflow-selection.png | Bin 0 -> 514075 bytes .../developer-guide/workflows/ci/overview.md | 239 ++ .../workflows/ci/private-repository.md | 177 + .../workflows/ci/workload-descriptor.md | 174 + .../developer-guide/workload/overview.md | 172 + .../version-v1.2.0-rc.2/ecosystem/overview.md | 34 + .../getting-started/cli-installation.mdx | 203 ++ .../getting-started/deploy-and-explore.mdx | 144 + .../getting-started/examples-catalog.mdx | 271 ++ .../getting-started/qsg-backstage-ui.png | Bin 0 -> 619762 bytes .../qsg-install-status-success.png | Bin 0 -> 588941 bytes .../getting-started/quick-start-guide.mdx | 320 ++ .../try-it-out/on-k3d-locally.mdx | 971 +++++ .../on-k3d-locally/k3d-observability-plane.sh | 89 + .../on-k3d-locally/k3d-prerequisites.sh | 79 + .../try-it-out/on-your-environment.mdx | 1268 +++++++ .../overview/architecture.mdx | 419 +++ .../overview/what-is-openchoreo.mdx | 91 + .../air-gapped-installation.mdx | 535 +++ .../api-gateway/dataplane.png | Bin 0 -> 542044 bytes .../api-gateway/environment.png | Bin 0 -> 538302 bytes .../api-gateway/modules.md | 27 + .../api-gateway/topology.md | 230 ++ .../api-management.mdx | 0 .../platform-engineer-guide/authorization.md | 688 ++++ .../authorization/conditions.md | 141 + .../authorization/custom-roles.mdx | 185 + .../images/access-control-page.png | Bin 0 -> 416239 bytes ...-binding-creation-allow-deny-selection.png | Bin 0 -> 358127 bytes ...le-binding-creation-conditions-editing.png | Bin 0 -> 428987 bytes ...role-binding-creation-review-selection.png | Bin 0 -> 342291 bytes ...ole-binding-creation-role-mapping-view.png | Bin 0 -> 334888 bytes .../role-binding-creation-role-mapping.png | Bin 0 -> 366509 bytes ...ole-binding-creation-subject-selection.png | Bin 0 -> 377685 bytes .../authorization/images/role-binding-tab.png | Bin 0 -> 472179 bytes ...role-dialog-with-quick-start-templates.png | Bin 0 -> 435948 bytes .../authorization/overview.md | 195 + .../backstage-configuration.mdx | 799 +++++ .../backstage-ownership.mdx | 117 + .../backstage-plugins/catalog-sync.mdx | 101 + .../compatibility-matrix.mdx | 97 + .../backstage-plugins/entity-views.mdx | 208 ++ .../installing-into-existing-backstage.mdx | 925 +++++ .../migration-1.1-to-1.2.mdx | 138 + .../backstage-plugins/overview.mdx | 67 + .../backstage-plugins/permission-policy.mdx | 97 + .../backstage-plugins/troubleshooting.mdx | 327 ++ .../backstage-scaffolder-templates.mdx | 572 +++ .../cli-configuration.mdx | 143 + .../cluster-agent-rbac.mdx | 272 ++ .../component-types/overview.md | 357 ++ .../component-types/patching-syntax.md | 479 +++ .../component-types/schema-syntax.md | 472 +++ .../component-types/templating-syntax.md | 383 ++ .../component-types/validation-rules.md | 540 +++ .../container-registry-configuration.mdx | 201 ++ .../deployment-topology.mdx | 462 +++ .../external-ca-tls-setup.md | 480 +++ .../build-and-release-workflows.mdx | 107 + .../gitops/automations/bulk-promote.mdx | 80 + .../gitops/overview.md | 400 +++ .../gitops/using-flux-cd.mdx | 133 + .../identity-configuration.mdx | 258 ++ .../air-gapped-installation-overview.drawio | 90 + .../air-gapped-installation-overview.svg | 4 + .../images/catalog-import-wizard.png | Bin 0 -> 511885 bytes .../images/create-page.png | Bin 0 -> 604502 bytes .../images/custom-templates-in-scaffolder.png | Bin 0 -> 473715 bytes .../modules/building-a-module.md | 320 ++ .../observability-logging-adapter-api.mdx | 15 + .../observability-metrics-adapter-api.mdx | 15 + .../observability-tracing-adapter-api.mdx | 15 + .../modules/overview.md | 112 + .../multi-cluster-connectivity.mdx | 527 +++ .../namespace-management.mdx | 471 +++ .../observability-alerting.mdx | 646 ++++ .../platform-engineer-guide/resource-types.md | 301 ++ .../secret-management.mdx | 285 ++ .../upgrades/overview.mdx | 115 + .../upgrades/v1.0-to-v1.1.mdx | 105 + .../upgrades/v1.1-to-v1.2.mdx | 520 +++ .../workflows/auto-build.mdx | 89 + .../workflows/ci-governance.md | 244 ++ .../workflows/creating-workflows.mdx | 633 ++++ .../workflows/external-ci.mdx | 604 ++++ .../workflows/images/architecture.png | Bin 0 -> 287206 bytes .../workflows/images/ci-architecture.png | Bin 0 -> 256866 bytes .../workflows/overview.md | 151 + .../workflows/running-workflows.md | 108 + .../workflows/schema-syntax.md | 470 +++ .../workflow-workload-configuration.mdx | 145 + .../workflows/workload-generation.md | 154 + .../reference/api/application/component.md | 181 + .../reference/api/application/project.md | 92 + .../reference/api/application/resource.md | 164 + .../reference/api/application/workflowrun.md | 261 ++ .../reference/api/application/workload.md | 285 ++ .../reference/api/platform/authzrole.md | 85 + .../api/platform/authzrolebinding.md | 212 ++ .../api/platform/clusterauthzrole.md | 81 + .../api/platform/clusterauthzrolebinding.md | 192 + .../api/platform/clustercomponenttype.md | 427 +++ .../api/platform/clusterdataplane.md | 255 ++ .../api/platform/clusterobservabilityplane.md | 234 ++ .../api/platform/clusterresourcetype.md | 138 + .../reference/api/platform/clustertrait.md | 466 +++ .../reference/api/platform/clusterworkflow.md | 202 ++ .../api/platform/clusterworkflowplane.md | 204 ++ .../reference/api/platform/componenttype.md | 499 +++ .../reference/api/platform/dataplane.md | 420 +++ .../api/platform/deployment-pipeline.md | 104 + .../reference/api/platform/environment.md | 186 + .../api/platform/observabilityalertrule.md | 244 ++ .../observabilityalertsnotificationchannel.md | 202 ++ .../api/platform/observabilityplane.md | 334 ++ .../reference/api/platform/releasebinding.md | 319 ++ .../api/platform/resourcereleasebinding.md | 190 + .../reference/api/platform/resourcetype.md | 316 ++ .../reference/api/platform/secretreference.md | 258 ++ .../reference/api/platform/trait.md | 477 +++ .../reference/api/platform/workflow.md | 484 +++ .../reference/api/platform/workflowplane.md | 321 ++ .../reference/api/runtime/componentrelease.md | 258 ++ .../reference/api/runtime/renderedrelease.md | 163 + .../reference/api/runtime/resourcerelease.md | 162 + .../reference/cel/built-in-functions.md | 228 ++ .../reference/cel/context-variables.md | 462 +++ .../reference/cel/helper-functions.md | 786 ++++ .../reference/cli-reference.md | 3160 +++++++++++++++++ .../version-v1.2.0-rc.2/reference/faq.md | 146 + .../reference/helm/_category_.json | 5 + .../reference/helm/control-plane.mdx | 390 ++ .../reference/helm/data-plane.mdx | 147 + .../reference/helm/observability-plane.mdx | 232 ++ .../reference/helm/workflow-plane.mdx | 97 + .../reference/mcp-servers.mdx | 473 +++ .../releases/release-and-support-process.md | 54 + .../resources/ai-overview-diagram.svg | 2 + .../cell-runtime-with-portal-screenshot.png | Bin 0 -> 6018314 bytes .../resources/developer-api-overview.svg | 1 + .../openchoreo-architecture-components.svg | 2025 +++++++++++ .../openchoreo-cell-runtime-view.png | Bin 0 -> 217048 bytes .../openchoreo-development-abstractions.png | Bin 0 -> 264911 bytes .../openchoreo-diagram-v1-with-borders.png | Bin 0 -> 1041768 bytes .../openchoreo-platform-abstractions.png | Bin 0 -> 295345 bytes .../resources/openchoreo_components.svg | 1 + .../resources/platform-topology-overview.svg | 744 ++++ .../release-transformation-diagram.svg | 300 ++ .../resources/what-are-openchoreo-planes.svg | 467 +++ .../resources/what-is-openchoreo-overview.svg | 156 + .../tutorials/build-and-deploy-private.mdx | 457 +++ .../component-alerts-and-incidents.mdx | 735 ++++ ...nfigure-budget-alert-and-cost-analysis.mdx | 587 +++ .../tutorials/deploy-prebuilt-image.mdx | 274 ++ .../tutorials/deploy-with-configurations.mdx | 246 ++ .../version-v1.2.0-rc.2-sidebars.json | 728 ++++ versions.json | 1 + 183 files changed, 45642 insertions(+) create mode 100644 versioned_docs/version-v1.2.0-rc.2/_constants.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/finops-agent.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/mcp-prompt-scenarios.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/mcp-servers.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/overview.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/portal-assistant.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/ai/sre-agent.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/changelog.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/concepts/developer-abstractions.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/concepts/platform-abstractions.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/concepts/resource-relationships.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/concepts/runtime-model.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/endpoints.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/resources.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/deploy-and-promote.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/environment-overrides.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/logs-and-troubleshooting.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-component.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-project.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/auto-build.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/images/create-secret.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/images/secret-management.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/images/workflow-selection.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/private-repository.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/workload-descriptor.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/developer-guide/workload/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/ecosystem/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/cli-installation.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/deploy-and-explore.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/examples-catalog.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/qsg-backstage-ui.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/qsg-install-status-success.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/quick-start-guide.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/try-it-out/on-k3d-locally.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/try-it-out/on-k3d-locally/k3d-observability-plane.sh create mode 100755 versioned_docs/version-v1.2.0-rc.2/getting-started/try-it-out/on-k3d-locally/k3d-prerequisites.sh create mode 100644 versioned_docs/version-v1.2.0-rc.2/getting-started/try-it-out/on-your-environment.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/overview/architecture.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/overview/what-is-openchoreo.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/air-gapped-installation.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/api-gateway/dataplane.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/api-gateway/environment.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/api-gateway/modules.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/api-gateway/topology.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/api-management.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/conditions.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/custom-roles.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/access-control-page.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-allow-deny-selection.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-conditions-editing.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-review-selection.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-role-mapping-view.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-role-mapping.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-creation-subject-selection.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-binding-tab.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/images/role-dialog-with-quick-start-templates.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/authorization/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-configuration.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-ownership.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/catalog-sync.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/compatibility-matrix.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/entity-views.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/installing-into-existing-backstage.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/migration-1.1-to-1.2.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/overview.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/permission-policy.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-plugins/troubleshooting.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/backstage-scaffolder-templates.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/cli-configuration.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/cluster-agent-rbac.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/component-types/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/component-types/patching-syntax.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/component-types/schema-syntax.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/component-types/templating-syntax.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/component-types/validation-rules.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/container-registry-configuration.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/deployment-topology.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/external-ca-tls-setup.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/gitops/automations/build-and-release-workflows.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/gitops/automations/bulk-promote.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/gitops/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/gitops/using-flux-cd.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/identity-configuration.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/images/air-gapped-installation-overview.drawio create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/images/air-gapped-installation-overview.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/images/catalog-import-wizard.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/images/create-page.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/images/custom-templates-in-scaffolder.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/modules/building-a-module.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/modules/observability-logging-adapter-api.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/modules/observability-metrics-adapter-api.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/modules/observability-tracing-adapter-api.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/modules/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/multi-cluster-connectivity.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/namespace-management.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/observability-alerting.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/resource-types.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/secret-management.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/upgrades/overview.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/upgrades/v1.0-to-v1.1.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/upgrades/v1.1-to-v1.2.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/auto-build.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/ci-governance.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/creating-workflows.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/external-ci.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/images/architecture.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/images/ci-architecture.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/overview.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/running-workflows.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/schema-syntax.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/workflow-workload-configuration.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/platform-engineer-guide/workflows/workload-generation.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/application/component.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/application/project.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/application/resource.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/application/workflowrun.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/application/workload.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/authzrole.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/authzrolebinding.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterauthzrole.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterauthzrolebinding.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clustercomponenttype.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterdataplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterobservabilityplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterresourcetype.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clustertrait.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterworkflow.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/clusterworkflowplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/componenttype.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/dataplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/deployment-pipeline.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/environment.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/observabilityalertrule.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/observabilityalertsnotificationchannel.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/observabilityplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/releasebinding.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/resourcereleasebinding.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/resourcetype.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/secretreference.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/trait.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/workflow.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/platform/workflowplane.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/runtime/componentrelease.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/runtime/renderedrelease.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/api/runtime/resourcerelease.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/cel/built-in-functions.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/cel/context-variables.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/cel/helper-functions.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/cli-reference.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/faq.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/helm/_category_.json create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/helm/control-plane.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/helm/data-plane.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/helm/observability-plane.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/helm/workflow-plane.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/reference/mcp-servers.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/releases/release-and-support-process.md create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/ai-overview-diagram.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/cell-runtime-with-portal-screenshot.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/developer-api-overview.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo-architecture-components.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo-cell-runtime-view.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo-development-abstractions.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo-diagram-v1-with-borders.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo-platform-abstractions.png create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/openchoreo_components.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/platform-topology-overview.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/release-transformation-diagram.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/what-are-openchoreo-planes.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/resources/what-is-openchoreo-overview.svg create mode 100644 versioned_docs/version-v1.2.0-rc.2/tutorials/build-and-deploy-private.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/tutorials/component-alerts-and-incidents.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/tutorials/configure-budget-alert-and-cost-analysis.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/tutorials/deploy-prebuilt-image.mdx create mode 100644 versioned_docs/version-v1.2.0-rc.2/tutorials/deploy-with-configurations.mdx create mode 100644 versioned_sidebars/version-v1.2.0-rc.2-sidebars.json diff --git a/docs/changelog.md b/docs/changelog.md index b5b3eaf6..275dffab 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -9,6 +9,7 @@ description: Release history for OpenChoreo | Version | Date | Changelog | | ----------- | ---------- | ----------------------------------------------------------------------------------------- | +| v1.2.0-rc.2 | 2026-07-24 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.2.0-rc.1...v1.2.0-rc.2) | | v1.2.0-rc.1 | 2026-07-17 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.1.0...v1.2.0-rc.1) | | v1.2.0-m.1 | 2026-06-12 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.1.1...v1.2.0-m.1) | | v1.1.3 | 2026-07-22 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.1/CHANGELOG.md#v113) | diff --git a/docusaurus.config.ts b/docusaurus.config.ts index 6eb5e563..873ed805 100644 --- a/docusaurus.config.ts +++ b/docusaurus.config.ts @@ -126,6 +126,11 @@ const config: Config = { docs: { lastVersion, versions: { + 'v1.2.0-rc.2': { + label: 'v1.2.0-rc.2 (pre-release)', + banner: 'unreleased', + noIndex: true, + }, 'v1.2.0-rc.1': { label: 'v1.2.0-rc.1 (pre-release)', banner: 'unreleased', diff --git a/versioned_docs/version-v1.2.0-rc.2/_constants.mdx b/versioned_docs/version-v1.2.0-rc.2/_constants.mdx new file mode 100644 index 00000000..af1718a7 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/_constants.mdx @@ -0,0 +1,13 @@ +[//] # (This file stores the constants used across the documentation.) + +export const versions = { + dockerTag: "v1.2.0-rc.2", + githubRef: "release-v1.2.0-rc.2", // Used for the GitHub Raw URL references. Example: main, latest, v0.1.0 + helmChart: "1.2.0-rc.2", + helmSource: "oci://ghcr.io/openchoreo/helm-charts", +}; + +export const defaultCredentials = { + username: "admin@openchoreo.dev", + password: "Admin@123", +}; diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/finops-agent.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/finops-agent.mdx new file mode 100644 index 00000000..98ad0da6 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/finops-agent.mdx @@ -0,0 +1,195 @@ +--- +title: FinOps Agent +description: Configure the FinOps Agent in OpenChoreo for AI-powered cloud cost analysis and optimization. +--- + +import CodeBlock from "@theme/CodeBlock"; +import { versions } from "../_constants.mdx"; + +# FinOps Agent + +The FinOps Agent is an AI-powered component that analyzes Kubernetes resource usage and cloud cost data from your OpenChoreo components to generate reports with cost optimization recommendations. It integrates with Large Language Models (LLMs) to provide cost attribution, rightsizing suggestions, and actionable insights to help teams reduce cloud spend. + +## Prerequisites + +Before enabling the FinOps Agent, ensure the following: + +- OpenChoreo Observability Plane installed with at least a metrics module. +- An LLM API key from [OpenAI](https://platform.openai.com/) (support for other providers coming soon) +- [Metrics collection configured](../platform-engineer-guide/observability-alerting.mdx) for your components. + +## Enabling the FinOps Agent + +### Step 1: Install a FinOps module + +The FinOps Agent relies on a FinOps module to provide Kubernetes resource cost data. Install a FinOps module before enabling the agent — for example, the [OpenCost FinOps module](https://github.com/openchoreo/community-modules/tree/main/finops-opencost). + +### Step 2: Create the FinOps Agent Secret + +The FinOps Agent requires a Kubernetes Secret named `finops-agent` in the `openchoreo-observability-plane` namespace with the following keys: + +| Key | Description | +| --------------------- | -------------------------------------------------- | +| `LLM_API_KEY` | Your LLM provider API key | +| `OAUTH_CLIENT_SECRET` | OAuth client secret (only needed for external IdP) | + +You can create this secret using any method you prefer. If you followed the [Try It Out on k3d locally](../getting-started/try-it-out/on-k3d-locally.mdx) guide, you can follow along: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/finops-llm-api-key value="" +``` + +```bash +kubectl apply -f - < + {`helm upgrade --install openchoreo-observability-plane ${versions.helmSource}/openchoreo-observability-plane \\ + --version ${versions.helmChart} \\ + --namespace openchoreo-observability-plane \\ + --reuse-values \\ + --set finOpsAgent.enabled="true" \\ + --set finOpsAgent.llmName= \\ + --set finOpsAgent.remediationEnabled=true`} + + +:::note Supported Models +The FinOps Agent currently supports the [OpenAI](https://platform.openai.com/) GPT model series (e.g., `gpt-5.4`, `gpt-5.2-pro`, `gpt-5` etc.). Support for additional model providers is coming soon. + +The `gpt-5.6` series is not supported at the moment. Support is coming soon. +::: + +If the observability plane and control plane are in separate clusters, also set `finOpsAgent.openchoreoApiUrl` to the control plane API URL (defaults to `http://api.openchoreo.localhost:8080`). + +### Step 4: Register with the control plane + +Configure `finOpsAgentURL` in the `ClusterObservabilityPlane` resource so the UI knows where to reach the FinOps Agent: + + + {`kubectl patch clusterobservabilityplane default --type=merge -p '{"spec":{"finOpsAgentURL":"http://finops-agent.openchoreo.localhost:11080"}}'`} + + +### Step 5: Verify the installation + +Check that the FinOps Agent pod is running: + +```bash +kubectl get pods -n openchoreo-observability-plane -l app.kubernetes.io/component=finops-agent +``` + +If you are using the default identity provider (ThunderID) and the default SQLite report storage, your setup is complete. + +
+Authentication and Authorization (External IdP) + +By default, OpenChoreo configures ThunderID as the identity provider for the FinOps Agent with a pre-configured OAuth client for testing purposes. If you are using an external identity provider, follow the steps below. + +#### Authentication + +Create an OAuth 2.0 client that supports the `client_credentials` grant type for service-to-service authentication. + +Store your OAuth client secret in OpenBao: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/finops-agent-oauth-client-secret value="" +``` + +Then configure the Observability Plane Helm values with your client credentials: + +```yaml +security: + oidc: + tokenUrl: "" + +finOpsAgent: + secretName: "finops-agent" + oauth: + clientId: "" +``` + +See [Identity Provider Configuration](../platform-engineer-guide/identity-configuration.mdx) for detailed setup instructions. + +#### Authorization + +The FinOps Agent uses the `client_credentials` grant to authenticate with the OpenChoreo API as a service account. The API matches the `sub` claim in the issued JWT to identify the caller, so the new client must be granted the `finops-agent` role via a bootstrap authorization mapping. + +Add the following to your Control Plane values override, replacing `` with the same client ID used above: + +```yaml +openchoreoApi: + config: + security: + authorization: + bootstrap: + mappings: + - name: finops-agent-binding + roleRef: + name: finops-agent + entitlement: + claim: sub + value: "" + effect: allow +``` + +
+ +
+Report Storage + +By default, FinOps reports are stored in **SQLite** with a persistent volume — no external database required. + +For production deployments that need horizontal scaling or shared storage, you can use **PostgreSQL** instead. + +Store the PostgreSQL connection URI in OpenBao: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/finops-sql-backend-uri value="postgresql+asyncpg://:@:/" +``` + +Add the `SQL_BACKEND_URI` key to the ExternalSecret from [Step 2](#step-2-create-the-finops-agent-secret): + +```bash +kubectl patch externalsecret finops-agent -n openchoreo-observability-plane --type=json \ + -p '[{"op":"add","path":"/spec/data/-","value":{"secretKey":"SQL_BACKEND_URI","remoteRef":{"key":"finops-sql-backend-uri","property":"value"}}}]' +``` + +Then set the report backend in your Helm values: + +```yaml +finOpsAgent: + reportBackend: postgresql +``` + +
diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/mcp-prompt-scenarios.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/mcp-prompt-scenarios.mdx new file mode 100644 index 00000000..dba491ce --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/mcp-prompt-scenarios.mdx @@ -0,0 +1,70 @@ +--- +title: MCP Prompt Scenarios +--- + +# MCP Prompt Scenarios + +This page provides a collection of practical prompt scenarios for using the OpenChoreo MCP server with AI assistants. Each scenario is a hands-on guide that walks you through real-world tasks using natural language prompts. + +## Prerequisites + +Before trying these scenarios: + +1. **Configure your MCP server** — follow the [Configuring MCP Servers with AI Assistants](./mcp-servers.mdx) guide +2. **Review available tools** — see the [MCP Servers Reference](../reference/mcp-servers.mdx) for the full list of MCP tools + +## Scenarios + +### 1. Getting Started + +Learn the basics of connecting your AI assistant to OpenChoreo and performing simple operations like listing namespaces and projects. + +**Time:** ~2 minutes + +[View guide on GitHub →](https://github.com/openchoreo/openchoreo/tree/main/samples/mcp/getting-started) + +--- + +### 2. Service Deployment + +Deploy a complete service from source code to production using the OpenChoreo MCP server. Choose between a step-by-step guided walkthrough or a natural conversation-based deployment. + +**Time:** ~15-20 minutes + +[View guide on GitHub →](https://github.com/openchoreo/openchoreo/tree/main/samples/mcp/service-deployment) + +--- + +### 3. Log Analysis & Debugging + +Debug a cascading failure in the GCP Microservices Demo (Online Boutique) application. You'll intentionally break the product catalog service by scaling it to zero replicas, then use AI-assisted observability — logs, distributed traces, and deployment inspection — to diagnose and fix the issue across service boundaries. + +**Key MCP tools:** `list_components`, `query_component_logs`, `query_traces`, `query_trace_spans`, `get_release_binding`, `update_release_binding` + +**Time:** ~10 minutes + +[View guide on GitHub →](https://github.com/openchoreo/openchoreo/tree/main/samples/mcp/log-analysis) + +--- + +### 4. Build Failure Diagnosis + +Debug a Docker build failure in the Go Greeter service. You'll trigger a build with a misconfigured Dockerfile path, then use AI-assisted workflow inspection and log analysis to diagnose the root cause, compare with the previous successful build, and apply the fix. + +**Key MCP tools:** `list_workflow_runs`, `get_workflow_run`, `query_workflow_logs`, `create_workflow_run` + +**Time:** ~10 minutes + +[View guide on GitHub →](https://github.com/openchoreo/openchoreo/tree/main/samples/mcp/build-failures) + +--- + +### 5. Resource Optimization + +Detect and fix over-provisioned deployments in the GCP Microservices Demo (Online Boutique). You'll intentionally allocate excessive CPU and memory to several services, then use AI-assisted analysis to compare allocation vs actual usage, get right-sizing recommendations, and apply optimized configurations. + +**Key MCP tools:** `list_components`, `list_release_bindings`, `get_release_binding`, `query_resource_metrics`, `update_release_binding` + +**Time:** ~10 minutes + +[View guide on GitHub →](https://github.com/openchoreo/openchoreo/tree/main/samples/mcp/resource-optimization) diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/mcp-servers.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/mcp-servers.mdx new file mode 100644 index 00000000..596afa35 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/mcp-servers.mdx @@ -0,0 +1,350 @@ +--- +title: Configuring MCP Servers with AI Assistants +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; +import { defaultCredentials } from "../_constants.mdx"; + +# Configuring MCP Servers with AI Assistants + +This guide shows how to connect your AI assistant to OpenChoreo MCP servers using the pre-configured default OAuth applications or custom setups. + +## Prerequisites + +Before following this guide: + +1. Review the [MCP Servers Reference](../reference/mcp-servers.mdx) to understand the available MCP servers and tools +2. Note your MCP server URLs from the MCP Servers Reference page +3. Have your AI assistant installed and ready to configure + +## Quick Start (Default Setup) + +This guide uses the pre-created OAuth applications available in the default ThunderID IdP that ships with the k3d setup. No manual IdP configuration is required. The following AI agents are supported: + +| AI Agent | Browser-Based | Client Credentials | +| ------------------------ | :-----------: | :----------------: | +| Claude Code | Yes | Yes | +| Codex CLI | — | Yes | +| Cursor | Yes | Yes | +| Gemini CLI | Yes | Yes | +| OpenCode CLI | Yes | Yes | +| VS Code (GitHub Copilot) | — | Yes | + + + + +Browser-based authentication uses the `user_mcp_client` OAuth application (public client with PKCE, authorization code grant), which is pre-created in the default ThunderID IdP. + +When prompted, log in with the default credentials: + +- **Username**: {defaultCredentials.username} +- **Password**: {defaultCredentials.password} + + + + +Add both MCP servers using the Claude Code CLI: + +```bash +claude mcp add --transport http \ + --client-id user_mcp_client --callback-port 8075 \ + openchoreo-cp http://api.openchoreo.localhost:8080/mcp + +claude mcp add --transport http \ + --client-id user_mcp_client --callback-port 8075 \ + openchoreo-obs http://observer.openchoreo.localhost:11080/mcp +``` + +Then, inside a Claude Code session, run `/mcp` to verify the servers are connected. Select a server and authenticate in the browser when prompted. + + + + +Add the following to `~/.cursor/mcp.json` (or via **Cursor Settings > Tools & Integrations > Add Custom MCP**): + +```json +{ + "mcpServers": { + "openchoreo-cp": { + "protocol": "http", + "url": "http://api.openchoreo.localhost:8080/mcp", + "auth": { + "CLIENT_ID": "user_mcp_client" + } + }, + "openchoreo-obs": { + "protocol": "http", + "url": "http://observer.openchoreo.localhost:11080/mcp", + "auth": { + "CLIENT_ID": "user_mcp_client" + } + } + } +} +``` + +On first use, a browser window opens for authentication. Log in with the default credentials above. + + + + +Add both MCP servers using the Gemini CLI. The CLI will automatically detect the OAuth 2.0 configuration from the server. + +```bash +gemini mcp add --transport http \ + openchoreo-cp http://api.openchoreo.localhost:8080/mcp + +gemini mcp add --transport http \ + openchoreo-obs http://observer.openchoreo.localhost:11080/mcp +``` + +On first use, a browser window opens for authentication. Log in with the default credentials above. Note that Gemini CLI uses `http://localhost:7777/oauth/callback` as its default redirect URI, so ensure it is added to your OAuth application's allowed redirect URIs. + + + + +Add the MCP server using the OpenCode CLI interactive wizard: + +```bash +opencode mcp add +``` + +When prompted, provide the following values: + +| Prompt | Value | +| -------------------------------------------------- | ------------------------------------------------------------------- | +| **Server name** | `openchoreo-cp` | +| **MCP server URL** | `http://api.openchoreo.localhost:8080/mcp` (or your MCP server URL) | +| **Does this server require OAuth authentication?** | `yes` | +| **Enter client ID** | `user_mcp_client` | +| **Do you have a client secret?** | `No` | + +Repeat the process for the observability server: + +```bash +opencode mcp add +``` + +| Prompt | Value | +| -------------------------------------------------- | ------------------------------------------------------------------------- | +| **Server name** | `openchoreo-obs` | +| **MCP server URL** | `http://observer.openchoreo.localhost:11080/mcp` (or your MCP server URL) | +| **Does this server require OAuth authentication?** | `yes` | +| **Enter client ID** | `user_mcp_client` | +| **Do you have a client secret?** | `No` | + +On first use, a browser window opens for authentication. Log in with the default credentials above. + + + + + + + +Client credentials authentication uses the `service_mcp_client` OAuth application (confidential client, client_credentials grant), which is pre-created in the default ThunderID IdP. This method works with any AI agent that supports custom HTTP headers. + +**Step 1: Obtain an Access Token** + +```bash +curl -s -X POST "http://thunder.openchoreo.localhost:8080/oauth2/token" \ + -H 'Content-Type: application/x-www-form-urlencoded' \ + -u 'service_mcp_client:service_mcp_client_secret' \ + -d 'grant_type=client_credentials' +``` + +The response contains an `access_token`: + +```json +{ + "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...", + "token_type": "Bearer", + "expires_in": 3600 +} +``` + +Copy the `access_token` value and replace `` in the configuration below. + +**Step 2: Configure Your AI Agent** + + + + +```bash +claude mcp add --transport http \ + openchoreo-cp http://api.openchoreo.localhost:8080/mcp \ + --header "Authorization: Bearer " + +claude mcp add --transport http \ + openchoreo-obs http://observer.openchoreo.localhost:11080/mcp \ + --header "Authorization: Bearer " +``` + + + + +Set the access token as an environment variable: + +```bash +export OPENCHOREO_MCP_TOKEN="" +``` + +Then add the following to `~/.codex/config.toml`: + +```toml +[mcp_servers.openchoreo-cp] +url = "http://api.openchoreo.localhost:8080/mcp" +bearer_token_env_var = "OPENCHOREO_MCP_TOKEN" + +[mcp_servers.openchoreo-obs] +url = "http://observer.openchoreo.localhost:11080/mcp" +bearer_token_env_var = "OPENCHOREO_MCP_TOKEN" +``` + + + + +Add the following to `~/.cursor/mcp.json` (or via **Cursor Settings > Tools & Integrations > Add Custom MCP**): + +```json +{ + "mcpServers": { + "openchoreo-cp": { + "protocol": "http", + "url": "http://api.openchoreo.localhost:8080/mcp", + "headers": { + "Authorization": "Bearer " + } + }, + "openchoreo-obs": { + "protocol": "http", + "url": "http://observer.openchoreo.localhost:11080/mcp", + "headers": { + "Authorization": "Bearer " + } + } + } +} +``` + + + + +```bash +gemini mcp add --transport http \ + --header "Authorization: Bearer " \ + openchoreo-cp http://api.openchoreo.localhost:8080/mcp + +gemini mcp add --transport http \ + --header "Authorization: Bearer " \ + openchoreo-obs http://observer.openchoreo.localhost:11080/mcp +``` + + + + +Add the following to `~/.config/opencode/opencode.json`: + +```json +{ + "mcp": { + "openchoreo-cp": { + "type": "remote", + "url": "http://api.openchoreo.localhost:8080/mcp", + "headers": { + "Authorization": "Bearer " + } + }, + "openchoreo-obs": { + "type": "remote", + "url": "http://observer.openchoreo.localhost:11080/mcp", + "headers": { + "Authorization": "Bearer " + } + } + } +} +``` + + + + +Create a `.vscode/mcp.json` file in your workspace: + +```json +{ + "servers": { + "openchoreo-cp": { + "type": "http", + "url": "http://api.openchoreo.localhost:8080/mcp", + "headers": { + "Authorization": "Bearer " + } + }, + "openchoreo-obs": { + "type": "http", + "url": "http://observer.openchoreo.localhost:11080/mcp", + "headers": { + "Authorization": "Bearer " + } + } + } +} +``` + + + + +:::note Token Expiry +Access tokens expire (default: 1 hour). When your token expires, re-run the `curl` command from Step 1 to obtain a new token and update your AI agent configuration. +::: + + + + +:::tip To get the most out of MCP, pair it with skills +MCP gives your agent the **capabilities** to talk to OpenChoreo. **Skills** teach it how to use them well, so it gets it right first try instead of you correcting it along the way. You can stay in your agentic environment instead of jumping to the UI or CLI. To get started, install one (or both) of the skills below: + +- **[`openchoreo-developer`](https://openchoreo.dev/ecosystem/item/?id=skill-openchoreo-developer)**: daily developer tasks like deploying components, triggering builds, configuring workloads, promoting across environments, and troubleshooting. +- **[`openchoreo-platform-engineer`](https://openchoreo.dev/ecosystem/item/?id=skill-openchoreo-platform-engineer)**: daily platform engineering tasks like authoring ComponentTypes, Traits, and Workflows, setting up environments and pipelines, registering planes, and configuring identity and authorization. + +Browse [our ecosystem](https://openchoreo.dev/ecosystem/?group=skill) for all available skills. +::: + +## Advanced Configuration + +For custom AI agents, non-default IdP configurations, or custom OAuth application setups. + +### Creating a Custom OAuth Application + +If you need to create your own OAuth application (e.g., for a different IdP, custom redirect URIs, or non-default client settings), create an application in your IdP with one of the following configurations: + +**Client Credentials (Programmatic Access):** + +| Setting | Value | +| ----------------- | ------------------------- | +| **Grant Type** | `client_credentials` | +| **Client ID** | Your chosen client ID | +| **Client Secret** | Your chosen client secret | + +:::info Custom Role and Role Binding Required +Client credentials clients require a role binding to authorize access to OpenChoreo resources. See the [Custom Roles and Bindings](../platform-engineer-guide/authorization/custom-roles.mdx) guide for details on creating roles and role bindings via the UI. +::: + +**Authorization Code (Browser-Based Access):** + +| Setting | Value | +| ------------------------ | ------------------------------------------ | +| **Grant Types** | `authorization_code`, `refresh_token` | +| **Client ID** | Your chosen client ID | +| **Public Client (PKCE)** | `true` | +| **Redirect URIs** | Configure per your AI agent's callback URL | +| **Scopes** | `openid`, `profile`, `email` | + +:::note Redirect URIs +The redirect URI must match your AI agent's OAuth callback URL. Refer to your AI agent's MCP documentation for the correct callback URL and port. +::: + +### Configuring Non-Default Installations + +If your OpenChoreo installation uses a custom base domain or non-standard ports, refer to the [MCP Servers Reference](../reference/mcp-servers.mdx#finding-mcp-server-urls) for instructions on constructing the correct MCP server URLs. Replace the `openchoreo.localhost` URLs in the examples above with your actual MCP server URLs. diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/overview.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/overview.mdx new file mode 100644 index 00000000..b7eba19b --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/overview.mdx @@ -0,0 +1,38 @@ +--- +title: Working with AI +--- + +# Working with AI + +AI is reshaping how platform engineering teams build, operate, and troubleshoot software. OpenChoreo is built to meet that shift — with first-class AI integration that lets your developers and operators work alongside AI assistants naturally. Whether you're using Claude Code, Cursor, Gemini CLI, or another AI assistant, OpenChoreo gives it the context it needs to act. + +![](../resources/ai-overview-diagram.svg) + +## MCP Servers + +OpenChoreo exposes [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) servers for both the **Control Plane** and the **Observability Plane**. This means your AI assistant can: + +- **Discover and manage resources** — list namespaces, projects, components, environments, and deployment pipelines directly from a chat prompt +- **Trigger and monitor builds** — kick off workflow runs and check their status without leaving your editor +- **Query observability data** — fetch logs, metrics, traces, alerts, and incidents by asking questions in natural language +- **Deploy and promote** — update release bindings and promote components across environments through conversational workflows + +This turns your AI assistant into an active participant in your platform operations — not just a code suggester, but a collaborator that understands your infrastructure. + +See the [MCP Servers Reference](../reference/mcp-servers.mdx) for the full list of available tools and server configuration options, and follow [Configuring MCP Servers with AI Assistants](./mcp-servers.mdx) to connect your AI assistant to OpenChoreo. + +## Built-in Agents + +Beyond MCP, OpenChoreo ships with AI agents that run autonomously inside the platform. + +### FinOps Agent + +Cloud costs grow quietly until they become a problem. The **[FinOps Agent](./finops-agent.mdx)** analyzes Kubernetes resource usage of your OpenChoreo components and generates cost optimization reports with rightsizing recommendations. Instead of manually correlating utilization data, teams get clear, LLM-generated guidance on where spend can be reduced. + +### SRE Agent + +When something goes wrong in production, finding the root cause is expensive. The **[SRE Agent](./sre-agent.mdx)** automates that investigation by analyzing logs, metrics, and traces from your OpenChoreo deployments and generating a report with likely root causes. Instead of engineers spending hours correlating signals across dashboards, the SRE Agent delivers actionable insights the moment an alert fires. + +### Portal Assistant + +Platform context is scattered across CRDs, dashboards, and log stores — answering "what is this component doing?" often means jumping between five tools. The **[Portal Assistant](./portal-assistant.mdx)** is a read-only chat assistant embedded in Backstage that talks to the OpenChoreo and Observability MCP servers (and the SRE Agent when deployed) so your team can ask questions in natural language — list projects, inspect a component, query runtime logs, or pull an existing RCA report — without leaving the developer portal. The Portal Assistant is read-only by contract: it can list, get, and query, but never create, update, or delete. diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/portal-assistant.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/portal-assistant.mdx new file mode 100644 index 00000000..23c24a22 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/portal-assistant.mdx @@ -0,0 +1,262 @@ +--- +title: Portal Assistant +description: Configure the Portal Assistant in OpenChoreo for AI-powered chat over your platform and runtime telemetry. +--- + +import CodeBlock from "@theme/CodeBlock"; +import { versions } from "../_constants.mdx"; + +# Portal Assistant + +The Portal Assistant is an AI-powered, chat assistant for OpenChoreo. The Portal Assistant is read-only by contract. It can list, get, and query — it cannot create, update, delete, or trigger anything. Write-style requests are politely declined. + +## Prerequisites + +Before enabling the Portal Assistant, ensure the following: + +- OpenChoreo Control Plane installed (this is where the Portal Assistant runs). +- OpenChoreo Observability Plane installed if you want runtime-debug answers (the Portal Assistant's "what's going wrong in my logs?" path calls the Observer MCP). +- Backstage deployed and reachable — the Portal Assistant is a backend; users interact with it through Backstage's chat drawer. +- An LLM API key from OpenAI (support for other providers coming soon) +- (Optional) The [SRE Agent](./sre-agent.mdx) deployed if you want Tier-2 deep analysis from inside a chat. + +:::note +The Portal Assistant streams responses over HTTP/ndjson and can issue several MCP tool calls per turn. Choose a model with reasonable streaming + tool-use quality (e.g. `openai:gpt-5`). +::: + +## Enabling the Portal Assistant + +### Step 1: Create the Portal Assistant Secret + +The Portal Assistant requires a Kubernetes Secret in the `openchoreo-control-plane` namespace with the following key: + +| Key | Description | +| ------------------------------ | ------------------------- | +| `PORTAL_ASSISTANT_LLM_API_KEY` | Your LLM provider API key | + +The Secret name is referenced from the chart via `portalAssistant.llm.secretName`, so pick any name (the deployment fails fast at install time if the value is unset). + +You can create this secret using any method you prefer. If you followed the [Try It Out on k3d locally](../getting-started/try-it-out/on-k3d-locally.mdx) guide: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/portal-assistant-llm-api-key value="" +``` + +```bash +kubectl apply -f - <" +``` + +### Step 2: Upgrade the Control Plane + +Enable the Portal Assistant and point it at your secret + model. The `--reuse-values` flag preserves your existing configuration. + + + {`helm upgrade --install openchoreo-control-plane ${versions.helmSource}/openchoreo-control-plane \\ + --version ${versions.helmChart} \\ + --namespace openchoreo-control-plane \\ + --reuse-values \\ + --set portalAssistant.enabled=true \\ + --set portalAssistant.llm.secretName=portal-assistant-secret \\ + --set portalAssistant.llm.modelName=openai:gpt-5.4-mini`} + + +:::note Supported Models +Any `provider:model` string accepted by LangChain's `init_chat_model` works. Common picks: + +- `openai:gpt-5.4-mini` — lightweight, low-cost option for development and quick chats. +- `openai:gpt-5` — base model, good default for development. +- `openai:gpt-5.2-pro` — better tool-use reasoning, recommended for production. +- `openai:gpt-5.4` — latest, highest-quality option. + +The `gpt-5.6` series is not supported at the moment. Support is coming soon. +::: + +If your Observer or SRE Agent live outside the cluster, also point the Portal Assistant at them: + +```bash +--set portalAssistant.config.observerApiUrl=http://observer.example.com:8080 +--set portalAssistant.config.rcaAgentApiUrl=http://rca-agent.example.com:11080 +``` + +When `rcaAgentApiUrl` is empty, the Portal Assistant simply does not register the SRE Agent MCP — chat keeps working with just the OpenChoreo + Observability tools. + +### Step 3: Wire Backstage to call the Portal Assistant + +Backstage ships with the Portal Assistant chat drawer behind a feature flag. The control-plane chart exposes it as `backstage.features.assistant`: + + + {`helm upgrade --install openchoreo-control-plane ${versions.helmSource}/openchoreo-control-plane \\ + --version ${versions.helmChart} \\ + --namespace openchoreo-control-plane \\ + --reuse-values \\ + --set backstage.features.assistant.enabled=true`} + + +| Value | Default | Notes | +| ------------------------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| `backstage.features.assistant.enabled` | `false` | Toggles the "Ask Portal Assistant" affordance on the Logs tab and the chat drawer. Renders as `OPENCHOREO_FEATURES_ASSISTANT_ENABLED`. | +| `backstage.features.assistant.portalAssistantUrl` | `http://portal-assistant.openchoreo-control-plane.svc.cluster.local:8080` | Cluster-internal Service URL the Backstage forwarder calls. Renders as `OPENCHOREO_PORTAL_ASSISTANT_URL`. | + +If the Portal Assistant runs in a different cluster or namespace, override the URL: + +```bash +--set backstage.features.assistant.portalAssistantUrl=http://portal-assistant.example.com:8080 +``` + +After this, restart the Backstage pod so the new env vars take effect: + +```bash +kubectl -n openchoreo-control-plane rollout restart deployment/backstage +``` + +### Step 4: Verify the installation + +Check that the Portal Assistant pod is running: + +```bash +kubectl get pods -n openchoreo-control-plane -l app.kubernetes.io/component=portal-assistant +``` + +Hit the health endpoint over a port-forward: + +```bash +kubectl -n openchoreo-control-plane port-forward svc/portal-assistant 8088:8080 & +curl -sf http://localhost:8088/health # {"status":"healthy"} +``` + +Open Backstage in a browser, navigate to any component's Logs tab, and you should see an "Ask Portal Assistant" affordance — click it to send your first chat. The drawer streams tokens as the model produces them. + +If the drawer never appears, the most common cause is the Backstage feature flag — check the running pod's env: + +```bash +kubectl -n openchoreo-control-plane get deployment backstage \ + -o jsonpath='{range .spec.template.spec.containers[0].env[*]}{.name}={.value}{"\n"}{end}' \ + | grep -i 'assistant' +``` + +You should see both `OPENCHOREO_FEATURES_ASSISTANT_ENABLED=true` and `OPENCHOREO_PORTAL_ASSISTANT_URL=...`. + +### Where Portal Assistant appears in Backstage + +With the feature flag enabled, the Portal Assistant surfaces from two places in Backstage: + +- **Build page** — the Portal Assistant appears automatically when a component's latest build is failing, and is also available when you open any failed build run. The chat opens already scoped to that run, so you can ask about the failure without re-pasting context. +- **Logs page** — every log row has an Investigate button in its hover-action column. Clicking it opens the Portal Assistant scoped to that specific log line (with the surrounding trace, when present), so you can investigate the error without copy-pasting the line. + +
+Exposing the Portal Assistant externally via HTTPRoute + +By default, the Portal Assistant is only reachable in-cluster. To expose `/api/v1alpha1/portal-assistant/*` through the OpenChoreo Gateway, set a hostname: + +```bash +helm upgrade --install openchoreo-control-plane \ + oci://ghcr.io/openchoreo/helm-charts/openchoreo-control-plane \ + --namespace openchoreo-control-plane \ + --reuse-values \ + --set 'portalAssistant.http.hostnames[0]=portal-assistant.openchoreo.example.com' +``` + +The chart renders an `HTTPRoute` for `/chat` (with `backendRequest: 450s` to allow long LLM turns) and `/warmup`. A streaming `TrafficPolicy` (`streamIdle: 24h`, `request: 0s`) is also applied so the ndjson stream is not torn down mid-turn by the gateway. + +
+ +
+Authentication and Authorization (External IdP) + +By default, OpenChoreo configures ThunderID as the identity provider for the Portal Assistant with a pre-configured OAuth client (`openchoreo-backstage-client`) bound to the `backstage-catalog-reader` role — which is granted the `portal-assistant:invoke` action. Chat requests originate from Backstage with the end user's JWT; the Portal Assistant validates the JWT against the ThunderID JWKS and forwards it verbatim to the MCP servers, so the per-tool authorization is end-user scoped, not service-account scoped. + +If you are using an external identity provider, configure the JWKS / issuer / audience on the Portal Assistant via the standard control-plane security values: + +```yaml +security: + oidc: + jwksUrl: "" + issuer: "" + jwt: + audience: "" +``` + +For service accounts that need to call `/chat` programmatically (rather than via Backstage forwarding a user JWT), create an OAuth client that supports `client_credentials` and grant it the `portal-assistant:invoke` action: + +```yaml +openchoreoApi: + config: + security: + authorization: + bootstrap: + mappings: + - name: portal-assistant-service-binding + roleRef: + name: backstage-catalog-reader # or a dedicated role that has portal-assistant:invoke + entitlement: + claim: sub + value: "" + effect: allow +``` + +See [Identity Provider Configuration](../platform-engineer-guide/identity-configuration.mdx) for detailed setup. + +
+ +
+Tuning capacity and behaviour + +| Value | Default | What it does | +| -------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| `portalAssistant.replicas` | `1` | Increase for HA / higher chat throughput. The Portal Assistant is stateless — adding replicas is safe. | +| `portalAssistant.config.maxConcurrentChats` | `20` | Per-pod concurrent-chat semaphore. Bound to your LLM provider's per-minute rate limit; raising it past your provider quota will surface as 429s. | +| `portalAssistant.config.authzTimeoutSeconds` | `30` | Hard timeout for the per-request authz check. | +| `portalAssistant.config.logLevel` | `INFO` | Set to `DEBUG` for verbose MCP / tool-call logging during incident investigation. | +| `portalAssistant.cors.allowedOrigins` | `[]` | Only relevant if you hit `/chat` directly from a browser bypassing the Backstage forwarder. | +| `portalAssistant.extraEnvs` | `[]` | Inject additional env vars without editing the chart. | + +
+ +
+Bring-your-own auth-config + +The Portal Assistant ships with a default `/app/auth-config.yaml` baked into the image that defines its action catalog (`portal-assistant:invoke`) and subject-type rules. To override, create a `ConfigMap` with your replacement file and point the chart at it: + +```bash +kubectl create configmap portal-assistant-auth-config \ + -n openchoreo-control-plane \ + --from-file=auth-config.yaml=./my-auth-config.yaml + +helm upgrade --install openchoreo-control-plane \ + oci://ghcr.io/openchoreo/helm-charts/openchoreo-control-plane \ + --namespace openchoreo-control-plane \ + --reuse-values \ + --set portalAssistant.authConfigConfigMap=portal-assistant-auth-config +``` + +The chart will mount the ConfigMap at `/etc/openchoreo/auth-config.yaml` and set `AUTH_CONFIG_PATH` accordingly. + +
diff --git a/versioned_docs/version-v1.2.0-rc.2/ai/sre-agent.mdx b/versioned_docs/version-v1.2.0-rc.2/ai/sre-agent.mdx new file mode 100644 index 00000000..73ad604a --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/ai/sre-agent.mdx @@ -0,0 +1,196 @@ +--- +title: SRE Agent +description: Configure the SRE Agent in OpenChoreo for AI-powered root cause analysis. +--- + +import CodeBlock from "@theme/CodeBlock"; +import { versions } from "../_constants.mdx"; + +# SRE Agent + +The SRE Agent is an AI-powered component that performs root cause analysis (RCA) by analyzing logs, metrics, and traces from your OpenChoreo deployments to generate reports with likely root causes of issues. It integrates with Large Language Models (LLMs) to provide intelligent analysis and actionable insights. + +## Prerequisites + +Before enabling the SRE Agent, ensure the following: + +- OpenChoreo Observability Plane installed with at least a logs module. +- An LLM API key from [OpenAI](https://platform.openai.com/) (support for other providers coming soon) +- [Alerting configured](../platform-engineer-guide/observability-alerting.mdx#ai-powered-root-cause-analysis) for your components with `triggerAiRca` enabled. + +:::note +Enable automatic RCA only for critical alerts to manage LLM costs. +::: + +## Enabling the SRE Agent + +### Step 1: Create the SRE Agent Secret + +The SRE Agent requires a Kubernetes Secret named `rca-agent-secret` in the `openchoreo-observability-plane` namespace with the following keys: + +| Key | Description | +| --------------------- | -------------------------------------------------- | +| `RCA_LLM_API_KEY` | Your LLM provider API key | +| `OAUTH_CLIENT_SECRET` | OAuth client secret (only needed for external IdP) | + +You can create this secret using any method you prefer. If you followed the [Try It Out on k3d locally](../getting-started/try-it-out/on-k3d-locally.mdx) guide, you can follow along: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/rca-llm-api-key value="" +``` + +```bash +kubectl apply -f - < + {`helm upgrade --install openchoreo-observability-plane ${versions.helmSource}/openchoreo-observability-plane \\ + --version ${versions.helmChart} \\ + --namespace openchoreo-observability-plane \\ + --reuse-values \\ + --set rca.enabled=true \\ + --set rca.llm.modelName=`} + + +:::note Supported Models +The SRE Agent currently supports the [OpenAI](https://platform.openai.com/) GPT model series (e.g., `gpt-5.4`, `gpt-5.2-pro`, `gpt-5` etc.). Support for additional model providers is coming soon. + +The `gpt-5.6` series is not supported at the moment. Support is coming soon. +::: + +If the observability plane and control plane are in separate clusters, also set `rca.openchoreoApiUrl` to the control plane API URL (defaults to `http://api.openchoreo.localhost:8080`). + +### Step 3: Register with the control plane + +Configure `rcaAgentURL` in the `ClusterObservabilityPlane` resource so the UI knows where to reach the SRE Agent: + + + {`kubectl patch clusterobservabilityplane default --type=merge -p '{"spec":{"rcaAgentURL":"http://rca-agent.openchoreo.localhost:11080"}}'`} + + +### Step 4: Verify the installation + +Check that the SRE Agent pod is running: + +```bash +kubectl get pods -n openchoreo-observability-plane -l app.kubernetes.io/component=ai-rca-agent +``` + +If you are using the default identity provider (ThunderID) and the default SQLite report storage, your setup is complete. + +For a full, end-to-end walkthrough of setting up alerting with AI-powered root cause analysis, refer to the [URL Shortener sample](https://github.com/openchoreo/openchoreo/tree/main/samples/from-source/projects/url-shortener). + +
+Authentication and Authorization (External IdP) + +By default, OpenChoreo configures ThunderID as the identity provider for the SRE Agent with a pre-configured OAuth client for testing purposes. If you are using an external identity provider, follow the steps below. + +#### Authentication + +Create an OAuth 2.0 client that supports the `client_credentials` grant type for service-to-service authentication. + +Store your OAuth client secret in OpenBao: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/rca-oauth-client-secret value="" +``` + +Then configure the Observability Plane Helm values with your client credentials: + +```yaml +security: + oidc: + tokenUrl: "" + +rca: + secretName: "rca-agent-secret" + oauth: + clientId: "" +``` + +See [Identity Provider Configuration](../platform-engineer-guide/identity-configuration.mdx) for detailed setup instructions. + +#### Authorization + +The SRE Agent uses the `client_credentials` grant to authenticate with the OpenChoreo API as a service account. The API matches the `sub` claim in the issued JWT to identify the caller, so the new client must be granted the `rca-agent` role via a bootstrap authorization mapping. + +Add the following to your Control Plane values override, replacing `` with the same client ID used above: + +```yaml +openchoreoApi: + config: + security: + authorization: + bootstrap: + mappings: + - name: rca-agent-binding + roleRef: + name: rca-agent + entitlement: + claim: sub + value: "" + effect: allow +``` + +
+ +
+Report Storage + +By default, RCA reports are stored in **SQLite** with a persistent volume — no external database required. + +For production deployments that need horizontal scaling or shared storage, you can use **PostgreSQL** instead. + +Store the PostgreSQL connection URI in OpenBao: + +```bash +kubectl exec -n openbao openbao-0 -- \ + env BAO_ADDR=http://127.0.0.1:8200 BAO_TOKEN=root \ + bao kv put secret/rca-sql-backend-uri value="postgresql+asyncpg://:@:/" +``` + +Add the `SQL_BACKEND_URI` key to the ExternalSecret from [Step 1](#step-1-create-the-sre-agent-secret): + +```bash +kubectl patch externalsecret rca-agent-secret -n openchoreo-observability-plane --type=json \ + -p '[{"op":"add","path":"/spec/data/-","value":{"secretKey":"SQL_BACKEND_URI","remoteRef":{"key":"rca-sql-backend-uri","property":"value"}}}]' +``` + +Then set the report backend in your Helm values: + +```yaml +rca: + reportBackend: postgresql +``` + +
diff --git a/versioned_docs/version-v1.2.0-rc.2/changelog.md b/versioned_docs/version-v1.2.0-rc.2/changelog.md new file mode 100644 index 00000000..5d71d9fa --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/changelog.md @@ -0,0 +1,48 @@ +--- +title: Changelog +description: Release history for OpenChoreo +--- + +# Changelog + +## v1.x Releases + +| Version | Date | Changelog | +| ----------- | ---------- | ----------------------------------------------------------------------------------------- | +| v1.2.0-rc.2 | 2026-07-23 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.2.0-rc.1...v1.2.0-rc.2) | +| v1.2.0-rc.1 | 2026-07-17 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.1.0...v1.2.0-rc.1) | +| v1.2.0-m.1 | 2026-06-12 | [Changelog](https://github.com/openchoreo/openchoreo/compare/v1.1.1...v1.2.0-m.1) | +| v1.1.3 | 2026-07-22 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.1/CHANGELOG.md#v113) | +| v1.1.2 | 2026-07-08 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.1/CHANGELOG.md#v112) | +| v1.1.1 | 2026-05-29 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.1/CHANGELOG.md#v111) | +| v1.1.0 | 2026-05-18 | [Changelog](https://github.com/openchoreo/openchoreo/blob/main/CHANGELOG.md#v110) | +| v1.0.3 | 2026-07-22 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.0/CHANGELOG.md#v103) | +| v1.0.2 | 2026-07-08 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.0/CHANGELOG.md#v102) | +| v1.0.1 | 2026-05-07 | [Changelog](https://github.com/openchoreo/openchoreo/blob/release-v1.0/CHANGELOG.md#v101) | +| v1.0.0 | 2026-03-22 | [Changelog](https://github.com/openchoreo/openchoreo/blob/main/CHANGELOG.md#v100) | +| v1.0.0-rc.2 | 2026-03-20 | [Changelog](https://github.com/openchoreo/openchoreo/blob/main/CHANGELOG.md#v100-rc2) | +| v1.0.0-rc.1 | 2026-03-15 | [Changelog](https://github.com/openchoreo/openchoreo/blob/main/CHANGELOG.md#v100-rc1) | + +## v0.x Releases + +| Version | Date | Release Notes | +| ------- | ---------- | ------------------------------------------------------------------------------ | +| v0.17.0 | 2026-03-05 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.17.0) | +| v0.16.0 | 2026-02-23 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.16.0) | +| v0.15.0 | 2026-02-17 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.15.0) | +| v0.14.0 | 2026-02-07 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.14.0) | +| v0.13.0 | 2026-01-30 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.13.0) | +| v0.12.0 | 2026-01-23 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.12.0) | +| v0.11.0 | 2026-01-16 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.11.0) | +| v0.10.0 | 2026-01-09 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.10.0) | +| v0.9.0 | 2025-12-23 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.9.0) | +| v0.8.0 | 2025-12-18 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.8.0) | +| v0.7.0 | 2025-12-08 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.7.0) | +| v0.6.0 | 2025-12-01 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.6.0) | +| v0.5.0 | 2025-11-23 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.5.0) | +| v0.4.0 | 2025-11-17 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.4.0) | +| v0.3.2 | 2025-09-18 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.3.2) | +| v0.3.1 | 2025-09-02 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.3.1) | +| v0.3.0 | 2025-07-31 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.3.0) | +| v0.2.0 | 2025-06-06 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.2.0) | +| v0.1.0 | 2025-03-20 | [Release Notes](https://github.com/openchoreo/openchoreo/releases/tag/v0.1.0) | diff --git a/versioned_docs/version-v1.2.0-rc.2/concepts/developer-abstractions.md b/versioned_docs/version-v1.2.0-rc.2/concepts/developer-abstractions.md new file mode 100644 index 00000000..1e39b786 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/concepts/developer-abstractions.md @@ -0,0 +1,160 @@ +--- +title: Developer Abstractions +description: Developer abstractions for building and running applications +--- + +# Developer Abstractions + +Developer abstractions in OpenChoreo enable teams to build, deploy, and operate cloud-native applications without +managing infrastructure complexity. These abstractions provide a declarative model for expressing application +architecture, dependencies, and operational requirements while the platform handles the underlying Kubernetes resources, +networking, and security configurations automatically. + +## Project + +A **Project** represents a bounded context in Domain-Driven Design terms - a cohesive collection of components that +together implement a specific business capability or application domain. It serves as the primary organizational unit +for developers, defining clear boundaries for code ownership, deployment coordination, and operational responsibility. + +Projects establish both logical and physical boundaries in the platform. Logically, they group related components that +share common business logic, data models, and team ownership. Physically, they translate into isolated deployment units +with dedicated namespaces, network boundaries, and security policies. This alignment between organizational structure +and technical architecture enables teams to work autonomously while maintaining clear integration points with other +projects. + +The project boundary also defines the scope for internal communication and shared resources. Components within a project +can communicate freely with each other. This locality principle reduces complexity for +developers while maintaining security and isolation between different application domains. + +## Component + +A **Component** represents a deployable unit of software - the fundamental building block of applications in OpenChoreo. +Each component encapsulates a specific piece of functionality, whether it's a microservice handling business logic, a +web application serving user interfaces, or a background job processing data. + +Components use a **ComponentType** reference to determine their deployment characteristics. This reference is a structured +object with `kind` and `name` fields, where `kind` specifies the resource type (`ComponentType` or `ClusterComponentType`, +defaulting to `ComponentType`) and `name` follows the `{workloadType}/{componentTypeName}` format, such as +`deployment/web-service` or `cronjob/data-processor`. The default platform setup provides ClusterComponentTypes that are +visible to all namespaces, so you should set `kind: ClusterComponentType` explicitly when referencing them. Namespace-scoped ComponentTypes can be used for isolation. This explicit typing allows +platform engineers to define multiple variations of deployment patterns for the same workload type, each tuned for +different use cases. + +The Component resource connects four essential elements: + +**ComponentType Reference** specifies which platform-defined template governs this component's deployment. The +ComponentType defines the available configuration schema, resource templates, and allowed workflows. This separation +of concerns means developers work with a simplified interface while platform engineers maintain control over +infrastructure patterns. + +**Parameters** provide the component-specific configuration values that conform to the schema defined in the +ComponentType. When a ComponentRelease is created, these parameter values are captured in the release snapshot. The +same values then apply wherever that release is deployed—if you deploy the same ComponentRelease to dev, staging, and +prod, the parameters are identical across all environments. To change parameter values, you update the Component and +create a new ComponentRelease. Environment-specific values (like resource limits or storage sizes) are handled +separately through `environmentConfigs` in ReleaseBinding resources. + +**Traits** enable composition of additional capabilities into the component. Each trait instance adds specific +functionality like persistent storage, caching, or monitoring. Traits can be instantiated multiple times with +different configurations using unique instance names. For example, a component might attach multiple persistent volume +traits for different storage needs, each with its own size, storage class, and mount configuration. Traits use the +same schema-driven approach as ComponentTypes, with parameters set in the Component and environment-specific overrides +applied through ReleaseBinding resources. + +**Workflow Configuration** optionally specifies how to build the component from source code. This references a +Workflow and provides the developer-configured schema values needed to execute builds. The workflow integration +enables automated container image creation triggered by code changes or manual developer actions. + +The component abstraction thus becomes a declarative specification that combines: + +- A ComponentType that defines _how_ to deploy +- Parameters that configure _what_ to deploy +- Traits that compose _additional capabilities_ +- A Workflow that defines _how to build_ + +This composition-based approach enables developers to assemble complex applications from reusable building blocks +while the platform ensures consistency, governance, and operational best practices through the underlying ComponentType +and Trait templates. + +## Workload + +A **Workload** defines the runtime contract of a component - specifying what the component needs to run. The workload +focuses on application requirements rather than infrastructure details, which are handled by the platform through ComponentTypes. + +Each component has one workload that describes its runtime needs through several key specifications: + +**Container** defines the container image to deploy, along with its commands, arguments, and environment variables. +This tells the platform what code to run and how to configure it. + +**Endpoints** specify the network interfaces that the component exposes - the ports and protocols it listens on. Each +endpoint declares its type (HTTP, gRPC, GraphQL, Websocket, TCP, or UDP) and port number. These definitions tell the +platform what network services the component provides, enabling automatic service creation and network policy generation. + +**Dependencies** declare the component's dependencies on other services and managed infrastructure. Endpoint +dependencies target endpoints exposed by other components: the platform resolves the +target address, injects it through environment variables, and configures network policies for the traffic. Resource +dependencies target a Resource in the same project: the outputs declared by the +referenced ResourceType are injected as environment variables or file mounts in +the consuming container. + +This declarative specification can be generated from configuration files in the source repository or applied directly +to the cluster. The separation between workload (what the application needs) and ComponentType (how the platform provides it) +enables platform teams to control infrastructure policies while developers focus on application requirements. Resource +limits, scaling parameters, and operational policies come from the ComponentType and Traits, while the +workload simply declares what the application needs to function. + +## Resource + +A **Resource** represents a managed-infrastructure dependency declared by a developer—a database, queue, cache, object +store, or any other piece of stateful infrastructure the application needs. A Resource is project-bound: it belongs to +exactly one Project, and the components in that project consume it through Workload dependencies. + +A Resource is a thin declaration. It references a **ResourceType** (or **ClusterResourceType**) by name, mirroring how +Components reference ComponentTypes. The Resource provides parameter values that conform to the ResourceType's +schema—such as database engine version, replica count, or storage size—and the platform handles provisioning through +the resource templates defined on the ResourceType. Developers do not write the provisioning logic; that lives in the +ResourceType authored by platform engineers. + +Each Resource is bound to environments through one **ResourceReleaseBinding** per environment. A binding selects a +specific **ResourceRelease**—an immutable snapshot of the Resource and ResourceType spec taken at release time—and +carries per-environment configuration overrides. + +Components consume Resource outputs through their Workload's dependencies. Each output declared by the ResourceType +can be mapped to an environment variable or a file mount inside the consuming container. Sensitive values remain on +the data plane: the consuming pod reads them from a Secret resolved at pod-start, and only the reference travels +through the control plane. + +The Resource abstraction follows the same separation of concerns as Components. Developers express what +infrastructure they need; platform engineers control how that infrastructure is provisioned, what defaults apply, and +which resource templates are available. + +## WorkflowRun + +A **WorkflowRun** represents a runtime execution instance of a Workflow. While Workflows define the automation template +and parameter schema, WorkflowRuns represent actual executions with specific parameter values. + +WorkflowRuns bridge the gap between developer intent and automation execution. Developers create WorkflowRun resources +to trigger workflows, providing parameter values. The platform handles all the complexity of rendering the final +workflow specification, resolving external references, synchronizing secrets, and managing execution in the workflow plane. + +Each WorkflowRun captures: + +**Workflow Reference and Parameters** specify which Workflow to execute and provide the developer-supplied parameter +values. These values are validated against the Workflow's parameter schema, which can include repository URLs, build +configuration, test modes, resource allocations, and any other fields the platform engineer has defined. + +**Component Ownership** (for CI workflows) is tracked through metadata labels (`openchoreo.dev/component` and +`openchoreo.dev/project`) that link the execution to a specific component and project. This enables traceability, +build history per component, and component-specific operations. These labels are accessible in the Workflow's CEL +expressions for injecting component context into the execution. + +**Execution Status** tracks the workflow through conditions (`WorkflowRunning`, `WorkflowCompleted`, +`WorkflowSucceeded`, `WorkflowFailed`), individual task status with a vendor-neutral step abstraction, timestamps, +and references to the actual workflow plane resources for debugging and cleanup. + +This abstraction provides a unified interface for both generic automation and component builds, where developers +interact with curated parameter schemas rather than complex CI/CD pipeline definitions. The separation of concerns +allows platform engineers to control workflow implementation and security policies through Workflow templates while +developers manage parameter values through WorkflowRun instances. WorkflowRuns can be created manually for ad-hoc +executions or automatically by platform controllers in response to code changes, supporting both interactive development +and fully automated CI/CD pipelines. diff --git a/versioned_docs/version-v1.2.0-rc.2/concepts/platform-abstractions.md b/versioned_docs/version-v1.2.0-rc.2/concepts/platform-abstractions.md new file mode 100644 index 00000000..3f1ee9f5 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/concepts/platform-abstractions.md @@ -0,0 +1,315 @@ +--- +title: Platform Abstractions +description: Platform abstractions for managing infrastructure +--- + +# Platform Abstractions + +Platform abstractions in OpenChoreo provide the foundational infrastructure layer that platform engineers use to build +and manage Internal Developer Platforms. These abstractions establish logical boundaries, manage infrastructure +resources, and define the operational policies that enable developer self-service while maintaining security and +compliance. + +## Namespace + +OpenChoreo uses Kubernetes namespaces to organize and isolate groups of related resources. By default, platform resources like ComponentTypes, Traits, Workflows, DataPlanes, and WorkflowPlanes are created as cluster-scoped resources (ClusterComponentType, ClusterTrait, ClusterWorkflow, ClusterDataPlane, ClusterWorkflowPlane, ClusterObservabilityPlane), making them automatically visible to all namespaces. This means any new namespace has access to them out of the box. Platform teams can create namespace-scoped variants when they need to customize or isolate resources for a particular namespace. Resources like Projects, Environments, and DeploymentPipelines remain namespace-scoped since they are inherently tied to a specific namespace context. + +OpenChoreo identifies and manages namespaces through a label (`openchoreo.dev/control-plane: true`). The control plane uses this label to discover namespaces, perform list/get operations, and organize platform resources. When an OpenChoreo cluster is created, the default namespace is automatically labeled with this identifier, enabling immediate platform resource creation. + +## Infrastructure Planes + +OpenChoreo separates infrastructure concerns into specialized planes, each serving a distinct purpose in the platform architecture. This separation enables independent scaling, security isolation, and operational management of different platform functions. + +### Control Plane + +The **Control Plane** is the Kubernetes cluster where OpenChoreo itself runs. It hosts the platform's custom resource definitions (CRDs), controllers, and the OpenChoreo API. Platform engineers interact with the control plane to define platform abstractions—ComponentTypes, Traits, Workflows, Environments, and more—and the controllers running here reconcile those resources into running workloads on the connected data planes. + +The control plane does not run application workloads directly. Instead, it orchestrates all other planes: rendering releases and applying them to DataPlanes, dispatching builds to WorkflowPlanes, and collecting observability signals from across the fleet. Because all platform state lives here, the control plane is the single source of truth for the entire OpenChoreo installation. + +### DataPlane + +A **DataPlane** represents a Kubernetes cluster where application workloads run. It abstracts the complexity of cluster +management, providing a unified interface for deploying applications across multiple clusters regardless of their +location or underlying infrastructure. + +DataPlanes encapsulate all the configuration needed to connect to and manage a Kubernetes cluster, including connection +credentials, TLS certificates, and cluster-specific settings. They enable platform teams to register multiple clusters - +whether on-premises, in public clouds, or at edge locations - and manage them through a single control plane. + +Each DataPlane can host multiple environments and projects, with OpenChoreo managing the creation of namespaces, network +policies, and other cluster resources automatically. This abstraction allows platform teams to treat clusters as +interchangeable infrastructure resources, enabling strategies like geographic distribution, compliance-based placement, +and disaster recovery. + +### WorkflowPlane + +A **WorkflowPlane** provides dedicated infrastructure for executing continuous integration and build workloads. By +separating build operations from runtime workloads, WorkflowPlanes ensure that resource-intensive compilation and testing +processes don't impact production applications. + +WorkflowPlanes integrate with Argo Workflows to provide a scalable, Kubernetes-native CI/CD execution environment. They +handle the complete build lifecycle, from source code retrieval through compilation, testing, and container image +creation. This separation also provides security benefits, isolating potentially untrusted build processes from +production environments. + +Platform engineers configure WorkflowPlanes with the necessary tools, credentials, and policies for building applications. +This includes container registry credentials, build tool configurations, and security scanning policies. WorkflowPlanes can +be scaled independently based on build demand and can be distributed geographically to reduce latency for development +teams. + +### Observability Plane + +The **Observability Plane** provides centralized infrastructure for collecting and analyzing logs, metrics, and traces +across all other planes - Control, Data, and Workflow - providing a unified view of platform operations and +application behavior. + +The Observability Plane uses a pluggable adapter-pattern, with OpenSearch as the default backend for logs and traces, +and Prometheus for metrics. The Observer API provides authenticated access to observability data, enabling integration +with external monitoring tools and dashboards. Module authors can swap in alternative backends by implementing the adapter API. + +Platform engineers can configure alerting rules to define conditions on logs and metrics, and route notifications via email or webhook. + +Platform engineers configure the Observability Plane during initial setup, establishing collection pipelines, +retention policies, and access controls. This centralized approach ensures that all platform activity is auditable and +debuggable while maintaining security boundaries between namespaces. + +## Environment + +An **Environment** represents a stage in the software delivery lifecycle, such as development, staging, or production. +Environments provide the context for deploying and running applications, defining the policies, configurations, and +constraints that apply to workloads in that stage. + +Environments are not just labels or namespaces - they are first-class abstractions that define where applications +should be deployed (which DataPlane) and serve as targets for deployment pipelines. This abstraction enables platform +teams to organize different stages of the delivery pipeline. + +Each environment represents a distinct deployment target. Development environments might target smaller clusters or +shared infrastructure, while production environments target dedicated, high-availability clusters. The Environment +resource primarily defines the mapping to infrastructure (DataPlane) and serves as a reference point for deployments +and promotion workflows. + +## DeploymentPipeline + +A **DeploymentPipeline** defines the allowed progression paths for applications moving through environments. It +represents the organization's software delivery process as a declarative configuration, encoding promotion rules and quality gates. + +DeploymentPipelines go beyond simple environment ordering to define complex promotion topologies. They can specify +parallel paths for different types of releases and conditional progressions based on application characteristics. +This flexibility allows organizations to implement sophisticated delivery strategies while maintaining governance and +control. + +The pipeline abstraction also serves as an integration point for organizational processes. Automated testing can be +triggered at promotion boundaries, and compliance checks can be enforced before production deployment. This ensures +that all applications follow organizational standards regardless of which team develops them. + +## ReleaseBinding + +A **ReleaseBinding** connects a ComponentRelease to a specific Environment. This is what actually deploys a release to +a data plane—but it's more than just a reference. + +ReleaseBinding provides a way to override configuration for a specific environment. Values defined in the ComponentType's +`environmentConfigs` and Trait's `environmentConfigs` can be customized per environment through the ReleaseBinding. +This is where environment-specific differences reside: scaling in production versus development, resource limits, +storage classes, and so on—while the ComponentRelease itself remains unchanged. + +Once a ReleaseBinding is created, the ReleaseBinding controller renders the ComponentType and Trait templates with the +combined configuration, generates the necessary Kubernetes resources (Deployments, Services, ConfigMaps, etc.), and +produces a **RenderedRelease** resource that is applied to the target plane. + +## ResourceReleaseBinding + +A **ResourceReleaseBinding** is the resource-side counterpart of ReleaseBinding. It pins a specific **ResourceRelease** +to an Environment and carries per-environment configuration overrides for the referenced ResourceType. This is where +settings like dev-vs-prod replica counts, storage sizes, or admin-UI enablement live—while the ResourceRelease +snapshot itself remains unchanged across environments. + +ResourceReleaseBindings are authored by platform engineers or GitOps tooling, not by the Resource controller. The +controller for the parent Resource never creates or modifies bindings on its own; this keeps developer-facing Resource +spec edits from triggering automatic redeploys. Promoting a release into an environment is always an explicit step. + +A binding can also override the ResourceType's retention policy for its environment, allowing production environments +to preserve underlying state (volumes, databases) on deletion while dev and staging defer to the type-level default. + +When a ResourceReleaseBinding is created, the binding controller renders the ResourceType template with the combined +Resource parameters and environment overrides, produces a RenderedRelease that is applied to the target plane, and +resolves the declared outputs so consuming workloads can read them. + +## Component Types + +A **ComponentType** is a platform engineer-defined template that governs how components are deployed and managed in +OpenChoreo. It represents the bridge between developer intent and platform governance, encoding organizational +policies, best practices, and infrastructure patterns as reusable templates. + +ComponentTypes separate developer intent from platform governance. While developers create Components +that express their application requirements, platform engineers define ComponentTypes that specify how those +requirements should be fulfilled. This separation enables developers to focus on application logic while platform +engineers maintain control over infrastructure policies, resource limits, security configurations, and operational +standards. + +OpenChoreo also provides **ClusterComponentType**, a cluster-scoped variant of ComponentType. The default platform setup uses ClusterComponentTypes so that all namespaces can reference them without duplication. Namespace-scoped ComponentTypes are available when platform engineers need to customize or override the defaults for a specific namespace. + +Each ComponentType (or ClusterComponentType) is built around a specific **workload type** - the primary Kubernetes +resource that will run the application. OpenChoreo supports four fundamental workload types: + +- **deployment**: For long-running services that need continuous availability +- **statefulset**: For applications requiring stable network identities and persistent storage +- **cronjob**: For scheduled tasks that run at specific times or intervals +- **job**: For one-off tasks that run to completion +- **proxy**: For proxy workloads that route traffic without running application containers + +The ComponentType uses a **schema-driven architecture** that defines what developers can configure when creating +components. This schema consists of two types of parameters: + +**Parameters** are configurations captured in the ComponentRelease and applied uniformly wherever that release is +deployed. These include settings like replica counts, image pull policies, and container ports. When you deploy the +same ComponentRelease to multiple environments, the parameter values are identical. To change parameters, you update +the Component and create a new ComponentRelease. + +**EnvironmentConfigs** are configurations that can be overridden on a per-environment basis through ReleaseBinding resources. +These typically include resource allocations, scaling limits, and environment-specific policies. This flexibility +allows platform engineers to provide generous resources in production while constraining development environments to +optimize infrastructure costs. + +The schema uses standard [OpenAPI v3 JSON Schema](https://swagger.io/specification/) format (`openAPIV3Schema`), making +configuration requirements explicit and self-documenting. For example, `type: integer` with `default: 1` declares an +integer parameter with a default value, while `type: string` with `enum: [Always, IfNotPresent, Never]` restricts a +string to specific allowed values. This format supports validation rules like minimum/maximum values, required fields, +and enumerated choices. + +ComponentTypes define **resource templates** that generate the actual Kubernetes resources for components. Each +template uses CEL (Common Expression Language) expressions to dynamically generate resource manifests based on +component specifications. Templates can access component metadata, schema parameters, and workload specifications +through predefined variables like `${metadata.name}` and `${parameters.replicas}`. + +Templates support advanced patterns through conditional inclusion and iteration. The `includeWhen` field uses CEL +expressions to conditionally create resources based on configuration, enabling optional features like autoscaling or +ingress. The `forEach` field generates multiple resources from lists, useful for creating ConfigMaps from multiple +configuration files or managing multiple service dependencies. + +ComponentTypes can also restrict which **Workflows** developers can use for building components through the +`allowedWorkflows` field. This enables platform engineers to enforce build standards, ensure security scanning, or +mandate specific build tools for different component types. For instance, a web application ComponentType might only +allow Workflows that use approved frontend build tools and security scanners. + +This schema-driven approach ensures consistency across the platform while providing flexibility for different +application patterns. Platform engineers create ComponentTypes that encode organizational knowledge about how to run +applications securely and efficiently, while developers benefit from simplified configuration and automatic compliance +with platform standards. + +## Traits + +A **Trait** is a platform engineer-defined template that augments components with operational behavior without modifying +the ComponentType. Traits enable composable, reusable capabilities that can be attached to any component—such as +persistent storage, autoscaling, network policies, or sidecar injection. + +Similar to ComponentTypes, OpenChoreo provides **ClusterTrait**, a cluster-scoped variant. The default platform setup uses ClusterTraits so that shared cross-cutting concerns are available to all namespaces. Namespace-scoped Traits can override or extend these defaults within a specific namespace. + +Traits use the same schema-driven approach as ComponentTypes, with `parameters` for static configuration and +`environmentConfigs` for environment-specific values. Developers attach Traits to their Components with instance-specific +parameters, while platform engineers can override environment-specific values through ReleaseBindings. + +Each Trait defines two types of operations: + +**Creates** generate new Kubernetes resources that don't exist in the base ComponentType. For example, a storage Trait +might create PersistentVolumeClaims, or a monitoring Trait might create ServiceMonitor resources. + +**Patches** modify existing resources generated by the ComponentType. Using JSON Patch operations with array filtering, +Traits can inject environment variables, add volume mounts, attach sidecar containers, or add labels and annotations +to existing resources. + +This separation between ComponentTypes (base deployment patterns) and Traits (composable capabilities) enables platform +engineers to define orthogonal concerns independently. Rather than creating separate ComponentTypes for every combination +of features, platform engineers define focused Traits that developers can mix and match as needed. + +## Resource Types + +A **ResourceType** is a platform engineer-defined template that governs how managed infrastructure is provisioned in +OpenChoreo. Where ComponentTypes describe how a code component is deployed, ResourceTypes describe how a Resource is +provisioned: what Kubernetes manifests the platform emits on the data plane, what parameters the developer can set, +what outputs (hostnames, credentials, connection URLs) the resulting infrastructure exposes to consumers, and what +retention behavior applies on deletion. + +OpenChoreo also provides **ClusterResourceType**, a cluster-scoped variant that is available across all namespaces. +Use ClusterResourceType for templates intended to be shared platform-wide and namespace-scoped ResourceType for +templates that are only relevant to a single namespace. + +ResourceTypes use the same schema-driven approach as ComponentTypes, with two schemas: + +**Parameters** capture developer-supplied values that become part of the immutable ResourceRelease snapshot—engine +version pin, schema name, replica count, anything that should stay identical across environments for a given release. +The schema is `openAPIV3Schema` with support for defaults, enums, validation, and CEL-based rules. + +**EnvironmentConfigs** capture per-environment overrides applied through ResourceReleaseBinding's +`resourceTypeEnvironmentConfigs`—storage size, request limits, dev-only admin UI toggles. The same ResourceRelease +deployed to dev and prod uses different environment overrides while the snapshot itself is unchanged. + +A ResourceType defines two further sections: + +**Resources** are the Kubernetes manifest templates the provisioner emits on the data plane—StatefulSets, Services, +Secrets, or third-party CRs like Crossplane claims. Each entry has a unique `id`, an optional `includeWhen` CEL +expression for conditional rendering, the manifest template, and an optional `readyWhen` CEL expression for per-Kind +health beyond the default heuristic. Templates have access to `metadata.*`, `parameters.*`, `environmentConfigs.*`, +and `dataplane.*` through the same CEL surface ComponentTypes use. + +**Outputs** declare the named values that consuming workloads bind to environment variables and file mounts. Each +output is one of three source kinds: a literal or CEL-rendered `value`, a `secretKeyRef` to a data-plane Secret, or a +`configMapKeyRef` to a data-plane ConfigMap. Output CEL has access to the same context as the resource templates plus +`applied..status.*` for reading fields populated by the provisioner. Sensitive material—passwords, tokens, private +keys—is declared with `secretKeyRef`; only the `{name, key}` reference transits to the control plane while the +underlying credential never leaves the data plane. + +The **`retainPolicy`** field on a ResourceType sets the default retention behavior for bindings of that type. The +default is `Delete`. Platform engineers opt into `Retain` for templates that emit non-recoverable state (production +databases, persistent volumes), and individual environments can still override the policy through the binding. + +## Workflow and ClusterWorkflow + +A **Workflow** is a platform engineer-defined template for running automation tasks in OpenChoreo. Workflows provide +a unified mechanism for both component builds and generic automation — infrastructure provisioning, data pipelines, +end-to-end testing, and more. + +OpenChoreo also provides **ClusterWorkflow**, a cluster-scoped variant of Workflow. While Workflows are +namespace-scoped and available only within their namespace, ClusterWorkflows are available across all namespaces. +This is useful when platform engineers want to define shared workflow templates once and allow WorkflowRuns in any +namespace to reference them, eliminating duplication. Because ClusterWorkflows are cluster-scoped, they can only +reference ClusterWorkflowPlanes (not namespace-scoped WorkflowPlanes) for their workflow operations. + +Each Workflow defines: + +**Parameter Schema** provides complete freedom for platform engineers to define developer-facing parameters. The schema +can include any structure — repository URLs, build version numbers, test modes, resource allocations, timeout settings, +and more. The schema supports all types including integers, booleans, strings, arrays, and nested objects, with full +validation through defaults, minimums, maximums, and enums. + +**Run Template** contains the actual Kubernetes resource specification (typically an Argo Workflow) with CEL template +variables for dynamic value injection. These expressions access context variables like `${metadata.workflowRunName}` +and `${metadata.namespaceName}` for runtime information, developer parameter values through `${parameters.*}` for +configuration, and resolved external references through `${externalRefs..spec.*}` for secret data. Platform +engineers can also hardcode platform-controlled parameters directly in the template, such as builder images, registry +URLs, and security scanning settings. + +**External References** declare references to external CRs (currently `SecretReference`) that are resolved at runtime +and injected into the CEL context. This enables secure access to credentials without embedding them in the Workflow +definition. + +**Resources** are additional Kubernetes resource templates (e.g., ExternalSecrets, ConfigMaps) created in the workflow +plane alongside the workflow run, with optional `includeWhen` conditions for conditional creation. + +Workflows can be triggered directly via WorkflowRun resources and are suitable for +tasks like infrastructure provisioning, data pipelines, scheduled jobs, and other automation that is not tied to a +specific component's build lifecycle. A Workflow becomes a **component workflow** when it is bound to a Component. +Component workflows carry the label `openchoreo.dev/workflow-type: "component"`, which identifies the workflow +as component-scoped and is used by developer tooling (e.g., `occ`) for categorization. To enable auto-build via +Git webhooks and UI integration, the Workflow's `openAPIV3Schema` uses vendor extensions +(e.g., `x-openchoreo-component-parameter-repository-url: true`, `x-openchoreo-component-parameter-repository-branch: true`) +to map schema fields to logical parameter keys. Component workflows must also be listed in a ComponentType's +`allowedWorkflows` field to be available for components. + +ComponentTypes govern which Workflows developers can use through the `allowedWorkflows` field. This enables platform +engineers to enforce build standards per component type, ensuring that web applications use approved frontend build +tools, backend services use appropriate security scanners, and different component types follow their specific build +requirements. + +Components reference Workflows in their `workflow` field, providing parameter values for build configuration. The +platform handles template rendering, external reference resolution, secret synchronization, and execution management +in the workflow plane, with WorkflowRun resources tracking individual executions. diff --git a/versioned_docs/version-v1.2.0-rc.2/concepts/resource-relationships.md b/versioned_docs/version-v1.2.0-rc.2/concepts/resource-relationships.md new file mode 100644 index 00000000..ef832746 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/concepts/resource-relationships.md @@ -0,0 +1,211 @@ +--- +title: Resource Relationships +description: How resources connect and interact within the platform +--- + +# Resource Relationships + +OpenChoreo's resources form a carefully designed hierarchy that balances flexibility with governance. Understanding how +these resources relate to each other is essential for designing applications, troubleshooting issues, and managing the +platform effectively. These relationships define ownership, inheritance, and dependencies throughout the system. + +## Ownership Hierarchy + +The ownership model in OpenChoreo follows a strict hierarchical structure where each resource has a clear parent and +defined scope. This hierarchy provides namespace isolation, access control boundaries, and lifecycle management. + +### Namespace-Scoped Resources + +OpenChoreo organizes most resources within **Namespaces**, which serve as containers for related platform and application resources. Namespace-scoped resources include DataPlanes, Environments, Projects, Components, and runtime resources generated by the platform. These resources are isolated within their namespace boundaries, leveraging Kubernetes RBAC and network policies to maintain security and access control. + +OpenChoreo also supports cluster-scoped resources that exist at the cluster level. This hybrid model enables both namespace-level isolation for team-specific resources and cluster-wide sharing of common platform infrastructure. + +### Project Ownership + +Within a namespace, **Projects** form the next level of ownership for application resources. Projects own both +**Components** and **Resources**, establishing team boundaries and application domains. This ownership relationship +means that components and resources cannot exist without a parent project, and deleting a project removes all its +components and resources. + +The project ownership boundary also defines the scope for internal communication and resource sharing. Components +within the same project can reference each other's endpoints directly and consume the project's Resources through +Workload dependencies. This locality enables teams to work efficiently within their domain while maintaining +isolation from other projects. + +### ComponentType + +**ComponentTypes** define platform-level templates that govern how components are deployed. Each ComponentType +specifies a workload type (deployment, statefulset, cronjob, job, or proxy) and provides a schema for configurable +parameters. Platform engineers create ComponentTypes that encode organizational policies, resource templates, and +operational best practices. + +ComponentTypes serve as the platform's contract for component deployment. They define what developers can configure +through parameter schemas, what resources will be created through resource templates, and which workflows are allowed +for building components. By centralizing these definitions in ComponentTypes, platform teams ensure consistency across +all applications while maintaining control over infrastructure patterns. + +### Component to ComponentType + +**Components** reference a ComponentType using the format `{workloadType}/{componentTypeName}`. This relationship +establishes which template governs the component's deployment. The component provides parameter values that conform to +the schema defined in the ComponentType, and these parameters are validated automatically. + +The component-to-ComponentType relationship is fundamental to OpenChoreo's governance model. Components cannot be +created without referencing a valid ComponentType, ensuring all deployments follow platform standards. This +relationship also determines which resource templates will be used to generate the actual Kubernetes resources. + +### Trait Composition + +**Traits** attach additional capabilities to components through composition. Each component can instantiate multiple +traits, such as persistent storage, caching, or monitoring. Traits use the same schema-driven approach as +ComponentTypes, with parameters that can be overridden per environment through ReleaseBinding resources. + +Traits maintain an independent lifecycle from components but are applied together during deployment. This separation +enables platform engineers to define reusable capabilities that can be composed into different component types. The +trait relationship provides flexibility while maintaining governance through schema validation. + +### Workflow Integration + +**Workflows** define build and automation templates that components can reference through their workflow configuration. +ComponentTypes can restrict which workflows are allowed, ensuring components use appropriate build strategies. This +relationship between components, ComponentTypes, and Workflows enables platform teams to enforce build standards and +security policies. + +When a component references a Workflow, it provides schema values that configure the workflow execution. The Workflow +template uses these values along with platform-controlled parameters to generate the actual CI/CD pipeline. This +separation enables developers to trigger builds with simple configuration while platform engineers maintain control +over build infrastructure and security. + +### Component to WorkflowRun + +Components trigger **WorkflowRuns** through their workflow configuration to build container images. Each WorkflowRun +represents an execution instance of a Workflow template with specific parameter values. WorkflowRuns maintain a +relationship back to their originating component and the specific commit that triggered them. + +This relationship provides complete traceability from running containers back to source code. Platform operators can +trace any container image to the WorkflowRun that built it, which links to the component and ultimately to the source +repository and commit. This traceability is essential for debugging, compliance, and security auditing. + +### ComponentRelease and ReleaseBinding + +A **ComponentRelease** is an immutable snapshot of a component at a specific point in time. It captures the Component, +its Workload configuration, the selected ComponentType, and any attached Traits into a single versioned resource. This +snapshot works like a lock file—preserving the exact state so that the release remains stable and repeatable, even if +the original resources are later updated. + +To deploy a ComponentRelease to an environment, you create a **ReleaseBinding**. This resource binds the release to a +specific environment and provides environment-specific overrides for ComponentType and Trait parameters. The +ReleaseBinding controller renders the final Kubernetes manifests and produces a **RenderedRelease** resource that is applied +to the target plane. + +This relationship chain—Component → ComponentRelease → ReleaseBinding → RenderedRelease—ensures complete governance while +enabling environment-specific customization. Because ComponentRelease is immutable, the same release can be promoted +across environments (dev → staging → prod) without drift. + +### Resource and ResourceType + +**ResourceTypes** define platform-level templates for managed infrastructure—databases, caches, queues, object stores. +Each ResourceType declares a parameter schema, the Kubernetes manifests the provisioner emits on the data plane, the +named outputs that consumers bind to, and a default retention policy. The cluster-scoped variant +**ClusterResourceType** is available across all namespaces. + +**Resources** reference a ResourceType (or ClusterResourceType) by name. The Resource provides parameter values that +conform to the referenced template's schema; validation failures surface on the Resource itself. The reference is +fixed at create time, so a Resource cannot be retargeted to a different template after it is created. + +The Resource-to-ResourceType relationship mirrors Component-to-ComponentType. Resources cannot be created without a +valid ResourceType reference, ensuring every Resource is governed by a template the platform team has approved. + +### ResourceRelease and ResourceReleaseBinding + +A **ResourceRelease** is an immutable snapshot of a Resource and its referenced ResourceType at a specific point in +time. When either input changes, a new ResourceRelease is cut; existing releases remain unchanged so the same +snapshot can be promoted to multiple environments without re-rendering. This mirrors how ComponentRelease works for +components. + +To deploy a ResourceRelease to an environment, a platform engineer or GitOps process creates a +**ResourceReleaseBinding**. The binding pins a specific ResourceRelease, targets an Environment, and provides +per-environment configuration overrides. The captured ResourceType template is rendered with the combined +configuration, producing a **RenderedRelease** that is applied to the target data plane, and the declared outputs +are resolved so consuming workloads can read them. + +The chain—Resource → ResourceRelease → ResourceReleaseBinding → RenderedRelease—is the resource-side parallel of the +Component chain. The Resource controller never authors bindings on its own; promoting a release into an environment +is always an explicit action by a platform engineer or GitOps tooling. This isolation keeps developer-facing Resource +spec edits from triggering automatic redeploys against stateful infrastructure. + +## Network Relationships + +Network relationships in OpenChoreo define how components communicate and expose functionality. These relationships +translate into concrete network policies, routing rules, and security configurations. + +### Endpoint Exposure + +**Endpoints** establish relationships between components and their network interfaces. A component can expose multiple +endpoints with different visibility scopes and protocols. These relationships define how the component can be accessed +and by whom. + +The endpoint relationship includes more than just network configuration. It establishes contracts about protocols, +expected behavior, and API compatibility. Other components can depend on these contracts, with the platform ensuring +that communication follows the declared patterns. + +### Dependencies + +**Dependencies** create explicit relationships from a Component to the things it consumes. A Workload declares +two kinds of dependency: + +**Endpoint dependencies** (`dependencies.endpoints[]`) target endpoints exposed by other components. The platform +resolves the target address, injects it into the consuming container through environment variables, and configures +network policies for the resulting traffic. + +**Resource dependencies** (`dependencies.resources[]`) target a Resource in the same project. The named outputs +declared by the referenced ResourceType—hostnames, ports, credentials, connection URLs—are bound to environment +variables or file mounts on the consuming container. Sensitive outputs are delivered through Kubernetes Secret +references; the underlying values never transit the control plane. + +Both forms of dependency make the consuming relationships explicit. The platform uses them for service discovery, +network policy generation, resource readiness gating, and to track which components depend on which endpoints and +Resources. + +## Environment Progression + +Environment relationships define how applications move through the delivery pipeline from development to production. +These relationships ensure consistent progression while maintaining appropriate governance. + +### Pipeline Definition + +**DeploymentPipelines** establish relationships between environments, defining allowed transitions and promotion rules. +These relationships create a directed graph of environment progression, potentially with multiple paths for different +scenarios. + +Pipeline relationships include more than just ordering. They define testing gates and promotion rules. These +relationships ensure that applications follow organizational processes while enabling automation where appropriate. + +## Lifecycle Dependencies + +Resource relationships in OpenChoreo include lifecycle dependencies that ensure proper initialization, updates, and +cleanup. + +### Creation Order + +Some resources must be created before others can exist. DataPlanes must be registered before environments can target +them. Projects must exist before components can be created within them. These relationships ensure that the platform +maintains consistency during resource creation. + +### Update Propagation + +When resources are updated, changes propagate through relationships to dependent resources. Updating a ComponentType +triggers reconciliation of all components that reference it, creating new ComponentReleases that capture the updated +template. Since ComponentReleases are immutable snapshots, existing deployments are not affected until a new release is deployed. + +### Deletion Cascades + +Resource relationships define deletion behavior. When a project is deleted, all its components and Resources are +removed. When a Component is deleted, its WorkflowRuns, ComponentReleases, ReleaseBindings, and RenderedReleases are +cleaned up. When a Resource is deleted, deletion is blocked while any ResourceReleaseBinding still references it; once +all bindings are gone, the owned ResourceReleases are removed. These cascading relationships ensure that resources are +properly cleaned up without leaving orphaned objects. + +The retention of the underlying data-plane state on a binding deletion is governed by the binding's retention policy. +Production environments can preserve volumes and databases on deletion while dev and staging dispose of them, the same +shape as Kubernetes PersistentVolume reclaim policy applied per binding. diff --git a/versioned_docs/version-v1.2.0-rc.2/concepts/runtime-model.md b/versioned_docs/version-v1.2.0-rc.2/concepts/runtime-model.md new file mode 100644 index 00000000..8ad5a2e8 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/concepts/runtime-model.md @@ -0,0 +1,122 @@ +--- +title: Runtime Model +description: Runtime execution and deployment model +--- + +# Runtime Model + +The runtime model describes how OpenChoreo's abstractions transform into running systems. When developers declare +projects and components, the platform orchestrates a sophisticated runtime environment that provides isolation, +security, and observability. Understanding this transformation from declaration to execution helps teams design better +applications and troubleshoot issues effectively. + +## Cell Architecture + +At runtime, each **Project** transforms into a **Cell** - a secure, isolated runtime boundary that encapsulates all +components of an application domain. This transformation represents a fundamental principle of OpenChoreo: +organizational boundaries in code become physical boundaries in infrastructure. + +Cells provide complete isolation between different application domains. Each cell operates as an independent unit with +its own namespace, network policies, and security boundaries. Components within a cell can communicate freely using +cluster-local networking, but all communication across cell boundaries must flow through well-defined gateways. This +architecture ensures that failures, security breaches, or performance issues in one cell cannot directly impact others. + +The cell model aligns with microservices best practices and Domain-Driven Design principles. By mapping bounded contexts +to isolated runtime units, OpenChoreo ensures that architectural boundaries are enforced by infrastructure. This +alignment reduces the cognitive load on developers - the same mental model used for designing applications applies to +their runtime behavior. + +## Traffic Flow Patterns + +OpenChoreo implements a structured approach to network traffic through directional gateways, each serving a specific +purpose in the overall communication architecture. These patterns provide clarity about how data flows through the +system while enabling sophisticated security and routing policies. + +### Northbound Ingress + +Northbound ingress handles traffic entering cells from the public internet. This gateway serves as the entry point for +external users, customers, and third-party integrations accessing public APIs and web applications. The platform +automatically configures load balancing and TLS termination at this boundary. + +The northbound gateway translates friendly DNS names into internal service endpoints, handles HTTP routing based on +hostnames and paths, and enforces public-facing API policies. This abstraction means developers can focus on application +logic while the platform manages the complexities of internet-facing services. + +### Southbound Egress + +Southbound egress manages traffic leaving cells to reach external services on the internet. This gateway provides +controlled access to third-party APIs, cloud services, and external databases. By channeling all outbound traffic +through a managed gateway, the platform can enforce security policies, manage credentials, and provide observability for +external dependencies. + +The southbound gateway enables capabilities like egress filtering to prevent data exfiltration, credential injection for +authenticated external services, and circuit breaking for unreliable external dependencies. This controlled approach to +external communication reduces security risks while improving reliability. + +### Westbound Ingress + +Westbound ingress handles traffic from other parts of the organization entering the cell. This gateway manages internal +API consumption, service-to-service communication across projects, and administrative access. Unlike public northbound +traffic, westbound traffic comes from trusted sources within the organization but still requires authentication and +authorization. + +The westbound gateway enables internal service discovery, allowing components in other cells to locate and communicate +with services using logical names. It enforces organization-wide policies while allowing more permissive communication +than public interfaces. This balance enables productive development while maintaining security boundaries. + +### Eastbound Egress + +Eastbound egress manages traffic leaving the cell to reach other cells or internal services within the organization. +This gateway handles inter-project dependencies, shared service consumption, and platform service integration. By +managing internal outbound traffic, the platform can track dependencies, enforce quotas, and provide circuit breaking +between internal services. + +The eastbound gateway makes internal service dependencies explicit and observable. Teams can understand which other +projects they depend on, platform engineers can track usage patterns across the organization, and the system can prevent +cascading failures through circuit breaking and retry policies. + +## Network Security + +OpenChoreo manages network security through the cell architecture and gateway pattern. Communication within a cell is +allowed between components of the same project, while all cross-cell communication must flow through the defined +gateways. This provides security boundaries between different application domains. + +The platform uses Cilium for network policy enforcement and kgateway for ingress traffic management. Developers +declare their components' endpoints and dependencies in the workload specification, and the platform handles the +underlying network configuration. + +## Workload Execution + +When components deploy to a runtime environment, OpenChoreo orchestrates a complex series of transformations. The +abstract component definition combines with workload specifications, environment configurations, and platform policies +to produce concrete Kubernetes resources. + +The platform manages the complete lifecycle of these resources. It creates deployments with appropriate resource limits +and scaling parameters, configures services for network access and load balancing, injects configuration and secrets +from secure stores, and establishes health checks and readiness probes. This orchestration happens transparently - +developers see their components running while the platform manages the underlying complexity. + +Workload execution is environment-aware. The same component might run with different resource allocations, replica +counts, or configuration values in different environments. The platform manages these variations through the binding +system, where environment-specific bindings override default values from ComponentTypes, Traits, and workload specifications. + +Resources follow the same render-and-apply path. Each ResourceReleaseBinding renders its ResourceType template with +the combined Resource parameters and environment overrides, produces a RenderedRelease, and applies the resulting +manifests to the data plane. The provisioned outputs are resolved back to the binding so consuming components can read +them at pod-start. The render gate on the consuming Component waits for its declared Resource dependencies to be +Ready before producing its own RenderedRelease, so a component is not deployed until the infrastructure it depends on +is healthy. + +## Service Discovery and Load Balancing + +Service discovery uses standard Kubernetes DNS, allowing services to communicate using service names. Within a cell, +components can discover each other using simple service names. Across cells, services require routing through the +appropriate gateways. + +## Observability + +The runtime model includes centralized logs, metrics, and traces collected through the Observability Plane. Logs from all containers flow through the platform's collection pipeline configured in the Observability Plane, enriched with metadata about the source component, project, and environment. This enrichment enables queries across the entire platform while maintaining clear attribution of log entries to their sources. + +The Observer API provides access to these logs, metrics, and traces, allowing developers to search and analyze application behavior across environments. This centralized observability approach ensures that debugging information is always available without requiring additional configuration from developers. + +The platform also provides a centralized alerting and RCA (Root Cause Analysis) system through the Observability Plane. Developers can define alert rules in their components' traits, and the platform will automatically create alert rules in the Observability Plane. When an alert fires, incident creation and AI RCA can be triggered if they are enabled for that rule. diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/endpoints.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/endpoints.md new file mode 100644 index 00000000..d49118ac --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/endpoints.md @@ -0,0 +1,225 @@ +--- +title: Endpoint Dependencies +description: Learn how to define dependencies between components by consuming endpoints from other components +--- + +# Endpoint Dependencies + +Components in OpenChoreo can consume endpoints exposed by other components by declaring **endpoint dependencies** in their Workload. When a dependency is declared, the platform automatically: + +- Resolves the target component's endpoint address +- Injects connection details as environment variables into the consuming component's container +- Configures network policies to allow traffic between the components + +## Defining a Dependency + +Dependencies are defined in the `spec.dependencies.endpoints` array of a Workload. Each entry specifies which component endpoint to consume and how to inject the connection details: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Workload +metadata: + name: frontend-workload + namespace: default +spec: + owner: + projectName: my-project + componentName: frontend + container: + image: myregistry/frontend:v1.0.0 + endpoints: + http: + type: HTTP + port: 3000 + visibility: + - external + dependencies: + endpoints: + - component: backend-api + name: http + visibility: project + envBindings: + address: BACKEND_API_URL +``` + +In this example, the `frontend` component declares a dependency on the `backend-api` component's `http` endpoint. The platform resolves the endpoint address and injects it as the `BACKEND_API_URL` environment variable. + +## Dependency Fields + +| Field | Type | Required | Description | +| ------------- | ------ | -------- | ----------------------------------------------------------------------------- | +| `component` | string | Yes | Name of the target component | +| `name` | string | Yes | Name of the target endpoint on the component | +| `visibility` | string | Yes | Visibility level at which to consume the endpoint (`project` or `namespace`) | +| `project` | string | No | Target component's project name. Defaults to the same project as the consumer | +| `envBindings` | object | Yes | Maps connection details to environment variable names | + +### Environment Bindings + +The `envBindings` field controls which connection details are injected as environment variables: + +| Field | Type | Description | +| ---------- | ------ | -------------------------------------------------------------------------------------------------------------------------------- | +| `address` | string | Env var name for the full connection string. For HTTP/HTTPS/WS/WSS: `scheme://host:port/basePath`. For gRPC/TCP/UDP: `host:port` | +| `host` | string | Env var name for just the hostname | +| `port` | string | Env var name for just the port number | +| `basePath` | string | Env var name for just the base path | + +You can specify any combination of these bindings. At minimum, `address` is the most commonly used. + +## Visibility + +The `visibility` field determines at which network scope the dependency is consumed. Only `project` and `namespace` are supported for dependencies: + +| Visibility | When to Use | +| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `project` | The target component is in the **same project**. Traffic stays within the project's data plane namespace. | +| `namespace` | The target component is in a **different project** but the same namespace. Requires the target endpoint to have `namespace` (or broader) visibility declared. | + +:::note +The target endpoint must have a visibility level that is equal to or broader than the dependency's visibility. For example, if you consume an endpoint at `namespace` visibility, the target endpoint must include `namespace` (or `internal` or `external`) in its visibility array. +::: + +## Same-Project Dependencies + +When both components are in the same project, use `project` visibility. The `project` field can be omitted since it defaults to the consumer's project: + +```yaml +dependencies: + endpoints: + - component: user-service + name: http + visibility: project + envBindings: + address: USER_SERVICE_URL +``` + +## Cross-Project Dependencies + +When the target component is in a different project, you must specify the `project` field and use `namespace` visibility: + +```yaml +dependencies: + endpoints: + - component: shared-auth-service + name: http + visibility: namespace + project: platform-services + envBindings: + address: AUTH_SERVICE_URL + host: AUTH_SERVICE_HOST + port: AUTH_SERVICE_PORT +``` + +For cross-project dependencies to work, the target component's endpoint must have `namespace` (or broader) visibility declared in its Workload: + +```yaml +# Target component's Workload (in the platform-services project) +spec: + endpoints: + http: + type: HTTP + port: 8080 + visibility: + - namespace # Required for cross-project access +``` + +If a cross-project dependency is failing with a connection refused error, verify that the target endpoint has the appropriate visibility level configured. + +## Multiple Dependencies + +A single Workload can declare up to 50 endpoint dependencies: + +```yaml +dependencies: + endpoints: + - component: user-service + name: http + visibility: project + envBindings: + address: USER_SERVICE_URL + - component: notification-service + name: grpc + visibility: project + envBindings: + address: NOTIFICATION_SERVICE_ADDR + - component: analytics-api + name: http + visibility: namespace + project: analytics + envBindings: + address: ANALYTICS_API_URL + basePath: ANALYTICS_API_BASE_PATH +``` + +## Complete Example + +Here is a full example with a `greeter-service` that depends on both a same-project component and a cross-project component: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Component +metadata: + name: greeter-service + namespace: default +spec: + autoDeploy: true + componentType: + kind: ClusterComponentType + name: deployment/service + owner: + projectName: default +--- +apiVersion: openchoreo.dev/v1alpha1 +kind: Workload +metadata: + name: greeter-service-workload + namespace: default +spec: + owner: + projectName: default + componentName: greeter-service + container: + image: ghcr.io/openchoreo/samples/greeter-service:latest + args: + - --port + - "9090" + endpoints: + http: + type: HTTP + port: 9090 + visibility: + - external + dependencies: + endpoints: + - component: hello-world-service + name: http + visibility: project + envBindings: + address: HELLO_WORLD_SERVICE_URL + - component: bar-service + name: http + visibility: namespace + project: test-project + envBindings: + address: BAR_SERVICE_URL +``` + +After deployment, you can verify the injected environment variables by exec-ing into the pod: + +```bash +kubectl exec -n -- env | grep -E 'HELLO_WORLD|BAR_SERVICE' +``` + +You should see the resolved addresses: + +```text +HELLO_WORLD_SERVICE_URL=http://hello-world-service..svc.cluster.local:9090 +BAR_SERVICE_URL=http://bar-service..svc.cluster.local:9090 +``` + +## Related Resources + +- [Workload](../workload/overview.md) - How to define Workloads +- [Workload API Reference](../../reference/api/application/workload.md) - Full CRD specification +- [Component](../../reference/api/application/component.md) - Components that own workloads diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/resources.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/resources.md new file mode 100644 index 00000000..cf9f5db0 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/dependencies/resources.md @@ -0,0 +1,235 @@ +--- +title: Resource Dependencies +description: Learn how a Component consumes a project-bound Resource by binding its outputs to environment variables and file mounts +--- + +# Resource Dependencies + +Components in OpenChoreo consume managed infrastructure—databases, queues, caches, object stores—by declaring **resource dependencies** in their Workload. Each entry references a project-bound [Resource](../../concepts/developer-abstractions.md#resource) and wires the named outputs declared on the referenced ResourceType into the consuming container. + +When a resource dependency is declared, the platform automatically: + +- Resolves the active `ResourceReleaseBinding` for the consumer's environment and reads its `status.outputs` +- Gates the consumer's deployment until that binding reports `Ready` +- Injects the resolved outputs as container environment variables (`envBindings`) and file mounts (`fileBindings`) +- Keeps sensitive values on the data plane: only the underlying `{name, key}` reference to a Secret or ConfigMap transits the control plane + +## Defining a Dependency + +Resource dependencies are defined in the `spec.dependencies.resources` array of a Workload. Each entry references a Resource by name and picks which of its outputs to inject: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Workload +metadata: + name: doclet-document + namespace: default +spec: + owner: + projectName: doclet + componentName: doclet-document + container: + image: ghcr.io/openchoreo/samples/doclet-document:latest + dependencies: + resources: + - ref: doclet-postgres + envBindings: + host: DB_HOST + port: DB_PORT + username: DB_USER + password: DB_PASSWORD + database: DB_NAME +``` + +In this example, the `doclet-document` Workload declares a dependency on the `doclet-postgres` Resource (a Postgres database provisioned from a `ClusterResourceType`). The platform resolves five outputs—`host`, `port`, `username`, `password`, `database`—and injects each as the corresponding environment variable. The credential is delivered through a `valueFrom.secretKeyRef`; the container reads the actual value from the data-plane Secret at pod-start. + +## Dependency Fields + +| Field | Type | Required | Description | +| -------------- | ----------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| `ref` | string | Yes | Name of the target Resource. Must live in the same project as the consuming Component | +| `envBindings` | map[string]string | No | Maps a ResourceType output name (key) to a container environment variable name (value) | +| `fileBindings` | map[string]string | No | Maps a ResourceType output name (key) to a container mount path (value). The referenced output must be backed by `secretKeyRef` or `configMapKeyRef` | + +An entry must specify `ref` and at least one of `envBindings` or `fileBindings`. Outputs the ResourceType declares but the Workload does not list are ignored—each Workload picks only the outputs it actually needs. + +### Environment Bindings + +The `envBindings` map binds each output to an environment variable on the consuming container. The shape of the resulting `env` entry depends on the ResourceType output's source kind: + +| Output source kind | Resulting env var shape | +| ------------------ | ----------------------------------------------------------------------------------------------------- | +| `value` | Literal value: `env: [{name: DB_HOST, value: "postgres.doclet.svc.cluster.local"}]` | +| `secretKeyRef` | Secret reference: `env: [{name: DB_PASSWORD, valueFrom: {secretKeyRef: {name: ..., key: ...}}}]` | +| `configMapKeyRef` | ConfigMap reference: `env: [{name: APP_REGION, valueFrom: {configMapKeyRef: {name: ..., key: ...}}}]` | + +Sensitive outputs are declared as `secretKeyRef` on the ResourceType. Only the `{name, key}` reference transits the control plane; the underlying credential is never serialized to the consumer's `ReleaseBinding.status` or to the rendered `Deployment` manifest. + +### File Bindings + +The `fileBindings` map mounts each output into the consuming container's filesystem. The referenced output must be backed by `secretKeyRef` or `configMapKeyRef`—`value`-kind outputs cannot be file-mounted because there is no data-plane object to mount. + +```yaml +dependencies: + resources: + - ref: my-mtls-creds + fileBindings: + ca-bundle: /etc/ssl/certs/ca-bundle.pem + client-cert: /etc/ssl/certs/client.crt + client-key: /etc/ssl/private/client.key +``` + +The platform synthesizes one Volume per `(resource, output)` pair (deduplicated when the same output is referenced multiple times) and one VolumeMount per declared path. The container reads each file at the declared path; underlying Secret or ConfigMap updates propagate through the standard kubelet projection. + +## Multiple Dependencies + +A Workload can declare up to 50 resource dependencies. Each entry's `ref` must be unique within the Workload, and endpoint and resource dependencies can be declared together: + +```yaml +dependencies: + endpoints: + - component: doclet-collab + name: ws + visibility: project + envBindings: + address: COLLAB_URL + resources: + - ref: doclet-postgres + envBindings: + host: DB_HOST + port: DB_PORT + username: DB_USER + password: DB_PASSWORD + database: DB_NAME + - ref: doclet-nats + envBindings: + url: DOCLET_NATS_URL +``` + +The render gate waits for both endpoint and resource dependencies before proceeding. + +## Complete Example + +A full example: a `doclet-document` Component depending on a Postgres Resource for storage and a NATS Resource for pub/sub. + +The Component, Workload, and matching Resources: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Component +metadata: + name: doclet-document + namespace: default +spec: + owner: + projectName: doclet + autoDeploy: true + componentType: + kind: ClusterComponentType + name: deployment/service +--- +apiVersion: openchoreo.dev/v1alpha1 +kind: Workload +metadata: + name: doclet-document + namespace: default +spec: + owner: + projectName: doclet + componentName: doclet-document + container: + image: ghcr.io/openchoreo/samples/doclet-document:latest + endpoints: + http: + type: HTTP + port: 8080 + visibility: [project] + dependencies: + resources: + - ref: doclet-postgres + envBindings: + host: DB_HOST + port: DB_PORT + username: DB_USER + password: DB_PASSWORD + database: DB_NAME + - ref: doclet-nats + envBindings: + url: DOCLET_NATS_URL +--- +apiVersion: openchoreo.dev/v1alpha1 +kind: Resource +metadata: + name: doclet-postgres + namespace: default +spec: + owner: + projectName: doclet + type: + kind: ClusterResourceType + name: postgres + parameters: + database: doclet +--- +apiVersion: openchoreo.dev/v1alpha1 +kind: Resource +metadata: + name: doclet-nats + namespace: default +spec: + owner: + projectName: doclet + type: + kind: ClusterResourceType + name: nats +``` + +And the `ResourceReleaseBinding` for the `development` environment (one per Resource per environment, authored by the platform engineer or GitOps tooling): + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: ResourceReleaseBinding +metadata: + name: doclet-postgres-development + namespace: default +spec: + owner: + projectName: doclet + resourceName: doclet-postgres + environment: development + resourceRelease: doclet-postgres-abc12345 # advanced via `occ resource promote` +``` + +After the bindings report `Ready` and the consuming Component is deployed, verify the injected variables from inside the pod: + +```bash +kubectl exec -n -- env | grep -E 'DB_|NATS' +``` + +The output shows the resolved connection details, with the password loaded from a Secret reference: + +```text +DB_HOST=postgres-doclet.openchoreo-dp-doclet-development.svc.cluster.local +DB_PORT=5432 +DB_USER=doclet +DB_NAME=doclet +DB_PASSWORD= +DOCLET_NATS_URL=nats://@nats-doclet.openchoreo-dp-doclet-development.svc.cluster.local:4222 +``` + +## Inspecting Outputs on the Binding + +To inspect the resolved outputs without deploying a consumer, view the `ResourceReleaseBinding.status.outputs` directly: + +```bash +kubectl get resourcereleasebinding doclet-postgres-development -o yaml +``` + +Each entry in `status.outputs[]` carries the output name and one of `value`, `secretKeyRef`, or `configMapKeyRef`. For credential outputs, only the Secret name and key are shown—the actual value never appears here, matching the shape that lands in the consuming pod. + +## Related Resources + +- [Workload](../workload/overview.md) - How to define Workloads +- [Endpoint Dependencies](./endpoints.md) - Consuming endpoints exposed by other components +- [Resource concept](../../concepts/developer-abstractions.md#resource) - Overview of the Resource abstraction +- [Workload API Reference](../../reference/api/application/workload.md) - Full Workload CRD specification diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/deploy-and-promote.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/deploy-and-promote.md new file mode 100644 index 00000000..f10757ed --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/deploy-and-promote.md @@ -0,0 +1,79 @@ +--- +title: Deploy and Promote +description: Deploy components to environments and promote across the deployment pipeline +--- + +# Deploy and Promote + +## Deploy to the First Environment + +### Via Backstage UI + +1. Navigate to your Component in the Backstage console +2. Go to the **Deploy** tab +3. The first environment card shows your deployment status + +For components with `autoDeploy: true`, the deployment happens automatically when the Component and Workload are created. For manual deployment, use the CLI or configure overrides through the environment card settings. + +### Via CLI + +Deploy the latest release to the root (first) environment: + +```bash +occ component deploy my-service --namespace default --project default +``` + +Deploy a specific release: + +```bash +occ component deploy my-service --release my-service-5d7f658d9c +``` + +## Promote to the Next Environment + +Promotion moves a deployment from one environment to the next in the pipeline (e.g., development to staging). + +### Via Backstage UI + +1. On the **Deploy** tab, find the environment card for the deployed environment +2. Click the **Promote** button +3. The deployment promotes to the next environment in the pipeline + +### Via CLI + +```bash +# Promote to staging +occ component deploy my-service --to staging + +# Promote to production +occ component deploy my-service --to production +``` + +## View Deployment Status + +### Via Backstage UI + +The **Deploy** tab shows: + +- Deployment status per environment (Ready, NotReady, Failed) +- Last deployed timestamp +- Container image reference +- Release name +- Endpoint URLs + +Click **View K8s Artifacts** on an environment card to see the full resource tree, including Deployments, Pods, Services, and HTTPRoutes. + +### Via CLI + +```bash +# Check ReleaseBinding status +occ releasebinding list --namespace default --project default --component my-service + +# Check component status +occ component get my-service --namespace default +``` + +## What's Next + +- [Environment Overrides](./environment-overrides.md): customize configuration per environment +- [Logs and Troubleshooting](./logs-and-troubleshooting.md): view runtime logs and manage deployments diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/environment-overrides.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/environment-overrides.md new file mode 100644 index 00000000..2370c645 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/environment-overrides.md @@ -0,0 +1,105 @@ +--- +title: Environment Overrides +description: Customize component configuration per environment using overrides +--- + +# Environment Overrides + +The same ComponentRelease is deployed to all environments, but each environment can have different configuration. Environment overrides let you customize parameters, trait settings, and workload configuration per environment without creating separate releases. + +## Types of Overrides + +| Override Type | What It Configures | Example | +| --------------------------------- | ----------------------------- | ---------------------------------- | +| `componentTypeEnvironmentConfigs` | ComponentType parameters | Replicas, resource limits, port | +| `traitEnvironmentConfigs` | Trait parameters per instance | Alert thresholds, storage class | +| `workloadOverrides` | Workload-level settings | Environment variables, file mounts | + +## Configuring Overrides via Backstage UI + +1. Navigate to your Component's **Deploy** tab +2. Click the **gear icon** on an environment card header +3. The Overrides page shows tabs for: + - **Workload**: edit environment variables, file mounts, and endpoints + - **Component**: edit parameters defined by the ComponentType schema + - **Traits**: edit parameters for each attached trait instance + +## Configuring Overrides via CLI + +Use `--set` with `occ component deploy` to apply overrides during deployment or promotion: + +```bash +# Set replicas for production +occ component deploy my-service --to production \ + --set spec.componentTypeEnvironmentConfigs.replicas=3 + +# Set resource limits +occ component deploy my-service --to production \ + --set spec.componentTypeEnvironmentConfigs.resources.requests.cpu=500m \ + --set spec.componentTypeEnvironmentConfigs.resources.requests.memory=512Mi +``` + +The `--set` flag uses dot-notation paths to target specific fields in the ReleaseBinding spec. + +## Configuring Overrides via YAML + +You can also create or edit a ReleaseBinding directly: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: ReleaseBinding +metadata: + name: my-service-production + namespace: default +spec: + owner: + projectName: default + componentName: my-service + environment: production + releaseName: my-service-5d7f658d9c + state: Active + componentTypeEnvironmentConfigs: + replicas: 3 + resources: + requests: + cpu: "500m" + memory: "512Mi" + traitEnvironmentConfigs: + high-error-rate: + enabled: true + threshold: 50 +``` + +## Common Override Patterns + +### Different replicas per environment + +```bash +# Development: 1 replica +occ component deploy my-service --set spec.componentTypeEnvironmentConfigs.replicas=1 + +# Production: 3 replicas +occ component deploy my-service --to production \ + --set spec.componentTypeEnvironmentConfigs.replicas=3 +``` + +### Environment-specific environment variables + +Environment variables can be overridden at the workload level: + +```bash +occ component deploy my-service --to production \ + --set spec.workloadOverrides.env.LOG_LEVEL=warn +``` + +### Disable a trait in development + +```bash +occ component deploy my-service \ + --set spec.traitEnvironmentConfigs.high-error-rate.enabled=false +``` + +## What's Next + +- [Logs and Troubleshooting](./logs-and-troubleshooting.md): view runtime logs and manage deployments +- [ReleaseBinding API Reference](../../reference/api/platform/releasebinding.md): full spec for ReleaseBinding overrides diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/logs-and-troubleshooting.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/logs-and-troubleshooting.md new file mode 100644 index 00000000..e486ca2e --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/logs-and-troubleshooting.md @@ -0,0 +1,108 @@ +--- +title: Logs and Troubleshooting +description: View runtime logs and manage deployments in OpenChoreo +--- + +# Logs and Troubleshooting + +## Runtime Logs + +### Via Backstage UI + +1. Navigate to your Component +2. Click the **Logs** tab to search and filter logs by environment, time range, and keywords + +When the observability plane is configured, additional tabs are available: + +- **Metrics**: view metric graphs for your component +- **Alerts**: view triggered alerts + +### Via CLI + +```bash +# Get recent logs +occ component logs my-service --namespace default --project default + +# Follow logs in real-time +occ component logs my-service -f + +# Logs from a specific environment +occ component logs my-service --env production + +# Last 100 lines +occ component logs my-service --tail 100 + +# Logs since a specific time +occ component logs my-service --since 1h +``` + +## Viewing Kubernetes Resources + +### Via Backstage UI + +Click **View K8s Artifacts** on any environment card to see the resource tree: + +- Deployments and ReplicaSets +- Pods (with status and events) +- Services +- HTTPRoutes +- Kubernetes events + +This view helps diagnose issues like pod crashes, image pull errors, or routing problems. + +### Via CLI + +```bash +# Check the underlying deployment +kubectl get deployment -A -l openchoreo.dev/component=my-service + +# Check pod status +kubectl get pods -A -l openchoreo.dev/component=my-service + +# View pod logs directly +kubectl logs -l openchoreo.dev/component=my-service -n + +# Check events +kubectl get events -A --field-selector involvedObject.name=my-service +``` + +## Undeploy and Redeploy + +### Undeploy + +Undeploying removes the running workload from an environment without deleting the ReleaseBinding. This lets you redeploy later without reconfiguring. + +In the Backstage UI, click **Undeploy** on the environment card. + +### Redeploy + +To restore an undeployed component, click **Redeploy** on the environment card. This reactivates the existing ReleaseBinding. + +## Rollback + +To roll back to a previous release, deploy an older ComponentRelease by name: + +```bash +# List available releases +occ componentrelease list --namespace default --project default --component my-service + +# Deploy a specific older release +occ component deploy my-service --release my-service-a1b2c3d4e5 +``` + +This creates a new ReleaseBinding pointing to the older release, effectively rolling back the deployment. + +## Common Issues + +| Symptom | Possible Cause | What to Check | +| ----------------------------- | ---------------------------------------------- | -------------------------------------------------------- | +| Component stuck in "NotReady" | Data plane connectivity | Check ClusterAgent pod logs in the data plane | +| Pods in CrashLoopBackOff | Application error | Check pod logs via Backstage or `kubectl logs` | +| Image pull error | Wrong image reference or missing credentials | Verify container image URL and registry access | +| No endpoints accessible | HTTPRoute not created or gateway misconfigured | Check `kubectl get httproute -A` and gateway pod logs | +| Deployment not appearing | ReleaseBinding not created | Check `occ releasebinding list` and Component conditions | + +## What's Next + +- [Deploy and Promote](./deploy-and-promote.md): deploy and promote across environments +- [Environment Overrides](./environment-overrides.md): customize configuration per environment diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/overview.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/overview.md new file mode 100644 index 00000000..2518ee46 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/deploying-applications/overview.md @@ -0,0 +1,49 @@ +--- +title: Overview +description: Understand how OpenChoreo deploys your components across environments +--- + +# Deploying Applications + +OpenChoreo deploys your components through a chain of resources that separate what you want to deploy from where and how it gets deployed. This model enables reliable rollbacks, environment-specific configuration, and controlled promotion across environments. + +## How Deployment Works + +When you create a Component and Workload, OpenChoreo creates a deployment chain: + +```mermaid +flowchart TD + A["Component + Workload"] --> B["ComponentRelease
(immutable snapshot)"] + B --> C["ReleaseBinding
(per environment)"] + C --> D["Deployment + Service + HTTPRoute
(on data plane)"] +``` + +- **ComponentRelease**: an immutable snapshot that freezes your Component's configuration, including the ComponentType spec, Traits, parameters, and Workload. A new release is only created when something changes. +- **ReleaseBinding**: binds a ComponentRelease to a specific environment (e.g., development, staging, production). This is where environment-specific overrides are applied. +- **Kubernetes resources**: the ReleaseBinding produces the final Deployment, Service, and HTTPRoute resources on the data plane. + +## Deployment Pipeline + +Each Project references a DeploymentPipeline that defines the promotion order between environments. For example, the default pipeline promotes through: + +```text +development → staging → production +``` + +When you deploy a component, it goes to the first environment in the pipeline. You then promote it to subsequent environments either manually or automatically. + +## autoDeploy + +When `autoDeploy: true` is set on a Component, OpenChoreo automatically creates a ReleaseBinding for the first environment in the deployment pipeline whenever a new ComponentRelease is created. This means: + +- Creating a Component with `autoDeploy: true` deploys it to the first environment immediately +- Updating the Workload creates a new release and deploys it automatically +- You still need to promote to subsequent environments manually or via GitOps + +Without `autoDeploy`, you deploy manually using the CLI or Backstage UI. + +## What's Next + +- [Deploy and Promote](./deploy-and-promote.md): deploy to environments and promote across the pipeline +- [Environment Overrides](./environment-overrides.md): configure environment-specific parameters +- [Logs and Troubleshooting](./logs-and-troubleshooting.md): view logs and manage deployments diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-component.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-component.md new file mode 100644 index 00000000..f5432a5c --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-component.md @@ -0,0 +1,246 @@ +--- +title: Creating a Component +description: Create and configure components for deployment in OpenChoreo +--- + +# Creating a Component + +A Component is a single deployable unit within a Project. It references a platform-defined ComponentType template that determines how it gets deployed, and optionally includes workflow configuration for building from source code. + +## Choosing a ComponentType + +Before creating a component, check which ComponentTypes are available. ComponentTypes are defined by platform engineers and determine the workload type (Deployment, StatefulSet, CronJob, etc.) and default configuration. + +```bash +# List cluster-scoped ComponentTypes (available to all namespaces) +occ clustercomponenttype list + +# List namespace-scoped ComponentTypes +occ componenttype list --namespace default +``` + +Common built-in ComponentTypes: + +| Component Type Reference | Workload Type | Use Case | +| ---------------------------- | ------------- | -------------------------------------------- | +| `deployment/service` | Deployment | Backend services and APIs | +| `deployment/web-application` | Deployment | Frontend or full-stack web apps | +| `deployment/worker` | Deployment | Background workers without exposed endpoints | +| `cronjob/scheduled-task` | CronJob | Periodic batch jobs | + +The reference format is `{workloadType}/{componentTypeName}`. This is used in the Component's `spec.componentType.name` field. + +## Creating via Backstage UI + +The Backstage UI provides a 3-step wizard for creating components. + +### Step 1: Component Metadata + +- Select your **Namespace** and **Project** +- Enter a **Component Name** (must be a valid Kubernetes name) +- Optionally add a **Display Name** and **Description** + +### Step 2: Build & Deploy + +Choose how your component gets its container image: + +**Build from Source**: OpenChoreo builds your code using a workflow + +- Select a **Workflow** from the available build workflows +- Enter your **Git Repository URL**, **Branch**, and **Application Path** +- Optionally select a **Git Secret** for private repositories + +**Container Image**: Deploy a pre-built image directly + +- Enter the **Container Image** reference (e.g., `ghcr.io/myorg/myapp:v1.0.0`) +- Toggle **Auto Deploy** to deploy automatically + +**External CI**: Use your existing CI system (Jenkins, GitHub Actions, GitLab CI) + +- Select your **CI Platform** +- Enter the platform-specific identifier (job path, repo slug, or project ID) + +### Step 3: Type-Specific Details + +This step is dynamically generated from the selected ComponentType (e.g., "Service Details", "Web Application Details"): + +- **Parameters**: configure values defined by the ComponentType schema (e.g., replicas, port, resource limits) +- **Endpoints**: define HTTP, gRPC, or WebSocket endpoints your service exposes (for deployment workload types) +- **Environment Variables**: set key-value pairs or reference secrets (available for container image deployments) +- **File Mounts**: mount configuration files into the container (available for container image deployments) +- **Traits**: attach reusable cross-cutting concerns from the platform's available traits + +Click **Create** to finish. + +:::note +For "Build from Source" deployments, environment variables and file mounts are configured via a [`workload.yaml` descriptor](../workflows/ci/workload-descriptor.md) in your source repository rather than in the wizard. +::: + +## Creating via YAML + +### Scaffolding with occ CLI + +The `occ component scaffold` command generates a Component YAML from a ComponentType, pre-filling defaults and documenting available fields: + +```bash +# Generate from a ClusterComponentType +occ component scaffold my-service \ + --clustercomponenttype deployment/service \ + --namespace default \ + --project default \ + -o my-service.yaml + +# Include traits +occ component scaffold my-service \ + --clustercomponenttype deployment/service \ + --clustertraits observability-alert-rule \ + -o my-service.yaml + +# Include a workflow for building from source +occ component scaffold my-service \ + --clustercomponenttype deployment/service \ + --clusterworkflow dockerfile-builder \ + -o my-service.yaml + +# Minimal output without comments +occ component scaffold my-service \ + --clustercomponenttype deployment/service \ + --skip-comments --skip-optional +``` + +Edit the generated file to fill in your values, then apply it. + +### From a Pre-built Image + +Create a Component and Workload together: + +```yaml +# component.yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Component +metadata: + name: greeter-service + namespace: default +spec: + owner: + projectName: default + componentType: + kind: ClusterComponentType + name: deployment/service + autoDeploy: true + +--- +apiVersion: openchoreo.dev/v1alpha1 +kind: Workload +metadata: + name: greeter-service-workload + namespace: default +spec: + owner: + projectName: default + componentName: greeter-service + container: + image: "ghcr.io/openchoreo/samples/greeter-service:latest" + env: + - key: LOG_LEVEL + value: "info" + endpoints: + http: + type: HTTP + port: 9090 + visibility: [external] +``` + +```bash +kubectl apply -f component.yaml +``` + +:::tip +The `componentType.kind` defaults to `ComponentType` (namespace-scoped) if omitted. Since the built-in types are `ClusterComponentType` (cluster-scoped), always specify `kind: ClusterComponentType` explicitly. The same applies to traits (`kind` defaults to `Trait`, so specify `kind: ClusterTrait` for cluster-scoped traits). Workflows default to `ClusterWorkflow`, so omitting `kind` is fine for cluster-scoped workflows. +::: + +### From Source Code + +Create a Component with workflow configuration: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Component +metadata: + name: my-service + namespace: default +spec: + owner: + projectName: default + componentType: + kind: ClusterComponentType + name: deployment/service + autoDeploy: true + autoBuild: true + workflow: + kind: ClusterWorkflow + name: dockerfile-builder + parameters: + repository: + url: "https://github.com/myorg/my-service" + revision: + branch: "main" + appPath: "/" + docker: + context: "." + filePath: "./Dockerfile" +``` + +```bash +kubectl apply -f component.yaml +``` + +The workflow builds the container image and creates a Workload automatically. If `autoBuild` is enabled, subsequent Git pushes trigger new builds via webhooks. + +## Adding Traits + +Traits are reusable extensions that add cross-cutting functionality to your component. Add them to the `traits` array: + +```yaml +spec: + traits: + - kind: ClusterTrait + name: observability-alert-rule + instanceName: high-error-rate + parameters: + enabled: true + condition: "error_count > 100" +``` + +Each trait instance needs a unique `instanceName` within the component. List available traits: + +```bash +occ clustertrait list +occ trait list --namespace default +``` + +## Managing Components + +### List components + +```bash +occ component list --namespace default --project default +``` + +### View component details + +```bash +occ component get greeter-service --namespace default +``` + +### Delete a component + +```bash +occ component delete greeter-service --namespace default +``` + +## What's Next + +- [Define Your Workload](../workload/overview.md): learn more about workload specifications +- [Build Your Code](../workflows/ci/overview.md): configure CI workflows and auto-build +- [Workload Descriptor](../workflows/ci/workload-descriptor.md): customize what your CI build produces diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-project.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-project.md new file mode 100644 index 00000000..02e803b6 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/creating-a-project.md @@ -0,0 +1,100 @@ +--- +title: Creating a Project +description: Set up a project to organize your components in OpenChoreo +--- + +# Creating a Project + +A Project is a logical boundary that groups related components together. Each project is linked to a deployment pipeline that defines how components are promoted across environments (e.g., development, staging, production). + +## What a Project Defines + +A Project resource is intentionally minimal: + +| Field | Description | +| ---------------------------- | --------------------------------------------------------------------------------------------------- | +| `metadata.name` | Unique name within the namespace (must be a valid Kubernetes name) | +| `metadata.namespace` | The namespace this project belongs to | +| `spec.deploymentPipelineRef` | Reference to a DeploymentPipeline that controls environment promotion | +| Annotations | Optional `openchoreo.dev/display-name` and `openchoreo.dev/description` for human-readable metadata | + +## Creating via Backstage UI + +1. Navigate to your namespace in the Backstage console +2. Click **Create Project** +3. Fill in the form: + - **Namespace**: pre-selected from your current context + - **Project Name**: a valid Kubernetes name + - **Display Name**: optional human-readable name + - **Description**: optional description + - **Deployment Pipeline**: select from available pipelines in the namespace (defaults to `default`) +4. Click **Create** + +The project appears immediately in the Backstage catalog. + +## Creating via YAML + +Create a `project.yaml` file: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Project +metadata: + name: online-store + namespace: default + annotations: + openchoreo.dev/display-name: "Online Store" + openchoreo.dev/description: "E-commerce application components" +spec: + deploymentPipelineRef: + kind: DeploymentPipeline + name: default +``` + +Apply it: + +```bash +kubectl apply -f project.yaml +``` + +Or using the occ CLI: + +```bash +occ apply -f project.yaml +``` + +:::tip Default Resources +When OpenChoreo is installed, a `default` project with a `default` deployment pipeline is created automatically. You only need to create additional projects if you want separate deployment pipelines or logical groupings. +::: + +## Managing Projects + +### List projects + +```bash +occ project list --namespace default +``` + +### View project details + +```bash +occ project get online-store --namespace default +``` + +### Delete a project + +```bash +occ project delete online-store --namespace default +``` + +:::warning +Deleting a project also deletes all components within it. +::: + +### Change deployment pipeline + +In the Backstage UI, click the edit icon on the Deployment Pipeline card on the project page to select a different pipeline. + +## What's Next + +- [Creating a Component](./creating-a-component.md): deploy a service, web app, or scheduled task within your project diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/overview.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/overview.md new file mode 100644 index 00000000..664f20fb --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/projects-and-components/overview.md @@ -0,0 +1,101 @@ +--- +title: Overview +description: Understand how Projects and Components organize your applications in OpenChoreo +--- + +# Projects and Components + +OpenChoreo organizes applications using a simple hierarchy: **Namespaces** contain **Projects**, and Projects contain **Components**. As a developer, you create Projects to group related services and Components to define each deployable unit. + +## Resource Hierarchy + +```mermaid +flowchart TD + A["Namespace"] --> B["Project"] + B --> C1["Component"] + B --> C2["Component"] + C1 --> D1["Workload"] + C2 --> D2["Workload"] +``` + +- **Namespace**: your team or organization boundary, managed by platform engineers +- **Project**: a logical grouping of related components that share a deployment pipeline +- **Component**: a single deployable unit (service, web app, scheduled task) that references a platform-defined template +- **Workload**: the runtime specification for a component (container image, endpoints, environment variables, dependencies) + +## ComponentTypes and Traits + +Platform engineers define **ComponentTypes** that serve as templates for how your application gets deployed. When you create a Component, you reference one of these templates by name: + +```yaml +componentType: + kind: ClusterComponentType + name: deployment/service +``` + +The ComponentType determines the workload type (Deployment, StatefulSet, CronJob, etc.), default parameters, and what Kubernetes resources get created. + +You can list available ComponentTypes using: + +```bash +occ clustercomponenttype list +occ componenttype list +``` + +**Traits** are reusable cross-cutting concerns (like observability alerting or OAuth2 proxy) that you can attach to your Component. Each trait instance has a unique name and its own parameters: + +```yaml +traits: + - kind: ClusterTrait + name: observability-alert-rule + instanceName: high-error-rate + parameters: + condition: "error_count > 100" +``` + +## Deployment Patterns + +There are two ways to deploy a component, depending on whether you have a pre-built container image or want OpenChoreo to build from source. + +### From a Pre-built Image + +You provide the container image directly in a Workload resource. OpenChoreo deploys it without any build step. + +```text +Component (references ComponentType) + + +Workload (container image, endpoints, env vars) + ↓ +OpenChoreo deploys to environments +``` + +This pattern is used when: + +- You have an existing CI system (GitHub Actions, GitLab CI, Jenkins) +- You are deploying a third-party or pre-built image +- You want full control over the build process + +### From Source Code + +You configure a workflow on the Component that builds your source code into a container image. OpenChoreo runs the build and creates the Workload automatically. + +```text +Component (references ComponentType + Workflow) + ↓ +WorkflowRun (builds container image) + ↓ +Workload (generated from build output + workload.yaml descriptor) + ↓ +OpenChoreo deploys to environments +``` + +This pattern is used when: + +- You want OpenChoreo to handle your CI pipeline +- You are using Docker, Buildpacks, or other supported build methods +- You want auto-build on Git push + +## What's Next + +- [Creating a Project](./creating-a-project.md): set up a project to organize your components +- [Creating a Component](./creating-a-component.md): deploy your first service or web application diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/auto-build.md b/versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/auto-build.md new file mode 100644 index 00000000..4f39c432 --- /dev/null +++ b/versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/auto-build.md @@ -0,0 +1,82 @@ +--- +title: Auto-Build +description: Trigger builds automatically when code is pushed to your Git repository +--- + +# Auto-Build + +Auto-build triggers a CI workflow automatically whenever you push code to your Git repository. Instead of manually creating a WorkflowRun for each build, OpenChoreo creates one for you based on the workflow configuration in your Component. + +:::info Platform Engineer Setup Required +Auto-build requires your platform engineer to set up a webhook secret and configure the webhook endpoint. See the [PE Auto Build guide](../../../platform-engineer-guide/workflows/auto-build.mdx) for infrastructure setup. +::: + +## Enable Auto-Build on Your Component + +Add `autoBuild: true` to your Component spec: + +```yaml +apiVersion: openchoreo.dev/v1alpha1 +kind: Component +metadata: + name: patient-management-service +spec: + owner: + projectName: default + componentType: + kind: ClusterComponentType + name: deployment/service + autoBuild: true + autoDeploy: true + workflow: + kind: ClusterWorkflow + name: ballerina-buildpack-builder + parameters: + repository: + url: "https://github.com/openchoreo/sample-workloads" + revision: + branch: "main" + appPath: "/service-ballerina-patient-management" +``` + +Apply it: + +```bash +occ apply -f component.yaml +``` + +## Key Fields + +| Field | Description | +| ------------------------------------------------ | ----------------------------------------------------------------------------------------------------- | +| `autoBuild: true` | Enables webhook-triggered builds. Pushes to the configured branch create a WorkflowRun automatically. | +| `autoDeploy: true` | Automatically deploys the generated Workload after a successful build. | +| `workflow.parameters.repository.revision.branch` | The branch that triggers builds on push. | +| `workflow.parameters.repository.appPath` | Only pushes that change files within this path trigger builds. | + +## What Triggers a Build + +When a push event is received, OpenChoreo matches it to Components by checking: + +1. The **repository URL** matches the Component's `repository.url` +2. The **branch** matches the Component's `repository.revision.branch` +3. The push includes changes within the Component's `appPath` + +If all conditions match, a WorkflowRun is created automatically with the commit SHA from the push event. + +## Verify Auto-Build + +After pushing a change to your repository: + +```bash +# Check if a WorkflowRun was created +occ workflowrun list + +# View build logs +occ workflowrun logs -f +``` + +## See Also + +- [CI Overview](./overview.md) — How CI workflows work, monitoring builds, error conditions +- [Auto Build Setup](../../../platform-engineer-guide/workflows/auto-build.mdx) — Platform engineer guide for webhook infrastructure setup diff --git a/versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/images/create-secret.png b/versioned_docs/version-v1.2.0-rc.2/developer-guide/workflows/ci/images/create-secret.png new file mode 100644 index 0000000000000000000000000000000000000000..805ceaa43d22d9a30c5d3d44626a6915ce4318bc GIT binary patch literal 499580 zcmcG$2UJsg_bnQXf(laPD7`D#=n8}q5EMjdf&x-Q?=|$^6p;?nn+Sr4G-=XNdhZR0Tq>eY$RDbNBQT zUeOLi*QrQX*?!eJy3@6Ky#f^ig;WJ{M$LbWhs;WE8F2qN`IenV0v_$~cDZz1r!@Xx>H`@|yy4bz<%%guCsA_qllS|i?xT;|$-cH4bT{MamfepL zMV;PjIxVTxXxN|~%+E4v5!ZbWt1B02jU&sbUm~rEN|$;h{`CsvUEpiqlc(+NihlLk zN6%@z6A8`m*LlU*T9A6q5^h0Z45au4O!ZU|<8L>ZQa%#dg!K{(lH`!0wtq7Y`hJ(Z zK@vo_1!GMh;}u>nCZdYrtrwkZ!`D+-@>HI!N4_|Wb%=h;pB@UkZrE^BT;{dF2eCv_ zyaN1svaQ77e5RECpZ6^5^Fw9G(%Ku&>Bc>|x{}x1)XJ;>kXz!UVtcl(Ot0x%edSvw zx7(t3IUc>dyAtjtN-5fYVVPW7V6#q;*;(`Fadm2B0<(*g4H2V(d<-WLUkxa(N161d*jpd`HfW1Td>%S}J=6p?rFFR_!p zTgoi~QUU`#c=|-Uw_n=U>b>0-qilXX^zPx!mM7;gO_Ax>pT2!dk^q?xBTV>qI=Nl9 zLh%L~x4|UIEzA-KUy&xdfln&&_TeM-amfCUsts$W9p4TrqZ#@-XSYo@VOG1V^rX30 z%P;b01_tVx@LCxq^gWXBn^e~c>s>WUtLFx$Sjx{e)aX5*|~cww+4fX_57op^czH|HEik%8;BYsq4tt_ah#6IucXVA zl9WRGq+V)Q8#`2Q>CAS9b%qD<7H++wJ!l{;aJuR}BM@B}Sx5tET@PK_-MizmQQ^Zi zR%89l16qq&sI9H-(*E9^202?Y9iCAsJmMoRrtY=?}z{%V0#A{pOr-}B7VPlzId(wXAk zo`24`9d`2x)i>&_Mb^oDuf@MjZ#LywrlrHfe0>x{VLVr3JbX-JMgGH0Y` zak$EG6(fBk0^X85EBo%orZhVJYA=2-eed($n~MC4?5~2&#T{4)!j+n@+LDzeRwlUl zxdj~g9eID0E=YG$WWFulO51w5mG~*)+__H)Ti#zD7iYLA&3qo|Z|J{lDzf|6F7a+u ze0+RdyuVseysAaG+UrqxF|X~g0?Bys8mGO z%6-eBO{ul1ozvAUeca;(%>^x+4NifZw`Wv>MMX{uBeRuq`-n5g$cxzg7 z@(S&TpRX9K@Y0_r@eF-UpF_VcJR!U-vhR8_JvS9KQ#`i3cw<@mqqGpK$PVB3ysOpr z+LjJF9c{k5xb*W1?$*H)qT4BE%oyu0%U#DAl zr;grl>~-5ub=I^<3AVgAH@2TV;#YhAy2R`pZpOSW`AVuI=nxi9Nq0^2+OdG4czb1R zUqR6DVnKzTb`!cO=WPX@6?xuUt7jkB(pmdiE2W8^Xe13s-G`fa^rH2B^=)jCmv7%> zGBA32`o8w|a)w=Ic}8@)R)VRB{a$}zA5q4foRDx5JW2FejGBp^+rn{F%Hu{&{A5b! zv&qF?>I4Wo>9-+_9|@eTbAus~0C+dFESJ3|rmH`yB`8v@@kc#t2-Y6eIK ze4RZk)taY>}abjoGqf(zr+uyWfQ81YVbPC&ZrHJ_7;2QBvh}NmpwyA@AmJ$+ts`_eXWD$9^cpT zjRKzREz>B^mvxeL!O!@sM{bPUnG{aWAF@lTx0X~+-udOCr@m3rXqvsWwOYS=VT@t@ z$*|2K!5&Gm&=rU1)uWn83lbqhY$>`Lwzq1eY?vB-7&JULiv zert?i@AjSuhI_Ti3{$sPN{3+heA=>=8B9P$@QfzB5f;1_o@w=Jd&+gJ}ISS@BFTY8{gckedOXc zN!867v*YT`*ByZ=CmWcwD$cAt%%_#P+LSh}#hTIGm^N*Td1q9)0H@rrx|Q!bx{TqqHhr^UigL>efrm+J)^J$kJV3N*OA<>p*?+&OlY#P*xUl z2Y7!GavqNwashaU2mHeEX#RdLiFXSE#cn?bf%uz1&Yyip4tNEB!hm0JAKWW6;thlV z`0Fb0>-6E=pKo6o{Q&**o4RaQn&&&q;BN8d`}=M8Ri1ruFZndlPI>BXBrMum9n>z}GWinqPj!oa^);5(CK4GB5E7uAyx zx~ckQZ)|sKt3jVK^H?(3lUA~;zzgpU=NC;Lw4V53A_M86{t`?OT9JsC!F&IG>q79Q zf*+ZpGUgB-uLaYrpkJ|M35WRlT5=*y*SKmSOXV-L;O}LGW+Xdz6UUz!>_kiCwr)2l z*q$7bjv1Xlu=Bgrv{4m^e?DMwlr61PE$0BIusvsGOHZ9oT|cY0G6)KeCm8E4 zL2orxy4{L%piT0o>x8sjG>mRtu;5)Q*jvBRM_+cW<4reRAIThWe`7&|ZcprLUPyNJ zfHwA~>wsVgB%@o4=`j$JtDksW#@-n!9UP7G)+0E^+)^BEmhznY-*4gGCVbWAELK?ToxUX*<=RjAQfG5d6bUUhthFn)-()bZ@ zjjHi@tdJ`>b7WbG&tc8vLWwv?Q@)eNZ&;t<=Cae$kJPvT4PwlF0IY(UWc8(04C}S3 zsE;=E3k6g?4w{w$zy4B$WyvQW<^N3gIS%h<8v`y%$7>`|?d+mNeJ(YnQXyVtPAF1U zakKF4T1Yv=c4?D<<>;eA_^*Fs;beH4LDytZo$?m*40-NOVF(dk(HZkb5t#)QlGu>Lc)u4k_C;BN54i~u$thcutc$oxV-C|{Q-VOeF~qi%`d9^*c>Z$Y=>Ef)B*1=EBJOcHr}%ea7*KxT?56y^yqOe=F+_ z739nW4w!!fDnk+vm|LB#7lpH<9e%W-*CjCtTqKP1>ulLB<(y{<3e0&rFBr0&%=ufh zh2TjU?TE#WPQ|(|^X_lDtTd#-yy0knse7o4{wZn#-?z8hobq=+jsacp)F*q^*ab~m1;hfOJu zf$Fb~!aH%x0)o)a+7Lr8140%c)=H>s-cKD@D8d%vWK&&B+&FG1SxNi#K31^;BN$wh z5u*Huhz6f6{U3SDMqBie*Pe&m4>zmgh&zhXJx(NLnIe-;c-BPHLP2os|^8vwK{qyVqg@bjQaeQ5R4)}7AM8U7Fl zSkHKBm&ADgteqyXc7P4qcxv~=h0c6ZJO9lkDIVxIs6 zZ-09Q-E^@G2kZjSKBD}evM4^qGi>$Hq44(x7ELESdn>Reuw}e%o}yG&u38kh&arFF z8+F5==P~iM|3Y}eeQ1E26vK}_x&P{Ht=b-gaE(WafONmW1^pIYKbumQ4FQ1Q4e!q~ zg*NAwaBWK!aHFuP8-XDQTpm7|yHU5l?foWDq5Nc-VHRo*! z*vx|W_mdxz&+ib36yCP$%9zxks`4g#x;-)pT;_j)wbr6X*g8Z1nDYIE`Bl+a;Tb2D zp@jEFJZQ3Km(rB8;4TQQl%om%HRC2#Yiw2lGU-M9dBM?~v2&edJ88I;JeegS4}tR5 zZe))9iP{vLM{f~vNf?^|`J}eeean&e=Otd@v-}cc;MMa?2KVEoX2^rKf5}yQaj9GxyPVIMC*RLcrth`%)9~6pGpr6Mrb_dO+}gU6 z^1Mvuaef}HNYg|kq>#JXF29T>xeI}ZTI-Ud_%*VH83#mVSxA|yKTQ&D+Q2qs{{%<+k4>BTgL}~%`@CVcu3=&>OS?{Q7=MMN^^%F2iO}B1R(sC&Z>I80bgXq(X_fu(afewM7L@z(Fk@DU7-7 zkuBxZj3fZHBi9MQ`vpBt`^@7aNsMEyHMjx5=1@O#={`0rig&j`ptNVOJ4c=_pYiJ! zsn23Nc;(SG7k|{^m9b6O>X+W;jEr&3oic6Ht+9XZyZm8&frzLP(nfye44_lQfKy&3 z-G0uFO}}+sJKgCZ!xDfV6Xi309Jv`;<4{EG$1jmncP|l<&ve@=ybiU{Uup~2Ci#NY z2Qofs?<4(?`dJ>AI+lBK6Qpe#${Cv2<>?{HuR-VoQOCy;LSC6NbTd-)wV~2NGP^$$ zKt!yhi=Dj_@Aki7;!w4M0yIc@ECNXL(mi3%kH9O)V5r_M#v$ z6kjMMqaUSunmfjQG0W=Q-)zR=-)sg5DFd(7*C3Gsz5jsB_~THPJkFsSJ0U^>$Fv|D zvG))q+YghF8-rh}O%~6M0K#_I!*#-qZQEZ~I~vdR$~4 zSZyrdPNSOAyBxF<(ZWCuPzm%BkOM(ZRFQxw5i$qsI%K}%P>B^CY5;%uXDQ0)Z zWY7=*WKrM2O`0*SAfGfEZ$_s?x&qE-+x*fwm-1=Is&x}4nX?-?{+ftOgd4>6B%oB- zO8`!AS8QQg5jgnrqr=3$p}%4FpD+$1G+gjd41v8^jcCypT`M>s5v(pmz3K5OPd=YjCMonFLaF?K zu)H+`_-AK)xRtonW=B@A+hN~c$8FJv&9=Ur%X zZ!F_3`@^`|D@C67fLV?&5zpa(gLi8Sn<(co0pDxxvwh%!xdyc`m?_`j{q1(@4Nw^H z*7^MLlC2hOaOvF5(+6ZW&jGJivH~liwiE^@IY=3sL|ajXhrVYj8_tUs#zFyKpoXp) zmp=Ga8I47_&f0iNYbhZX520j&?!17VuRopp zV8r)&ZDgw~Os484*;5M5sy`<)4_Px%R1QYDn|A}W;Th^KWPKf{PqnFcSm>ztEPCvh&Boc?tsiq(tKEi;Vz;8Oz%W>kh>Zx zW>}^ZM1y{&j-Np_d zU*Bs{J`#ta>5eUg$j%JCOK-n9TR?tYG9jo^M^a5~L8{bpnt3-w^rf zzOUQq2N00&Tb=%xrE01u|CKHr*%GSyl(P8gZVz<(aCRMTCnePAgQH5FwUS(1Lp_!g z*R4Z7FyLZwi_P=jV4P)jh^wuljl8JDp8;?JonMWtNg$Jpoj%JU-|s<#v_==P>C{M4 zGYhPV^6x`sE>hpY)*nif=OsXw3j-j200i!!;eLL1(2sI`@FW$6im~wd>MAe^el^r)%nf5qe8e?gC&tYv-?^C9GCpG z@F}Nm{o%@tfKy9)lgbzYMR`-f(c_Ew#LXg$(JU3R&T*CF$O|g!*D*PP)EWft0QPX< zt0o)gpT+8VfEuZI#fD;KL{b|3bbnSlY#p*-X1C>bI=V zgR(o`c-P8in*)$`Kp9~cdI3Z$U_SFJfHq=Q>CAq{K^}1DzfYq>K@UT~bP3Z2oCl}^ z0X^b=Y8<#MOnf`CY#^4+M{#{eJmS3D7oxWhh0*SJ45zl0<<68-Aso6xaSO;U=z{ga z4hXs58vu-&{B}mzNM!_aWAQSeF<^PZtE)7*Bi>aIjJIa39=C;cb}E=$_2;h`u-^Ym zye4>~yHmk-BIMx{Y-$AgeI2BbrufW7`qHp!RQ^4vIcE`227nsuk5Do4Dmeg@L6qTg zz#smxhyshK+--n^c;08D--~RSC>e{a9HL^-lc57;!kxd%MUajRHbF3iBDb&rDN#L% zOEUAkwdm@$2ps8BId$u?&*SGzIrdGCITJ}d!SKrU2mD%{WZmQCnLXLTa#AueS?Wm# zYK@yM2IOQ|v6evzp^xLps5X^9_3tWC{>LAnS9>{QW_1_=;}w_0exz-C zswr@9hc=-Np55rEp)4BMDA!%7j>qiny9sw~=O+BEe?o9&9vu)@{=Bt3zHz{r#)1SC z^C2vyA4f>b_1YfYT90U%G%T8z&>aBI3K|ZOaiw@`PrSoF+2%CC!`4ZF@GZI4k+Fd; za~>>SmFm%V_4wSzW_w)ObJJ=SeNLAy0rk1rnxi6(t`d|RLD|auJoFiCl--O8b@@+{ z7i|H+q81eYN^sE)cSroRal((q(_}|sYk^_mC%kjO5$ZBN-?H;GVCqYN6J#gK!W19@uE$u1xfRy{eYMV`YJ z5)->Dfd7Np8dUE+{E}w|0ajg+vpHW3tN=Db>b-#Bb`o*wO>7&a@A^%k zv2(c61_ci;6pW|Lyh06DxXujvk`D3LN?wG7GYeejpx*R$X=7r^Safg4s;mHVyXc)E z_aVOBA8r;)50>^|Orx>(12P(bC0=J(QfBoNbjnNOloaYw?e?@N5$*bl2$C>c*tfBT zI)}*#P$Rz*Y`{|<4wO+Gg=8+Nt;(nLK_ne3FO(fvWfQkjk zB%Xg{lB<=TDrZ7qbk}SBCH4S?@`WW?fXZSEp9DOAiTp(KQ%hYU=)$&MM(o)Y2Gn=$une=c&znt{rop`*nYW( z`E#VbuIOSUm@gzIPaN=2+Bte9>j$ER1YjS{!kR`R_g?Mt_M) zm;P0Ag4d37b=jk6F(IjYH$A~RTjlhl@cz|ysf!@^#8V-hKhXTa@7!uYlfnNi=taEWP9`%YpKfnAEN*_DK=D82Z zdfeA>k}XjA`*@cD-dy5^Jbw(PEneM7QS^U-vHO`mnlB7(CshOC1k|7=f~ccr>)O1W!S>em2&0~lqelp{88 zBVOSTYZ+&rE#Mwl!-4vTgCH~!l8^j{_RdC+d5AZrux9K{+$B{E6%VA{jR#Wtk_D&S zS6*b1e8hhgk;XPi7yr%~*L24`$0A}a)zs0ihDTT}GNc0Se|cD&(jBMZC$4jX<=Ge& zfYT0I!b7;q5s?o1^hB(EB8rOSpZw!Cer$>U83)!Twgr_`0JrYrt9q9oCSh%{Cdmx9 zhCB*+!VyjjHmm?mWES%fWDj(;kUk0}oX5>Vsaa`N+5tJF4LDj3`66<1rv+OP?C{h= zq|QVQ^mzc}`7iq+1bM+G(;TNbm$lfMbScn$Y;BvJOTPPk^xLqNX8)An0?v<5@eYjZ)eW0qX1D_K6T`; za-%k#W`J_3U(IoO5}4cg$H|QVJ;Wy}uH6TIyTn#RzTaf4?xuaQWQYr3|FL%t#3(eI z23sU7bJ<_z1svCvoFltFk_tpCh}mWUW~#P^!Tw8|JDfq(X|1*pV;Gk{DS>bB#zJ)@YWK`iGpJQU>~ynI1RR$ zpCYqXGLbx6)pJA0*8Ytk~s1Sk@5@%Pwm<)7LLjb9S^pC&+hX5c8%LO(m41L1YH2k(FsJ|in z?wbfbq4CXX6d}-zWP-}b%YY6CCWK>F&dgtzNnO5!txdsjIBIA2Z%9}-&b&&wiRPX+uVAJeraHNk8sT@>;eO=!3b-;4WMheV(Nyg?@h{xx z0wr#kM4eUK8Nf}xv1lVf$qZFEn?0LI+0^GGU$C&SRS64@2u7&73*0F5~ zO?db+DwxCBtc!wld01HL@f+j_-Re{Ww!sM`N)}4h)*j1OgfNxAle{!ND=mQ`&=fDk zMfcVslsu%4x0YzZ6ownin12`_M{FzqCfa1krh*D#hs3rBftoW|@(j`ejTN*zFt@|{ z0t<>C`~m62^b9UC0m&2Bp!}@{klI4S*@W)rZb(y>l#?QNWUG9X&v?XR1I)(<)jao2 z7$d<93Dk8fkF#;!1y={c1{g$IGbzPwAheQyVFwm$SB$*qs@sBa)n5$SgL#d!%MNX6BfzTv-SPrkG?O7HyfxHE6yQ^UyVq&}*nNu$>_y9})5}e`=LZYh1j&++7hVQ`27otL zf-w85S!-n|G5cl1m&Xo!E~h`00j9oFnEe0{w6W7|fC$5wulvyX+tko5uXAY2yIWH6&BaxXZ7>AL&d_ zxrX*m$|TeY2BoZ5B<25wVjvkJOipk@&|esIqlFda3A*TO+b>Kdn@*lo4FoTjW@qoM zAE9Xm`Z2%NXS=dtP)#E9t(|G7U$2K{oPp?_ndjcvt>pD|i3L#EVG3m6seCPl6%*>u^z)?MSQF~P`gL=PP(y;s?*#92x{{K2Ov*1cU ztz`xO=83?@Gg-pcaRwkv%?N;e%;p!xdK9ibB}M=h8t7T053JnTfck$SZz3sD^BDWc zeSFY1u(Z6MD^vEa=#Fa!@Z}X3{DWaAGpO$b4d*vEUG5TfciZ|{CB3&xb*~4%0)|<1cc5p1t3!%Ovg&~25rybAV6rq4? z3DtesAD!vcrmW#b&vUPeBsW3&7(hS=*h6xm#`!3)GYkdfsSjCa1swDE1?z z3dr|Ein+2F1YEosNDG7QkT&Q9P?7+xvT&c^k1vQ_z)1($nlc_2D9ii#!Wyjb_^(`9 z)qBi1Df$^ql^tWAT+WS~xw-teyY#nMqOVHkD7x>|R>IL0BojR$1yfiO3y>f(qIUZt z_0kK8EDKWJ_o6cCMb$*CCv`d7WK&5?sOw-4sS*J?4K6NT&999-&Ql8~9@=tn~f+zYo!t zF}Tm>!;mir=I)k$>8h^OkLK3nvd!k%t*sQ(J99CwH+uy?`;X{$%#~Q2bkm z9Y-~Wtb*(>S}to&H#2!Yb=8zBYF54~RgM9Aa19+{aDQ5y`ejnA;d9S2D{t9$| z1U83BE)T!C>UktzS>N=Q54QARF0iG4Oe;~zpx~y)^8s zgcqPRR|N*(xJE?{0+qbs2KF)$zu@Sx&=pRNtp9Uo#XD)(@ z#xW|1UK#$;yUXyM;{h~&VN5x_HM{12~mu!qin=|rO|LZ7Y&ZmV~ zH|wrm+7wsU|l5RblY(H?iv_*(SF4sA||$zP_YKSTzuAvwl6 z7Jp!f<0KsIh{_Q??!QilX?-1@j9wyJ{g{?yDDSNJNceH1lA7!dQRNT0aO|js(i@CL zXP4k;0nZg*DdP1UE!W%0>h+RAPd?YYF{#O+V}=iBpPLrM_IxDD+`1dP;mG+iQ?nan zv~Nnp+i&$)zCS!0?8A;(jh&+}asFaFHIz5jJSKkJ;$<-9z{L0b*Ud;Pr`e6_rI*ML zEGHwBxVV{82#e>ivw_<;1(QJ8OL?v1ff+#T(!^t@C|T8Gx!`+ zY$p*gZ=;9&ZHVx)4oTG0g5Q;L$ZE35mY1}u5-^i(H*uNdz_yd)@uJ!Or49e1ZDcEW zR(xCI;{)B$a7mERhYVt;X=7hsCE~j=$8LK-n{=q$iCTVW9|BTe@dLLSzyqB88nS3~ z;hw1rd%=#N$z!-pw< zsXK4IKAH7@wqzPi3l3qGUR$N@+?M|?$BL92BFXve#ykX8xos@z-z0$Q2vkZ2yXjU- z8WV$RicC8fvNyaW|t551>-_vybR z7d1Ny&fdbb7AFjht*h(%&iYOxuA@R3gHCH$0@9LPmO~t7s*H%FTvUN51>5}P*ba=? z_8m$Mu8%iu*oi$G?+=XtowJl;F^(&M`Kp=66Q__7~fNmUeXIuonY^1s8&xR3<4(*Lh? zvZj}rWp@O3Tfi1vXYDW*f*>;m%v8h2mpBvmlb)1l(eH<5z%iFp*04GyAlASCAH`87 zzFCYcP8@{;;%LRa7F?@#w~In^-(`DiM05dQk6>A~bqvmT&$xhs|L(?`ialB8qfKBhVheM}V*Ix*{>y;D~AZC%jjgUjX!Ir)6`!II+c4Cl~C zcv-)coOR9X$KRJc(XDg{ck>_AcAmFelG7+xGdyPW<|PykzQ{jgBYhw~%Y6 z5plJU%}2;IAlcTRS(eW3xHNAokR&Ym&z}m56btFYO1jsC(Oy?bVV&(pk=IM!P}iwf z;dgeAiR{fP*xfzqDqNumE+Z;iqPwU#+l7t{EwO=Zf}@QTp|8VSTfMCtjs)m z`|5Z{LTToFlh4GZCCm!_IJesTgxN^qdS-bwb8oorqoNbc+iL5$w+_@oPt_u$rA;5x z?i91QT2W56-Oxmp%)6uF=~dDfj2%sFZ$ExlE@Gem@$I#HxdwuPvisp%w0ify7%-30 z^`Gdw$FHe)Bw0G=t&-~`5hK5;bFM)OG}ywv8*;EJaY`js6WVLXu?1WuTl$veqL5Qn@xe8 zBE104rcPY8G?>f~t1K*f-_|@vNY0r>J|=SgJ0Z}t*L4BB5zs$)HvrvF*{i$m6QDN& z6O_*NJH3ywTIC~r)u{z=IuP*L`FkHyG19Pkg<=2}k10hxtqyxYDa3gmI&uUZW#nfs zSw_uOJU4vL<%QGZntG^)fyUWagduPQzO?GL%aa!dhaOr!E@b*V!b8>ypl~Tw4mwP= z7^@{?OT4qLPU0#-WlC*zH}C)uP7<+znvI2iY{5QVqwfk2bJn!QY>THcv@!J-B#;Os z{7RkZA=dhb;0-8$(Zyj%d~h)1rqkpELAKGUwcA#~q7Xx3Hiveci(Z*WZ)xDoT`i!w z1RFOI5_Mg3*`3a$V4EXR?XU1>5d6z!A2UKTFl3bbVn0cc@+3*vxi-;h??p-*Y+VJ} zIU|g4`h9S+6iz_=(lJR`b+tIg+rCVF&nJU<#r?EzVCksgh9dFq&zHYF{fSp{>ZIx2YtUxf|_zppOL~ zlMCB?UCh&}R#{qEPgg!=$g8yOJ*3}-AJsDxjuIAzQl|V?AL_rff4Do=*tWa0Y?aqO z{9aqfLo|;nkALgV07HrpnbzRTA5tk=Uvjldxx+s-F+5mX6C}0uJVdMQRvowXwagIQ z^BD*ab01l&W7F1ZeW^Als@nf3nBbFDmqw09q0J5AwUSNb?_qE2X3yng>wCLS^vw)?Ce# z>AC1MpLNnxurCciuCNjvesI48^~p6S_q&kcwmec^+TzGG(cQ{HyKU{|uR)uNNs?Eh zM&E06$4?zZiXw*=BjrWi>doKj?MN+4SYNcvN-x-Z9}N4HtzxCZp#G?u<>_`OKRcG%tT^OfjIIr(`Z{pZ3%^JbE~pW-9d7)f zIrGJ4KQbp~a?Cw8hB!X5(^pyLxZnLnxgm0&zk*t>M>e}AxyDGEp+#*ULv?FHQQ%=oB^mdzvg_~bs_eW~g48JE{p#S1LK-)Wz zJ3{@nZ0C4?sQuNVgVdvnMGq5A&mE@1rpB=zmH-jMK=Z>Ndu46xd(G7Tz|r|ZEX7+1Lj9XCt*e+O7Dh5OqYMK46awK3sQdXBVhb3i<~x67n%(V7aTqA zVE))(B3GklhCbaNrWLSo&5WRZ@Y|eU&T>3ipxt=#q;)S-xMk134$jbZ?#|Gt?TOEx zvwE6ENYN*Pwc_2m9|ECv^9P3&**d3XlUv37rd2hTL#OCcvjN8vqenttmhypa^vi{2 z*nB0-8StUt!3UpK1*gq7Kd;_(NU}Z+rVpNAu;{|Hc<3_YnoAv2=3?r|UFpG5=o-w;~PVV&GRib8kYYFSQ_hz4D z6mP)P$)!_Gjwt3aTB5HYhT_Nmv z-^~`kn+o0gX>t3?63NI>4@_cFg=XEz!?;r`o@7xN-8kRk-k7)dBQl4N% zGxvW7=2>v|@t%Y7^C{64VQfZouXSA$rZ6TcGeNfRSycG{*bH-dKwp5~Y>(vbEp3cW zwz;oT%;yBZkTyQ-XhBR$&V6af#lmhiavs2Xp1Y%X<&B^i)vlH#5^!uuVD+(4ED`G z9dmx7H?@x}?BYoMc3NtHAB%quMX;ecPJ6(ZE`?yk=OL04F5au={Vf;lfB^c zlZSb>iYbX!s=Pc`08Y70h1mNDJh=)!(^7qW3G5<0$5$Q90-l7#Zd(2G?18x;bY(-+ z0oRrpb`xum#-B~LYqRk*0UCIFP1gp|FQ@ATm#YtDe;)#+0pjdQQm934hf(T-L& ze=T2HH(9nyfDd8X4>8E&h&w}ff1GX}W|_)cG}TNBsf(mi=&sUdXYwDV|X zwez#Rac}y;I%>P+`$4<2YhG~uHoe?f>vw$&*qn4L1XxjvDMY26E&DYZ!)<6o-4Vc60Z)7JS1489b_~4sI(Tc2Go_~sz^0x!g&zvm!EkEPRQHuF}K5-kVSEF>)Z0*6!%VPX+NnxY|j#@=p>n!$z-^`pvEv4 z^c6;zG#&47pJCFC-Lc55)gBR{G)W#xccoL_`7~@!ALAV?+o28jdhZ)T8#aFvrqn_f z<{Njo9kjaazFgGlWh?$}_cI-aoj`iRmk+)exF)NSxtL?^asDmA$~|rttlr0eUq)6tA_H{o2HQUY%Pym*t06L zx*_em{?ioMt&t zL=P3`>N*(4Su;R=m2S1S*^#>L)Tv^`Y@w#J)^>J%^llGkF!{o)bsQV)wP@q;gUo)0 zl~K-->QQ@E(e1u#$i@TwL)V>2~M&RTW_&oL5li0#bm(J_hIe;Th z=ES#%{t9#bW)IA(_4`|>oz?Jqn(er!1^)S52-*)Aq4--j0=l0Ie+I8R|LeJsOW<=M zI=JUT+&(m+X20f7Y}tm#B{=w-`y)kD_5m^I|1_@lMCt$SxZ0rO+@h;FdD$SK&1&Bs zMany!}2q zO#x#0${HcOg-2(R`&OWN5sK(6gViJ<`W-4LGq@Ek9Y(9MFPGT(F}-=^I9}rjoGh0u zHf96=sJYf|-Fs|s8RbNd<5~L?%aB`pwZR#lD!iNTy?}}*+TS8;xll~6-VApC3s(PdRY_#^tc}Jc z$uo=ckx4neeNIpava2~%6pT}hlf&iG4;fMM zdgj)7^zt}#36KMamtHJg^WLWLB{4LwXP1g{YIGY}#qngV*uynEwBoH1W(&9Zt)N)# zeslInid;v&Mz6k}U!SFgH@k+bhf!r z6GociL+#_U;J=~2lO>D5kp;Ukkzpa{IemS zQXGe3wLpYHG}2p5p9Qyk1@C!L+7L-#F>!xe^{7yBBSkT?CL!-w8y!6)V>m4om-72s z_n>FhamnfCkX#G zgQW7p?l3OPhlLxId}RyXHuKqcje6}bwP*jhK+m%7Pw7tNY zGPOxte4CpaI-2Oc3AS!_aTe6VSD|+?N>5$Z9;gj99n^xoCkJWvOz4g6%Sc(Bh@9=| ztWI+W*W>rfAbxx(DZe1zg6?tJlDOT2@!(@MxJjCMeqV%`XKBJP9WsaXVWo!P=|BkB z)6aiX4`lMf4#C-J~t25e@R%CRV>6+4<>N(a=ANHbq3$1&{Q>^$% zUr(FOhxGm2(9Dw-^lMGF4hvT4^$}&Vol<0QNfT>T6+sf{E4{IRrj2}L1*HnE#q{&t z*1%kh9bF-CaC|Nu!Ro_bZry&6<~}J`+AF*V^X?nF5&eES11yU*9LFR(6}QXO&h`{@ zPPU_ydCfiCN~s!`k~t!mdj_o1MY%%g-U-+a;@a+z@C zylu+DJozrrC1cQB-bxLi+^O2iD<30xU&B}Tz_?YVM$%AM7uehz19Ez|qU3qbusJU1 zw+j8%P>utX93H_ifU=nP${U&G9ut{woOOF{UA0H$*>T+Ak9z;NqcwEiR9Wl9+F<(( za*-XV1y_A9_{v)xQq2i`4yPx%ww-G-?b`AFxERO?Xo*h5N_=OEX{Cq%DwzH)qy7(L zkk(WgNvqp_^N-|V6yQm~XHGO2H-er|x1ZU|*-)=^1QW+%#EYy2cjJ}*P*s3p@*m~n ziG&#)vUB6p(QUea1c3+ix^n9Z=j6t1C;xaZP6W7q|1H2>`)ic@|KLfKGjb4jaXrUC z@}Ia$+iFYaCZY=F$a3V( z+^hlxPkDLORr@_3u`DgN#C5-@v58KDX?&esAx&qLV- zlgHhwMRkiz$L63+JNEj)`IhKJpnZ_F-@q+lHiveS!`dwvvxE?UhJ(Sh9n-$VBHyAZ z2}bDl%gqoOk7^Sx`=M)Ckg#q0tEKPibv?S7U`K|KSBF5&<1QId8ZnfGIR1cOGdX}s zzft5Dj&>9b1p5RY?&p5vcvJgch^;I&n;-N;l&8;UpQf?vO1G=&!Rii^Cx6UC-)?I{ z@NAYvS03>0{W6GJibdIj&uvy#wmW(x#2P!LFXUwxxHn3dB&W~Stk}Z}c=RARac(RM z(IVCHX(Kv@eYes-l_^3Wv^)Rsee%y^yVz$&OAu|o8MN%Ro}DjU?TNaM)AM4X--|TKH5z5^_c%sz=x^%?&2S6e zwJ}`b`nXr_)Xv(_Qk~o|wYEr4^l~4Zat?0VC(U5ZTX9&0>o;v2CysP@y%!DfPdJHe z%ddfV)^+vty)B{WL4ZktSt!U%UQ1j!wN>VmpWUTkZ&<ZI(XtHovMy4NXITX}X1v27&DfbiYl}VVPNAb2Td?+zAJoHspT>SjFZL%{ zfGUBr2M92^DyB7tc`+J<-I#PJN62j(m1*IA|)!rx6i1YQYO7j3B^b z3W8IN4ickc(T>U1di{@!ue-(`=6sMLC9xvCNh_;L-=$0C#!{&5(4$(bD+} zCH&lPw`3I})xeh+!8-EwdXYt}oJxs|oSRJV?~=6T(>g0s6KtUSk%aAd4YME3J$@Tn zX>cc#I)fHWc*AvhYZooUy4iporoMZ@Rb&I?8PPOSchEH|um(6vxKshgXF@6d(KNvr zJZEH64-4|^<4KA;BDqi5eJOw1p0v#z_%_^A#GT`+U2yLu(br*x-ngX)UfmI7!#0DD z(b&xeK6X5fDJxWp7<(LN_t4gT*!|wvx#1%jyn|RWyC=}DsS%N1uSE!?9u6fIuIZ|8 z8?6fcC}}t70_}Di{*F#gTM4zWai0!X?zoJk5B3uU9E9^r_0{h9XRa|Y%(AspU)+2y zIwEoVxb+P3`Ie|5PTGBO;p&FjU7CX%i)!i^n(-Q%fG387+o2bIhSYQ#7p9LF7ZT*z zoaboZr8PQx1ytd?!9R=jBZBOtUT4%1^?VyQ9PROFj>-FmOh&g#Yk-xLYIIY1rEY zS%sby82-H02gevUY{dncgmebC`i=B|a`NxbTD;-FT=;t!gy+^SOr*uJZANNzno8^B zKD4{9_VjG%NQi!&MgGu@h$lt)JKz|g6aTA(`>f$m<9Kz}7Sctvr-bL}kd4*6RTQPD zgThJo%MbSKB34*UdY~Y;NnLq8v&l@$IlsL&f2LJMQBkn{uLuEiiCpduGz`GBKriK4 z-apay%tZmj^_wa;0{>B&k4K^R{AK~w3ea46O4eyYTt>Tr{wQV^!1?|E8-8Ovcl2Ub z_S*K~AD04s+W(tnfFye@s3r19+E%#kPcJgyS#LVU_W;KfL@U6U;Y<&ak@~c$atpY_ znybXo7r)6`f$4uux~czv>KWE)I^V&E$M>mI^|p4ob;o@6*65rn$nO&*EkL7CV51P< zasYIluh9IfxWwKHP@EdPEkA*u2YGe+zc({P2pz>Vb#kH~cTVsqZNAYMDGx`O(CJgb zJg)J!D)Z6WwQ8-?AY-M4mtT%L#DyERPC~IpIS$mzl$?kCXn{ zPX+tP;jlJVW+2}s{lIQB|1naJ&hI+7(a6e?a$-T6r%mgj-7=*!%Uz>Bqjsxf zEb^s@FPITjPLR=DGsneq3?vYU_kvKRQt z2Cf&6T|kP)bP5ka=UQC!>4(hZx8gP0t2}l0G1*qH9XrnGMjBw$G1(8$`B=`yH$VGQ zM0K-0tjNIf!+%*z!pG&k;;M*6EDxi|>euDqlx#~c)q>}65j#Gg^$W}dWw3_u5Ve0H zW88#pHgm**GxZ+kdc*cmOI%i3K?}cAcsN2(?CDiQyM=4l^2ZdqP>d}6P^~XK(Rbd; zC(G#bRlJ$xCR`eAnm|WUc8w8C1xKBajEJ(k zl3NEv!zN$}GJMaZCn_JLR|_8*oB!5Win<@EZwTXy=UUYd|HR5Mi1*8l6Uefvrzps= zJlgsY9ct)>aVFf;<1fKRX67)Zcc%vOJRUn%w4-O7yfrW9#?d>h31928T9lr##I$QQuoq%OE;{C0ex0FM6#GhIPfy7Xy|Q?l zOdG->jg6ugc5matX)H>+t10q<8LeuoB}>Y(vSs`2YsXZ;la3pCE#9MdGyE%LrlJ_2 zpFZBPE=q}DTjiqXKvjnO3=JLqY!_uQtwDS;JT_i7Djj`U$e6zLnnQG$FTdy(p_d=B zpvedQY*}4J2xOUwZ8EWstn}XLCirRKpN+BmUewQYNP|yF(h~_`TwD=`dn3nm(P8k2 zP|v5vb8~8wK&W=oZFEKvhMoNh5laaiK9fDo8*f{Ng2vFj9wv#!GG_EIv?A3nVpDAx z1=Y^w2zSTIr`xpF)k!EPX`m0au`59s?*x*})4An<9Os_X;;*CX6^P4xJe-UVX=jlG zt=a>M+OR8^`X^x?x}^q>&1AAy;^Zg{1o9T0FZmT9K9AE~T2Q_nD?hku5sbta_t{j- zr5iO(WA&Ghx5KoRr1HPpFkJ=dSgdGpv`!+jlpr2(05r)%{U^V)*cn^&KjVSw77 z{Y0(L9^q;KlE?V+rKF`VSauF6hJ>Rgn&ODx8v{BS1od2D@n9zaBi=p5y&P-*n0y1qx2exgZBN@M%mglEJzxd^{&3GVUu)z5n$?GkB7*(Lr!U z^7_v0gJl-ig^$s^8eZLaWwg5IP!HKJ;p5=_j^M)V^!4$Mct_=7W|pezh>!7mGTtHy z@K@I15_msemGyPZJ|>g6g#8|kG!&r0^=97#z{Tw z`2xDEWk3*)_`aT+| zI-Cbtp^xB-;Q0A#yi^4%*BCjCBFYEvYbZ6Pg8&(!m9*$iPkluY8@9B^@a|Y%KgWY=o+Y?-DNo9=XSkrRamezo}zyb$nTiP8}7q;8Wc1pjjKbByC zs;XB2C{_Oj1*NXIp3nOiYykLv?w-4U*$D#(afptbgUP2|$>;xS7^l-n_qdd)#v|E{ zPkfrOaOF8xudW5UnDVz95C22NIOhFt5hFPtIL(DVz;fYgvrF~Cb^^83HDbx|J^>Hp z#3zzCKtKH-I!5R3W07pb|0QH}?Vuh2jm-g=xhK$}|CBO{{Xa+BxI|#5AF=+B_di z`PFhvwv4*hi*trnf7zv|wQ3(GX|_%11XI??hXdI9J)2i zA2l(UG*sXhS>w6XafR$q5wxD(xShHcS(RCuE}xMunGbj-VrYeqYgivK=W4`OCM?L^ zNqE2fPTqP3Dbjqm^t_x|85P}&%<(;&HOFLm`Xa7j^14|Yt!yoEt8C<&3FDj!9yPTt{dAtRNy$ouQs)_}s$UQ|A3_QlqDmm}H|;I*CmZ39$4bzXzCgK)~jTRnWA7%WBw{2;H51)<_q0`Jzv!> zR2xRJ+|UZVt34HiGW7>r@kX|te{F-y)fdHLC8AZvWMFnjq99rN#u5MSFZT2A#Xnm} z+3nr8p}S|G$Tj|pX6$&c4}}Ss4J}8%-?KC+JX2FDx*5xp!oV*sk9$ z)c0kW6ME4UKa#$7gcSQ>$7v{3gO%HqAv4jiT$%cW@% zyE;-k?>`kR9 zuG2zJF{Fj9CoVoag`&Yi4MsrNQ#qPwHIS#3Zl0}#&p&=X*yT?_s5sbWjgn(i_83id zjy;eMDo~rZ+S_^ajz+o`vxI9IrZX5Q`=(%C3xYcHOzQJ|wBLH^m*bfosHNp_OGAN{ zEW2;c9j?G3`%j$z$?g(D*Gr>=v!2d=_# zwKl$a;9IZGKSS&qYV+b=b|!KXUwD_H9u(DyEUOsAp5CSU4eY7^xXl7jqzSSBUd+|L z`!5AxL2f^*a@_$C5*nQ+%Vw!T)VjPO=dLd@o;ZO18&|WVm;6WJ|3B;lK;ww-Ee(1D zgxeE^;6uX2^3dUd6LEx#lFE;q)1UWuyF8B_aSqDgyHCt&cU%DM-M2r9t6`0KbM{X$xAP}hAGs&v^DUH>8jednaYU9LJab`@d7c6J#BbF6fOBv|&KJXfB3@KTo`CtOWIb z2vCLGaGDU_B= z)HX1@;O(9`1b>NMTzoc^6kbzQDZOb|l!p4^M^D_`PpxU4TiwYbV?AZ$aWc+LXw^L4 zE)A$+x@BAJ?oIS7N66O|pbgm9ZEx8090*`5f>21{N@CF6RMb4CsktSsAmo?-+7vvp z{T@Vjd1q1XAc{ELf@AZQnwBDgqYr;bAT;$ki`n||fOBV!qu9}09yzEVX`t+fsUPT9 z*M?yrTVFyppi*%B)w4d#^(KeXjdIsqDq0`JfW~XqRlJ%Dkh7Tb=6_vTtiH(M%)Ki7l?ToDhWTt~O`sQEOqar!8wSpA5wtmp+o5dA}fgNpV`U@ ze4IWbfUE-JtSmlb*Z1>dOnt`tOO+3G7dqPy56eQA2%^gm$(^5X%zw^vau{NZD@&_F zA&RdL_vQFAc6_nmmwP`JsI)boxRxgM3#y!F86MaF2`8$RIshv&0RmLD#$0y)b$DkG zIrroG>fugpZT&#qi0Aahb8oG;PY6c3;g`hNPjpzkfR#?&>~FDfF|jsF>SFd6jebb8=>z;I)Ub-9p{F?DXN zhFc?kM{TqM@T?55TWyZL16>=d@IysjT&{%RTBH64?gvoLW-3`=rI{y&M9O1ZL*MP& zAGy%MM}P=n(kvM-i0skaet!FXF>42RZGp5=hI-6=mx3zTpz-i?SC{>dp2;gRQ83(( z+ykFkWgDv&W}AK!zRYFur!ROWa<#ahOxFiO)Gf3)9}5I(C+P6tx*|51@t}z2Ys+$@ zHgK_eYskd~|2nk1N-pYt-8&AVZ=r%WhP^Taq=@8EM=>O~8h$#un-CMOO3L z>G3qsY6kn>AhE+2Nc~6rUjnl7@s#;(1Nba&LV5%Pf_d;&FjQ1Mm*I!CIJ&p%5Qr_|7&t+ltM++!Lkc ztbbf0V}lJyVDZKnr#CHm!!`=#47zZ*Q1{AqWeBbM`mhEzXC$z ziaCsd*NH&h;)K5_Z~$(&G==Jc1Dk_iG(B@7bXNmET)mjP z{l1S_wdD2Tq+Aykx8lee`8in7fI!zjGm@*`J6HKDGkEn%KIpaeDiqqVq&o!ct5J94 ztPA@Uj=n0&kC)EKhRxC#A0&6LH}@xr^ZwQaY_7Rt3AnS1@L{x?!{@Wo3(nw zx3_kI&N_A3Gd%Rfk#{cf*+oWdm)5x~H^OI&wtACo3^sV3+7wF)BR9tr+t}%!d4rCa z$4UwmWuXo42_oK@VQT>gXs#$ie-IpO?JpH!odU%`oSZYWy2Nj`0IR5U)(j5{0XSk@ zGV_&rE@yEbq17>VJR`pgz-;eIvpKu#4$eSy3JUD@YpV8x=JCI6WuMS?7oJI%;Ln!{ zy!$TS1muGmMn4*eh41;y`9N^1K{>I#JJ~eyj3Q10fyJoyi6th@45mxFJMwXY0|E(P zi4K+emV&=9&|XR6$!7JxnZzmPKtL1Km{%CIlgc61Pcw+uiwN#?S+QT)0eTL%0ajYU zi6{ALMWK7}?GvPC!s#UMwSUbUf&5D@l?G6^1Clxa=0>vI_dVnOVAZE>Qa5kbkv(5} z2blt7`TDY;_>V!!y3PLz7^EvF)XNh``^g3VhkCj3$o2%C^nXDV|3!2IBS8pq(1zoK ztb|Dvhg>HDPbiQyGV?YuWqxu=9Fum@zVXO_HR3vV@ihWy5i z!e5z=12Urh_!UnylJO$eE8OMbK2XZ6eaawqQBfvqUiP5Q zbOtSb&lKoK7x-^h{HEB*F{oJl*}}m$D+#5R*qRfkC(>Gay&f-VY3*%n(X#A0s?$_& zkcC2a;=}3X%k_WNKV07afoEKjF5EuqSpJ}s?53AB9r%0l<>f9N`^ci)%< z#9FrnFLnL?OaiRR7@=E4dR>!RaiHdV$;lZvJX7ci3kiN zL9Y6#5ZNlWYVC#w_a_rGKTA(cw)l4*1x@8eo@K2?KJdeyhf1=K_alE^ zV+RH*ml9}3uX)V+O@{y?pkP zD4=LKAg<(e=8*<|>D>|Iek&oNqw)E8c$7vqX(38FD?PpS#Dn{R z7gBi7QoX^8qXT?v;8B7%H>ANn+E3#L#4jL2V=R4!cdka$6c+co_`*D$Jf;hXM$ina zBK%kUz#U8!Ub%^~frBW-3e~RTfDXe&cX1}rLz}stL@QzmoxOJpFm6Z$nyy-3P zHwQo1qb8p0Rf^^sPDxCXgEm612+O%Uy!$c>=v^zMTouk7Ix74AT)frSyzBAfew~Z@ z-#A7S+fCpo%|UL*Tt0gSJZA&Z`fz{smL+#{t<-v@{?=XN((Ng}VO6r($_>y9=ImUzg zsxp@Mwtcm!ykxty z+7pYj;KK%wM#M(-n~zuIIjwp|QS7dSLT}%(%+Th+UBb@gL1%(&hF_+uMd1)N&jT7F1SYFE)I6jGdT)>x4*_7GDDZCuSb!&eoBY@yI|3-J=ZS;@Jyu(~no4 zq}kKUYEm4vi{W}#94ZonLOjiPZcAF;sHXA*n`gZHuIk!`FrpG9vTuY1X0K*!(^{C6 z+*;r>95?c66*U`$!g%ufW3lvZK-ZY1(fD zE14%-2Rt|@q}hTn>1ej2{g+1jx(m_@j1PCrZ{XlOlV*M6LLz*;i)@UFWaEvjDe$Kv zH0Q^rji4y3ih_V4snJPa3n;iDfscQfq(d>rkTALpnuAy5t45YUsiNPG!#Vw^fe3CC zR8cHF8F)g+d`aJseH6s6L1mnw1LVH7rQwJ*=e4%HKx_I7$1ciCncUtV5oE+pQhL|s z_UEb&wFR5ST8&1XoeaOV0u9P&)V|Y*!fGQ!|EZ;yPEK6GfIOvmh?Fl4sAjnp!;#oZ z6k-@*Rizn3G~^ql)5tPbm(QIp1;{|FFDmPu66DFJJ{h#&uGl^kc-w!;N-EmXCr0BE zAI1Bn+GsK!ZLyt^Jq7L6)mb-Ksik!-k6@vYH=avYDI_>z^PpR5B&ZPD~jIk;?&cK$E6J?GsF4= z?#k?civ*PA)VT&32z&=i;dlf&2iOCiOr81z(Ek^K>_6F{`$3e9C`b%|;kqyT&!8)z z;O1|o?jn`JgU7%Y5P(4M5S35N${k$*g)4fi*|+Q;*sc7P31Jk%e?R>^*G*Hy(Anda z+aBnY#v3Ki7dQeZvK}^-7BI@+u!uV|29%R71>;+!H+3zyKSpGce>H+Hr5B~Y>Ob_& zC6O8gZbFOlMTPw%&HaR34m=%>ZyOx0j^LWGT$TdpBSytu3 zn5U?V8|gAswJSo#4rlMzv^ZH>1%nu{>()^s0&d$B8!6mr$jqR%MeCWGlAo!&L_N6w zqiW-%e{G`%B{^68tF_UCK;N$eypt#^IvV9ba zWnam^4b5p?oLzq9d#hg;p+8>6hK#k&Q}QIJ2$5FDKa4uq$bd@mF9`>0tUp`G-z)n* zECmfES+|Yp%&6a;(pHDy;^0?73OyCeR`AZcG24cvcIPFRx{gwD=%uDBCE)@vN#vGH z)REoZgVi%g@F8Iz(XdE~=tSR6SO|xW@@NQ0Afv_W@}?`qtZWo2rk{cMrbR_&Zta%o z(FIC+yC}Htse@W;M|Gx>v(wS>gOJCd!HpY?_8WRgC!5%tPRdsv9iNrxP0`$g3P&^D z0^YxHqZ{Hsu1pdg&CWVFSuc%~u2}C5pp#^3d~amn^T~W=G>e+gjHMM8>bHE~xOHoI zy+)z(oFF(l>HI{dU#l~yghCNQGc`UElfSQLA)A$s&I;B>XF1h+&5ih}$6iR>euV1h&Dnl% zgA+>|9bQFG{9zIQG?q>VZ*nrkYXbd1UpJ?$xV-+M zN4ytAbxoU6l@b46K$HFNn;WdP_YS7PzOD=O@TyJRa3|bms&4yry3W)#=6UUM&;BI(3BI68+49ALO>l#C z(_Ls_g5KO@^Ylo}nBrS?t`Q5q<%dLgj$OU`4VmG2Y0bdV?ak zYF^))*@?De4FGVcU9T_4Js1OG)9?1H*VT2hzsT%PR)=TLR0TV73}enY##}(vbxGC_ zaFD|Dbj*MZIMqh|aQNz~sK%krTG*=nCV&old-zdx-O-E*TiWcq=9s^=&JSk5`zMh& zh?thcUoROGg9vJ?JwR!9YoS=mT1fXOt)gOn(xJVb3us?>cF6G6y(NP|Su^yIGc8Y% z5pnG#D#U;UM$Q-?`-}g;^IX>G)1bh2=#W}qtFOE{NFcvwJeorR!yjLV{LKn`g8+Br zvK%l~<5cS3i4gI0l5ZzocAomW4RirchMwAhV5Q5S088dck#2oCs*`rH?g(@7i-J4Q zK838GI`>xTEx@?@$HUp=s091+>Qf=gPqKcdH~Q8?*21zqFDu))+W`aQ27R! zqqP+P-+#dQ?|F6v(mTrbG#G4Y4jUABa`pd(!W!}6Ak9Ypp<$U+k>K{*=k7R5@YUM@ z-(ltA;W44fe4OKJ;t{*11Hcg_;}h?v)*r0V7%rtp97B zg4qnwZD69kYRc8+jBNIm-fJ)DuZf+OjT=<>&IqXtN-;M~VK#JsVtQw^0WQAkml4hx zTaoeldXsOZn>*HNJIrbq$TLCTd`vxtgqNS*u#A6mJ=1!vFVeErxM9!XE>ocTYg4fUXQ^?V`-hT+n{Eggs!{K@ ze@Ef2y_uhCh8kjMVy0HqB#HN^W#h98E6C|bPfe3MvZ1CkitP+>$J%XC!E~Y}+sgY> zM#6_Ftp4VYDJLHR1<@(1ud5bV{eIq7Q$eD4(ld7D^0H|z>MRRe8@8k)Lt0J7vyyb;TaNT3>`X4?lJ_v~2@p=Y!blEnd5|e!R2nznDO(>%1$vJkffU||QWTz9xjAko-PO|!kbTXC8UH)n7t-SW!kQCYz zc`dHkqb|@p!~y-}er7tiG(X+vn=u>Jrz;rVUT>h#ezWR1B@l5o;oBW>(e}3hCD~Ec zHBqW`>fp7yz?znAs&nGzn89e7u4JX_sxi<5#E!z(bM#Z*o#$#RbW&d@cVxKPK_Qc- zrq81)={??ax-V}pL7fb0;&UnEZ{9xoj-V0QxFCr5ib5Pq?i@*OemUyg?+!WnSp!m; zB7zJtXy0LAAK?vqtuy(kfJ5QBO>TGpTgp0!+JQ|)Hs?;9@c{pu&x6-+EFZBe6ptCl)q9<|JAN>}NvUpNOmGA;NlQ1(6b$JI@`OaRz11l+KX%PMg{n z4V5;3K9US=3XApg+BmX%;tl%LygnVpGWqQm9um^IsYY4(B^8eGp>R4{lpTC`Kw0OI zgmk!?!GRxRQH8}FzfR|l;RJ2w($-PQbsT-7=#V=@#BDk|W^$so?^2j721V}ol-N_n zuwzZuP47)s>yiqJle21Hp9XKLKo0gMndf!ZJfZHuGlxZ$&Bl0Xk{)KJQ;Iq4kzf+k zwtbM&k+U)ddQ7YPQjta}?E82@a`!|>CxNU?MlLw3?e0I$dw5v0 z5Y@se3X!FaqndeluN_7(x)Wgg`zwsVB>4W6V!`@@pdd@MKUJbrR0h5-PGH9S|Bvj%e8Y z*J%hIRUFT8A_M9Rz1_RCMQ8!6S%Ppn0Qt& z|66SYw1*BU$a~tIp;4eApsUoXa=5p9xN_!Wo4)nOq)%sRqnP&+S*S6LX#sm5jhc^t ztngQcYPbu6{7!4FP1PR1pt*Rz{&7g)uOQE6_H?xI^K;YsqBS#7M4h_vL_{>$#V0$C z^$8u9{OWTXzE2C=iBW`go{hy;e!sMwklfJpscE}N*Np~%FyTX_X-3W|J_-dzCKEu_ zr@213BBdI#nmP#jH+8g$i2;6vR*1*)(^IVmd)6C1^WQrJ=_|$PG*D9j?(s!W{gppZG+3uom z!@;ajbGL+4BcB+^R7Yj9;lKcQYG|+b!UrtLdy+N~CNcSIU5j1Q9^@L-`THX&@WUp^ zV1eIn^0Y?=5u%eMQ4U&j2+?PHLpt8HpjYQv&EKIo%9-oD4ieQ-!5Olfluzu6Qs0Zk z^M|}9+?FZNw$jM7x@Y;T1z)Qu0)e5aejX(>t>4_e z(!ZJ2KqzR_`Sqigki>`Uxo`Dk`89mNI<@5@NY!Ovg0+0{r3dc2^KsH8j~NFaHLw$z zH8_x1t7I3-sJ&Q7HPcqz7`Q9EDL)xov3IjseChc%?W9uOU_WPNLuN0%y&f4@zZCBx z$b5r#E$K9meG19OdL`$!)>$(x59L6TN;>*n{;Scxbj4%f3dt-v8cK9$opV5$|0 z$1c}$DPyqXP-={G(HU2<4Gs}kh9=r6rK7w{MsZ4@jsw@h3y?!i90Z}CETj570d&Y-Ro;L?BthdSy&xaw7Ir`z8uu^yCva z`x^mxC}kzuEmY6`9oJIKh#@ToX@uZptIX5<`75{So1L&I*_y&>F=Iu9n8eu+%&m{+ zl>O=^0m1Dz>_;deOOzOjAJFE-e`>ZJ*RU&dKASw3CXG*_PRzDIgl%(<93v#Ad^ zzx>yGwp4Tgg+hJ%JgS97Arb>QC_f+H`ZRk0b}zpNR{8&otj}rTu_TjJ~LY$i&U8;)~yR2T$5#(5MkrSgfuYRl{jrnd0o5t@cxSj z!<&Y*jtO4ad+Vnwxk2~8k