feat: restrict Grafana public network access and document VPN requirement

[Cliff Schomburg (cssjr)]

Summary

Prerequisite change for restricting Azure Managed Grafana public network access (ARO-28693). This PR adds the config parameter, pipeline wiring, and documentation without changing current behavior — all environments remain Enabled. The actual restriction to Disabled will be coordinated with the production access mechanism (PE and/or crossTenantSecurityGroup) in a follow-up sdp-pipelines change.

Changes:

• Add grafanaPublicNetworkAccess config parameter (default: Enabled — no behavior change)
• Wire publicNetworkAccess through pipeline YAML and Go reconciler to grafanactl
• Bump ARO-Tools dependency to include merged PublicNetworkAccess support (ARO-Tools#279)
• Update 6 documentation files with VPN access notes
• Add new SOP for Grafana VPN access troubleshooting

How the defaults work

The grafanactl tool default is Enabled (backward compatible — an independent ARO-Tools dependency bump won't accidentally lock anyone out). The config.yaml base default is also Enabled (no behavior change on any environment). When the team is ready to restrict access, a coordinated change will:

1. Confirm the production access mechanism (PE and/or crossTenantSecurityGroup) is in place
2. Set grafanaPublicNetworkAccess: Disabled for the target environments (via sdp-pipelines or config overlay)

Design decision: Private Endpoint vs Network ACL

Evaluated per ARO-28696. Azure Managed Grafana does not expose inbound IP allowlist or network ACL properties in the ARM API (ARM template reference). The two viable inbound access restriction mechanisms are:

1. Private Endpoint — Creates a private IP for Grafana in a VNet. VPN users access Grafana through the private IP rather than the public endpoint. Production environments likely already have PE infrastructure configured outside this repo.

2. crossTenantSecurityGroup — An Azure Managed Grafana platform feature (AMG.CrossTenant.SecurityGroup tag) that grants access to an Entra ID security group in the MSFT Corp tenant. Already wired through the pipeline; the value is configured in sdp-pipelines for production environments.

Either mechanism (or both) must be confirmed as in place before setting publicNetworkAccess: Disabled, otherwise all users will be locked out. This PR does not change publicNetworkAccess from its current state — it only adds the config parameter and pipeline wiring so the restriction can be enabled in a follow-up change.

Full evaluation documented on ARO-28696.

Integration test results

Tested against live Azure Managed Grafana instances in the dev subscription:

• ✅ publicNetworkAccess property correctly set via Go SDK BeginCreate (no silent drop)
• ✅ Toggling between Enabled and Disabled works on existing instances
• ✅ Default (no --public-network-access flag) correctly sets Enabled (backward compatible)
• ✅ publicNetworkAccess: Disabled blocks public internet access (verified off-VPN)
• ✅ publicNetworkAccess: Enabled allows public access (verified in browser)

Jira

• ARO-28693 — Restrict Grafana to MSFT Corp VPN and add safeguards (parent story)
• ARO-28696 — Update Grafana network restriction config
• ARO-28697 — Add CI policy check to prevent Grafana public re-exposure
• ARO-28771 — Update existing documentation with Grafana VPN access requirements

Test plan

• cd config && make materialize passes (schema validation + rendering)
• go build ./... passes for tooling/templatize
• All rendered configs show grafanaPublicNetworkAccess: Enabled (no behavior change)
• Integration tests against live Azure Managed Grafana (see above)
• CI passes

🤖 Generated with Claude Code

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: cssjr
Once this PR has been reviewed and has the lgtm label, please assign geoberle for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• config/OWNERS
• dev-infrastructure/OWNERS
• docs/OWNERS
• test/OWNERS
• tooling/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[Cliff Schomburg (cssjr)]

/hold still a work in progress but I wanted to open the draft to run GitHub Copilot for feedback

[Copilot]

Pull request overview

This PR prepares the codebase to optionally restrict Azure Managed Grafana public network access by introducing a new grafanaPublicNetworkAccess config parameter (defaulting to Enabled), wiring it through the pipeline and templatize Grafana reconciler, and documenting potential MSFT Corp VPN access requirements—without changing current defaults/behavior.

Changes:

• Add monitoring.grafanaPublicNetworkAccess (schema + defaults + rendered configs) and wire it into the GrafanaManage pipeline step as publicNetworkAccess.
• Extend the templatize Grafana reconcile step to pass publicNetworkAccess through to grafanactl reconcile options, plus add unit tests.
• Bump ARO-Tools dependencies in relevant Go modules and update multiple docs + add a new Grafana VPN access SOP.

Reviewed changes

Copilot reviewed 25 out of 30 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tooling/templatize/pkg/pipeline/grafana_reconcile.go	Resolves and forwards publicNetworkAccess into grafanactl reconcile options.
tooling/templatize/pkg/pipeline/grafana_reconcile_test.go	Adds tests covering default PublicNetworkAccess and config resolution/override behavior.
tooling/templatize/go.mod	Bumps ARO-Tools pipelines and grafanactl modules for PublicNetworkAccess support.
tooling/templatize/go.sum	Updates dependency checksums for the ARO-Tools bumps.
tooling/pipeline-documentation/go.mod	Bumps ARO-Tools pipelines dependency.
tooling/pipeline-documentation/go.sum	Updates checksums for the ARO-Tools pipelines bump.
tooling/helmtest/go.mod	Bumps ARO-Tools pipelines dependency.
tooling/helmtest/go.sum	Updates checksums for the ARO-Tools pipelines bump.
tooling/grafanactl/go.mod	Bumps ARO-Tools grafanactl dependency.
tooling/grafanactl/go.sum	Updates checksums for the ARO-Tools grafanactl bump.
test/go.mod	Updates indirect ARO-Tools dependency versions.
test/go.sum	Updates checksums for the ARO-Tools bumps used by tests.
dev-infrastructure/grafana-pipeline.yaml	Wires publicNetworkAccess into the GrafanaManage step via configRef.
dev-infrastructure/global-pipeline.yaml	Wires publicNetworkAccess into the global GrafanaManage pipeline step.
dev-infrastructure/global-pipeline-stg.yaml	Wires publicNetworkAccess into the staging global GrafanaManage pipeline step.
config/config.yaml	Adds grafanaPublicNetworkAccess default and a dev cloud override value.
config/config.schema.json	Adds schema enum for public network access and requires grafanaPublicNetworkAccess under monitoring.
config/rendered/dev/pers/westus3.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
config/rendered/dev/perf/westus3.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
config/rendered/dev/dev/westus3.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
config/rendered/dev/cspr/westus3.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
config/rendered/dev/ci01/centralus.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
config/rendered/dev/ci00/centralus.yaml	Materialized config output includes grafanaPublicNetworkAccess: Enabled.
docs/sops/grafana-vpn-access.md	New SOP documenting VPN access expectations and troubleshooting for restricted Grafana.
docs/ops/kubernetes-provisioning-diagnostics.md	Adds a note that non-dev Grafana access may require MSFT Corp VPN.
docs/network-security.md	Extends the access overview table to include Managed Grafana and its restriction mechanism.
docs/monitoring.md	Adds a note about potential VPN requirement when Grafana public access is disabled.
docs/grafana-dashboards.md	Adds prerequisites note about potential VPN requirement for non-dev environments.
docs/alert-verification.md	Adds a VPN access note for troubleshooting Grafana reachability in non-dev environments.
docs/ai/grafana-debugging.md	Adds AI-facing guidance about verifying VPN connectivity when Grafana is unreachable.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (1)

docs/sops/grafana-vpn-access.md:5

• The overview sentence implies that setting publicNetworkAccess: Disabled directly “restricts access to users connected to the MSFT Corp VPN”. In practice, publicNetworkAccess: Disabled only disables the public endpoint; users still need whatever private connectivity is configured (often via a Private Endpoint reachable from Corp VPN). Rewording this avoids readers assuming VPN is the only/guaranteed access mechanism and keeps the SOP accurate across environments.

Azure Managed Grafana instances in staging, integration, and production environments can be configured with `publicNetworkAccess: Disabled`, which restricts access to users connected to the MSFT Corp VPN. Dev environment Grafana instances are publicly accessible.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (3)

docs/sops/grafana-vpn-access.md:5

• publicNetworkAccess: Disabled does not by itself restrict Grafana to Corp VPN users; it blocks the public endpoint. VPN access only works when a Grafana private endpoint, private DNS, and routing from the Corp network are configured. Because that access mechanism is still described as needing confirmation, this SOP currently promises an access path that may not exist and could misdirect an outage response.

Azure Managed Grafana instances in staging, integration, and production environments can be configured with `publicNetworkAccess: Disabled`, which restricts access to users connected to the MSFT Corp VPN. Dev environment Grafana instances are publicly accessible.

docs/network-security.md:69

• This row treats PublicNetworkAccess as the mechanism connecting Corp VPN users, but disabling public access only closes the public endpoint. A private endpoint (plus DNS and VPN routing) supplies that connectivity, so the table should distinguish the network path from the public-access switch.

| Global | Managed Grafana | arohcp-{env} | MSFT Corp VPN | PublicNetworkAccess (when Disabled) |

tooling/templatize/pkg/pipeline/grafana_reconcile_test.go:105

• This test reimplements the new assignment instead of exercising runGrafanaManageStep; deleting the production wiring at lines 77–83 would leave every added test green. Please extract option construction behind a testable helper (or inject the reconcile runner) and assert that a Disabled step reaches the actual manage.ReconcileOptions.

	if resolved != "" {
		opts.PublicNetworkAccess = resolved

[Cliff Schomburg (cssjr)]

/unhold ready for review

[Cliff Schomburg (cssjr)]

/test e2e-parallel