docs: Reorganize README and architecture documentation

README.md

@@ -1,138 +1,101 @@
-## datum dns-operator
-
-Kubernetes operator for managing DNS zones and records, with a pluggable backend architecture. This repository provides:
-- Custom resources to model zones, recordsets, and zone classes
-- Controllers for two runtime roles:
-  - "downstream" agent that programs a DNS backend (PowerDNS supported)
-  - "replicator" that mirrors resources from an upstream cluster to a downstream cluster and synthesizes status
-- Kustomize overlays to deploy either role
-
-### CRDs
-- **`DNSZoneClass`** (cluster-scoped)
-  - `spec.controllerName`: selects backend controller (e.g., "powerdns")
-  - `spec.nameServerPolicy`: currently supports `Static` with `servers: []`
-  - `spec.defaults.defaultTTL`: optional default TTL for zones
-
-- **`DNSZone`** (namespaced)
-  - `spec.domainName`: required zone FQDN (e.g., `example.com`)
-  - `spec.dnsZoneClassName`: optional reference to a `DNSZoneClass`
-  - `status.nameservers`: authoritative nameservers (derived from class policy)
-  - `status.conditions`: `Accepted`, `Programmed`
-
-- **`DNSRecordSet`** (namespaced)
-  - `spec.dnsZoneRef`: `LocalObjectReference` to a `DNSZone` in the same namespace
-  - `spec.recordType`: one of `A, AAAA, CNAME, TXT, MX, SRV, CAA, NS, SOA, PTR, TLSA, HTTPS, SVCB`
-  - `spec.records[]`: owners with typed fields per record type (or `raw` strings). TTL per-owner optional.
-  - `status.conditions`: `Accepted`, `Programmed`
-
-### Controllers and Roles
-
-- **Downstream role** (`--role=downstream`)
-  - `DNSZoneReconciler`: when `DNSZone.spec.dnsZoneClassName` references a class with `controllerName: powerdns`, ensures the zone exists in PowerDNS and honors static nameserver policy.
-  - `DNSRecordSetReconciler`: for PowerDNS-backed zones, applies recordsets to PDNS using an authoritative mode that REPLACEs desired owners and DELETEs extraneous owners of the same type. Requeues while the zone is not ready.
-
-- **Replicator role** (`--role=replicator`)
-  - Multicluster manager discovers one or many upstream clusters (single-cluster or Milo discovery) and mirrors `DNSZone`/`DNSRecordSet` into a configured downstream cluster using a mapped-namespace strategy.
-  - `DNSZoneReplicator`:
-    - Mirrors upstream `spec` into a downstream shadow object
-    - Ensures an operator-managed upstream `DNSRecordSet` named `soa` exists (typed SOA targeting `@`) for PowerDNS-backed zones
-    - Updates upstream `status`: sets `Accepted=True` and currently treats `Programmed=True` optimistically; fills `status.nameservers` from `DNSZoneClass` when `Static` policy is set
-  - `DNSRecordSetReplicator`:
-    - Mirrors upstream `spec` into a downstream shadow object
-    - Updates upstream `status`: `Accepted` reflects `DNSZone` presence; `Programmed=True` once downstream shadow ensured
-
-### Backends
-- **PowerDNS (Authoritative)**
-  - Enabled when `DNSZoneClass.spec.controllerName: powerdns`
-  - The downstream agent uses environment variables to connect:
-    - `PDNS_API_URL` (default `http://127.0.0.1:8081`)
-    - `PDNS_API_KEY` or `PDNS_API_KEY_FILE`
-  - Recordset translation supports typed fields for all declared RR types and sensible normalization of names and quoting for TXT/targets.
-
-### Deployment Overlays
-
-- `config/agent/`
-  - Namespace: `dns-agent-system`
-  - Runs the operator with `--role=downstream`
-  - Merges a `pdns` sidecar container into the controller Deployment to run PowerDNS alongside the manager
-  - Mounts a shared `emptyDir` to exchange an auto-generated API key, and sets `PDNS_API_KEY_FILE` in the manager
-  - Provides a `Service` exposing PDNS ports 53/udp, 53/tcp, and 8081/tcp
-  - ConfigMap `server-config` wired to `--server-config`
-
-- `config/overlays/replicator/`
-  - Namespace: `dns-replicator-system`
-  - Runs the operator with `--role=replicator`
-  - Requires a Secret `downstream-kubeconfig` containing key `kubeconfig` to target the downstream cluster
-  - ConfigMap `server-config` sets discovery mode (defaults to `single`) and points `downstreamResourceManagement.kubeconfigPath` to `/downstream/kubeconfig`
-
-### Quickstart: Agent with embedded PowerDNS
-1. Install CRDs and default manifests:
-   - `kubectl apply -k config/agent`
-2. Create a `DNSZoneClass` for PowerDNS with static nameservers, for example:
-```yaml
-apiVersion: dns.networking.miloapis.com/v1alpha1
-kind: DNSZoneClass
-metadata:
-  name: powerdns
-spec:
-  controllerName: powerdns
-  nameServerPolicy:
-    mode: Static
-    static:
-      servers: ["ns1.example.net.", "ns2.example.net."]
-```
-3. Create a `DNSZone` and a `DNSRecordSet`:
-```yaml
-apiVersion: dns.networking.miloapis.com/v1alpha1
-kind: DNSZone
-metadata:
-  name: example-com
-  namespace: default
-spec:
-  domainName: example.com
-  dnsZoneClassName: powerdns
----
-apiVersion: dns.networking.miloapis.com/v1alpha1
-kind: DNSRecordSet
-metadata:
-  name: www-a
-  namespace: default
-spec:
-  dnsZoneRef:
-    name: example-com
-  recordType: A
-  records:
-    - name: www
-      a:
-        content: ["192.0.2.10", "192.0.2.11"]
-      ttl: 300
+# DNS
+
+Manage authoritative DNS the Kubernetes way. Declare a domain and its records as
+ordinary Kubernetes resources, and the DNS service programs them into an
+authoritative DNS backend and publishes them to a globally distributed serving
+layer — no zone files, no backend API calls, no manual nameserver wiring.
+
+You manage DNS through native Kubernetes resources, so it works with `kubectl`
+and any Kubernetes client, and inherits your platform's identity, RBAC, and audit
+controls.
+
+This repository provides the **DNS operator**, the control-plane component that
+reconciles those resources and programs the backend. See the
+[Architecture Overview](docs/architecture/README.md) for how the operator fits
+into the wider service.
+
+## What it does
+
+- **Zones and records as resources** — Model domains with `DNSZone` and records
+  with `DNSRecordSet`, covering `A`, `AAAA`, `CNAME`, `ALIAS`, `TXT`, `MX`,
+  `SRV`, `CAA`, `NS`, `SOA`, `PTR`, `TLSA`, `HTTPS`, and `SVCB` types.
+- **Pluggable backends** — A cluster-scoped `DNSZoneClass` selects the backend
+  and nameserver policy, keeping backend choice out of individual zones.
+  [PowerDNS](https://doc.powerdns.com/authoritative/) is supported today.
+- **Multi-tenant by design** — Each tenant authors DNS in their own control
+  plane; the operator discovers and serves many control planes from one shared
+  authoritative backend, with per-domain ownership accounting.
+- **Automatic zone bootstrap** — Default `SOA` and `NS` records are created for
+  every zone from its nameserver policy, without clobbering user-authored apex
+  records.
+- **Clear status** — `Accepted` and `Programmed` conditions report whether a
+  zone or record is valid and actually serving, mirrored back from the
+  authoritative backend.
+
+## How it works
+
+Users declare `DNSZone` and `DNSRecordSet` resources in their own control plane.
+A **replicator** mirrors that desired state into a shared authoritative cluster,
+where a **downstream agent** programs it into the DNS backend. The authoritative
+data is then replicated to a read-only serving layer that answers live queries.
+
+For the full picture — components, control planes, and the serving layer — see
+the [Architecture Overview](docs/architecture/README.md).
+
+## Documentation
+
+**Architecture**
+- [Architecture Overview](docs/architecture/README.md) — System design and core
+  concepts
+- [Deployment Topology](docs/architecture/topology.md) — Roles, control planes,
+  and the serving layer
+- [Replication Model](docs/architecture/replication.md) — How desired state and
+  status move between clusters
+- [API Reference](docs/architecture/api-reference.md) — Full resource schema and
+  conditions
+
+**Guides**
+- [Service Catalog](config/components/service-catalog/README.md) — DNS as a
+  billable platform service
+
+## Deploying
+
+The operator runs in one of two roles, deployed with the Kustomize overlays in
+[`config/`](config). See [Deployment Topology](docs/architecture/topology.md) for
+how the roles fit together.
+
+### Agent with embedded PowerDNS
+
+Runs the operator as a downstream agent alongside PowerDNS and a storage backend
+— the quickest way to a working DNS service:
+
+```sh
+kubectl apply -k config/overlays/agent-powerdns
 ```
 
-### Quickstart: Replicator (upstream → downstream)
-1. Create Secret on the replicator namespace containing the downstream kubeconfig (`data.kubeconfig`):
-```bash
+Then create a `DNSZoneClass`, `DNSZone`, and `DNSRecordSet` (see
+[`config/samples`](config/samples) and the
+[API Reference](docs/architecture/api-reference.md)).
+
+### Replicator (upstream → downstream)
+
+Runs the operator as a replicator that mirrors DNS resources from tenant control
+planes into a downstream authoritative cluster:
+
+```sh
+# Provide the downstream cluster kubeconfig
 kubectl -n dns-replicator-system create secret generic downstream-kubeconfig \
   --from-file=kubeconfig=/path/to/downstream/kubeconfig
-```
-2. Deploy replicator overlay:
-```bash
+
 kubectl apply -k config/overlays/replicator
 ```
-3. Create `DNSZoneClass` (cluster-scoped), `DNSZone` and `DNSRecordSet` on the upstream cluster. The replicator will mirror them into the downstream cluster and update upstream `status` conditions.
-
-### Conditions
-- `Accepted`: resource is valid and has required dependencies (e.g., `DNSRecordSet` sees its `DNSZone`)
-- `Programmed`: desired state is realized (shadow exists downstream; for downstream agent, recordsets applied to backend)
-
-### Configuration CRD (server config)
-- `kind: DNSOperator` (internal config consumed by the binary via `--server-config`)
-  - `discovery.mode`: `single` or `milo`
-  - `downstreamResourceManagement.kubeconfigPath`: path inside the Pod to the downstream kubeconfig
-  - `controllers.dnsRecordSetPowerDNS.maxConcurrentReconciles`: concurrent reconciles for the PowerDNS recordset controller (default: 4)
-  - `controllers.dnsRecordSetPowerDNS.rateLimiterBaseDelay`: exponential backoff base delay (default: `1s`)
-  - `controllers.dnsRecordSetPowerDNS.rateLimiterMaxDelay`: exponential backoff max delay (default: `30s`)
-
-### Development
-- Build: `make docker-build` (see `Makefile`)
-- Generate code/manifests: `make generate` and `make manifests`
-- Local e2e: see `test/e2e/chainsaw-test.yaml` and sample manifests under `config/samples/`
+
+The replicator mirrors upstream `DNSZone` / `DNSRecordSet` resources downstream
+and synthesizes their status back upstream. See
+[Replication Model](docs/architecture/replication.md).
+
+## Development
+
+- **Build:** `make docker-build` (see the [`Makefile`](Makefile))
+- **Generate code / manifests:** `make generate` and `make manifests`
+- **End-to-end tests:** see `test/e2e/` and the samples under `config/samples/`

docs/Taskfile.yaml

@@ -0,0 +1,56 @@
+version: '3'
+
+# Renders the architecture diagrams from PlantUML sources using Docker,
+# mirroring the datum-cloud/enhancements convention. Each diagram is committed
+# next to the document that embeds it. The shared brand theme
+# (datum-theme*.puml) is an include, not a diagram, so it is excluded.
+#
+# The whole docs/architecture tree is mounted so relative includes (e.g.
+# ../datum-theme.puml from backends/) resolve, and each PNG is written back
+# beside its source.
+
+vars:
+  ARCH_DIR: "{{.USER_WORKING_DIR}}/docs/architecture"
+  OUTPUT_FORMAT: "png"
+  PLANTUML_IMAGE: "plantuml/plantuml:latest"
+
+tasks:
+  diagrams:
+    desc: Render all architecture diagrams from PlantUML
+    cmds:
+      - task: diagrams:render
+
+  diagrams:render:
+    desc: Render PlantUML diagrams to PNG using Docker
+    cmds:
+      - |
+        set -e
+        cd "{{.ARCH_DIR}}"
+        find . -name '*.puml' ! -name 'datum-theme*.puml' | while read -r puml; do
+          echo "Rendering: $puml"
+          docker run --rm -v "$(pwd)":/data "{{.PLANTUML_IMAGE}}" \
+            -t{{.OUTPUT_FORMAT}} -o "/data/$(dirname "$puml")" "/data/$puml"
+        done
+
+  diagrams:validate:
+    desc: Validate PlantUML syntax using Docker
+    cmds:
+      - |
+        set -e
+        cd "{{.ARCH_DIR}}"
+        find . -name '*.puml' ! -name 'datum-theme*.puml' | while read -r puml; do
+          echo "Validating: $puml"
+          docker run --rm -v "$(pwd)":/data "{{.PLANTUML_IMAGE}}" \
+            -syntax "/data/$puml"
+        done
+
+  diagrams:clean:
+    desc: Remove generated diagram PNGs
+    cmds:
+      - |
+        set -e
+        cd "{{.ARCH_DIR}}"
+        find . -name '*.puml' ! -name 'datum-theme*.puml' | while read -r puml; do
+          rm -f "${puml%.puml}.{{.OUTPUT_FORMAT}}"
+        done
+        echo "Removed generated diagrams"