feat(kafka): multi-cluster support (named clusters + --cluster + v3 migrate)

[HumanBean17]

Summary

Adds support for multiple named Kafka clusters in one agctl.yaml, mirroring the existing database.connections model. Previously Config.kafka was a single object (one broker set, one TLS block) — asymmetric with database.connections / services / logs.sources, which are all named maps.

What changed

• Config schema v3 — kafka: restructured from a single object to kafka.clusters.<name> (named map, each with brokers/ssl/timeout_seconds/default_consumer_group/schema_registry_url) + kafka.default_cluster. KafkaCluster model; KafkaSSL reused unchanged.
• Cluster resolution — new resolve_cluster_name mirrors resolve_connection_name (db_commands.py): precedence --cluster > pattern/reactor .cluster > kafka.default_cluster > single-cluster auto-default. new_kafka_client(cluster, …) now takes a resolved KafkaCluster.
• CLI — --cluster <name> on kafka produce/consume/assert; kafka assert --pattern resolves both topic and cluster from the pattern.
• Mock reactors — MockEngine takes a per-reactor client map (kafka_clients: dict[reactor_name→client]); each mocks.kafka.reactors.<name> can target its own cluster via an optional cluster field.
• config migrate — lifts a flat kafka: block into kafka.clusters.default, handles v1→v3 (jq + structural, one pass) and v2→v3 (structural only — v2 exprs are already envelope-rooted, so no double-prefix). Bumps version to "3". Envelope key already_v2 → already_current.
• Discover — kafka-patterns item detail surfaces the resolved cluster.
• Docs — DESIGN §2.1/§3.2/§3.7/§5, ARCHITECTURE §5/§8/§15, and consumer skills/ synced to v3.

⚠️ Breaking change (config version 2 → 3)

A version: "2" (or "1") config is now rejected at load with a pointer to migrate. Run:

agctl config migrate        # backs up to <path>.bak, lifts kafka → clusters, bumps to v3

The migration is non-mutating on input, idempotent on v3, and safe for v1 and v2 sources.

Testing

• 962 unit tests passing (was ~947 pre-feature). New coverage: cluster resolution precedence, unknown/no-cluster errors, migrate_config (v1/v2→v3 one-pass, idempotency, non-mutation), validator per-cluster cross-refs, per-cluster mock-reactor wiring + shared-cluster client reuse, discover cluster surfacing, and the v1/v2 cli_flags_note gate.
• Integration: existing require_kafka suite runs against the default cluster unchanged; a new self-skipping require_kafka_second_broker round-trip is gated on AGCTL_TEST_KAFKA_BROKER_2.
• Unit tests run without confluent_kafka installed (fakes), per convention.

Review

Per-task reviews (sonnet) all Approved; final whole-branch review (opus) verdict Ready to merge — with fixes. The single Important finding (config migrate's cli_flags_note unconditionally told v2→v3 migrators to prefix .body |/.value | on CLI flags, which would double-prefix working scripts) was fixed by gating that advice on v1 sources only (commit 55e606c). No Critical issues.

Spec / Plan

• Spec: docs/superpowers/specs/active/2026-07-09-agctl-kafka-multi-cluster-design.md
• Plan: docs/superpowers/plans/active/2026-07-09-agctl-kafka-multi-cluster.md

(These stay in active/ until the PR merges, then move to archive/ as ADRs.)

---

🤖 Generated with Claude Code