Skip to content

fix: Stop gateway DNS dual-write for v4/v6 aliases - #316

Draft
mattdjenkinson wants to merge 1 commit into
mainfrom
fix/315-gateway-dns-dual-write
Draft

fix: Stop gateway DNS dual-write for v4/v6 aliases#316
mattdjenkinson wants to merge 1 commit into
mainfrom
fix/315-gateway-dns-dual-write

Conversation

@mattdjenkinson

Copy link
Copy Markdown
Contributor

Summary

Gateway v4. / v6. hostnames were getting two writers: ExternalDNS A/AAAA via the downstream DNSEndpoint, and Gateway DNS CNAMEs to the apex. PowerDNS rejects that mix with a 422, and dnsrecordset-powerdns retries forever. Staging hit this on project datum-cloud after a Domain NS flap deleted then recreated the CNAMEs into still-present A/AAAA records (see #315).

This makes Gateway DNS the only writer for those aliases. ExternalDNS keeps the apex A/AAAA only. When Domain verification or DNS authority is temporarily missing, we still keep the hostname in the desired set so GC does not delete a record the Gateway still wants. Conflict detection now treats any other managed DNSRecordSet at the same FQDN/zone as a blocker, not only a different managed-by.

Test plan

  • Unit: authority miss retains an existing CNAME
  • Unit: same-FQDN peer with platform managed-by surfaces Conflict (no second record)
  • Unit: v4. / v6. claimed hostnames create CNAMEs to the canonical hostname
  • Unit: gatewayDNSEndpointEntries emits apex A/AAAA only
  • go test ./internal/controller/ -run 'DNS|GarbageCollectDNS|Reap'
  • Gateway chainsaw e2e: DNSEndpoint assert is apex-only; status still lists v4. / v6. Hostname addresses

Fixes #315

ExternalDNS was emitting A/AAAA for v4./v6. aliases while Gateway DNS
also created CNAMEs at the same FQDNs, which PowerDNS rejects with 422.
A Domain NS flap then GC'd those CNAMEs and recreated them into the
conflict, hot-looping dnsrecordset-powerdns.

Key changes:
- ExternalDNS DNSEndpoint covers the apex only; v4./v6. stay CNAME-owned
- Authority / verification misses retain desired DNSRecordSets (no GC)
- Conflict detection blocks any other managed record at the same FQDN

Fixes #315
@mattdjenkinson
mattdjenkinson requested a review from scotwells July 23, 2026 12:28
@mattdjenkinson mattdjenkinson assigned ecv, savme and JoseSzycho and unassigned ecv, savme and JoseSzycho Jul 23, 2026
@JoseSzycho
JoseSzycho removed their request for review July 23, 2026 12:33
@JoseSzycho

Copy link
Copy Markdown
Contributor

I'm sorry for this one, I don't really have the knowledge to help with this review

@ecv

ecv commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

I feel like ExternalDNS should maybe be the sole writer? Then the responsibility isn't split.

Point a contextless Opus model at this and ask it if the implementation has more elegant alternatives. I think there's also a few code issues w/ the unverifiedDomain retain branch never being traversed and retain-on-flap retaining forever, but these may be moot if we want ExternalDNS to do this.

We might want to merge a guidance-and-reasoning doc on what bits should handle what, first, and then the code.

@scotwells

Copy link
Copy Markdown
Contributor

I'd agree we should make sure there's an architecture doc we're working from that captures the responsibilities of each component in the system and make sure that's well understood here before we make a change. It's helpful to rationalize these types of situations where there's dual ownership over something.

@mattdjenkinson
mattdjenkinson marked this pull request as draft July 23, 2026 18:39
@mattdjenkinson

Copy link
Copy Markdown
Contributor Author

@ecv @scotwells yeah sure thing. Let me take this away and work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway DNS dual-writes CNAME and A/AAAA for v4./v6. — PowerDNS 422 hot-loop

5 participants