Skip to content

DNSRecordSet reports Programmed=True while an unresolvable ALIAS target SERVFAILs #78

Description

@ecv

What needs to happen

A DNSRecordSet whose ALIAS target does not resolve should say so in its status. Today it reports success.

Programmed=True currently means "record successfully applied to PDNS" — the write landed. For ALIAS that is only half the story: PowerDNS expands ALIAS by resolving the target's A/AAAA at query time, so if the target is NXDOMAIN the authoritative servers answer SERVFAIL for the name's A/AAAA while every condition on the object stays True.

Observed on a live record — control plane green, apex dark:

status:
  conditions:
  - type: Accepted    status: "True"   message: DNSRecordSet accepted for PowerDNS zone
  - type: Programmed  status: "True"   message: All records programmed

$ dig @ns1.datumdomains.net <apex> A     -> SERVFAIL
$ dig @ns1.datumdomains.net <apex> SOA   -> NOERROR      # zone itself healthy
$ dig <alias target>                     -> NXDOMAIN

Non-ALIAS types at the same name (SOA/NS/TXT/MX/CNAME) answer normally, so nothing else hints at the breakage. A dangling CNAME degrades more gracefully — NXDOMAIN at the end of the chain — but is equally invisible in status.

Why this matters

This is the failure mode that has actually taken customer hostnames down, and the one users cannot self-diagnose. Two recent cases: an apex ALIAS pointing at a torn-down proxy canonical (datum-cloud/infra#3407) and a fleet sweep that found six more stranded records across four projects (datum-cloud/infra#3477). Most recently a user hit it directly and filed it as an ALIAS bug, because every signal available to them said the record was healthy: datum-cloud/infra#3717.

Preventing the creation of orphans on the proxy path is datum-cloud/network-services-operator#283. This issue is the missing signal for orphans however they arise — hand-authored records, a target deleted later, or a target that never existed.

Desired outcome

An operator or user looking at a DNSRecordSet can tell a resolving ALIAS from a SERVFAILing one without reaching for dig. A record whose target does not resolve is not reported as fully healthy, and the message names the unresolvable target.

Whether that is a narrowed meaning for Programmed or a separate condition is a design call — worth deciding alongside how often the target is re-checked, since a target can die long after the write succeeded.

Related to datum-cloud/infra#3717
Related to datum-cloud/infra#3477
Related to datum-cloud/network-services-operator#283

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions