A FedRAMP requirement, a Route 53 NSEC edge case, and a cached DS record: the most instructive DNSSEC failure ever written up.
The setup
In September 2021 Slack needed DNSSEC on slack.com to meet FedRAMP Moderate. Their DNS estate was split: Amazon Route 53 held the primary slack.com zone, NS1 served delegated subzones for traffic management, and the domain was registered through MarkMonitor, which is where the DS record would be published into the .com parent zone.
DNSSEC has two halves, and both must agree: the zone signs its records (Route 53 manages the zone-signing key, you hold the key-signing key in KMS), and the parent zone publishes a DS record that anchors your KSK into the global chain of trust. Signing without the DS accomplishes nothing; a DS pointing at a zone that stops signing breaks everything.
Slack's split DNS estate and the two halves of DNSSEC
Trace
A FedRAMP Moderate requirement drives DNSSEC onto slack.com. The estate is split three ways: Route 53 holds the primary zone and signs records (managing the ZSK, with the KSK in KMS), NS1 serves delegated traffic-management subzones, and MarkMonitor publishes the DS into the .com parent. DNSSEC only works when both halves agree: the zone signs, AND the parent anchors the KSK with a DS record.
What broke
Slack's third rollout attempt hit an edge case in how Route 53 answered non-existence proofs for wildcard records. When resolvers asked for a AAAA record under *.slack.com that did not exist, the NSEC response failed to say that A records DID exist. Resolvers doing aggressive NSEC caching (RFC 8198), notably Google Public DNS, cached that answer as 'nothing exists here at all' and stopped resolving slack.com A records entirely.
The rollback made it worse in the most instructive way possible: the team pulled the DS record from the registrar and assumed the change was immediate. But the .com zone had served that DS with a 24-hour TTL, so validating resolvers kept demanding signatures from a zone that had just turned signing off. The result was SERVFAIL for roughly a day for affected users. AWS later fixed the NSEC type-bitmap behavior in Route 53.
Two failures: the NSEC wildcard bug and the 24-hour rollback trap
Trace
The third rollout attempt failed twice over. First, Route 53's NSEC proof for a non-existent AAAA under *.slack.com omitted A from the type bitmap; resolvers doing aggressive NSEC caching (RFC 8198), notably Google Public DNS, cached it as 'nothing exists here' and stopped resolving A records. Then the rollback made it worse: the DS was pulled at the registrar but .com kept serving the cached DS for its 24-hour TTL, so validating resolvers demanded signatures a now-unsigned zone could not produce, returning SERVFAIL for roughly a day.
What this teaches for the exam
Enabling DNSSEC on Route 53 is two steps in a strict order: enable signing on the hosted zone (KSK backed by an asymmetric KMS key), then publish the DS at the parent through your registrar. Disabling is the same walk in reverse with a mandatory wait: remove the DS record, wait out the parent's DS TTL (86400 seconds for .com), and only then disable signing.
Wildcards plus DNSSEC deserve their own test plan, because non-existence proofs (NSEC) behave differently around them, and resolver behavior varies. And when a question mentions per-subdomain DNSSEC or partial rollout, remember Slack's lesson: the chain of trust is zone-wide, and you cannot enable it for only part of a zone.
Enable and disable DNSSEC in strict order (with a mandatory wait)
Trace
Enabling is two ordered steps: sign the zone first (KSK backed by an asymmetric KMS key), then publish the DS at the registrar. Disabling is the same walk in reverse with a mandatory wait: remove the DS, wait out the parent's DS TTL (86400s for .com), and only then disable signing. Slack's mistake was disabling signing before the cached DS expired, which is exactly the path that produces SERVFAIL.
