Chapter 04 · Part I · The namespace

Caching, TTLs & negative answers

Every DNS answer comes with an expiry date, and almost every answer you ever get comes out of somebody's cache rather than from the source. The basics, what a cache is and what the TTL number means, are in networks · DHCP & DNS. This chapter goes underneath: how the countdown runs in each layer between you and the authoritative server, why "no such name" is cached too, what a resolver does when the source goes quiet, and how to plan a change so the switch-over takes minutes instead of a day.

2³¹ − 1 sthe largest TTL RFC 2181 allows: 2,147,483,647 seconds, about 68 years. RFC 8767 now recommends resolvers cap TTLs at 7 days.
1998the year RFC 2308 declared that negative caching "should no longer be seen as an optional part of a DNS resolver"
min(TTL, MIN)a negative answer's lifetime: the lower of the SOA record's own TTL and its MINIMUM field (RFC 2308 §5)

Common mix-up: DNS changes do not "propagate." Nothing is sent outward when you edit a record. Your authoritative servers start giving the new answer almost at once; everyone else keeps the copy they already fetched until its TTL runs out, then asks again. "Waiting for propagation" really means waiting for other people's caches to expire, and the longest you can be made to wait is set by the TTL that was on the record before you changed it.

Pull, not push

Nothing propagates

Picture the record www.example.com pointing at 192.0.2.10. You log in to your DNS host and change it to 198.51.100.20. Within seconds, or a few minutes if your provider has secondary servers to update (Chapter 6 covers that hand-off), every authoritative server for example.com is answering with the new address. That part is fast.

The slow part is everyone else. Thousands of recursive resolvers around the world asked about www.example.com at some point in the last day and are still holding the answer they got. None of them is told about your change. The DNS has no mechanism for an authoritative server to reach out to caches and tell them to forget; it only answers questions. Each cache keeps its copy for as long as the record's time to live allowed when it was fetched, and only when that time is up does it ask again.

So a change becomes visible cache by cache, in a ragged wave set by when each one happened to last ask. That ragged wave is what people call propagation. It has a hard ceiling: if the old record had a TTL of 3600 seconds, no well-behaved cache can still be serving it more than an hour after your change.

A countdown, not a label

The TTL is a whole number of seconds, set by whoever runs the zone, attached to every record. It is not a date. When a resolver caches an answer it notes when it got it, and from then on the TTL it hands out is the time remaining. Ask the same resolver twice, twenty minutes apart, and you can watch it count down (simulated output; all names and addresses here are reserved examples):

; first query, 10:00:00: the resolver had to fetch it, so the full TTL comes back
www.example.com.    3600   IN  A   192.0.2.10

; same question, same resolver, 10:20:00
www.example.com.    2400   IN  A   192.0.2.10

; 11:00:01: the copy expired at 11:00:00, so the resolver asked again
www.example.com.    3600   IN  A   198.51.100.20

That decreasing number is the most useful clue you have when debugging a change. If a resolver answers with the old address and a TTL of 2400, you know it will switch in 40 minutes, and nothing you do at your DNS host can make it switch sooner.

Three rules sit around the number. First, RFC 2181 (§8) says the TTL "specifies a maximum time to live, not a mandatory time to live": a cache may throw an answer away early, but it should never keep it longer. Second, all the records in one RRset, the set of records with the same name, class and type, must carry the same TTL (RFC 2181 §5.2). If your two A records for www had TTLs of 60 and 3600, a cache could end up holding only half the set after a minute and hand out a partial answer, so differing TTLs within a set are deprecated and a receiver should treat them as an error. Third, a TTL of 0 means "use this answer for the question in hand and don't cache it at all."

The ceiling on the number has moved over the years. RFC 2181 fixed it at 2³¹ − 1 seconds, about 68 years. RFC 8767 (2020) now recommends that resolvers cap whatever they receive at 604,800 seconds, seven days, which it says reflects what major resolvers already did. Resolvers apply their own caps on top: the Unbound resolver, for example, defaults to keeping nothing longer than one day (cache-max-ttl: 86400). Some can also be configured with a floor, a minimum TTL that stretches short answers longer than the zone owner asked for; Unbound's manual warns that values above an hour or so can cause trouble. A floor is one of the ways a cache ends up serving old data past the deadline you calculated.

Go deeper: signed or unsigned? The sign bit's 30-year story

RFC 1035 (1987) contradicts itself. Section 3.2.1 calls the TTL "a 32 bit signed integer," and Section 4.1.3 calls it "a 32 bit unsigned integer." Implementations disagreed, so a TTL with the top bit set might be read as a huge positive number or as negative.

RFC 2181 (1997) settled it: the TTL is unsigned, from 0 to 2,147,483,647, sent in the low 31 bits with the top bit zero, and any value received with the top bit set should be treated as if it were 0. RFC 8767 (2020) changed that last rule: such values should now be read as positive and then clamped by the resolver's cap, on the grounds that a typo producing an enormous TTL should not silently disable caching.

In practice zone files rarely use TTLs above a day or two, and nothing on the public internet should depend on the seven-day cap being exactly seven days. But the history explains why you will see different behavior from very old software if you feed it a very large TTL.

Who is holding the old answer

Caches stacked between you and the source

The resolver at your ISP or a public service is the cache most people think of, but a single lookup from a laptop can pass through four or five of them, each with its own copy and its own clock:

LayerWhere it livesHow it keeps timeHow to empty it
ApplicationBrowsers, language runtimes, long-running servicesIts own rules. Some honor the TTL, some keep answers for a fixed time, and an open connection to the old server needs no lookup at allRestart the program; browsers have internal pages for it
OS stubWindows DNS Client, macOS mDNSResponder, systemd-resolvedUsually honors the TTL it receivedipconfig /flushdns · sudo killall -HUP mDNSResponder · resolvectl flush-caches
Home routerThe forwarder your Wi-Fi hands out by DHCPInherits the remaining TTL from the resolver it askedReboot it, or point your machine past it
Recursive resolverYour ISP, your company, or a public serviceCounts down the TTL from the authoritative answer, within its own caps and floorsOnly its operator can; some public services offer a flush form
AuthoritativeThe servers named in your NS recordsNo cache: it holds the zone itselfNothing to empty: this is the source

Two facts follow. A layer that inherits the remaining TTL can never outlive the cache it asked: if the resolver's copy has 40 minutes left, the router's copy fetched from it also has at most 40 minutes. Layers that keep their own time, such as an application with a fixed cache or a resolver with a minimum-TTL floor, are the ones that can push past your deadline.

And your colleague sees the old address while you see the new one because you are not asking the same cache. You might use your ISP's resolver while they use a public one; even if you both use the same public address, large services run many independent caches behind it, so two people, or one person asking twice, can land on caches that fetched at different moments. One copy expires at 10:12, another at 10:47, and for those 35 minutes the internet genuinely disagrees with itself. Nobody is wrong; they are just at different points in their countdowns.

The instrument below runs that disagreement on a clock. Each lane is one cache. Colored blocks are cached copies: orange is the old address, blue the new one, and each block ends where that copy's TTL runs out and the cache asks again.

Instrument 1

The cache clock

Set the record's TTL and the moment you change it, then scrub or play the clock. Six caches fetched the record at different times before you started watching. The app keeps its own 60-second cache and asks the OS; the OS asks the home router; the router asks the ISP resolver. The public resolver has two independent caches behind one address. Every number is computed from those rules. The data is simulated, with reserved example addresses.

Record changed at–
Last old copy expires–
Stale window–
Caches still old now–

Remembering a no

Caching the absence of a record

Plenty of DNS questions have the answer "there's nothing here." A mistyped name, a browser probing for a name that doesn't exist, a mail server looking for an IPv6 address a domain never published. If resolvers forgot every no immediately, they would ask the authoritative servers the same pointless question again and again. So they remember the no as well. RFC 1034 described negative caching as an optional extra; RFC 2308 (1998) rewrote the rules and said it "should no longer be seen as an optional part of a DNS resolver."

There are two kinds of no, and the difference matters when you read a dig answer:

  • NXDOMAIN ("non-existent domain," response code 3): the name itself does not exist. Not as an A record, not as anything. RFC 8020 adds that it also means no name underneath it exists, so a resolver that cached NXDOMAIN for old.example.com may answer NXDOMAIN for a.old.example.com without asking.
  • NODATA: the name exists, but has no records of the type you asked for. There is no response code for this; it arrives as NOERROR with an empty answer section. RFC 2308 calls it a "pseudo RCODE." Asking for the AAAA record of a name that has only an A record gets you NODATA.

A positive answer carries its TTL on the record. A negative answer has no record to carry it, so the authoritative server puts the zone's SOA record in the authority section of the reply and uses that record's TTL as the lifetime of the no. RFC 2308 §5 says how that TTL is set:

negative TTL = min( SOA record's TTL , SOA MINIMUM field )
The last number in the SOA record, once a "default TTL" for the zone, now means only this. A resolver then applies its own cap on top (Unbound's default is 3600 seconds), and counts the result down like any other cached answer.

Here is what that looks like for a name that doesn't exist yet, in a zone whose SOA has a TTL of 3600 and a MINIMUM of 300:

; simulated: dig new.example.com A
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4711
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;new.example.com.            IN  A

;; AUTHORITY SECTION:
example.com.   300  IN  SOA  ns1.example.com. hostmaster.example.com. (
                2026102201 7200 3600 1209600 300 )
; the 300 on the left is min(3600, 300): how long this NXDOMAIN may be cached

The trap is easy to fall into. You are about to launch new.example.com, so you type it into a browser to check it isn't live yet. Your resolver dutifully caches NXDOMAIN. Five minutes later you create the record, reload, and it still doesn't resolve. Nothing is broken; your resolver is honoring a no it was given before the name existed. A monitoring system, a certificate authority doing a validation check or a teammate who looked a little early can plant the same no in other resolvers.

RFC 2308 suggests that resolvers should not cache a negative answer longer than they would cache a positive one, that one to three hours works well, and that values over a day "have been found to be problematic." For a zone you change often, a MINIMUM of a few minutes keeps the trap small.

Go deeper: how a cached no is passed on

When a resolver answers from a cached negative response, RFC 2308 §6 says it must include the cached SOA record with its TTL decremented by the time it has spent in the cache. That way a forwarder downstream caches the no only for the time that is left, exactly as it would with a positive answer, and a no cannot circulate forever between servers that keep resetting each other's clocks.

NODATA responses come in a few shapes, which RFC 2308 numbers as types 1 to 3. The authority section may hold the SOA, the zone's NS records, both, or nothing. Only a response with the SOA tells the resolver how long it may cache the no; one without it should not be cached as negative at all. RFC 9499, the current DNS terminology document, gives the modern definitions of NXDOMAIN, NODATA and negative response. Chapter 14 shows how to tell them apart at the command line.

Instrument 2

The negative-cache lab

Someone looks up new.example.com before you create it. Choose the zone's SOA numbers, the resolver's cap on negative answers, and how long before the creation the first lookup happened. Tick the monitor box to simulate a checker that keeps asking every time the cached no expires. The lab works out how long the resolver keeps answering NXDOMAIN after the name really exists. The data is simulated.

Negative TTL from server–
Resolver keeps the no–
Still NXDOMAIN after creation–

      
When the source goes quiet

Stale answers, failed answers and early answers

The rules so far assume the authoritative servers answer when a cache comes back. Sometimes they don't: an outage, a misconfiguration, a flood of attack traffic. Strict TTL rules then turn a server outage into a total outage, because every cache in the world drops the record within one TTL and has nothing to replace it with. Three newer mechanisms soften that.

Serve-stale (RFC 8767)

A resolver may keep an expired record around and, if it cannot get a fresh answer, hand out the stale one instead of failing. RFC 8767 amended the definition of TTL itself to allow this: "If the data is unable to be authoritatively refreshed when the TTL expires, the record MAY be used as though it is unexpired." The RFC sets out how to do it carefully:

  • A client response timer, recommended at 1.8 seconds: if a fresh answer hasn't arrived by then, reply with the stale one and keep trying in the background.
  • Stale records are sent with a TTL greater than zero, recommended 30 seconds, so downstream caches don't hold them long.
  • A failure recheck timer: after a failed refresh, don't hammer the servers again more often than every 30 seconds.
  • A maximum stale timer, suggested at one to three days, after which even a stale copy is thrown away.

Serve-stale only ever helps when the authoritative servers are unreachable. It never applies when they answer, so it cannot delay a change you make while your servers are up.

Caching the failure itself (RFC 9520)

The opposite problem: a name whose servers are dead gets asked about constantly, and each question sends a resolver off on a slow, fruitless hunt. RFC 9520 (2023) updated RFC 2308 to require that resolvers cache resolution failures: for at least 1 second, never longer than 5 minutes, and preferably with a backoff that lengthens the wait while the failure persists. Its example starts at 5 seconds and grows toward the 5-minute ceiling. That is why a zone that comes back from an outage may take a few minutes to start working everywhere, even though no TTL is involved.

Prefetching

Busy resolvers often refresh a popular record a little before it expires, triggered by a query that arrives near the end of its TTL, so the next user never waits for a miss. Unbound's version fires in the last 10 percent of the TTL and its manual estimates the cost at about 10 percent more traffic. Prefetching doesn't change how long a copy can be stale; it just hides the refresh from users.

Aggressive use of NSEC and NSEC3 (RFC 8198)

A DNSSEC-signed zone proves that a name doesn't exist by returning a signed record that says, in effect, "there are no names between alpha and delta." A validating resolver that has cached such a proof can answer NXDOMAIN for beta, bravo or anything else in that gap without asking. RFC 8198 encourages this; it saves queries and blunts floods of random-name lookups. The cost is the one you already know: a newly added name inside a cached gap stays invisible until the proof expires, and the RFC says outright that "zone operators should have no expectation that an added name would work immediately." The NSEC record's TTL and the SOA MINIMUM decide how quickly a new name starts working. Chapter 11 explains how those proofs are built.

Doing it on purpose

Moving without the long wait

Since the old TTL sets the worst case, the trick is to make the old TTL small before you need it to be. The procedure is old; RFC 1912 (1996) gave it in one sentence: turn the value down beforehand, "wait the previous minimum value, make your changes, verify their correctness, and turn this value back up." In steps:

  1. Lower the TTL on the records you will change, from its normal value (say 86,400, a day) to something short (say 300, five minutes). Do this at least one old TTL before the change. Caches that fetched just before you lowered it are still holding the one-day copy, and you have to wait for those to expire.
  2. Wait the full old TTL. Now every well-behaved cache holds a copy with at most five minutes left.
  3. Make the change. The worst-case stale window is now the temporary TTL, not the old one. Keep the old server running through that window.
  4. Verify from several vantage points, including the authoritative servers directly.
  5. Raise the TTL back once you are sure you won't need to roll back. A short TTL also makes rollback fast, so many operators keep it low for a few hours.

A short TTL is not free. Every time a copy expires, a cache asks again, and those questions land on your authoritative servers. For one resolver whose users ask about your name at an average rate of q queries per second, each cache cycle is one miss, then TTL seconds of hits, then a wait for the next question. That gives an average refetch rate per resolver, and for N resolvers:

load on authoritative ≈ N × q ÷ (1 + q × TTL) queries / second
When q × TTL is large (a busy name, a long TTL), this is about N ÷ TTL: each resolver asks once per TTL. When it is small (a quiet name), it is about N × q: caching saves almost nothing. Cutting the TTL from a day to five minutes multiplies the load on a busy name by up to 288.

A worked example. Say 20,000 resolvers each see 0.05 queries per second for www.example.com (one every 20 seconds). At a one-day TTL your servers see 20,000 × 0.05 ÷ (1 + 0.05 × 86,400) ≈ 0.23 queries per second for that name. At 300 seconds it is 1,000 ÷ 16 = 62.5 per second, about 270 times as much. Keep the TTL low for half a day of ramp-up, two hours after the change and a few minutes of ramp-down, and the extra traffic comes to roughly 3 million queries: nothing for a large DNS provider, worth knowing if you run your own small server.

Instrument 3

The migration planner

Enter your record's normal TTL, the temporary TTL you plan to use, when you want to make the change, how long you will keep the TTL low afterwards, and a rough guess at how many resolvers ask about the name and how often. The planner tells you when to lower the TTL, what the worst-case stale window will be, and what it costs your authoritative servers. It assumes caches fetched at random times and honor the TTL.

Lower the TTL by–
Worst-case stale window–
Raise it again at–
Load: normal → low TTL–
Extra queries, whole plan–

Go deeper: where the load formula comes from, and what it leaves out

Treat the questions reaching one resolver as random arrivals at an average rate q. When one arrives and the cache is empty, the resolver asks your server and keeps the answer for TTL seconds. After it expires, the next question arrives on average 1/q seconds later. So one cycle lasts TTL + 1/q seconds and contains exactly one query to you. The rate per resolver is 1 ÷ (TTL + 1/q), which rearranges to q ÷ (1 + q × TTL). Prefetching shifts this slightly: a resolver that refreshes busy names before they expire asks about once per TTL, never waiting for the next arrival.

The extra-query total assumes the caches' expiry times are spread evenly. When you lower the TTL, caches switch to the short one gradually over one old TTL, so the load ramps up linearly; after you raise it, it ramps down over one temporary TTL. The total extra is (low rate − normal rate) × (old TTL ÷ 2 + time kept low + temporary TTL ÷ 2).

What it leaves out: caches with a minimum-TTL floor won't honor a very short temporary TTL, so values below about a minute buy less than they appear to. Applications that keep connections open don't look the name up again at all, so keep the old server answering for a while. And the NS records that delegate your zone live in the parent zone, with a TTL set by the registry that you usually cannot lower; moving to new nameservers is a different, slower job (Chapter 6).

Green ticks on a world map

What a propagation checker actually measures

Websites that show a map of "DNS propagation" send your question to a list of public resolvers in different countries and show what each one answered at that instant. That is useful, but it is a sample of a few dozen caches out of millions, and it says nothing about the ones that matter most to you: your users' ISP resolvers, their routers, their operating systems and their browsers.

  • A green map does not mean everyone sees the change. It means those particular caches either never had the record, or their copies expired. An ISP cache that fetched one minute before your change still has nearly a full TTL to run.
  • A red pin is not an error. It is a cache in the middle of its countdown. If the checker shows the TTL, it tells you when that pin will turn.
  • Checking can change what you check. A query to a resolver that had nothing cached makes it fetch and cache the current answer. Checking a name before you create it plants exactly the negative answer described above.

The precise way to know where you stand takes two facts. Ask each of your authoritative servers directly, with recursion turned off, to confirm they all serve the new data (dig @ns1.example.com www.example.com +norecurse, covered in Chapter 14). Then add the old TTL to the moment the last of them switched. After that moment, any cache still serving the old answer is breaking the rules: a floor, a stuck application, or a forwarder that ignores TTLs. If you need one particular public resolver to catch up sooner, some operators offer a form to flush a single name, such as Google Public DNS and Cloudflare's 1.1.1.1.

You seeWhat it meansWhat to do
Old answer, TTL counting downA cache honoring the old TTLWait the number of seconds shown
Old answer, TTL larger than the old record'sA cache with a minimum-TTL floor, or a stale answer served while the source was unreachableCheck your authoritative servers are up; otherwise only the cache's operator can help
NXDOMAIN for a name you just createdA cached negative answer, from someone who asked too earlyWait up to min(SOA TTL, MINIMUM), capped by the resolver
SERVFAIL, then success minutes laterA cached resolution failure from an outage (RFC 9520)Wait up to 5 minutes
New answer everywhere except one machineAn application or OS cache, or a long-lived connectionFlush the OS cache, restart the program
Cheat sheet

Terms from this chapter

TTL
Time to live: the number of seconds a record may be cached. A maximum, not a minimum; caches count it down and hand out the time remaining.
Propagation
The popular name for the delay before a DNS change is seen everywhere. Nothing is sent outward: it is the time it takes for cached copies of the old answer to expire.
RRset
All the records with the same name, class and type. They are cached together and must share one TTL (RFC 2181 §5.2).
Negative caching
Remembering that a name or record does not exist, for a time set by the zone's SOA record (RFC 2308).
NXDOMAIN
Response code 3: the name does not exist, and neither does anything under it (RFC 8020).
NODATA
The name exists but has no records of the type asked for. Sent as NOERROR with an empty answer section.
Negative TTL
How long a negative answer may be cached: the lower of the SOA record's TTL and its MINIMUM field.
Serve-stale
Answering from an expired cache entry when the authoritative servers cannot be reached, with a short TTL such as 30 seconds (RFC 8767).
Prefetching
A resolver refreshing a popular record shortly before it expires so users never wait for a cache miss.
TTL floor
A resolver setting that keeps answers for at least a fixed time even when their TTL is shorter. One of the ways a cache outlasts your plan.
Where this comes from

Sources