HN Debrief

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

  • Infrastructure
  • Programming
  • Developer Tools
  • Networking

Cloudflare’s post explains how it reduced memory use in the cache for 1.1.1.1, its public recursive DNS resolver, by stripping out structure it no longer needed after insert time. Cached DNS responses were being held in rich Rust objects built for mutability and convenience, even though once a response entered cache it was effectively read-only and mostly needed to be sent back out on the wire. The big changes were straightforward but high leverage: replacing `Vec` with boxed slices where capacity was useless, storing DNS records in a compact byte encoding instead of parsed structs, and deduplicating repeated owner names. At Cloudflare’s scale, that turned into roughly 100 TB less RAM across the fleet.

If you run a high-volume service, revisit any hot-path data that is written once and read many times. The easiest win may be changing the representation, not the algorithm, and those wins can stay invisible for years until scale makes them finance-level numbers.

Discussion mood

Mostly positive and appreciative. People liked seeing old-school systems engineering produce a huge real-world cost win, with some irritation that the optimizations looked obvious in hindsight and should perhaps have happened sooner.

Key insights

  1. 01

    Why obvious wins sit in production

    What looks trivial in a postmortem is often the part you defer on purpose when launching infrastructure. A recursive resolver has to be correct under ugly edge cases, survive attacks, and be fast enough everywhere. Saving a few gigabytes per machine loses to that until the service is stable enough that memory waste becomes worth migration risk and engineering time.

    When you review a mature system, separate “bad design” from “deliberate deferral.” Keep a list of scale-triggered rewrites tied to concrete thresholds so obvious cleanup does not wait for a blog-post-sized number.

      Attribution:
    • 0xAstro #1
    • yieldcrv #1
    • sophacles #1 #2
  2. 02

    The in-memory format was the real optimization target

    The useful framing here is not “use smaller containers.” It is that native object graphs are optimized for mutation and random access, while this workload mostly wanted compact storage and cheap replay to the network. Once you see the cache as a specialized serialization problem, the decision to store raw bytes instead of parsed records becomes the main win, and the better locality can make it faster as well as smaller.

    Audit hot-path state by asking what operations actually happen after creation. If objects are mostly immutable and repeatedly emitted, prototype a packed representation before chasing allocator or algorithm tweaks.

      Attribution:
    • edflsafoiewq #1
    • rfgplk #1
    • 9bot #1
  3. 03

    Why a DNS resolver needs huge caches

    The cache is not replacing some internal database that Cloudflare could simply centralize onto flash. A recursive resolver learns records on demand from authoritative name servers run by everyone else on the internet, then holds them only for their allowed lifetime. Large aggregate RAM use is the natural result of many edge servers caching a slice of global DNS close to users, which is exactly how DNS avoids hammering origins and hides slow or flaky authoritative servers.

    Do not read “100 TB cache” as evidence the architecture is wrong. In distributed edge systems, duplicated local state can be cheaper than adding a slower shared tier, especially when the source of truth is remote and inconsistent.

      Attribution:
    • mannyv #1
    • bastawhiz #1
    • toast0 #1
    • otterley #1
    • eggnet #1
  4. 04

    Rust can do this, but not gracefully

    Several comments zeroed in on the language tradeoff. Rust does support many low-level layout tricks, but packed variable-sized records and arena-style storage fit awkwardly with its standard containers and type system. The result is not that Rust blocks this kind of work. It is that the shortest path often involves more custom machinery than in C or Zig, especially when you want dynamically sized payloads inside generic collections like `HashMap`.

    If you pick Rust for infrastructure, expect some classes of memory-layout optimization to arrive later and cost more engineering effort. That does not negate the choice, but it should affect staffing and timeline assumptions for very high-scale services.

      Attribution:
    • irdc #1
    • mkeeter #1
    • f311a #1
    • afdbcreid #1
    • cakoose #1
    • esterna #1
  5. 05

    Packed layouts shift safety to API design

    Collapsing multiple logical arrays into one byte buffer does not automatically throw away Rust’s safety model, but the safety stops coming for free. Once correctness depends on offsets and internal invariants, the protection moves from the container to the wrapper API. Private fields and slice-returning methods can preserve safe use at the edges, but only if someone does the extra design work.

    When you introduce a custom packed layout, budget time for the safe abstraction around it. The optimization is only finished when misuse is hard or impossible for the next engineer.

      Attribution:
    • vinkelhake #1
    • ratorx #1
    • afdbcreid #1
    • FpUser #1
    • asgraham #1
  6. 06

    This cache layout mirrors network protocol design

    The byte format Cloudflare chose is basically the same type-length-value pattern used all over networking. That is a good sign, not a coincidence. TLV-style encodings let a reader step through variable-sized records, estimate needed resources, and skip data without inflating it into a larger object graph first. For serving wire protocols, borrowing wire-format ideas for memory layout is often the shortest route to density.

    For protocol-heavy systems, look at how the protocol itself solves compactness and partial parsing. Those patterns often transfer directly into efficient in-memory cache formats.

      Attribution:
    • OptionOfT #1
    • jandrewrogers #1
    • pocksuppet #1

Against the grain

  1. 01

    Resolvers are not as bound to TTLs as presented

    One pushback was that DNS caching rules are looser than the simplified story suggests. RFC 8767 allows serving stale records under failure conditions, and a commenter argued that caches can refetch more aggressively or retain data longer internally as long as they do not misrepresent TTL on output. That does not make “ignore TTLs” a good idea for a public resolver, but it does undercut the notion that the cache’s behavior is mechanically fixed by record owners.

    If you work on resolver infrastructure, distinguish protocol compliance, internal retention, and customer trust. There is more design space than “strict TTL or broken DNS,” but public behavior still has product consequences.

      Attribution:
    • inopinatus #1
    • ButlerianJihad #1
    • seiferteric #1
    • pbhjpbhj #1
    • otterley #1
  2. 02

    Allocator and arena choices may dwarf container tweaks

    A few low-level comments argued that the post may still be leaving memory and performance on the table by sticking with conventional allocator-driven layouts. They would have reached for large virtual-memory reservations, demand paging, or more intrusive layouts earlier, especially in a service with predictable lifetime and read patterns. That view makes Cloudflare’s changes look solid but still conservative.

    After obvious representation wins, check whether your allocator strategy matches object lifetime and access patterns. In systems code, page-level layout decisions can matter as much as per-object fields.

      Attribution:
    • irdc #1
    • cobalt #1
    • senderista #1

In plain english

1.1.1.1 ↩
Cloudflare’s public DNS resolver service, which answers DNS lookups for users and applications.
allocator ↩
The part of a runtime or library that manages requests for dynamic memory allocation.
arena ↩
A region or shard of allocator-managed memory, often used so multiple threads can allocate with less contention.
C ↩
A low-level programming language widely used for operating systems, networking, and other systems software.
demand paging ↩
An operating system technique where memory pages are only loaded or backed when they are first accessed.
DNS ↩
Domain Name System, the internet service that translates website names into network addresses.
HashMap ↩
A data structure that stores key-value pairs and looks them up using a hash function.
locality ↩
The tendency for data close together in memory to be accessed together, which usually improves CPU cache efficiency.
pointer chasing ↩
Following references from one memory location to another, which often hurts performance because data is scattered.
RAM ↩
Random Access Memory, the short-term working memory a computer uses while running programs.
recursive DNS resolver ↩
A DNS server that fetches answers from other DNS servers on behalf of a client and caches the results.
RFC 8767 ↩
An Internet Engineering Task Force specification that permits DNS resolvers to serve stale cached data in certain failure cases.
Rust ↩
A systems programming language focused on memory safety and performance.
TLV ↩
Type-Length-Value, a compact encoding format where each field carries its type, size, and raw content.
TTL ↩
Time To Live, the maximum time a DNS record may be cached before it should be refreshed.
Vec ↩
Rust’s growable array type, which stores a pointer, a length, and a capacity.
Zig ↩
A low-level programming language designed for systems programming with manual control over memory layout and allocation.

Reference links

Rust memory layout references

Related systems and protocol patterns

  • MaraDNS memory optimization note
    An older example of dramatic memory savings from replacing many small allocations with one large block in a DNS-related workload.
  • Netlink man page
    Referenced as another protocol family that uses length-prefixed binary structures similar to Cloudflare’s packed cache records.
  • Type-Length-Value on Wikipedia
    Background on the TLV encoding pattern several comments said Cloudflare’s design resembles.

DNS and captive portal references

Broader references