HN Debrief

Where did the old web go? We followed 657,607 links to find out

  • Web
  • Infrastructure
  • Open Source
  • Developer Tools
  • AI

The post comes from the creator of 0.mk, a URL shortener that ran from 2009 to 2014 and was recently restored from an old database backup. Using that recovered corpus, the author crawled 657,607 historical links and found that roughly three quarters no longer return a loading page. Even after removing duplicate destinations, the failure rate stayed almost the same, so this was not a quirk of a few heavily reused URLs. The author is careful about the wording. "Did not load" includes hard failures and also pages that may now block crawlers, while links that still return 2xx or 3xx count as live even if they now point to parked domains, login walls, or removed content. The result is not a clean census of the whole internet because the service was concentrated in one regional community, but it still lands as a stark sample of how quickly shared web history disappears.

Treat any external URL you depend on as fragile infrastructure. If a link matters to your product, docs, research, or compliance trail, archive it, snapshot it, or store the content you are allowed to keep instead of assuming the web will stay put.

Discussion mood

Nostalgic and frustrated. People largely accepted the post's core result and used it to complain about centralization, broken URL permanence, and how modern web incentives make preservation worse, even if some rolled their eyes at romanticizing the early web or at calling 2009 to 2014 the "old web."

Key insights

  1. 01

    What disappeared was independent community space

    The loss people actually care about is not Geocities-style design. It is the disappearance of self-governed places like local gaming forums, IRC servers, LiveJournal circles, personal homepages, and small standalone sites. Reddit, Discord, Slack, Facebook, and Instagram did not just offer nicer interfaces. They absorbed the social layer of the web into systems that are easier to monetize and easier to shut off, which means the web lost a lot of niche knowledge, local memory, and community continuity even when "content" still exists somewhere.

    If your company or community depends on discussion, documentation, or user history, do not assume a large platform is a durable home. Own an export path and a canonical archive you control.

      Attribution:
    • greazy #1
    • morganf #1
    • clickety_clack #1
    • ajsnigrutin #1
  2. 02

    URL permanence quietly died as a norm

    People used to treat a URL as a stable identifier and expected at least a 301 redirect when something moved. That norm collapsed. Sites now reorganize, redesign, or migrate with little regard for incoming links, which turns the web from an interconnected document system into a stream of temporary pointers. The practical damage is bigger than nostalgia. Bookmarks fail, citations decay, and accumulated references stop compounding.

    When you redesign docs or ship a migration, preserve old paths and add redirects as a product requirement. Measure broken inbound links the same way you would measure API breakage.

      Attribution:
    • ryandrake #1
    • z_rho_one #1
  3. 03

    Modern web architecture is harder to preserve

    Static HTML is easy to mirror and keep usable for decades. Today's web is often hidden behind login walls, paywalls, heavy client-side rendering, and aggressive blocking layers, so even pages that still "exist" are harder to crawl, archive, or replay. Content-addressed systems like InterPlanetary File System help with integrity and replication, but they do not solve the basic requirement that someone must keep hosting the bytes. Preservation got worse because the default publishing model got more brittle.

    For anything you may need to preserve, publish a simple static version alongside the dynamic one. If you rely on JavaScript-heavy delivery, assume future retrieval and compliance archiving will be weaker than you think.

      Attribution:
    • Gormo #1 #2
    • rcxdude #1
    • ChadNauseam #1
  4. 04

    Link rot is mostly incentives, not storage physics

    The interesting split was between physical data decay and operational neglect. Bit rot is real, especially on ordinary consumer systems without error correction, but the web loses far more material because nobody is paying to maintain servers, domains, software, moderation, and legal risk. Content-addressed storage, error-correcting codes, and replication can make corruption rare. They cannot make people care enough to keep something online. Preservation fails first on ownership and incentives, then on disks.

    Budget for stewardship, not just backup. Domains, maintainers, moderation, and migration plans are part of data retention, not optional extras.

      Attribution:
    • shevy-java #1
    • efskap #1
    • marginalia_nu #1 #2
    • tekne #1
  5. 05

    The rebuilt service is really a moderation economics test

    The revived 0.mk is not just a nostalgia project. The author explicitly says the original service died because there was no revenue and too much spam, abuse review, and support work, and now wants to see whether AI can push those operating costs low enough to make a tiny utility sustainable. That makes the post a live experiment in whether small consumer web services can come back if moderation and maintenance become cheap enough.

    Watch small services that retry old business models with AI-heavy operations. The key question is not product novelty. It is whether support and abuse handling costs have actually moved enough to change the unit economics.

      Attribution:
    • Lord_Zero #1
    • Gormo #1
    • tdx #1

Against the grain

  1. 01

    The old web was not objectively better

    A blunt pushback cut through the romance. Early web culture also meant ugly pages, scams, trolls, weak moderation, and lots of low-value content. If people now get what they need faster from YouTube, social feeds, or even AI answers, then the shift away from scattered personal sites may be less a cultural loss than a change in interface. That does not erase link rot, but it does challenge the idea that decentralization by itself produced better outcomes.

    Do not confuse independence with quality. If you are building outside large platforms, you still need discovery, usability, and trust or users will prefer more centralized tools.

      Attribution:
    • Razengan #1
  2. 02

    A smaller web may re-emerge after AI saturation

    One commenter argued that pervasive AI could push casual users deeper into closed platforms and leave the open internet to a smaller, more technical audience again. The bet is not on a mass return to the old web. It is on niche spaces, alternative networks, and harder-to-join communities becoming more attractive because they filter out spam, low-effort participation, and corporate pressure. Another commenter pushed back that these newer spaces often feel more gated than the exploratory web they replace.

    Expect open-web communities to get smaller and more intentional, not broadly mainstream again. Products for that audience should optimize for depth, ownership, and interoperability rather than viral growth.

      Attribution:
    • MetaWhirledPeas #1
    • goodmythical #1
    • Avicebron #1
  3. 03

    Modern design did improve some things

    Not everyone bought the nostalgia. One view was that the old web was clumsy and that modern design raised the baseline for readability and polish. The comeback was that many modern pages are so bloated they perform worse on fast broadband than old pages did on dial-up. That reframes the issue. The problem is not that the web evolved. It is that it often traded simplicity for needless overhead.

    Take performance as a first-class design constraint. Better visual design is not a win if it costs reliability, load time, and archivability.

      Attribution:
    • prmoustache #1
    • juleiie #1

In plain english

2xx ↩
A family of HTTP status codes that usually indicate a request succeeded.
301 ↩
An HTTP status code meaning a resource has permanently moved to a new URL.
3xx ↩
A family of HTTP status codes that usually indicate a redirect to another URL.
Client-side JavaScript ↩
Code that runs in the user's browser to render or control a webpage instead of serving the full page directly from the server.
Content-addressed ↩
A way of naming data by a hash of its contents rather than by where it is stored.
InterPlanetary File System ↩
A peer-to-peer system for sharing files by content hash so copies can be served from multiple machines.
IRC ↩
Internet Relay Chat, an older real-time text chat system often used in technical communities.
PHPBB ↩
A widely used open source forum software package popular for self-hosted discussion boards.
URL ↩
Uniform Resource Locator, the web address used to locate a page or file online.

Reference links

Web history and preservation references

Protocols and alternative networks

Example old-web sites still online

Cultural references