HN Debrief

The Valley of Webhooks

  • Infrastructure
  • APIs
  • Developer Tools
  • Programming

The post says webhooks work fine for side effects like “send a receipt” but fall apart when they are used as the primary mechanism to mirror a provider’s state into your own database. It walks through the usual failure modes: duplicate deliveries, missing events, hard bootstrap problems, poor local development ergonomics, and the ugly reality that consumers end up building dedup tables, replay jobs, cron backfills, and hand-rolled recovery logic anyway. The proposed fix is SCROLL, a draft spec for a cursor-based event feed that can be polled or streamed so consumers can ask for everything after a known point and rebuild state deterministically.

If your product exposes webhooks, add a cursor-based events or state-diff API before promising reliable sync. If you consume webhooks, stop treating them as authoritative data delivery and build reconciliation against the provider’s durable log or current state.

Discussion mood

Broadly sympathetic to the problem statement and skeptical of webhooks as a source-of-truth sync mechanism. The mood turns pragmatic around the proposed solution: people want durable logs, polling, and reconciliation more than a brand new protocol, and several were put off by what they saw as LLM-polished writing around a not-especially-novel idea.

Key insights

  1. 01

    Reconciliation should be the main path

    Building the disaster recovery path first changes the whole architecture. If you can always compare remote state or replay a durable change log, that same mechanism can run continuously in production instead of sitting unused until the day you need it and discover it is broken. Webhooks then become a latency optimization that triggers reconciliation sooner, not the thing your correctness depends on.

    Design one sync loop that can recover from missed events, outages, and bad deploys without special-case tooling. Exercise that path continuously so recovery is routine behavior, not an emergency feature.

      Attribution:
    • qlkzy #1
    • jallmann #1
    • cyberax #1
  2. 02

    Webhook as poke plus history API

    The cleaner pattern is to separate notification from data transfer. Gmail was called out as the concrete example: push tells you there is new history, then a cursorable history endpoint tells you exactly what changed since your last checkpoint. That avoids dedup and retry drama on the notification channel, keeps polling logic simple, and still gives low-latency updates.

    If you own an integration surface, ship a pollable history or change-data-capture endpoint and make webhook payloads minimal. If you are consuming an API, prefer providers that expose both a change log and deletions, not just event callbacks.

      Attribution:
    • tlonny #1
    • evolve-maz #1
    • AgentME #1
    • throwaway7783 #1
  3. 03

    Request failure does not prove no write happened

    One commenter’s QuickBooks experience highlighted a nasty but common integration fact. A create call can fail from the client’s point of view after the provider already committed the write, so treating an error response as “nothing happened” is unsafe. The useful distinction is between network ambiguity and true application-level rejection, but many APIs blur that line badly enough that callers must verify resulting state either way.

    Use idempotency keys where available and add post-write verification for any external system that matters financially or operationally. Your error handling should branch on ambiguous transport failure versus confirmed business-rule rejection.

      Attribution:
    • alt227 #1
    • lelanthran #1
    • tux3 #1
  4. 04

    The protocol idea already has neighbors

    SCROLL did not read as a greenfield invention. People linked it to Braid-HTTP Subscriptions at the IETF, Linked Data Event Streams, CouchDB replication, and existing industry APIs like RESO. That does not weaken the need. It suggests the opportunity is standardizing familiar replication patterns over plain HTTP rather than inventing a brand new conceptual model.

    Before drafting a bespoke sync protocol, survey adjacent standards and proven product patterns. Compatibility with existing HTTP tooling and prior art will matter more than novelty if you want adoption.

      Attribution:
    • toomim #1
    • Joeri #1
    • WorldMaker #1
    • bobtheborg #1
    • delusional #1
  5. 05

    Streaming is cheaper than it sounds

    The objection that one open connection per consumer is inherently too expensive got strong pushback. Mobile push systems and old Twitter firehose consumers were cited as proof that large fleets of long-lived connections are operationally normal when done well. The harder issue is not raw socket count. It is whether your CDN, hosting stack, and rate-limit model are built for long-lived internet-facing streams.

    Do not reject streaming feeds on intuition alone. Check whether your actual bottleneck is connection cost, or edge infrastructure and product-policy constraints like CDN timeouts and tenant fairness.

      Attribution:
    • bytesandbots #1
    • oasisbob #1
    • weli #1
  6. 06

    Webhooks win because buyers ask for them

    Providers are not only shipping webhooks out of ignorance. People who run webhook infrastructure said customers explicitly make buying decisions based on webhook availability, while Kafka, S3, and queue-based alternatives see lower adoption even when offered. The constraint is market habit and developer ergonomics, not just technical merit.

    If you sell into mainstream developer teams, keep webhooks in the product even if you also expose cleaner event streams. The winning move is offering a better backstop behind familiar onboarding, not assuming buyers will switch transport models on principle.

      Attribution:
    • zbentley #1 #2
    • tasn #1 #2

Against the grain

  1. 01

    A simple counter may solve most of this

    One practical objection was that the article overreaches toward persistent feeds when many of the stated problems disappear with a monotonically increasing sequence number in each webhook and an events API for backfill. That keeps delivery simple, avoids permanent open connections, and still lets consumers detect gaps and fetch missing records.

    If you own a webhook API, try sequence numbers plus backfill before designing a streaming protocol. You may get most of the reliability benefit with much less rollout risk.

      Attribution:
    • bytesandbots #1
  2. 02

    Just ship a downloadable SQLite snapshot

    Instead of pushing harder on event protocols, one commenter argued for a blunt but workable export model. A periodically refreshed SQLite database with stable row IDs would give consumers the provider’s actual current data in a form they can query directly, and let them compute their own deltas. That is clunky for real time, but very strong for correctness and bootstrap.

    For low-frequency or audit-heavy integrations, a durable snapshot export can be more valuable than elaborate event semantics. Consider batch state distribution as a first-class integration product, not just an internal backup.

      Attribution:
    • zie #1
  3. 03

    Pull has its own scaling and semantics limits

    Another pushback was that switching from push to pull does not erase the hard parts. Bulk change queries can be expensive for providers, especially on large datasets, and SCROLL still only covers part of true replication. You still need snapshots, ordered ranges, and a clean way to establish what history a client is entitled to see.

    When you design a pollable sync API, budget for provider-side indexing, retention, and authorization complexity up front. A cursor endpoint alone is not a complete replication story.

      Attribution:
    • sandeepkd #1
    • Elucalidavah #1

In plain english

Braid-HTTP Subscriptions ↩
A draft proposal to extend HTTP so clients can subscribe to resource updates over a long-lived response stream.
CDN ↩
Content Delivery Network, infrastructure that sits between users and servers to cache, route, and protect web traffic.
CouchDB replication ↩
The synchronization protocol used by CouchDB to copy document changes between databases.
HTTP ↩
Hypertext Transfer Protocol, the standard protocol used by web browsers and APIs to send requests and responses over the internet.
IETF ↩
Internet Engineering Task Force, the standards body that develops many core internet protocols such as HTTP.
Kafka ↩
Apache Kafka, a distributed system for storing and streaming ordered logs of events.
QuickBooks ↩
An accounting software platform whose APIs are often used to sync invoices, customers, and other financial records.
RESO ↩
Real Estate Standards Organization, which publishes API standards used in real estate listing systems.
S3 ↩
Amazon Simple Storage Service, an object storage system whose API is widely copied by other storage providers.
SCROLL ↩
The proposed protocol in the post for reading a cursor-based stream of changes over HTTP so clients can sync state.
Stripe ↩
A payments platform whose APIs and webhook tooling are widely used by software companies.
webhook ↩
An HTTP callback where one service sends another service a request when an event happens.

Reference links

Standards and protocol proposals

Industry examples and APIs

Background reading