Webhook reliability: retries, idempotency, and dead-letter queues
The younifyd team
July 8, 2026 · younifyd.com
Webhooks look simple — an HTTP POST when something happens — and that simplicity is a trap. Providers deliver at-least-once, which means duplicates. They deliver out of order. They burst: a bulk import upstream becomes ten thousand deliveries at your endpoint in a minute. And when your handler errors, most providers retry a few times over a few hours, then silently drop the event. Here are the five practices that separate webhook consumers that quietly lose orders from ones that survive production.
1. Acknowledge fast, process async
The most common failure mode is doing real work inside the webhook handler. The provider's timeout is typically 5–10 seconds; your CRM write plus email send plus ERP update won't always fit, and every timeout counts as a failed delivery against you. The fix: validate the signature, enqueue the payload, return 200 in milliseconds, and process from the queue. Your ingestion capacity is now decoupled from your processing capacity — bursts fill the queue instead of returning 503s.
2. Treat every event as a duplicate
At-least-once delivery guarantees you will eventually process the same event twice — after a timeout the provider retries a delivery your system actually completed. Idempotency is the defence: derive a key from the provider's event ID, record processed keys, and skip repeats. For handlers that write downstream, pass the key through so the downstream write is idempotent too. Without this, retries — the mechanism that makes webhooks reliable — become the mechanism that double-charges customers.
3. Retry with backoff, then dead-letter
When processing fails — a downstream 500, a rate limit — retry with exponential backoff and jitter rather than immediately or never. But bounded retries must end somewhere that isn't the void: a dead-letter queue. Events that exhaust their retries land in the DLQ with their payload and error history, where a human can inspect, fix the cause, and replay them. A DLQ turns 'we think we lost some orders in March' into a queue with three inspectable items in it.
4. Don't assume order
Providers rarely guarantee ordering, and retries scramble whatever order existed. If you apply an 'order.updated' event that arrived before the 'order.created' it depends on, you corrupt state. Two defences: prefer fetching current state from the provider's API on receipt (the event is a hint, the API is the truth), or track per-entity versions and park out-of-order events for reprocessing. The fetch-on-event approach is simpler and right for most teams.
5. Make deliveries observable
When a customer asks why their order didn't sync, you need per-event answers: when it arrived, what the payload was, which processing steps ran, where it failed, when it was retried. That's an execution timeline per delivery — not a grep through application logs. In younifyd, webhook triggers get all of the above out of the box: signature verification, a managed queue with back-pressure, per-step retries, a browsable dead-letter queue with one-click replay, and a full timeline for every event that ever hit the endpoint.
Make your webhooks boring
Managed queueing, retries, DLQ, and per-event timelines — configured, not coded.
Try for Free