Webhooks

Webhooks arrive twice, out of order, or never

Every payment provider sends webhooks at least once, which is not the same as exactly once. Two rules do most of the work: verify the raw bytes, and make the handler idempotent. This is the verification section of chapter 39, in full.

Every provider's docs say the same three things in different words: we retry until you return a 2xx, we don't promise order, and we may send the same event twice. What they don't share is the schedule. Stripe gives up after three days, Square after 24 hours, and Adyen keeps trying for up to 30 days (Stripe, Square, and Adyen docs, 2026). Those numbers set the size of your idempotency store, the length of your replay window, and how late a "late" event can be.

Webhook delivery states, from received through verified and deduplicated to processed, with the retry path

How Five Providers Deliver

The handler code later in this section is provider-agnostic on purpose. The numbers it runs on aren't. The two tables below are what each provider's documentation said in September 2026; retry schedules change without an API version bump, so check the docs before you hard-code any of them. Where a provider's docs don't state a policy, the cell says so instead of guessing.

ProviderRetry schedule and windowOrderingHow duplicates present
StripeLive: exponential backoff for up to 3 days. Sandbox: 3 tries over a few hours. Manual resend for 15 days in the Dashboard, 30 days via the CLINot guaranteed. created has one-second resolution, so don't order by itSame event.id on every retry. Occasionally two distinct events for one change; dedupe those on data.object.id plus type
PayPalUp to 25 attempts over 3 days, then marked Failed. Manual resend from the Webhook Events dashboardNot stated in the docs. Treat as unorderedRetries redeliver the event; dedupe on the event id
PaddleLive: 60 attempts over 3 days, 20 of them in the first hour. Sandbox: 3 attempts in 15 minutes. Respond 200 within 5 secondsNot guaranteed. Docs say to store occurred_at and compare before applying a changeSame event may arrive more than once; dedupe on the event ID, not the ntf_ notification ID
AdyenThree quick tries at 9, 18, and 27 seconds, then a per-endpoint queue that backs off to 8-hour intervals and keeps trying for up to 30 days. Acknowledge with a 2xx within 10 secondsDocs tell you to order by timestamp yourself; some webhook types carry a sequenceNumberDuplicates share eventCode and pspReference; eventDate can differ. Adyen says use the latest
Square11 attempts over 24 hours, backing off from 1 minute to 8 hours, then discarded. square-retry-number counts resendsNot guaranteedMay be sent more than once; dedupe on event_id
ProviderSignature scheme and headerReplay protectionLocal testing
StripeHMAC-SHA256 over t.body, hex, in Stripe-Signature: t=...,v1=.... Several v1 values for up to 24 hours after you roll the secrett is in the signed string; libraries default to a 5-minute tolerance. Each retry gets a fresh t and signaturestripe listen --forward-to for a signed tunnel, stripe trigger to fire events
PayPalCertificate-signed; five PAYPAL-* headers plus your webhook_id, checked by calling verify-webhook-signature. Self-verification against the cert and a CRC32 of the body is documentedTransmission time is in the signed string; the docs don't say how strictly the API checks itWebhooks simulator in the Developer Dashboard, a sandbox app, resend from Webhook Events
PaddleHMAC-SHA256 over ts:body, hex, in Paddle-Signature: ts=...;h1=.... More than one h1 during rotationts is in the signed string; Paddle's default tolerance is 5 secondsPaddle > Developer tools > Simulations sends signed events; pair with ngrok or the Hookdeck CLI
AdyenBase64 HMAC-SHA256 over eight colon-joined fields, not the body, inside additionalData.hmacSignature. Hex key from the Customer AreaNone in the signature. Your idempotency store and eventDate are all you haveCustomer Area > Developers > Webhooks > Test configuration; Troubleshoot > Retry to resend a failed event
SquareBase64 HMAC-SHA256 over notification_url + body in x-square-hmacsha256-signatureNone in the signature. Idempotency store onlyDeveloper Console > subscription > More > Send test event, or POST /v2/webhooks/subscriptions/{id}/test; square-environment header says Sandbox or Production

Sources: Stripe, PayPal, Paddle, Adyen, and Square webhook documentation, 2026.

Two things in those tables should change your design. First, four of the five retry windows are longer than the 48-hour Redis TTL in the code below, and Adyen's runs right up to the 30-day database retention. A retry that lands after your Redis key expires has to hit the database check or it gets processed twice. Second, Adyen and Square put no timestamp in what they sign, so a captured request stays valid forever. For those two, the idempotency store is your only replay protection.

Always verify signatures against the exact raw request bytes you received from the wire. Never json.loads(...) then json.dumps(...) and verify the re-serialized result - Python, Go, and Node all reorder keys, change whitespace, and normalize numbers (1.0 vs 1, "foo" vs "foo") differently from the sending provider. Even sort_keys=True won't save you. The signature was computed over the bytes the provider sent; verify against those bytes or expect silent rejection of every webhook in production.

In Flask use request.get_data() (not request.json); in FastAPI use await request.body(); in Express use the raw body-parser before any JSON middleware.

Idempotency Store Design

The key is the provider's stable event ID with the provider name in front, provider:event_id. Never a hash of the payload, because re-serialization changes the bytes and you'd stop catching duplicates without noticing. Never the payment or subscription ID either, because one payment intent produces several events and you'd drop all but the first. The prefix keeps two providers' ID spaces apart and makes the row readable when you're querying the table by hand at 3 AM.

The store has two layers because they fail differently. Redis is the fast path and expires keys after 48 hours; the database is slower and keeps rows for 30 days. A duplicate inside 48 hours never touches the database. A retry on day three, which Stripe, PayPal, and Paddle all still send, misses Redis and hits the database row instead.

If Redis is flushed, evicted, or replaced during a deploy, nothing is lost, only slowed. If the database check itself errors, fail closed: return a 500 so the provider retries later, and don't process. The one thing you must not do is treat "couldn't check" as "not seen".

The check-then-act in already_processed and mark_processed has a gap. Two deliveries of the same event at the same moment, on two workers, both pass the check and both process. Close it with an atomic claim: Redis SETNX so the first writer wins, and the primary key on the idempotency key in the database so the second insert fails instead of succeeding twice. Chapter 9's Stripe handler does this with event_store.claim(). Stripe's own guidance goes one further and records two marks, processing and then processed, so a crash mid-handler leaves a visible half-finished row rather than a silently lost event.

What happens at expiry is the design decision most teams never make. After 30 days the cleanup task deletes the database row, and the same event arriving after that is processed as new. For Stripe, PayPal, Paddle, and Square that can't be an automatic retry, because their schedules end at three days or less. It can be a manual resend from a dashboard, or an Adyen retry on its last day. Decide which you want: handlers that are safe to re-run, which Chapter 5 covers, or retention that outlasts the longest retry window you accept, with margin.

What the book adds

The rest of chapter 39

  • The deduplication schema
  • Ordering by event timestamp
  • Replay handling
  • Dead-letter queues
  • Monitoring, and silent delivery failures

30-day money-back · Instant PDF and EPUB

All 42 chapters