Webhook deliveries failing for past 2 hours · stripe.subscription.updated
Hey team — our webhook deliveries for stripe.subscription.updated have been failing for the past 2 hours. We're seeing 503s in our logs and our billing dashboard is now showing stale data. This is affecting customer onboarding.
I've attached the last 20 webhook attempts from our side — every single one is failing with the same 503.
Found a related incident: INC-2418 · we rotated the webhook signing key on May 18 at 23:14 UTC for ~12% of customers in cohort C. Helix Bio is in cohort C.
Suggested first action: regenerate webhook secret in customer's dashboard and provide the new value over the phone (per security policy).
Hi Jordan — thanks for the detailed logs, that helps. Looks like this is related to the webhook signing key rotation we did last night for a subset of customers. We'll get you a new secret now.
Calling you in 2 min on the number we have on file (415) ●●●-●●38 to share the secret over the phone per our policy. Standby.
Got it — standing by. Phone is on.
Hey Jordan — just called and left a voicemail with the new webhook secret. Confirm receipt & let me know when re-deployed. We'll keep this ticket open until your queue clears.