Design a distributed rate limiting service that protects APIs from abuse and overload. Support multiple algorithms (token bucket, sliding window), per-user and per-API limits, distributed counting across multiple data centers, and real-time configuration changes at the scale of millions of requests per second.
What to design
What are the core functional requirements for a distributed rate limiter?
What are the non-functional requirements? Consider decision latency, accuracy, and availability.
Design the APIs — rate limit check (inline), rule management, and usage queries.
Explain the token bucket algorithm and when you would use it.
Explain the sliding window algorithms and how they compare to fixed windows.
How would you implement distributed rate limiting across multiple servers?
How would you handle rate limiting across multiple data centers?
How would you propagate rule changes in real-time to all rate limiter nodes?
How should the rate limiter behave when it itself is experiencing failures?