Cybersecurity
Rate Limiting and Abuse Prevention for Your Own API
Updated October 16, 2018By the CalliArc team
Key takeaway
Rate limit per authenticated account rather than per IP address, return 429 with a Retry-After header so good clients back off correctly, and apply much stricter limits to the endpoints attackers care about — login, password reset, and anything that sends a message or costs money.
Rate limiting is usually added after an incident: a partner's runaway script, a scraper, or a credential-stuffing attack against the login endpoint. It's considerably cheaper to design in.
What to key the limit on
- Authenticated account or API key — the right default, because it survives changing IP addresses and matches your commercial terms.
- IP address for unauthenticated endpoints, with care: corporate networks and mobile carriers share addresses, so a strict per-IP limit blocks legitimate users in groups.
- A combination for sensitive endpoints: per account and per IP, whichever is hit first.
- Per-endpoint limits, because listing records and sending emails have very different risk profiles.
Algorithms, briefly
- Fixed window — simplest, but allows a double burst around the window boundary.
- Sliding window — smoother and fair, slightly more state to keep.
- Token bucket — allows a controlled burst then settles to a steady rate. Usually the best fit for APIs, because real clients are bursty.
- Whichever you choose, keep counters in a shared store; per-instance limits mean the effective limit multiplies by your server count.
Respond in a way clients can handle
- Return 429 Too Many Requests, not a generic error — clients have specific handling for it.
- Include Retry-After, and headers showing the limit, remaining quota, and reset time.
- Document the limits publicly. Undocumented limits produce support tickets and angry integrators.
- Consider warning before enforcing for a new limit — log what would have been blocked for a week first, so you don't discover your busiest customer was always over it.
Protect the endpoints that matter most
- Login and password reset — strict per-account and per-IP limits, plus progressive delays and lockout on repeated failure.
- Anything that sends email or SMS, because abuse there costs you money and your sending reputation.
- Signup and free-tier provisioning, which is where automated abuse concentrates.
- Expensive queries, reports, and exports — a per-account concurrency limit often protects the database better than a request-rate limit.
- Put a layer in front — a WAF or CDN rule — so volumetric attacks are absorbed before they reach your application at all.