Surviving the Self-Inflicted DDoS

Stop looking for hackers.

When your backend catches fire this week, it probably won't be a hacker. It'll be a teammate's script missing a sleep() call, a useEffect that fires on every keystroke, or a phone app retrying a failed request three times in the same second because nobody thought about what happens when the server says no. The call is coming from inside the house, and most of the time, the house built it by accident.

That's what rate limiting is actually for. Treated as a security checkbox — something you bolt on right before launch — it's easy to skip, which is exactly why it so often gets skipped. Treated as a shock absorber, it starts to make more sense: the thing that keeps one bad script or one panicked retry loop from taking the whole system down with it.

Here's the mechanism, in miniature. One noisy endpoint starts holding onto database connections a little longer than it should — not crashing, just slow. Under enough concurrent load, that slowness compounds: connections get requested faster than they're released, the pool drains, and everything behind it starts to queue. It doesn't matter that /health and /login are fast, lightweight routes at that point — they're stuck in the same line, and they start timing out too. From the outside, it looks like the whole platform is down. From the inside, it's one endpoint that never learned how to say not right now.

Three algorithms, and when each one earns its keep

Rate limiting sounds like it should be a whole infrastructure decision, but nearly every real system comes down to a choice between three patterns — and the choice mostly comes down to how you feel about bursts.

The simplest is the fixed window: pick a limit, say 100 requests a minute, and reset the counter every time the clock ticks over. In Redis this is about two lines — INCR the key, EXPIRE it after 60 seconds — which is exactly why it's so tempting to reach for first. The trouble shows up at the edges. A client that sends 100 requests at 12:00:59 and another 100 at 12:01:01 has pushed 200 through in two real seconds, and as far as either window's concerned, nothing went wrong. Your database will see it differently. Fixed windows are fine for internal tools and prototypes, where a burst costs nothing, and risky anywhere it does.

The token bucket fixes that edge problem by changing what "allowed" means. Picture a bucket that holds ten tokens and refills by one every second; every request spends a token, and an empty bucket means no request gets through. What makes it pleasant to use is that idle time becomes credit — a client that's been quiet shows up with a full bucket, so it can fire off ten requests at once to paint a dashboard, and whoever's using it never notices a delay. Once the bucket's spent, the client settles into the steady pace of the refill. Bursts feel free, sustained abuse doesn't — which is why it's the default most cloud APIs reach for.

Sometimes, though, even a brief burst is more than you can afford, because whatever sits behind your API is more fragile than your API is. That's where the leaky bucket earns its keep: requests can arrive in a rush, but they leave at a fixed, unhurried pace, and anything that doesn't fit just spills over and gets dropped. There's no such thing as a burst on the far side of a leaky bucket, which matters when what's downstream is a legacy payment gateway or an old database that chokes the moment traffic spikes. Put simply: token bucket protects your API. Leaky bucket protects whatever your API talks to.

What breaks once real traffic shows up

Building a rate limiter against a local test script is easy. The interesting failures only show up once real users and real networks get involved.

The first one catches almost everyone: keying the limit on req.ip. It seems obvious right up until you remember how many people can share one IP address. A corporate office might have hundreds of employees behind a single public IP, so a limit meant for one bad actor can lock out an entire company. Mobile traffic is worse — carrier-grade NAT puts thousands of phones behind the same small pool of addresses, so a limit tuned for "one user" ends up throttling a whole slice of a city. The fix is to save IP-based limits for routes where you genuinely have nothing else to key on — login, signup, password reset — and use a real identity, like a user ID, API key, or org ID, everywhere a request is authenticated.

Fix that, and the next failure only shows up once you're running more than one server instance against a shared Redis cache. Naive rate-limiting code checks the count, decides, then writes — three separate steps — and under concurrent load, two servers can race straight through the gap between them:

Server A reads count: 99
Server B reads count: 99
Server A: under 100, increments to 100, allows
Server B: under 100, increments to 101, allows

That's one extra request slipping through a limit of 100, and under real concurrency it's rarely just one — it's dozens, all sneaking through the same narrow window. The way around it is to never let the check and the write happen as two separate steps: a Redis Lua script, or an atomic INCR that returns the new count directly, closes the gap by removing it.

Solve that one, and there's a third failure waiting, quieter than the first two because it only shows up at real scale. If your service takes 20,000 requests a second and every single one means a network round-trip to Redis just to read a counter, the rate limiter itself becomes the slowest thing in the request path — you've rebuilt the very outage you were trying to prevent, just with extra steps. What works is keeping hot counters in a local in-memory cache and syncing back to Redis periodically, or only once a client gets close to its limit. You lose a little precision — a client might sneak a few extra requests through during a sync window — but a rate limiter's job is to be a guardrail, not a ledger, and that trade is almost always worth making.

Saying no without making things worse

Getting the "no" right matters as much as deciding when to say it. A bare 429 Too Many Requests, with nothing else attached, tells a client library nothing about when it's safe to try again — so most of them guess, and guess badly, hammering the server every 50 milliseconds to check whether the door's open yet. That's a second flood arriving right on top of the first one, self-inflicted all over again.

The fix is to make the response talk:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 30
Retry-After: 30

{
  "error": "rate_limit_exceeded",
  "message": "Quota exceeded. Retry in 30 seconds."
}

Retry-After does most of the work — it's the header client libraries are most likely to already respect, and it's what turns a frantic retry loop into a single scheduled one. RateLimit-Reset fills in the rest, though it's worth getting right: the IETF draft defines it as seconds remaining, not a Unix timestamp — that convention belongs to the older, unofficial X-RateLimit-Reset — and mixing the two formats is a reliable way to confuse whatever's on the other end. (The draft has since folded the three headers into two, RateLimit and RateLimit-Policy, though the version above is still what most APIs actually send.)

Defense in depth

None of this has to live entirely in your application code. A request headed for trouble can be caught at the edge, at the gateway, or in the app itself, and each layer is good at catching a different kind of problem:

LayerWhat it stopsTypical toolsEdge (CDN / WAF)Volumetric floods, scrapers, abusive IPsCloudflare, AWS WAFAPI gatewayPer-route limits, API key quotas, plan tiersEnvoy, Kong, TraefikApplicationBusiness logic — credit balances, expensive jobsRedis + middleware

The edge has no idea what a "credit balance" is, and it shouldn't have to — that's what the application layer is for. Let each layer catch what it's actually positioned to catch, and none of them has to work as hard as it would alone.

Rate limiting isn't a defense against your users. It's what keeps one enthusiastic client, or one honest mistake in a for-loop, from taking the whole platform down with it. Build it early, make it atomic, and always tell people when it's safe to come back.