Rate Limiting

The key parameter of rate limiting is burst tolerance (burst); setting it too strict will mistakenly penalize normal users' concurrent loading, while setting it too loose will not stop crawlers; in practice it usually requires combining log analysis, first observing the normal traffic baseline before setting the threshold, and pairing it with 429 status codes and Retry-After response headers to guide compliant crawlers to slow down.

Through reverse proxies such as Nginx or Caddy, the request frequency of a single IP or IP range can be limited, directly returning a 429 status code when the threshold is exceeded.

Rate limiting at the API layer increases the attacker's data collection cost by setting a maximum number of queries per unit time, greatly reducing the economic viability of distillation attacks.

There is an inherent tension between rate limiting and normal user experience; advanced defense systems typically keep loose limits for ordinary users and apply progressive restrictions (soft throttling) to accounts detected with anomalous patterns.

Hard rate limits extend the interval between actions by restricting the number of actions an agent can execute per unit of time, but at the cost of throughput and response speed.

Rate limiting based on single-account thresholds cannot defend against distributed distillation attacks that spread attack traffic across thousands of accounts.

Rate limiting sets call ceilings on a per API key or per account basis, and is effective against single-point high-frequency abuse.

Rate limiting must be applied at the user level rather than the key level; otherwise users can bypass the limit by continuously creating new keys.

If the breach originated from scraping, the root cause likely lies in insufficient API access controls and rate limiting.

Rate limiting prevents individual nodes from consuming excessive resources by restricting the request frequency from a single source.