Questions
What does a 429 response mean for my site?
429 Too Many Requests means the server refused a request because that client had made too many in a period. Receiving it from someone else's site means you are crawling them too fast. Serving it yourself is usually correct behaviour — it is how a rate limit is supposed to refuse — but it becomes a problem when the limit is tight enough to catch search engine crawlers or ordinary visitors behind a shared address.
What the code actually says
The request was well-formed and the client is not forbidden in general. It simply asked too often, and the refusal is temporary by definition.
`Retry-After` should accompany it, as either a number of seconds or an HTTP date, so the client knows when to come back rather than guessing.
It is distinct from 403, which refuses regardless of frequency, and from 503, which says the whole service is unavailable rather than this client's share of it.
Many implementations return 403 for rate limiting instead, which is unhelpful: the client cannot tell a temporary throttle from a permanent refusal.
A 429 is also cacheable in principle, which means a shared cache can hold the refusal and serve it to clients that never made a request — a reason to set caching directives on it deliberately.
When you are receiving it
A link checker or crawler hitting a destination too fast will collect 429s partway through a run, and the pattern is distinctive: early requests succeed and later ones fail.
This says nothing about whether those links are broken. The destination refused the rate, not the URL.
Slow the crawl, honour `Retry-After`, and re-check the affected URLs separately rather than treating them as failures.
If you are calling an API, published `RateLimit-Limit`, `RateLimit-Remaining` and `RateLimit-Reset` headers on successful responses let you pace yourself before hitting the limit at all.
Back off exponentially rather than retrying at a fixed interval. A tight retry loop against a limit is indistinguishable from the abuse the limit exists to stop.
When you are serving it correctly
Login endpoints, password reset, contact forms and anything that sends mail or costs money per call should be limited. Guessing passwords is useless at three attempts a minute.
Expensive endpoints — a search that scans a large table, an export, a report — protect capacity for everyone else by refusing the caller consuming it.
Scrapers and abusive automation, where the refusal is the entire point.
In all of these, 429 with `Retry-After` is the honest answer and 403 is the lazy one.
Publishing the policy in `RateLimit-*` headers is a courtesy to integrators and a small disclosure to everyone else — a trade most public APIs resolve in favour of publishing.
When your own limit is hurting you
Search engine crawlers can trip a limit keyed only on IP address. Google reduces its crawl rate in response to errors, so a limit that catches Googlebot slows indexing of the whole site.
Visitors sharing an address — a company network, a university, a mobile carrier's gateway — arrive as one client and can exhaust a per-address limit between them.
Legitimate integrations and your own monitoring get caught by limits nobody told them about.
A limit keyed only on address is the common cause of all three. Keying on account, API key or session where one exists is both fairer and more effective.
How to tell which situation you are in
Server logs are the complete answer: which addresses were refused, how often, and whether they resolve to a search engine.
Verify any address claiming to be a crawler before acting on it — a reverse DNS round trip, or the published address ranges — because the user agent alone proves nothing.
Search Console's crawl stats report shows the response codes Google received. 429s appearing there mean your limit is affecting indexing.
For a site you do not operate, repeated 429s in a crawl report are your own crawl rate, not their fault.
What a scan can and cannot see
VeriFixScan reports `api.ratelimit.headers` when a response advertises a rate-limit policy, which reveals the limit before it is reached.
`infrastructure.waf` reports when a firewall is present, which is the usual thing enforcing a limit without advertising one.
`availability.broken_links` reports the status of every link followed, so 429s collected during a crawl appear there with the URL that returned them.
The absence of rate-limit headers proves nothing: limits are frequently enforced upstream without any announcement, and only your own logs show whether they fired.
Frequently asked questions
- Is a 429 in a link report a broken link?
- No. It means the checker was refused for asking too often. Re-check the affected URLs at a slower rate before concluding anything.
- Should I return 429 or 403 for rate limiting?
- 429, with a Retry-After header. 403 gives the client no way to distinguish a temporary throttle from a permanent refusal.
- Can a rate limit hurt my search rankings?
- Indirectly. If crawlers are refused, crawl rate drops and pages are refreshed less often. Verify crawler addresses rather than trusting user agents.
Sources
Related
VeriFixScan crawls a site and applies its checks to every page it reaches, keeping the evidence behind each finding. Scanning one website is free.
Scan a website