Broken links on a website

404 and 410: the target does not exist. 410 additionally says the absence is deliberate and permanent. Both are genuinely broken if something on your site links to them.

5xx: the server failed while answering. Often transient, and a server problem rather than a link problem. Re-test before acting.

Redirect chains that end in a 404, and redirect loops that never end.

Not broken, though frequently reported: 401 and 403. On an external link these usually mean the other server blocked an automated client — rate limiting, bot protection, user-agent filtering — while a browser loads the page fine.

Also not broken: a link to a page that exists but contains the wrong thing. No status code will tell you that.

An internal broken link is unambiguous: you own both ends, so there is no question of a false alarm, and the fix is in your hands.

It costs twice. A visitor hits a dead end mid-journey, and a crawler following your internal links spends a request discovering nothing. On a large site, links pointing into 404s dilute the paths by which real pages are found.

External links decay on someone else's schedule. They matter, but they produce more false alarms and less certain fixes.

Collect the URLs — from the XML sitemap, and by walking the navigation. Anything reachable only from a page you never opened will be missed, which is exactly the limitation a crawler removes.

Test each target's status. `curl -I -L <url>` issues a HEAD request and follows redirects, printing each hop, so a chain is visible rather than collapsed into its final code. Some servers reject HEAD; `curl -sS -o /dev/null -w '%{http_code}' <url>` falls back to a GET.

Read your own server logs for 404s visitors already hit. That list includes links from other sites you cannot discover by crawling your own.

If the site is verified in Search Console, the Page indexing report lists URLs Google found and could not fetch — a second, independent view.

Internal link to a page that moved: update the link. Redirecting the old URL is also right, and often necessary for external traffic, but it does not excuse leaving your own links pointing at a redirect.

Internal link to a page gone for good: remove it or point it at the nearest useful page, and serve 410 rather than 404 on the old URL when the removal was deliberate.

External link that is dead: replace it, remove it, or point at an archived copy if the content mattered.

Case and typos: on most servers paths are case-sensitive, so `/About` and `/about` are different URLs. This is a common cause of internal 404s after a migration between systems.

The crawler follows internal links and records the status each target returns. `availability.broken_links` reports the broken ones together with the page each was found on — the part that makes a finding actionable, since a bare list of dead URLs does not say where to edit.

`transport.redirect_chain` reports chains separately, because a working link reached through four hops is a different defect from a dead one.

The crawl also reports what it could not reach and why, so an empty result reads as "nothing broken in what we covered" rather than as an unqualified all-clear.

Fixing a list of broken links is a one-off. The same list regenerates itself unless something changes about how links are created and content is removed.

Redirect on delete. Unpublishing content should offer, or require, a destination. Most content systems can do this and most installations never enable it, which is why deletion is the single largest source of internal 404s.

Link by identifier, not by URL. A link stored as a reference to a page — resolved to a path at render time — survives a slug change; a link stored as a literal path does not. This is the structural fix, and it is the one that has to be decided early.

Re-check after the events that break things in bulk: a migration, a redesign, a category restructure, a bulk import. Those are the days links break, and checking on that day turns a hundred findings into one afternoon.

Keep the 404 log. Server logs record the URLs visitors actually requested and failed to get, including links from sites you will never crawl, and it is the only source for that half of the problem.

Accept external decay rather than fighting it. Links to other people's sites will break at a steady rate whatever you do; a periodic re-check and a policy for what to do with a dead reference is the whole of the answer.