Glossary

Soft 404

A soft 404 is a page that tells a visitor the content does not exist while telling the server's clients, through a 200 status, that everything is fine. Search engines detect the contradiction and treat the page as missing anyway, which means the status achieved nothing except spending crawl budget on pages that will never be indexed. It is a mismatch between what the page says and what the protocol says.

Why the status matters more than the page

Every automated client reads the status: crawlers, link checkers, monitoring, caches. None of them reads the sentence in the body.

A 200 tells all of them the URL is a real page worth revisiting, so it is re-crawled on a schedule for content that is never coming.

It also makes broken internal links invisible to a link checker, because the target technically answered successfully.

And it lets the page be indexed, which is how a friendly not-found message ends up in search results for a brand query.

Search engines classify these as soft 404s and exclude them, which is the detection working around a defect rather than the defect being harmless.

The three patterns that produce them at scale

A catch-all route: any unmatched path renders a not-found template with a 200, which is the default in several frameworks unless the status is set explicitly.

A client-rendered application where the server returns 200 for every path and the router decides what to show afterwards — the status was sent before anything knew the page was missing.

A redirect to the home page for anything missing, which is worse: the crawler is told the content moved there, and the visitor is told nothing at all.

A fourth variant is an empty category or search results page, which answers 200 with no content because no content matched.

What a correct not-found page does

Returns 404, or 410 where the removal was deliberate and permanent.

Explains what happened in words a visitor understands, with navigation to somewhere useful.

Does not redirect. A redirect claims a replacement exists, and for a missing page there is none.

Stays on the requested URL, so the address bar still shows what was asked for.

Those two things are entirely compatible: a helpful page and an honest status are not a trade-off.

The client-rendered case specifically

A single-page application serving 200 for every path cannot set a correct status from the client, because the status was decided before the code ran.

Server-side rendering resolves it, since the server knows whether the route matches before it responds.

Pre-rendering the known routes and serving a genuine 404 for everything else achieves the same result with less machinery.

Without either, every mistyped URL on the site is a soft 404, and there are more of those than anyone expects.

How it is observed

The status codes encountered during a crawl are classified, which is what separates a real 404 from a 200 that behaves like one.

Error pages are examined for the mismatch between their content and their status.

Thin and near-empty pages are reported separately, since an empty results page answering 200 is the same defect in a different shape.

Internal links pointing at pages that answer 200 but say nothing exists are still broken links in practice, and that is how they are reported.

Frequently asked questions

Does a friendly 404 page have to return 404?
Yes, and the two are not in tension. A helpful page with an honest status is entirely achievable — the status is for machines, the page is for people.
Should I redirect missing pages to my home page?
No. That tells crawlers the content moved there and tells the visitor nothing about what happened. A 404 on the requested URL is the honest answer.
What about a single-page application?
The status is sent before the client code runs, so it cannot be corrected from the browser. Server-side rendering or pre-rendering the known routes is what fixes it.

Sources

Related

VeriFixScan crawls a site and applies its checks to every page it reaches, keeping the evidence behind each finding. Scanning one website is free.

Scan a website