Glossary
HTTP 500 Internal Server Error
HTTP 500 Internal Server Error means the server encountered a condition it could not handle and gave up on the request. It is the generic server-side failure: the request was acceptable, and the fault is on the server's side. The code is deliberately uninformative — it reveals nothing about the cause, which is a security property, not an oversight — so diagnosing a 500 always means reading the server's own logs rather than the response.
500, 502, 503 and 504 are different failures
`500 Internal Server Error`: the application itself failed. An unhandled exception, a failed database query, a null reference. The server that answered is the one that broke.
`502 Bad Gateway`: a proxy or load balancer got an invalid response from the server behind it. The front is healthy; the back is not answering correctly.
`503 Service Unavailable`: the server is working but deliberately not serving right now — overloaded, or in maintenance. It is the only one of the four that is expected to be temporary.
`504 Gateway Timeout`: a proxy waited for an upstream and gave up. The request may well still be running behind the scenes.
The distinction matters because it names which component to look at first, and a monitoring alert that only reports "the site is down" throws that away.
Why the page tells you nothing
A stack trace in a production error page discloses file paths, framework versions, library names and sometimes query fragments. That is a meaningful amount of information about the internals of a system, handed to anyone who can trigger the error.
So production configurations return a generic page, and the detail goes to the log. A 500 page that shows a stack trace is a configuration left in development mode, and it is worth fixing on its own.
The useful thing a 500 page can carry is a correlation identifier: a request id also written to the log, so a report of "I saw an error" can be matched to the exact failure.
What a 500 means for crawling and indexing
A crawler that receives 500 does not index the page. Google's documentation states that server errors cause the crawl rate to be reduced, because repeatedly requesting a failing server is unhelpful to everyone.
A transient 500 on one page costs a re-crawl. A sustained 500 across a site leads to pages being dropped from the index, and recovery is not instant once the fault is fixed.
If a server must be unavailable for a known period, 503 is the correct answer rather than 500: it states the condition is temporary, and it can carry a `Retry-After` header.
A 500 on the entry URL of an automated audit ends the audit. There is nothing to measure about a page that was never served.
How to find the cause
Start with the application log for the exact timestamp. A 500 is always an exception somewhere, and the trace names the line.
Check whether it reproduces. A 500 that appears on one specific URL points at that code path; one that appears at random on every URL points at a shared dependency — a database connection pool, a cache, a disk that filled.
Look at the web server log alongside the application log. A 500 that the application never recorded was produced by the server in front of it, which is usually a misconfiguration or a process that died.
Reproduce with `curl -v` to see the full response. Headers occasionally survive when the body does not, and the `Server` header names which component answered.
Frequently asked questions
- Does a 500 error hurt SEO?
- A brief one costs a re-crawl. A sustained one does real damage: crawl rate drops and affected pages are eventually dropped from the index.
- Should planned maintenance return 500?
- No. Use 503 with a Retry-After header. It states the outage is temporary, which 500 does not.
- Why does the error page not say what went wrong?
- Because the detail would disclose internal paths, versions and code structure. Production servers log it instead of displaying it.
Sources
Related
VeriFixScan crawls a site and applies its checks to every page it reaches, keeping the evidence behind each finding. Scanning one website is free.
Scan a website