Monitoring a website with recurring scans

Sites change without anyone deciding to change them: a certificate renews or does not, a plugin updates, a CDN setting is edited, a tag manager publishes a new container.

A one-off audit describes the site on one day. The regressions that cost most are the ones introduced afterwards and noticed late.

A scheduled scan gives each regression a time window, and a time window is what makes a cause findable in a deployment history.

It also catches failures that have nothing to do with a release — an expiring domain, a DNS record removed, a page starting to return errors.

Recurring scans are set per website, daily, weekly or monthly, and each can be switched on or off without losing its settings.

Daily suits a site that changes often or earns money every hour; the cost is more runs to read.

Weekly suits most sites: frequent enough to keep a regression window short, rare enough that each run is worth opening.

Monthly suits stable sites whose main risks are external — certificates, domains, third parties.

Deploy-triggered scans, through the API, complement a schedule rather than replacing it: a schedule catches what changes without a deploy.

Slack notifications are grouped: one message per scan and per kind of event, never one per issue, and delivered at most once even if a job is retried.

The kinds are a completed scan, critical issues — optionally high-severity ones too — and a failed crawl, including a scan that completed without reading a single page.

Recurring runs send nothing unless monitoring alerts, completed-scan or critical-issue notifications are switched on, so a schedule does not have to mean a message every run.

Webhooks deliver `scan.started`, `scan.completed`, `scan.completed_with_warnings` and `scan.failed` to your own endpoint, signed and retried, for teams that route alerts elsewhere.

A notification failure never fails the scan itself.

The comparison between two scans classifies every finding: resolved, newly verified, new, regression, persisting, now unverifiable, unchanged, removed, or inconclusive.

New and regression are the ones with a recent cause; persisting is the backlog; resolved confirms a fix actually landed.

Inconclusive marks a degradation the newer scan could not really observe, which keeps a partial run from raising a false alarm.

The history timeline keeps each run's score as it was computed, and shows a missing score as missing rather than inventing one.

A finding that disappears is worth checking against coverage before celebrating: a page the crawl did not reach is not a page that was fixed.

It detects configuration and content changes observable from outside: headers, certificates, redirects, DNS and mail records, status codes, page content, weight and loading behaviour.

It detects availability states precisely — maintenance pages, server errors, TLS failures, redirect loops — rather than a single up-or-down flag.

It is not an uptime monitor: runs are hours or days apart, so an outage between two runs is invisible to it. Minute-level availability needs a dedicated uptime service.

It sees what one anonymous visitor sees on the pages the crawl reaches, which is the same boundary as any single scan.

The scan history can be filtered by website, status and date range, and by score range.

Two filters answer the monitoring question directly: only scans whose score dropped against the previous one, and only scans whose score improved.

When a run did not produce every check the baseline had, the comparison marks it as partial, with the number of checks executed and the ids that are missing, rather than treating absent checks as passes.

Per category, the comparison shows the share of verifiable checks that passed before and after, and the number of failures on each side, so a degradation concentrated in one area stands out.