We were wiring a news panel into a site and pulling from eight feeds. Two were not working. Finding out why took longer than building the panel, and the reason is worth passing on.
The easy failure
The first feed returned HTTP 404. The URL had been retired at some point and nothing had been updated. That is the good kind of failure: anything watching it would have gone red immediately, and the fix was to find the current address.
The expensive one
The second feed returned HTTP 200. Valid XML. Twenty well-formed items, every field populated, no errors anywhere. By every check we had, it was healthy.
Its newest item was dated January 2025 — more than eighteen months before we looked.
Nothing was broken. It had simply stopped.
A monitor watching for errors would never have fired. A test asserting the response parses would have passed. A dashboard counting successful fetches would have been green the entire time. The feed was not down; it was abandoned, and from the outside those look identical.
What this costs in a business
Swap the feed for anything a business depends on and it reads the same way:
- A nightly sync that keeps completing but stopped picking up new records when a field was renamed.
- A report that still arrives every Monday, built from a query filtered to a date range that ended.
- A price list that updates from a supplier file that has not changed since the supplier switched systems.
- An integration that posts successfully to an endpoint quietly deprecated a year ago.
In each case the thing is running. Somebody is making decisions on it. Nothing anywhere is red.
What to check instead
Monitoring whether a job succeeded is the easy half. The half that catches this is monitoring whether the data moved:
- Freshness, not success. Alert when the newest record is older than it should be, not when the job errors.
- Volume against a normal range. Zero new rows is a problem; so is ten times the usual number.
- A number you already know. If the total should match the bank, the invoice, or last month, check that it does.
- An expiry date on the source itself. Anything you do not control gets a reminder to confirm it is still alive.
None of this is sophisticated. It is the difference between asking “did it run” and asking “is it right”, and almost everything we see in a first audit asks only the first one.
The uncomfortable part
We only found this because the output looked thin and somebody went and counted. There was no alert. There would not have been one next year either.
If you have a system that has run without complaint for a long time, that is not the same as evidence it is working. It is worth picking one and checking the data by hand, once. It takes an afternoon and it is usually the cheapest audit you will ever run.