All posts
Monitoring5 min readWatchFor Team

Seven API monitors every SaaS should be running (with the exact assertions)

Not a philosophy post — a checklist. Seven concrete API monitors, the exact assertion each one needs, and the failure it catches: the lying health endpoint, the silently expiring token, the queue that backs up on Friday night, and four more.

Seven API monitors every SaaS should be running (with the exact assertions)

There's plenty written about why to monitor APIs beyond status codes (we've written it too). This post is the other half: what to actually set up. Seven monitors, each with the specific assertion that makes it work and the production failure it exists to catch.

The examples use WatchFor's API monitor syntax — assert on any JSON field, header or body text; extract numbers into charted, alertable metrics — but the recipes translate to any tool that can read a response body. If yours can't, that's the first finding.

1. The health endpoint — asserted, not pinged

Every backend has a /health or /api/status endpoint, and most teams monitor it with a status-code check. The problem: health endpoints are built to aggregate status, and they routinely report their own degradation with a 200:

{ "status": "degraded", "database": "ok", "search": "timeout" }

A status-code monitor calls this healthy. The fix is one assertion:

Assert: JSON field status equals "ok"

Now "degraded" is an incident with the actual payload in the alert — you open the page already knowing it's the search cluster. If your health endpoint exposes per-dependency fields, assert on the critical ones individually; each becomes its own named alert rule, so the incident says what failed, not just "health check red."

Catches: the classic "everything is green but the app is broken" gap — a health endpoint doing its job while nobody reads its answer.

2. The auth flow — because tokens expire on weekends

The most common invisible API outage isn't the API at all — it's authentication in front of it. An OAuth client secret rotates, an API key hits its expiry date, an SSO certificate lapses. Every consumer starts getting 401s, and your unauthenticated health check sees nothing.

Set up a monitor that calls a real authenticated endpoint with a real credential (bearer token, basic or digest auth — stored encrypted):

Assert: status code is 2xx, and JSON field user.id (or any field only an authenticated response contains) exists

Catches: expired credentials, broken token issuance, an auth middleware deploy that started rejecting valid tokens — days before a customer emails you about it.

3. The queue depth — a number that predicts the outage

Some of the most valuable monitoring data is already in your API responses, waiting to be treated as a metric. Queue depth is the canonical example — most job systems expose it somewhere:

{ "queue": { "pending": 137, "oldest_age_seconds": 42 } }

Extract: queue.pending → charted over time Alert: value above 1000 (tune to your normal)

Extraction turns a JSON field into a first-class time series: you get the chart, the trend, and a threshold alert — without shipping a metrics pipeline. The Friday-night failure this catches is the slow one: workers died at 19:00, the queue grows quietly all evening, and either a threshold pages you at 21:00 — or customers page you Monday.

Catches: dead workers, stuck consumers, poison messages — before the backlog becomes a data-loss conversation.

4. The rate-limit budget — watch the headroom, not the wall

If your product consumes a third-party API (payment provider, LLM API, shipping calculator), you have a budget that can run out mid-day. The provider tells you your balance on every single response:

x-ratelimit-remaining: 8421

Extract: header x-ratelimit-remaining → charted Alert: value below ~20% of your quota

The chart alone is worth it — you see consumption climbing week over week and can plan the tier upgrade, instead of discovering the limit as a string of 429s during your busiest hour.

Catches: quota exhaustion from growth, a runaway retry loop chewing the budget, a new feature that tripled call volume.

5. The latency budget — with the phase breakdown

"The API is slow" is not actionable. "TTFB jumped 400ms while DNS, connect and TLS are flat" is — that's your backend, not the network. A proper API monitor times every phase (DNS → connect → TLS → first byte → download) separately, on every check, from multiple regions.

Alert: response time above your SLO (say 800ms), confirmed from multiple locations

Two details matter. Multi-location confirmation keeps one congested route from paging you at 3am. And the phase breakdown at the moment of the alert answers the first triage question — us or the network? — before you've opened a terminal.

Catches: slow database queries behind the endpoint, cold-start regressions, a CDN or DNS change that added a round trip.

6. The contract — schema drift as an alert

Your API's consumers — mobile apps, partners, your own frontend — depend on the shape of responses. A refactor that renames items to results, or starts returning null where an array lived, is a breaking change that no status code will ever report.

Pick your highest-traffic endpoint and assert the shape:

Assert: JSON field items exists · field items[0].id exists · header content-type contains application/json

It's not full schema validation — it's a tripwire on the fields your consumers actually read, which in practice catches the breakage that matters. (Building this by hand is tedious; WatchFor's Run & inspect fires the real request and lets you click any field in the live response to turn it into an assertion.)

Catches: the deploy that broke the mobile app but "passed" because the endpoint still returned 200.

7. The write path — because reads lie about writes

Almost all API monitoring exercises GETs, but your revenue usually travels through POSTs. A monitor that performs a safe, idempotent write — hitting a staging record, a sandbox environment, or a purpose-built echo endpoint — is the only way to know the write path works:

Method: POST with a realistic JSON body Assert: status is 2xx · response field id exists · field status equals "created"

Design the target so repetition is harmless (an upsert on a fixed test ID works well). If a sandbox is all you have, monitor the sandbox — a broken sandbox usually shares its root cause with production.

Catches: serialization regressions, validation rules that started rejecting valid payloads, write-only dependencies (queues, primary DB) failing while cached reads look fine.

The pattern behind all seven

Look at the list again and it's one idea applied seven times: the response body and headers are telemetry — your API is already telling you its queue depth, its auth state, its contract, its dependencies' health, on every single request. Status-code monitoring throws that information away. Assertion + extraction monitoring reads it.

Setting all seven up in WatchFor takes an evening: each is one API monitor — full HTTP under the hood (auth, redirects, HTTP/3, per-phase timings, SSL expiry included), with assertions and extractions built visually from a live response. Each failing assertion is its own named alert rule, so every incident opens with the reason in the title. Start with the health endpoint — it's ten minutes, and it's the one that ends the "but the monitor was green" conversations for good.

Share this article