Troubleshooting by symptom
Organised by what you observed, not by which feature is involved, because when something is wrong you know the symptom and not yet the cause. Causes are listed in the order they turn out to be responsible.
Start here
Most heartbeat problems are one question: did the request arrive? Answer it first.
# Does a ping reach us at all? Run this on the machine that runs the job.
curl -fsS -m 10 -w '\nHTTP %{http_code} in %{time_total}s\n' \
https://pulsewatcher.up.railway.app/ping/<token>
# Expected: HTTP 200. Anything else is the answer.
# 000 -> the request never completed: DNS, firewall or proxy
# 404 -> wrong token, or the monitor was deleted
# 429 -> rate limited: 60 pings/minute per tokenThe state looks wrong
My job ran successfully, but PulseWatch says it is down
1The ping never left the machine
How to tell: Look at the monitor page. If Last ping predates the run, nothing arrived. Run the curl by hand on the same host.
Fix: Usually egress: a firewall, a proxy that needs configuring, or DNS. Add --retry 5 so a single dropped packet does not cost you an alert.
2The ping ran only on success and the shell short-circuited
How to tell: A line shaped job.sh && curl ... sends nothing when the job exits non-zero, which reads as silence rather than as failure.
Fix: Report both outcomes. Use /exit/$? so the exit code decides, rather than making success the only thing you report.
3cron has a different PATH and curl is not on it
How to tell: Check the cron log for a "command not found" line.
Fix: Use the absolute path, usually /usr/bin/curl.
4The job takes longer than the grace window
How to tell: The ping arrives, but after the monitor already went down, so you get an alert and then a recovery a few minutes later.
Fix: Widen grace to cover the job's worst normal run, or ping /start at the beginning so the run is visible while it is still going.
The monitor has been in `new` for hours and never went down
1That is correct behaviour
How to tell: A monitor that has never received a ping has nothing to be late for.
Fix: Send one ping. The clock starts at the first one. If you want the monitor to alert before it has ever run, there is nothing to alert on yet, so create it as part of the deploy that also installs the job.
The uptime percentage does not match what I counted
1Three states are excluded from the calculation, not two
How to tell: blocked, paused and new are in neither the numerator nor the denominator. Only down is downtime.
Fix: See the state reference. A dependent that was blocked for six hours did not have six hours of downtime and did not have six hours of uptime either.
2Warns are counted as available
How to tell: A run that completed with problems still ran.
Fix: If a warn should count against you, map that exit code to down instead.
Monitors paused themselves
1A plan downgrade took you below the monitor cap
How to tell: The oldest active monitors are paused until the count fits, newest left running. Nothing is deleted.
Fix: Upgrade and they resume automatically, or delete monitors you no longer need and unpause the ones you want.
Alerts
A monitor is down and I received nothing
1No verified channel is attached
How to tell: An unverified channel is skipped silently by design, so a typo in an address cannot send your alerts to a stranger.
Fix: Verify the channel, then attach it to the monitor.
2The channel is not included in your plan
How to tell: Free accounts have email only. Telegram and webhooks need Pro or above.
Fix: Attach an email channel, or upgrade.
3A maintenance window is active
How to tell: The state still changed and the history still recorded. Suppressed sends are written to the alert history, so a silent window is auditable.
Fix: Check the window covers the time you expected an alert.
4The monitor is snoozed or the incident was acknowledged
How to tell: Both stop notifications without changing state.
Fix: Clear the snooze on the monitor page.
5The upstream took the blame
How to tell: If this monitor depends on one that is down, it went blocked, and the upstream's alert names it instead. That is one alert for one problem, working as intended.
Fix: Nothing. Read the upstream alert.
One broken thing produced a dozen alerts
1The chain is not declared
How to tell: Every job downstream of the broken one failed on its own account.
Fix: Declare the dependencies. Dependents go blocked and stay quiet, and the upstream alert names them.
2Many machines running the same job are separate monitors
How to tell: Two hundred monitors means two hundred alerts.
Fix: Use a fleet: one alert for the fleet, naming the machines that stopped.
3The repeat interval is short
How to tell: The same alert arriving over and over is the repeat, not new failures.
Fix: Raise the repeat interval, or acknowledge the incident to stop the repeats.
I got an alert and nothing was wrong
1Grace is tighter than the job's real variance
How to tell: The alert is followed by a recovery within a few minutes, repeatedly. The advisor flags this pattern.
Fix: Widen grace to cover the slowest normal run, not the average one.
2An uptime check hit a 200 that renders an error page
How to tell: Reason keyword_mismatch means the status was fine and the body was not.
Fix: That is the check doing its job. Read the body.
3The job is genuinely fine and its exit code is not
How to tell: Reason exit_code with a code you consider harmless.
Fix: Add an exit-code rule mapping that code to warn or ignore.
Warnings never reach me
1Warnings are batched by default
How to tell: warn_delivery defaults to digest_daily, so a warn arrives in the daily summary rather than immediately.
Fix: Set it to immediate if the warn is worth interrupting someone for. Consider whether it is.
2A quiet day sends nothing
How to tell: A digest with no warnings in it is not sent.
Fix: Nothing. That is the point.
Rules that do not fire
A metric rule has never fired
1The metric name does not match what the job sends
How to tell: Open a recent ping and read the metrics it carried. The name must match exactly, including case.
Fix: Rename the rule, or the metric. The rule editor suggests names already seen.
2The rule is disabled
How to tell: A disabled rule is stored and skipped.
Fix: Enable it.
3Nothing has crossed the bound
How to tell: The advisor tells a rule watching something healthy from a rule that has never matched anything, using the last time it fired.
Fix: Confirm the bound is where you meant it. A floor of 0 catches nothing.
A baseline rule is still learning
1Not enough comparable runs yet
How to tell: A bucketed baseline compares like with like, so an hour_of_day bucket on a daily job gets one sample a day for that hour.
Fix: Either wait, or widen the bucket. A daily job rarely needs anything finer than day_of_week.
2The job does not report the metric on every run
How to tell: Gaps in the samples slow learning down.
Fix: Report the number on every run, including the runs where it is zero.
My exit-code rule is being ignored
1A more specific rule matched first
How to tell: A rule covering exactly one code beats a rule covering a range that contains it.
Fix: Narrow or remove the competing rule.
2The code never arrived
How to tell: A malformed code is dropped rather than failing the ping, so /exit/ with an empty value looks like a plain success. Valid codes are 0 to 255.
Fix: Use /exit/$? immediately after the command, before anything else can overwrite it.
3Rules are cached briefly
How to tell: An edit can take up to a minute to apply to the ping path.
Fix: Wait a minute and send another ping.
Duration is never recorded
1Nothing pings /start
How to tell: Without a start there is no interval to measure.
Fix: Ping /start before the work and the plain URL after it. This is also what makes overlap detection and max-duration timeouts possible.
Fleets
A fleet says instances are missing and they are running
1The instance key changed
How to tell: Keys are case preserving and compared exactly. A host that started reporting as EDGE-07 after previously sending edge-07 is a new instance, and the old one goes quiet.
Fix: Pin the key. Use a stable identifier rather than something derived at boot, and retire the stale entry.
2The machine was decommissioned
How to tell: A retired instance leaves the denominator; a merely silent one does not.
Fix: Set a retirement window so departed machines drop out on their own.
A fleet filled up with instances I do not recognise
1Auto-register accepted every key that arrived
How to tell: Something is generating keys per run, per container or per boot, rather than per machine.
Fix: Fix the key at the source, then turn auto-register off so unknown keys are rejected instead of joining.
A completion budget is set and never says anything
1Instances are too new to judge
How to tell: An instance needs at least five scheduled runs behind it before a percentage means anything. Below that it is skipped rather than failed.
Fix: Wait for the history to build.
2Late runs count as runs
How to tell: A budget asks whether the work happened, not whether it was punctual. A machine that is always forty minutes late is at 100%.
Fix: Use duration regression or a tighter grace window for punctuality. They are different questions.
Delivery and integrations
My webhook signature check fails
1The body was parsed before it was verified
How to tell: The HMAC covers the raw bytes. Re-serialising the JSON changes whitespace and key order, and the digest no longer matches.
Fix: Capture the raw body first, verify, then parse. Express needs express.raw() on that route.
2Comparing with ===
How to tell: It works, and it leaks timing.
Fix: Use a constant-time comparison. The webhook page has a verified snippet in eight languages.
My webhook handler has no branch for an event it received
1The event set is open
How to tell: New events are added additively and the payload version stays at 1. Fleet deliveries have a fleet object and no monitor object.
Fix: Branch on details.reason, and default unknown events to logging rather than throwing. The full list is on the reference page.
2Branching on details.missing for a fleet
How to tell: A rule breach and a blown completion budget both report missing: 0, because nothing is missing in either. Every machine reported.
Fix: Branch on details.reason instead.
The REST API returns 401, 403 or 429
1401: the key is wrong or missing
How to tell: The header is Authorization: Bearer <key>.
Fix: Re-copy the key. It is shown once at creation.
2403: writing on a plan without write access
How to tell: Reads work on every plan. Writes need Pro or above.
Fix: Upgrade, or drive the change from the dashboard.
3429: 60 requests per minute per key
How to tell: The response body names the limit.
Fix: Back off and retry. When provisioning many monitors in a loop, add a small delay between calls.
A monitor is not on my status page
1It is not marked for publication
How to tell: Only monitors you explicitly publish appear.
Fix: Turn on the status page flag for that monitor.
2The page itself is not enabled
How to tell: A status page needs a slug set on the account.
Fix: Set the slug in settings. It must be 3 to 40 characters.
Data
Older runs and pings have disappeared
1Retention
How to tell: 7 days on Free, 30 on Pro, 90 on Business. Pruning is automatic.
Fix: Export what you need to keep, or upgrade. Rolled-up daily uptime and latency are kept for a year regardless.
My log excerpt is cut off
1The per-plan body cap
How to tell: 4 KiB on Free, 16 KiB on Pro, 64 KiB on Business.
Fix: Send the tail rather than the whole log. The last 50 lines contain the stack trace; the first 5,000 contain the banner.
Failures are not being grouped into an error group
1No parseable trace in the body
How to tell: Grouping needs a stack trace it can parse. A one-line message has no frames to fingerprint.
Fix: Send stderr as the ping body on failure. See reporting errors for the redirect that captures it.
2Error grouping is not on your plan
How to tell: It needs Pro or above.
Fix: Upgrade, or read the excerpt on the run directly.
Still stuck
The states and reasons reference explains any value you see on an alert. The configuration advisor finds monitoring that will not do its job before it fails to do it. On Pro and above, the in-app assistant answers questions about your own configuration.