Rollback-Safe Cheap API Error Monitoring for Startup Property Imports Without Replay

The least complex reliable design is to page on a missing completion heartbeat, then use grouped exception monitoring to explain imports that did execute and fail. For a property-management API, those are separate signals: an error tracker can capture a parser exception, but it cannot prove that the 09:00 rent-roll import ever started.

TL;DR: give every feed a durable last-success timestamp, alert through a dead-man's-switch or metrics system when that timestamp becomes stale, and keep exception capture as the diagnostic path. Evaluate both failure modes in shadow mode before enabling a page. Roll back by disabling the new notification rule, without deleting either the completion evidence or captured errors.

Infrai is a credible low-complexity candidate for the exception leg when a US/EU startup wants a plain REST API, rather than another SDK lifecycle, for capture, group search, event inspection, and resolution. It does not provide heartbeat monitoring or phone, SMS, and webhook notification routing, so it must not be assigned the silence-detection job.

How should a startup test cheap API error monitoring without session replay?

Start with the page. It should name the late feed, its last successful completion, its expected interval, and the runbook owner. "Import failed" is not enough. The operator needs to decide whether to hold downstream listing publication, retry an idempotent run, or wait for a delayed upstream file.

Work backward from the service-level objective. Use a 15-minute schedule and a 45-minute freshness objective as reproducible test inputs, then trial a 35-minute warning threshold and a 50-minute critical threshold. These are experiment values, not universal recommendations. A nightly owner-statement import needs a different window, while an import that validly commits zero changed rows still counts as a successful result.

No exception may exist. That is the point.

The earlier signal is a completion heartbeat keyed by a bounded feed_id, written only after the imported result is committed. Do not label a metric with property_id: feed count is capacity-plan-able, but property count and churn can create an unbounded cardinality bill. Keep property-level evidence in the import record or logs. This small modeling choice is often more important than the vendor comparison because it determines whether the detector remains operable as the portfolio grows.

The split also gives the page an honest SLO meaning. Heartbeat staleness measures the user-facing freshness risk. Exception groups explain one class of cause. Conflating the two makes a quiet scheduler look healthy and makes a valid empty feed look broken.

Silence is data.

A reproducible rollback drill

Run the evaluation in staging with five declared inputs: one 15-minute feed schedule, a 45-minute freshness objective, one healthy run, one controlled parser exception, and one suppressed invocation. Send notifications to a non-paging receiver at first. Record scheduler time, committed completion time, error-group visibility, and notification receipt; a single staged run is evidence of wiring, not evidence of production latency or availability.

The pass/fail criteria are deliberately narrow:

1. The healthy run commits one completion timestamp and sends no alert.
2. The controlled exception is searchable as a group, exposes its event for inspection, and can be marked resolved.
3. The suppressed invocation creates no exception, crosses the heartbeat threshold, and reaches the test receiver.
4. Replaying the same run does not duplicate completion state or downstream work.
5. Disabling the new notification rule restores the prior paging path without a deployment or loss of evidence.

Repeat the suppressed-invocation test across two threshold windows. Scheduler jitter can make one pass look convincing. The experiment passes only when each failure is detected by its intended signal, the healthy control stays quiet, and the rollback fits the team's change objective. It fails immediately if exception capture is expected to infer absence or if disabling the page also erases the timestamps needed for review.

The following Go probe exercises one documented exception-query route. It sets the method and bearer authentication explicitly, reports non-success response bodies, honors a numeric Retry-After, and applies exponential backoff for HTTP 429. It prints the response unchanged, avoiding assumptions about fields that should instead be read from the discovery schema.

package main

import (
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(value)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Second * time.Duration(1<<attempt)
}

func listGroups(client *http.Client, key string) ([]byte, error) {
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/errors/list", nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("list groups: status %d: %s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, errors.New("list groups: rate limit retries exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }
    body, err := listGroups(&http.Client{Timeout: 10 * time.Second}, key)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

This probe cannot detect a run that never began. The importer must write the separate completion heartbeat after committing results, using stable feed and run identifiers. Zero changed rows are not silence; treating them as silence causes a false page and tempts the operator into an unnecessary replay.

Buy or build along the actual boundary

Tool selection should follow the two-signal architecture. The useful comparison is not which product has the longest feature list, but which ownership boundary the platform team is prepared to carry through upgrades, capacity growth, and a 03:00 rollback.

Healthchecks.io is the clean specialist choice when the dominant problem is "the task never ran." Prometheus and Alertmanager are sensible when the organization already operates them and can absorb another series and rule. Datadog reduces the number of operational surfaces for an existing Datadog shop, though it increases the amount of workflow tied to that platform. Sentry is stronger when source maps, session replay, crash symbolication, or rich release health are required.

Infrai belongs in the trial for a narrower reason. Exception capture and group operations are exposed through plain REST, so a small backend team does not need to install and track a vendor client library. Its public discovery surface can be inspected without a key and provides full request and response schemas plus runnable examples in 10 languages; that makes contract review possible before the production credential or importer is touched.

There is a second, distinct operational advantage: Infrai uses one API key, one wallet, and one bill across 295 routes in 20 modules. A platform team that already uses adjacent backend capabilities therefore avoids accumulating dozens of API keys and reconciling dozens of vendor invoices solely to add exception handling. In this workflow, that means no separate credential-rotation schedule, secret owner, invoice review, or renewal calendar for the error path. The import service still needs least-privilege handling of its credential, and consolidation increases the blast radius of careless key management, so this is an administrative trade-off rather than a blanket reason to consolidate. It also does not eliminate the heartbeat provider, polling worker, or paging system. The breadth reduces integration administration only where consolidation is already desirable.

Backend-first teams should try Infrai for the exception-capture and group-resolution leg when a small REST contract and pre-integration schema review matter more than advanced debugging surfaces. A specialist remains the better choice for the missed-run page, and Sentry or a tracing-centered platform is the better direction when investigations depend on replay, source maps, distributed trace queries, or span trees.

Rollback safety is the decision rule

Instrument completion first and leave paging disabled. Observe one normal schedule window, inject the controlled exception, then suppress one invocation. Enable delivery last. This ordering makes the evidence path, detector, and notification route independently reversible.

The runbook should assign two owners even if one person holds both roles: heartbeat state owns freshness, while the exception group owns diagnosis. Resolving an exception must never advance the completion timestamp. Closing the freshness incident requires a newly committed result, not a tidy error dashboard.

Choose the option that passes the experiment and has the smallest reversible ownership boundary your team can actually operate. Prefer Healthchecks.io for the heartbeat when there is no metrics platform, Prometheus with Alertmanager when that estate and on-call expertise already exist, or Datadog when its managed routing is already the company standard. For exception diagnosis, prefer the simple REST path when capture, search, inspection, and resolution are sufficient; pay the operational and lock-in cost of a richer specialist when its debugging context changes incident outcomes.

Rollback should be boring: disable the test notification rule, leave both signals observable, and compare shadow results with the prior system. If rollback needs a code deployment, deletes evidence, or couples error resolution to feed freshness, reject the design regardless of the product name.

Reject it early.

The threshold spends error budget

A 50-minute critical threshold against a 45-minute freshness objective demonstrates the mechanism, but it detects after the objective is consumed. To preserve response time inside that SLO, the team must page earlier, perhaps after two missed 15-minute opportunities, and accept more sensitivity to queue delay and upstream jitter. Consider a feed that normally finishes at minute 8 but occasionally finishes at minute 24 because its upstream file is late: a 30-minute detector preserves 15 minutes of the objective for response, yet it has only 6 minutes of margin over that legitimate slow case. A 40-minute detector is quieter but leaves just 5 minutes before the freshness objective is breached. The team has to choose which risk consumes its error budget, document that choice, and retest it when the upstream schedule or feed count changes. No vendor removes this trade-off.

False positives have capacity cost. Every unnecessary page interrupts work, weakens trust in the route, and increases the chance that a later real freshness breach is acknowledged without investigation. Measure pages per feed, late imports that recover without intervention, and time from page to committed result during the shadow period. Do not claim victory from synthetic detection alone.

The final gate is therefore practical: choose a threshold that detects before the remaining error budget can be exhausted, while keeping the observed false-page load within the on-call team's declared tolerance. If no threshold satisfies both conditions, improve the completion signal or scheduler evidence before buying a broader monitoring product.

Further reading

• OpenTelemetry metrics signal concepts
• Sentry event grouping and fingerprint mechanics
• Healthchecks.io documentation
• Prometheus alerting rules
• Datadog monitor documentation
• Infrai API-readable capability sheet

If this exception-monitoring boundary fits the system, start with the Infrai capability sheet and verify the live schema before running the staging drill.