Upcoming: Monitor assertions and Incident

When we built Crystade's active monitoring engine, we aimed for maximum evaluation depth. By pairing synthetic multi-protocol probes with custom Rice DSL check scripts, developers could inspect response payloads, parse headers, and evaluate latency thresholds right after a check log was generated.

However, flexibility brought complexity. Writing imperative code to mix evaluation logic with action execution proved to be an operational drag for teams. Today, we are sunsetting imperative check scripts and rolling out Declarative Assertions with Automated Incident Management.

The Flaws of Imperative Check Scripts

Under the original system, a check script handled both evaluation logic and notification dispatching in a single script block:

if context.error == null && context.metadata.rttMs > 60 {
    alerting.send({
        header: "Monitor [%s] notices high RTT from %s"
            .format(context.monitor.name, context.location),
        message: "Measured a RTT of %.2fms\nCheck ID: %s"
            .format(context.metadata.rttMs, context.check.id),
        footer: "%s • Monitored by Crystade"
            .format(context.team.name),
        deduplication: {
            key: context.monitor.id + ":simple-uptime-check",
            ttl: "1h"
        }
    });
}

While powerful, this approach introduced severe friction:

1. High Cognitive Overhead: Developers had to learn platform-specific SDK functions like alerting.send(...) just to notify their team of a slow endpoint.

2. Zombie Incidents: Incidents were reported when check scripts threw uncaught errors, but they lacked automatic state tracking to resolve those incidents once the service recovered.

3. Alert Fatigue & Formatting Fragmentation: Inlining messages inside scripts made internationalization (i18n), consistent formatting, and centralized deduplication nearly impossible.

4. Per-Monitor Silos: Custom scripts were bound directly to individual monitors rather than shared across team workspaces.

The Solution: Declarative Assertions & State-Driven Incidents

We are decoupling data collection from evaluation and lifecycle tracking.

Monitors focus entirely on making network requests. The outcome of a check is now evaluated against one or more Assertions—pure boolean conditions (true or false).

                    ┌─────────────────────────┐
                    │    Monitor Execution    │
                    └────────────┬────────────┘
                                 │
                         Generates Check Log
                                 │
                                 ▼
                     ┌───────────────────────┐
                     │ Evaluates Assertions  │
                     └───────┬───────┬───────┘
                             │       │
             Assertion A =   │       │   Assertion B =
                 FALSE       │       │       TRUE
                             ▼       ▼
                    ┌─────────┐     ┌─────────┐
                    │  OPEN   │     │  CLOSE  │
                    │ Incident│     │ Incident│
                    └─────────┘     └─────────┘

1. Automated Incident Lifecycles

Each assertion on a monitor independently manages a single incident lifecycle:

• State Becomes FALSE: If an assertion evaluates to false for longer than its designated tolerance window and no active incident exists for that assertion, an incident is automatically reported.

• State Becomes TRUE: As soon as the target service recovers and the assertion evaluates back to true past the tolerance window, the associated incident automatically progresses to resolved.

No manual intervention required, and no persistent zombie incidents.

2. Targeted Multi-Layer Monitoring

Because a single monitor can run multiple assertions, you no longer need separate monitors to test distinct layers of service health. A single HTTP execution can power two separate incidents:

• Assertion 1 (Uptime): context.error == null && context.response.statusCode == 200 ➡️ Triggers a critical incident if unreachable.

• Assertion 2 (Performance): context.metadata.rttMs <= 100 ➡️Triggers a minor incident if response latency degrades.

Reusable Templates and Shared Team Assertions

Custom assertion logic is no longer restricted to high-tier plans or trapped inside single monitors.

• Official & Community Templates: Select from pre-built assertions (e.g., HTTP status code validation, JSON body verification, TLS certificate validity) without writing code.

• Team-Scoped Custom Assertions: All plans—including Free—can now author custom assertions. Custom assertions become reusable team assets that can be attached to any monitor across your workspace.

• Tier-Based Assertion Limits: To balance infrastructure workload while keeping features accessible, plan limits govern how many assertions can run simultaneously on a single monitor:

  • Free: Up to 1 assertion per monitor.

  • Starter: Up to 3 assertions per monitor.

  • Standard: Up to 5 assertions per monitor.

Fine-Grained Link Configuration

When attaching an assertion to a monitor, you can customize how failure states present themselves to your engineering team and public status pages:

• Dynamic Message Templates: Define clear alert context using dynamic variables:

"{{assertionName}} failed on monitor {{monitorName}} (Location: {{location}})"

• Incident Severity Override: Explicitly map assertion failures to minor, major, or critical severities.

• Assertion-Specific Tolerance Windows: Override the monitor’s default tolerance window with granular thresholds per assertion (e.g., tolerate high latency for 5 minutes, but trigger an uptime failure after 1 minute).

7-Day Migration Strategy

To ensure zero disruption to your active monitoring setup:

1. Dual Execution (Days 1–7): We will soon rolling out the feature, both legacy check scripts and new assertions will run in parallel.

2. Final Sunset: On August 15, 2026, legacy Rice check scripts will be formally disabled.

Explore the official assertion templates in your Crystade workspace dashboard or create your first team assertion today.