Snapshot & compare¶

Architecture

Snapshot pins the query executor to a single bitemporal coordinate; compare runs the winner rule at two coordinates and reports the difference as added, removed, and modified entities with per-property old/new values.

Overview & purpose¶

"What changed between Q1 and Q2" is, per the spec's problem statement, the product's marquee query, and it is easy to get subtly wrong three ways:

• Comparing version rows instead of selected states reports internal churn (a re-versioned but unchanged fact) as a real change.
• Unpaginated deltas on large tenants explode memory.
• Without a pinned definition of "modified," two implementations can disagree on identical data.

Snapshot and compare exist to answer this class of question without either failure mode: snapshot is query-DSL sugar for "state as of one coordinate," and compare is the delta between two such states, always computed through the same selection rule a plain read would use.

Design¶

Snapshot and compare are deliberately separate but related operations:

• Snapshot (POST /v1/snapshot) is not a distinct engine, it is the query executor with atValidTime (required) and optionally atTxTime pinned onto the request before dispatch. core-engine/src/api/v1.rs::post_snapshot builds a synthetic query-DSL JSON object from the typed SnapshotRequest body and re-dispatches through state.executor.execute, the identical executor POST /v1/query uses. There is no dedicated "snapshot module" in the engine: a snapshot is just a query whose temporal coordinates are fixed rather than defaulted to "now."
• Compare (POST /v1/compare) is a dedicated executor (query::compare::CompareExecutor) because a delta is not expressible as a single query: it requires evaluating the same entity at two independent coordinates and classifying the pair. Compare does not call the query executor or take two snapshots and diff them client-side; it walks storage once and resolves both coordinates per entity in the same pass (see Algorithms & invariants).

Both operations build on the same foundation: the winner rule from the tritemporal model. Among all versions of an entity whose valid interval and transaction interval both contain the queried (vT, tT), the winner is the version with the highest tx_time.start; a winning tombstone reads as absent. Snapshot uses this rule once, through the executor's normal read path, at one coordinate. Compare uses the exact same rule (the doc comment in compare.rs calls out explicitly that it is "the same winner rule as any read") independently at both the baseline and comparison coordinates, then classifies the two outcomes against each other.

Data model¶

Snapshot request/response¶

SnapshotRequest (core-engine/src/api/v1.rs) is a typed, deny_unknown_fields DTO:

#[derive(Debug, Deserialize, utoipa::ToSchema)]
#[serde(deny_unknown_fields, rename_all = "camelCase")]
pub struct SnapshotRequest {
    at_valid_time: serde_json::Value,        // RFC3339 string or unix-µs integer
    #[serde(default)]
    at_tx_time: Option<serde_json::Value>,   // RFC3339 string or unix-µs integer
    find: String,                            // label to find
    #[serde(default, rename = "where")]
    where_: Option<serde_json::Value>,       // query-DSL predicate tree
    #[serde(default)]
    limit: Option<u32>,
    #[serde(default)]
    cursor: Option<String>,
}

find is a required plain string here (snapshot always names a label), unlike /v1/query which parses a raw, more permissive query-DSL document. The response shape is identical to /v1/query's: {records, related, nextCursor, traceId, stats} (see REST API › POST /v1/snapshot), because both are served by execution_json over the same QueryResponse.

Compare request¶

CompareRequest (core-engine/src/query/compare.rs):

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CoordinateSpec {
    pub valid_time: TimeSpec,               // required: µs or RFC3339
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub tx_time: Option<TimeSpec>,           // default: now at execution
}

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CompareRequest {
    pub baseline: CoordinateSpec,
    pub comparison: CoordinateSpec,
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub find: Option<String>,                // label scope (either side may satisfy it)
    #[serde(default, rename = "where", skip_serializing_if = "Option::is_none")]
    pub where_: Option<serde_json::Value>,   // applies to the C-side selection ONLY
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub limit: Option<u32>,                  // 1..=1000, default 100
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub cursor: Option<String>,
}

Both baseline and comparison require validTime; txTime on either side defaults independently to wall-clock now, which is what makes "how did May change between what we knew in June vs. September" and plain valid-time-only deltas both expressible with the same shape.

The added / removed / modified diff structure¶

{
  "added": [
    { "kind": "node", "id": "...", "labels": ["RISK"], "properties": { "severity": 8 } }
  ],
  "removed": [
    { "kind": "node", "id": "...", "labels": ["RISK"], "reason": "tombstone" }
  ],
  "modified": [
    {
      "kind": "node", "id": "...", "labels": ["RISK"],
      "versions": {
        "baseline":   { "validTimeStart": 1, "txTimeStart": 1 },
        "comparison": { "validTimeStart": 2, "txTimeStart": 2 }
      },
      "props": {
        "added": {}, "removed": {},
        "changed": [{ "key": "severity", "old": 5, "new": 8 }]
      },
      "labelsAdded": [], "labelsRemoved": [], "touch": false
    }
  ],
  "stats": { "added": 1, "removed": 1, "modified": 1, "unchanged": 4, "rowsScanned": 812, "partial": false },
  "nextCursor": null
}

The Rust types behind that JSON (core-engine/src/query/compare.rs):

Algorithms & invariants¶

Computing a snapshot as of (vT, tT)¶

A snapshot has no separate algorithm. post_snapshot builds {"find": ..., "atValidTime": ..., "atTxTime"?: ..., "where"?: ..., "limit"?: ..., "cursor"?: ...} and parses it as an ordinary query-DSL document (ast::parse), then calls state.executor.execute(&scope, &query), exactly as POST /v1/query does. The temporal semantics (which version wins at (vT, tT)) are entirely the executor's and the winner rule's, not anything specific to the snapshot endpoint. This is why the response envelope, error codes (QUERY_INVALID), and cursor grammar are identical between /v1/query and /v1/snapshot.

Computing a compare between two coordinate pairs¶

Compare is a dedicated single-pass algorithm, and the as-built version deliberately departs from the spec's original architecture sketch (see As-built notes). The shape, from CompareExecutor::compare:

1. Validate the request into a ComparePlan: resolve both coordinates' TimeSpecs to concrete (Ts, Ts) pairs (defaulting missing txTime to wall-clock now), parse where into a Filter via the shared query grammar, clamp limit to 1..=1000 (default 100), and compute a plan_hash (xxh3 over the canonical request JSON with cursor stripped).
2. Resume, if a cursor was supplied: verify its HMAC, scope binding, and plan_hash binding, then decode (kind, last logical id) as the resume point.
3. Walk each primary column family once, node_versions then edge_versions, in key order. Version-row keys are laid out as [scope][logicalId][inv vT][inv tx], so one entity's entire version history is physically contiguous; the walk accumulates each contiguous run into a group and, on an id boundary, calls classify_group.
4. Classify each group by computing select_winner against the group twice, once for plan.baseline and once for plan.comparison, using the identical containment-plus-highest-tx_time.start rule the read path uses. The pair of Selected<T> outcomes (Present or Absent(reason)) drives the added/removed/modified classification in classify_node / classify_edge.
5. Diff properties for the Present/Present case with diff_props: walk the C-side BTreeMap<String, Value>, bucket each key as added (not in B) or changed (in both, bv != cv by exact Value equality) or matching (skip); then walk B for keys absent from C into removed. Node modifications additionally compute labels_added / labels_removed by set difference on the label vectors.
6. Edge endpoint presence is re-checked, not assumed: for each selected edge version, node_present scans that node's own version history at the same coordinate and side, memoized in a HashMap<(Uuid, u8), bool> per request. An edge whose winner is Present but has a missing/absent endpoint is downgraded to Absent(ValidityLapse) before classification.
7. Page and checkpoint: the scan ceiling (compare.max_scan_rows) and the limit are both checked only at completed group boundaries, never mid-group, so a page never contains a half-classified entity. When either limit is hit, next_cursor is signed over (kind byte, last completed entity's uuid).

fn select_winner<'a, T, I, F>(versions: I, fields: F, at: (Ts, Ts)) -> Selected<&'a T>
where
    I: Iterator<Item = &'a T>,
    F: Fn(&T) -> (Interval, Interval, bool),
{
    let mut winner: Option<(&T, Ts, bool)> = None;
    for v in versions {
        let (valid, tx, tomb) = fields(v);
        if valid.contains(at.0)
            && tx.contains(at.1)
            && winner.is_none_or(|(_, wt, _)| tx.start > wt)
        {
            winner = Some((v, tx.start, tomb));
        }
    }
    match winner {
        None => Selected::Absent(RemovedReason::ValidityLapse),
        Some((_, _, true)) => Selected::Absent(RemovedReason::Tombstone),
        Some((v, _, false)) => Selected::Present(v),
    }
}

Invariants¶

Compare request flow¶

sequenceDiagram
    autonumber
    participant C as Client
    participant API as POST /v1/compare
    participant CMP as CompareExecutor
    participant ST as Storage (node_versions / edge_versions)

    C->>API: {baseline, comparison, find?, where?, limit, cursor?}
    API->>CMP: parse_compare (pointered errors) → compare(scope, request)
    CMP->>CMP: validate() → ComparePlan (resolve coordinates, hash plan)
    alt cursor present
        CMP->>CMP: verify_cursor (HMAC, scope, plan_hash)
    end
    loop each contiguous entity group (nodes, then edges)
        CMP->>ST: scan_prefix(node_versions | edge_versions)
        ST-->>CMP: this entity's full version rows
        CMP->>CMP: select_winner(group, baseline) → Selected B
        CMP->>CMP: select_winner(group, comparison) → Selected C
        opt edge kind
            CMP->>ST: node_present(from/to, baseline) / node_present(from/to, comparison)
            ST-->>CMP: endpoint presence (memoized)
        end
        CMP->>CMP: classify (B, C) → added | removed | modified | unchanged
        opt modified
            CMP->>CMP: diff_props(B.properties, C.properties)
        end
        opt scan ceiling or limit reached
            CMP->>CMP: sign_cursor(last completed id) → next_cursor
        end
    end
    CMP-->>API: DeltaPage {added, removed, modified, stats, nextCursor}
    API-->>C: 200 JSON

Configuration¶

CompareConfig (core-engine/src/config.rs) has exactly this one field. It is layered through the standard figment precedence (defaults → TOML → TELHA_* env → CLI flags) shared by the rest of the engine's configuration. Page-size limits (limit, 1..=1000, default 100) are validated per-request in CompareExecutor::validate and are not separately configurable.

As-built notes¶

The spec's §14 changelog (v0.2, PR-053) records deviations from its own earlier sections. All are reflected in the code as read above:

1. Architecture changed from dual index-scan to single forward walk. §4's original sketch was two forward merges over valid_time_index (producing sorted (kind, logicalId) streams to merge-join). The as-built engine instead does one forward walk per primary column family (node_versions, then edge_versions) in key order, classifying each contiguous id-group against both coordinates in a single pass. The changelog explains why: the temporal indexes are time-ordered, not id-ordered, so producing "sorted (kind, logicalId) streams" from them would require a sort or materialization step that §9 already rejected. The compare.max_scan_rows ceiling bounds the cost that concerned §9 about chain-walking, and the read path's winner rule already walks candidate histories regardless.
2. Filters are node-scoped in v1, and edge deltas are skipped entirely (stats.edgesSkippedByFilter) whenever find or where is set. where applies to the comparison side only, exactly as later pinned in §6.
3. Edge endpoint presence is a real, memoized check, not merely an assertion, because deterministic-id ingestion intentionally creates self-healing dangling references (ingestion-provenance §17).
4. The delta shape is fully pinned: added entries carry full C-state properties; removed entries carry a reason of tombstone or validityLapse (the latter also covering "not yet known" on the tx axis); modified entries carry versions.{baseline,comparison}.{validTimeStart,txTimeStart}, a props.{added,removed,changed} diff (with values on added/removed, not just keys), additive labelsAdded/labelsRemoved, and touch. Property equality is exact Value equality, so an Int → Float type change on the same key counts as changed.
5. Cursor format is pinned: HMAC-signed per the query-language cursor rules; payload last is a kind byte followed by the UUID of the last completed entity group; the plan hash is xxh3 of the canonical request JSON with cursor stripped; limit (1..=1000, default 100) counts emitted delta entries, not scanned rows.
6. Config and surfaces: compare.max_scan_rows (plus its env key) is the only compare-specific config. The feature ships on POST /v1/compare (raw-body parse, pointered COMPARE_INVALID errors), gRPC Compare (same JSON grammar), MCP compareSnapshots (flat coordinate params, three shaped sections plus counts), and the TypeScript SDK's compare() on both transports.
7. Test posture: a brute-force differential over a roughly 50-event scripted history crossed with a coordinate-pair grid, the PRD's named Q1-vs-Q2 scenario, pagination exactness (including ceiling-partial composition), and cursor scope/plan binding, across tests/compare_e2e.rs, REST endpoint tests, gRPC roundtrip tests, and MCP conformance tests.

Related¶

• The version record model, the storage layout and winner-rule proof that both snapshot and compare read through.
• Global temporal scan, the general-purpose scan path that /v1/query (and therefore /v1/snapshot) uses when a request is not a point lookup.
• Concepts › The tritemporal model, the three clocks and the winner rule in plain terms.
• Developers › REST API, the POST /v1/compare and POST /v1/snapshot request/response contracts, status codes, and error codes.