Spec: Snapshot Compare — Temporal Delta Engine¶

File: .ai/specs/core-engine/2026-07-02-snapshot-compare.md Status: Approved Owners: Storage lane Related: PR-053; global-temporal-scan spec (dual scans), version-record-model spec (property diff), mcp-tools spec (shaping)

1. Overview¶

Defines delta semantics between two bitemporal coordinates: added/removed/modified nodes and edges, property-level diffs, streaming pagination of large deltas, and the exact meaning of "modified" in a versioned store.

2. Problem Statement¶

"What changed between Q1 and Q2" is the product's marquee query, and it is easy to get subtly wrong: comparing versions instead of selected states reports internal churn as change; unpaginated deltas on large tenants explode; and without pinned modified-semantics, two implementations disagree on the same data.

3. Proposed Solution¶

Compare(baseline B, comparison C) where each is a full bitemporal coordinate (validTime required, txTime defaults now — enabling both "how did May change between what we knew in June vs September" and plain valid-time deltas). For each logicalId in scope (optionally label-filtered): select the winning version at B and at C (same selection rule as any read). Classification: present@C ∧ absent@B → added; inverse → removed (tombstoned-or-expired collapses to removed, with a reason tag distinguishing tombstone vs validity-lapse); present@both ∧ same version → unchanged (not emitted); present@both ∧ different versions → modified, with property-level diff (added/removed/changed keys; changed = old≠new by Value equality; Float compared exactly — display rounding is a client concern). Edges classified identically; an edge is present only if both endpoints are present at that coordinate (dangling edges impossible by cascade invariants, asserted anyway).

4. Architecture¶

Two forward merges over valid_time_index at inv(B.vT) and inv(C.vT) restricted to scope (+label filter via post-check v1), producing sorted (kind, logicalId) streams of selected versions → merge-join → classify → property-diff modifieds → stream pages.

5. Data Models¶

Delta { added: [ref], removed: [{ref, reason}], modified: [{ref, versions: (B,C), props: {added, removed, changed: [{k, old, new}]}}], stats }. Page cursor: last (kind, logicalId) — HMAC-signed per query-language cursor rules, plan-hash bound to (B, C, filter).

6. API Contracts¶

POST /v1/compare {baseline: {validTime, txTime?}, comparison: {...}, find?, where?, limit, cursor} — where applies to the C-side selection (documented; filtering "risks that were severe in C" is the common intent; B-side filtering expressible by swapping and inverting). gRPC Compare mirrors (replaces the PR-026 UNIMPLEMENTED). MCP compareSnapshots returns the shaped tabular form.

7. UI/UX¶

Chat shaping: three sections with counts, modified rows showing per-property old→new (mcp-tools caps apply). The PRD's Q1-vs-Q2 risk scenario is the named e2e.

8. Configuration¶

compare.max_scan_rows (2M, partial-with-warning beyond), page size caps shared with query config.

9. Alternatives Considered¶

Version-chain walking per entity (rejected: touches every entity's history; dual selected-state scans touch only what existed at the two coordinates). Emitting unchanged entities (rejected: deltas are for change; counts in stats suffice). Text-diffing string properties (deferred: property-level old/new is the storage-layer contract; rich diffing is presentation). Snapshot materialization then diff (rejected v1: doubles memory; merge-join streams — materialized snapshot cache is EPIC-405's problem).

10. Implementation Approach¶

PR-053 commits as planned: this spec → dual-scan merge-join → classification + property diff → endpoints + MCP activation → scripted-history correctness fixtures incl. the named PRD scenario, plus the tombstone-vs-lapse reason tagging tests.

11. Migration Path¶

Greenfield; read-only feature, no stored state beyond standard traces when requested.

12. Success Metrics¶

Correctness fixtures: scripted 50-event history, every (B, C) pair matches brute-force reference deltas. Large-delta paging: 500k-change tenant streams within memory budget, cursors resume exactly. Determinism: identical inputs → identical pages.

13. Open Questions¶

• OQ-1: Resolved as stanced (v0.2, PR-053) — same-content different-version emits modified with an empty props diff and touch: true; visible but distinguishable.

14. Changelog¶

• 2026-07-02 — v0.1: initial draft for peer review.
• 2026-07-04 — v0.2: as-built (PR-053). Deviations and pinned details:
• Architecture: ONE forward walk per primary CF (node_versions then edge_versions) streaming id-ordered version groups, each group classified against both coordinates in one pass — NOT §4's dual valid_time_index merges. The temporal indexes are time-ordered, so "sorted (kind, logicalId) streams" cannot come from them without a sort/materialization (§9 rejected that); the §9 objection to chain-walking is bounded by compare.max_scan_rows (2M, partial-with-warning + resume cursor, checked at group boundaries), and the read path's winner rule already walks candidate histories. Winner selection = read rules exactly (containing version with highest txTimeStart; winning tombstone = absent).
• Filters are node-scoped v1: find is SCOPE, matched against either side's labels (removed entities stay reportable); where applies to the C-side selection only per §6 (absent-at-C never matches — swap coordinates to filter the B side). With any filter set, edge deltas are skipped (stats.edgesSkippedByFilter).
• Edge endpoint presence is a real check, not an assertion: deterministic-id ingestion (ingestion-provenance §17) creates intentional dangling self-healing edges, so compare validates both endpoints at each coordinate (memoized per request).
• Delta shape pinned: added entries carry full C-state properties; removed carry reason: tombstone|validityLapse (lapse also covers tx-axis not-yet-known); modified carries versions.{baseline,comparison}.{validTimeStart,txTimeStart}, props.{added,removed,changed:[{key,old,new}]} (added/removed include values), additive labelsAdded/labelsRemoved, and touch. Property equality is exact Value equality (a type change Int→Float counts as changed).
• Cursor: HMAC-signed per query-language rules; payload last = kind byte ‖ uuid of the last COMPLETED entity group; plan hash = xxh3 of the canonical request JSON minus cursor; limit (1..=1000, default 100) counts emitted delta entries.
• Config compare.max_scan_rows (+ env key). Surfaces: POST /v1/compare (raw-body parse, pointered COMPARE_INVALID errors), gRPC Compare (same JSON grammar), MCP compareSnapshots (flat coordinate params; three shaped sections + counts per §7), TS SDK compare() on both transports.
• §12 gates: brute-force differential over a ~50-event scripted history × coordinate-pair grid, the named Q1-vs-Q2 scenario, pagination exactness incl. ceiling-partial composition, cursor scope/plan binding (tests/compare_e2e.rs, rest_endpoints, grpc_roundtrip, mcp_conformance).