Spec: Memory Watch — Durable Subscriptions Over the Change Feed¶

File: .ai/specs/integration/2026-07-06-memory-watch.md Status: Approved (2026-07-06, J. Porter — all OQs resolved through v0.2/v0.3; ratified alongside the memory-cards OQ-1 security countersign that cleared the last Phase-3d gate. Durable WatchFire records in v1; NO channel content in v1, shared watches fan out to authorised DMs; cleared-channel deferred as a post-Phase-3d candidate) Owners: Core lane (store, evaluator, delivery jobs); app-layer lane (chat rendering rides chat-delivery-surface); reviewer: J. Porter Related: PR-087 (this spec), PR-088 (saved-answers consumes the internal delivery target), PR-091 (chat cards + weekly brief rendering — chat-delivery-surface addendum), global-temporal-scan spec (the tx-axis index IS the change feed), job-queue-leases spec (delivery + debounce ride jobs machinery), chat-delivery-surface spec (single-bot rendering, ACL floor, idempotent send), temporal-clarification spec (clarification lifecycle events), gdpr-erasure spec (watch owner/delivery refs join the surface register), mcp-tools spec (createWatch/listWatches/deleteWatch addendum)

1. Overview¶

Defines Telha's subscription primitive: a watch is a durable, tenant-scoped subscription to memory changes — an entity, a source, a label, a stored query, a clarification, or a saved answer — evaluated asynchronously and deterministically from the append-only transaction-time index, debounced, and delivered to chat (via the existing clarify-bot), to operator-registered webhooks, or to internal job queues. This spec owns the watch model, the evaluation semantics (checkpointed feed sweep, replayable), the event taxonomy, and the delivery/permission contract. It is the foundation for change alerts, weekly memory briefs, and saved-answer staleness (PR-088).

2. Problem Statement¶

Telha is currently a pull-only system: memory changes silently, and nothing tells a human, team, agent, or application that facts they rely on have moved. Concretely: (a) a steward who resolved a clarification never learns when the same entity's facts change again; (b) an agent that cached context from /v1/query has no signal to refetch; (c) "saved answers become stale" (PR-088) is impossible without change detection — answers would be passive records; (d) a supplier-risk team cannot say "tell me when anything about Meridian changes" — the core product promise of an active memory layer; (e) ad-hoc polling per consumer would re-scan the store N times with no shared checkpoint, no debounce, and no permission model at delivery time. Without a single, durable, permission-aware subscription primitive, every downstream feature reinvents change detection badly.

3. Proposed Solution¶

Normative decisions:

The change feed is the existing tx-axis covering temporal index. Every version write (node and edge) already lands an append-only tx-index row in the same batch (PR-016). Watch evaluation is a per-tenant sweep of that index from a durable checkpoint (last_evaluated_tx) — no write-path hooks, no new feed structure. Evaluation is therefore deterministic and replayable by construction: re-running a sweep from the same checkpoint over the same store yields the same events.
Evaluation is asynchronous, after commit. A core ticker (jobs/reaper pattern) sweeps each tenant's feed segment, matches changed rows against that tenant's compiled watch selectors, and coalesces matches into pending fires. Writers never block on watches, and a watch bug can never fail a write.
Exactly-once feed consumption, at-least-once delivery. The sweep enqueues delivery jobs AND advances the tenant checkpoint in one WriteBatch — a crash re-runs at most the un-checkpointed segment and re-produces the same jobs idempotently (job id derived from (watch_id, fire_window)). Delivery itself is at-least-once with the chat surface's existing idempotent-send (message_ref) and webhook retry semantics.
Watch scope types (v1): entity (logical id), label (all entities with a label), source (source_id — new versions of a document/connector item), query (stored query JSON, validated against the existing AST + ceilings at creation), clarification (lifecycle of one clarification), answer (reserved for PR-088; created only via the internal seam). Event kinds: fact_changed, fact_tombstoned, new_source_version, clarification_opened, clarification_bound, clarification_conflicted, query_results_changed, answer_stale (PR-088-emitted). confidence_dropped from the product plan is deliberately excluded: decay is computed at query time (confidence-decay spec), it is not a stored state transition and has no feed row to observe.
Debounce is mandatory. Every watch has a coalescing window (default 1h, floor 60s); events within a window collapse into one fire carrying the event set. Digest cadences (immediate | hourly | daily | weekly) layer on top: a weekly-cadence watch IS the Weekly Memory Brief (delivered per authorised recipient, §3.7). No per-event chat spam is expressible.
Noise-filtering is inherited, not built: connector resyncs and re-ingests that produce identical content take the upsert matched-no-write / content-hash dedup paths (ingestion-provenance §17) and write no version rows — so they generate no events by construction. Only genuine changes reach the feed.
Permissions apply twice, and every recipient is an RBAC-checkable identity (review decision, 2026-07-06). Creation requires the caller's scope to read the watched target (validated at create). Delivery re-checks at fire time per person: chat deliveries pass the clarify-bot's existing ACL-floor + role gate for the bound PERSON; a delivery the recipient may no longer see is skipped with a reason code, and N consecutive gate-denials (default 5) auto-suspend the watch with an audit event. No memory content is delivered to a chat channel in v1. Channel membership is administered in Teams/Slack, outside Telha's RBAC and audit — so it is not an access boundary, and even entity names/counts are content that would leak to unauthorised channel members (present and future). A shared/team watch therefore fans out to the DMs of the individually authorised members (each RBAC-checked as above), never to a channel message. Posting to a literal channel as a destination is deferred to a governed cleared-channel model (OQ-5): an admin explicitly binds a channel to a scope, ledgered, accepting membership-as-clearance — until then, the channel is at most a content-free "doorbell" and the facts always arrive through a per-person RBAC-checked surface (DM or the authenticated web app).
Webhook targets are operator-registered (admin-audience API, PR-083 boundary), referenced by id from watches; tenant API keys can never point a watch at an arbitrary URL. Webhook egress follows the generation-orchestration stance (https-or-loopback) and payloads are HMAC-signed.
One chat egress. Chat delivery is a notify:watch job leased by the existing clarify-audience bot; the chat-delivery-surface spec owns rendering (PR-091 adds the card/brief templates). Core never talks to Teams/Slack directly.
Watches are stored as append-only versioned documents in the schema CF under namespaced label hashes watch/* (the retention/* precedent) — definition changes and state transitions are new versions, giving the audit trail for free, and watches never appear in memory query results.
Every fire is a durable product record, not just an audit line (review decision, 2026-07-06 — OQ-2 stance changed). Fires power stale-answer alerts, briefs, steward escalation, and "why was I notified?" support questions — they must survive delivery-job cleanup. The sweep persists a minimal WatchFire row (§5) in the same WriteBatch as the delivery job + checkpoint; the rendered card stays in the job, the fact of the fire is durable. Downstream consumers reference fires by id (PR-088's check:answer payload carries the originating fire_id, so "which watch made this answer stale" is answerable).

4. Architecture¶

writes (NodeOps/EdgeOps/CascadeOps/ClarifyOps) ──▶ tx-axis index rows (existing, same batch)
                                                        │  append-only change feed
      [watch] ticker (per tenant, bounded rows/tick)    ▼
      checkpoint ──▶ sweep feed segment ──▶ match compiled selectors
                                               │ coalesce per watch (debounce window)
        ONE WriteBatch: pending-fire rows + delivery jobs + checkpoint advance
                                               │
        ┌──────────────────────────────────────┼───────────────────────────┐
   notify:watch job (jobs CF)          webhook executor (core,        internal job kind
   leased by clarify-audience bot      HMAC, https-or-loopback,       (e.g. check:answer,
   → per-person ACL/role gate →        operator-registered            PR-088)
     DM card / digest brief            targets only)
     (shared watch = fan-out to
      authorised members' DMs;
      no channel content in v1)

5. Data Models¶

Watch document (schema CF, watch/* namespace, msgpack, versioned): {watch_id (uuid), owner {person_id? , api_key_id?}, scope_type, scope_ref (uuid | label | query_json), events: [kind], debounce_secs, cadence, target {chat_person {person_id} | chat_shared {audience: role | [person_id]} (fan-out to authorised members' DMs, §3.7) | webhook {webhook_id} | internal {job_kind}}, enabled, suspended {reason?}, created_at, last_fired_tx?, result_fingerprint? (query watches: xxh3 of sorted (id, version_tx) pairs, row-capped)}. Tenant checkpoint row (one per tenant, same namespace): {last_evaluated_tx}. Fire/event payload (delivery job payload JSON, versioned {v:1}): {watch_id, window {from_tx, to_tx}, events: [{kind, entity_id?, source_id?, clarification_id?, label?, counts}], truncated?} — payloads carry ids and kinds, never property values or span text (rendering fetches what the recipient may see). WatchFire record (schema CF, watch-fire/* namespace, keyed (watch_id, fire_tx) for per-watch scans, msgpack {v:1}): {fire_id (uuid), watch_id, fire_tx, window {from_tx, to_tx}, events: [{kind, changed_ref_type, changed_ref_id, change_tx}] (id/kind coordinates only, bounded + truncated flag), delivery_target_kind, delivery_job_id?, status (fired | delivered | skipped {reason} | suppressed), created_at} — minimal by design (review decision): no rendered content, no property values; summary_hash optional for card-idempotence checks. Fires are pruned oldest-first beyond fire_keep_per_watch; the audit sink remains the unbounded record. Webhook registration (admin-scoped document): {webhook_id, url, secret_ref (env name), enabled, created_by}. Invariants: scope_ref validated at create (query JSON through the existing AST incl. ceilings); event payload rows are bounded by max_events_per_fire with an explicit truncated flag (no silent caps).

6. API Contracts¶

REST (API-key or tenant-bearer auth, same middleware family as /v1/records): POST /v1/watches (create; 400 WATCH_INVALID with pointered errors, 403 on unreadable target, 409 WATCH_LIMIT at tenant cap), GET /v1/watches[?scopeType=&enabled=], GET /v1/watches/:id, GET /v1/watches/:id/fires (durable WatchFire rows, newest first, cursor-paged), DELETE /v1/watches/:id, POST /v1/watches/:id/enable|disable. Owner or admin may delete; suspension is system-set, re-enable clears it. Internal seam: WatchOps::emit(tenant, event) for PR-088's answer_stale (never exposed over the wire). Jobs: new kind notify:watch; the clarify audience's poll-prefix set gains notify: (fence tests both directions — worker/connector audiences must NOT lease it, clarify audience must not gain ingest kinds). Webhook delivery: POST {url} with x-telha-signature: hmac-blake3-v1=<hex> over the raw body, 2 retries with backoff, dead-letter after max_attempts with watch suspension + audit. MCP (mcp-tools addendum): createWatch / listWatches / deleteWatch, shaped by the standard envelope; scope binds from the session tenant token as with all MCP tools.

7. UI/UX¶

Chat rendering is owned by chat-delivery-surface (PR-091): Memory Change card and the Weekly Memory Brief digest (counts of facts changed, clarifications resolved, answers gone stale, links into evidence), both delivered to authorised recipients' DMs — a shared/team watch fans out per member (§3.7), never a channel post. Until PR-091 lands, delivery targets webhook and internal are fully usable and chat fires park as leasable jobs (forward-compatible). Brief rendering deduplicates per (recipient, changed_ref_id, change_tx) — the core records every fire independently (a change legitimately fires an entity watch, a label watch, and a query watch; those are different subscriptions), dedup is a rendering concern (OQ-3 resolution). CLI (this PR): telha watch ls|show|fires|create|rm|enable|disable and telha debug watch-checkpoint --tenant (checkpoint position + feed lag). Every fire, skip (gate-denied, truncated, suspended), and suspension is a structured audit event.

8. Configuration¶

[watch]: enabled (false), eval_interval_secs (60), max_watches_per_tenant (1000), max_rows_per_tick (50_000 — bounded sweeps, backlog resumes next tick), debounce_default_secs (3600), debounce_floor_secs (60), max_events_per_fire (100), query_watch_max_results (1000), suspend_after_gate_denials (5), webhook_timeout_secs (10), webhook_max_attempts (3), fire_keep_per_watch (1000 — WatchFire pruning ceiling; audit sink stays complete). Deliberately NOT configurable: async-after-commit evaluation, checkpoint+enqueue atomicity, per-person RBAC at delivery (no channel content in v1), operator-only webhook registration, the audience fence, payloads-carry-ids-not-content.

9. Alternatives Considered¶

Write-path hooks (evaluate synchronously on write). Rejected: couples every write to watch-selector cost, a watch bug becomes a write failure, and replay is impossible. The feed sweep gets determinism and crash-safety for free from structures that already exist.
A new change-feed CF (CDC log). Rejected: the tx-axis index is already an append-only, same-batch, tenant-scoped record of every version write — a second log would double write amplification to store the same information.
Watches as graph nodes (CLARIFICATION precedent). Rejected: watches are operational subscriptions, not organizational memory — they must not surface in query results, briefs, or compare output. The retention/* schema-CF document precedent fits exactly.
Direct tenant-supplied webhook URLs. Rejected: arbitrary egress from core on tenant input is an SSRF/exfiltration surface; operator registration keeps the egress inventory auditable (generation-orchestration precedent).
Per-event immediate delivery without debounce. Rejected: a connector resync touching 500 entities must not send 500 DMs; coalescing windows are the difference between an active memory layer and a spam generator.
Fire history in the delivery job's event log only (the v0.1 draft position). Rejected at review (2026-07-06): the jobs CF permits deletes, so the only durable trace of a fire would be the audit sink — provable at system level but a weak product object once fires power staleness, briefs, escalation, and "why was I notified?" questions. Minimal durable WatchFire rows (§5) fix this without a heavy archive.
Polling /v1/compare per consumer. Rejected as the primitive: compare answers "what changed between two coordinates" on demand; it cannot share checkpoints, debounce, or delivery permissions across consumers. (Rendering MAY use compare to describe a fire.)

10. Implementation Approach¶

One core PR (PR-087), spec lands as the first commit: watch/webhook documents + checkpoint row + WatchOps (create/list/delete/suspend, selector compilation) → evaluator ticker (bounded sweep, coalesce, atomic enqueue+checkpoint) → notify:watch kind + audience-fence extension → webhook executor → REST + CLI + MCP addendum → §12 tests. PR-091 (separate, app-layer) adds card/brief rendering to clarify-bot; PR-088 consumes the internal target. Ordering: PR-087 has no dependency on 088/091 and must land first.

11. Migration Path¶

Greenfield and additive: new schema-CF namespace, new job kind, new REST routes; ships watch.enabled = false. No existing CF, wire message, or SDK surface changes shape. Watch documents carry a version byte for forward evolution; the fire payload is versioned JSON ({v:1}). Enabling on an existing store starts the checkpoint at the current tx watermark (no historical backfill storm — recorded, not configurable).

12. Success Metrics¶

watch_entity_fires_on_change: entity watch fires exactly once for a version write within one debounce window; payload carries ids/kinds only.
watch_no_fire_on_dedup_resync: re-ingesting identical content (upsert matched-no-write path) produces zero events (§3.6 proven, not assumed).
watch_debounce_coalesce: N changes to a watched entity inside the window produce one fire with N events (or truncated at the cap).
watch_checkpoint_replay: crash injected between sweep and delivery re-runs the segment with no lost and no duplicate delivery jobs (id = watch+window).
watch_tenant_isolation: tenant A's changes never match tenant B's watches; checkpoint rows are tenant-scoped by construction.
watch_delivery_gate_lost_access: recipient loses ACL-floor access between create and fire → delivery skipped with reason code; after the configured denials the watch suspends with an audit event.
watch_shared_fanout_rbac: a shared/team-target fire delivers to the DMs of exactly the authorised members and to no one else; an unauthorised member of the same audience receives nothing; no channel message is emitted for any watch (structural — no channel-post path exists in v1).
watch_query_diff: query watch fires only when the result fingerprint changes; entered/left sets are correct on a seeded transition.
watch_clarification_lifecycle: clarification watch fires on opened → bound → conflicted transitions and nothing else.
watch_webhook_signature_and_egress: webhook payload verifies against the shared secret; http:// non-loopback target refused at registration.
watch_fence: worker/connector audiences cannot lease notify:watch; clarify audience cannot lease ingest:/sync: (both directions, extending the existing fence matrix).
watch_fire_durable: a fire's WatchFire row survives delivery-job completion AND deletion; GET /v1/watches/:id/fires returns it with correct status transitions (fired → delivered / skipped); rows prune oldest-first at fire_keep_per_watch with the audit sink retaining the full record.

13. Open Questions¶

~~OQ-1: Should label watches support a where predicate filter (mini-query per changed row)?~~ Resolved 2026-07-06 (J. Porter): no in v1 — filtered subscriptions are query watches; label-level predicates create an awkward middle ground (row-predicate evaluation in the sweep without query-watch semantics, cost model, or explainability). Label watches stay cheap and simple.
~~OQ-2: Event history durability — is the audit sink + delivery-job event log enough?~~ Resolved 2026-07-06 (J. Porter, STANCE CHANGED): no — minimal durable WatchFire records are v1 (§3.11, §5). Fires become product-significant the moment they power stale-answer alerts, briefs, steward escalation, and "why was I notified?" — a system-level audit line proves something happened but is a weak product object. Keep records minimal (fact-of-fire + coordinates, rendered content stays in the job); "which watch caused this saved answer to become stale" must be answerable by id.
~~OQ-3: Cross-watch dedup for briefs?~~ Resolved 2026-07-06 (J. Porter): accept duplicate fires in v1 — an entity watch, a label watch, and a query watch firing on the same change are different subscriptions and the core records each independently. Dedup is a rendering concern: PR-091 briefs dedup per (recipient, changed_ref_id, change_tx).
~~OQ-4: system-namespace watches for operators?~~ Resolved 2026-07-06 (J. Porter): out of scope — memory watches answer "did organisational memory change?", operator monitoring answers "is Telha healthy?" (connector failures, sync lag, queue depth belong to PR-086b admin/ops endpoints). Do not mix the two.
OQ-5: Governed cleared-channel delivery — should a Telha admin be able to bind a channel to a scope (accepting membership-as-clearance, ledgered) so shared cards can post to a literal channel? Current stance (delegated review 2026-07-06): keep deferred as a named post-Phase-3d candidate (its own small design pass; paired with memory-cards OQ-3). v1 has NO channel-post path; shared watches fan out to authorised DMs (§3.7). If built, the cleared-channel model keeps the doorbell/room split — a channel post carries at most a content-free "activity" signal, and the facts still arrive per-person RBAC-checked. Making channel membership an implicit access boundary is rejected outright (it forks Telha RBAC to whoever administers the workspace); an explicit, audited admin binding is the only acceptable form and is not in v1.

14. Changelog¶

2026-07-06 — v0.4: Approved (J. Porter). Ratified with the Phase-3d set once the memory-cards OQ-1 write-action security countersign cleared the last gate; no design change from v0.3. PR-087 is buildable on dependency order alone.
2026-07-06 — v0.3: channel-delivery reconsidered at review (J. Porter): channel membership is NOT a Telha access boundary, so no memory content goes to a channel in v1. §3.7 rewritten — every recipient is an RBAC-checkable PERSON; a shared/team watch fans out to authorised members' DMs (target chat_channel → chat_shared {audience}, §5); even names/counts are content and channels are administered outside Telha RBAC/audit (and future members inherit backlog). Literal channel posting deferred to a governed cleared-channel admin binding (new OQ-5, doorbell/room split). Test watch_channel_summary_only → watch_shared_fanout_rbac; §3.5/§4/§7/§8 dechannelled. The v0.1/v0.2 "summary-only channel" carve-out is withdrawn as defending the wrong line.
2026-07-06 — v0.2: OQ review by J. Porter — all four resolved, one stance changed. OQ-2 reversed: fire history must be durable in v1 → new §3.11 normative decision, minimal WatchFire record in §5 (watch-fire/* namespace, keyed (watch_id, fire_tx), persisted in the same WriteBatch as delivery job + checkpoint; status fired/delivered/skipped/suppressed; fire_keep_per_watch pruning with the audit sink as the unbounded record), GET /v1/watches/:id/fires + telha watch fires, PR-088 linkage (check:answer carries fire_id), rejected-alternative bullet for the v0.1 job-log-only position, new test watch_fire_durable. OQ-1 (no label predicates — query watches), OQ-3 (duplicate fires accepted; PR-091 dedups per recipient+ref+change_tx, now pinned in §7), OQ-4 (no system watches — ops monitoring is PR-086b) resolved as drafted. Status → In Review.
2026-07-06 — v0.1: initial draft (PR-087). Feed = existing tx-axis index with checkpointed sweeps; async-after-commit; atomic enqueue+checkpoint; six scope types + eight event kinds (confidence_dropped excluded with rationale); mandatory debounce + digest cadences; dual permission check with auto-suspend; operator-registered webhooks; single-bot chat egress via notify:watch; watch/* schema-CF documents per the retention precedent.