Skip to content

Watch & subscriptions

Developers

A watch is a durable, tenant-scoped subscription to memory changes. It turns Telha from a pull-only store into an active memory layer: instead of re-querying to find out whether facts you rely on have moved, you subscribe once and Telha tells you when they do, with the same provenance and permissions as any other answer.

Build status (Phase 3d)

The watch engine is implemented in core-engine/src/watch/: the watch model, the change-feed evaluator, coalescing/debounce, delivery jobs, the in-core webhook runner, and durable WatchFire records. The public surface described below (the REST /v1/watches routes, the telha watch CLI, and the MCP createWatch / listWatches / deleteWatch tools) and the chat cards are specified in the memory-watch and memory-cards specs and are being wired. The subsystem ships dark: [watch] enabled = false by default. Check your build before relying on the endpoints.

How it works, in one picture

The change feed is not a new structure: it is the append-only transaction-time index every version write already lands. A per-tenant ticker sweeps that feed from a durable checkpoint, matches changed rows against compiled watch selectors, coalesces matches per watch inside a debounce window, and, in one write batch, persists the fire, enqueues delivery, and advances the checkpoint.

flowchart LR
    W["Version writes<br/>(node, edge, cascade, clarify)"] --> IDX["tx-axis index<br/>(the change feed)"]
    IDX --> SWEEP["Watch evaluator<br/>(per-tenant sweep from checkpoint)"]
    SWEEP --> MATCH["Match compiled selectors<br/>coalesce per watch (debounce)"]
    MATCH --> BATCH["ONE write batch:<br/>WatchFire + delivery job + checkpoint"]
    BATCH --> CHAT["notify:watch<br/>(chat DM via clarify bot)"]
    BATCH --> HOOK["webhook:watch<br/>(in-core HMAC runner)"]
    BATCH --> INT["internal job<br/>(e.g. check:answer)"]

Evaluation is asynchronous and after-commit, so writers never block on watches and a watch bug can never fail a write. It is deterministic and replayable: a crash re-runs at most the un-checkpointed segment and re-produces the same delivery jobs idempotently (the job id derives from (watch_id, fire_window)).

What you subscribe to: scope types

A watch names one target through its scopeType and scopeRef.

Scope type scopeRef Fires on
entity a logical id (UUID) any change to that one entity
label a label name any entity with that label changing
source a source_id (UUID) a new version of that document or connector item
query a stored query JSON the query's result set changing (entered/left)
clarification a clarification id that clarification's lifecycle transitions
answer reserved saved-answer staleness (internal seam only)

Query watches validate the stored query against the live AST and ceilings at creation time, and track a result_fingerprint (an xxh3 of the sorted (id, version_tx) pairs) so they fire only when the result set actually changes.

What you hear about: event kinds

A watch subscribes to one or more events. The v1 set:

Event Meaning
fact_changed a watched entity got a new version
fact_tombstoned a watched entity was tombstoned (deleted)
new_source_version a watched source produced a new version
clarification_opened a clarification was opened
clarification_bound a clarification was resolved and bound
clarification_conflicted answers to a clarification conflicted
query_results_changed a query watch's result set changed
answer_stale a saved answer went stale (emitted internally)

No confidence_dropped

Confidence decay is computed at query time (see Confidence & decay). It is not a stored state transition and has no feed row to observe, so it is deliberately not a watch event.

Not too loud: debounce and cadence

Every watch has a mandatory coalescing window (debounceSecs, default 3600, floor 60). Events inside a window collapse into one fire carrying the event set, so a connector resync touching 500 entities is one notification, not 500.

On top of debounce, a cadence sets the digest rhythm:

Cadence Behaviour
immediate deliver each fire as it coalesces
hourly / daily batch fires into a periodic digest
weekly the Weekly Memory Brief

No spam by construction

Connector resyncs and re-ingests that produce identical content take the upsert matched-no-write / content-hash dedup path and write no version rows, so they generate no events at all. Only genuine changes reach the feed.

Where it goes: delivery targets

A watch's target is one of four shapes. There is no channel target: channel membership is administered in Slack/Teams, outside Telha's RBAC and audit, so it is not an access boundary.

{"chatPerson": {"personId": "0193...-person-uuid"}}

Delivered to one bound person's DM, re-checked against that person's RBAC at fire time.

{"chatShared": {"audience": {"persons": ["0193...", "0194..."]}}}

Fans out a DM to each authorised member. A member who fails the fire-time gate receives nothing. No single shared message is ever posted to a channel.

{"webhook": {"webhookId": "0195...-webhook-uuid"}}

An operator-registered webhook (see Webhooks). Tenant keys can never point a watch at an arbitrary URL.

{"internal": {"jobKind": "check:answer"}}

An internal job kind (for example saved-answer staleness). Not creatable over the wire.

The API surface

Being wired (Phase 3d)

The engine methods (WatchOps::create_watch / list_watches / get_watch / delete_watch / emit) exist today; the REST, CLI, and MCP surfaces below are specified and landing. Treat this section as the target contract.

REST

Method & path Purpose
POST /v1/watches create a watch (owner bound from the caller, never the body)
GET /v1/watches list watches (?scopeType=&enabled= filters)
GET /v1/watches/:id fetch one watch
GET /v1/watches/:id/fires durable fire history, newest first, cursor-paged
POST /v1/watches/:id/enable · /disable toggle; re-enable clears a suspension
DELETE /v1/watches/:id delete (owner or admin)

Errors follow the uniform envelope: 400 WATCH_INVALID (with a JSON pointer to the offending field), 403 on an unreadable target, 404, and 409 WATCH_LIMIT at the per-tenant cap.

Create a watch

The request body is camelCase and rejects unknown fields. owner is never in the body; it is bound from the authenticated caller.

POST /v1/watches
{
  "scopeType": "entity",
  "scopeRef": "0193aaaa-bbbb-cccc-dddd-000000000001",
  "events": ["fact_changed", "fact_tombstoned"],
  "debounceSecs": 3600,
  "cadence": "immediate",
  "target": {"chatPerson": {"personId": "0193...-person-uuid"}}
}

Creation requires that the caller's scope may read the watched target (validated at create). "Tell me when anything about Meridian changes" is a query watch:

A query watch over open supplier risks
{
  "scopeType": "query",
  "scopeRef": "{\"find\": \"RISK\", \"where\": {\"supplier\": {\"$eq\": \"Meridian\"}, \"status\": {\"$eq\": \"open\"}}}",
  "events": ["query_results_changed"],
  "cadence": "daily",
  "target": {"chatShared": {"audience": {"persons": ["0193...", "0194..."]}}}
}

CLI and MCP

  • CLI: telha watch ls | show | fires | create | rm | enable | disable, plus telha debug watch-checkpoint --tenant <id> (checkpoint position and feed lag).
  • MCP: createWatch / listWatches / deleteWatch, shaped by the standard MCP envelope; scope binds from the session tenant token.

Fires and history

Every fire persists a minimal, durable WatchFire record (the fact of the fire plus id/kind coordinates, never property values or span text). Fires outlive their delivery job, so GET /v1/watches/:id/fires answers "what fired, and did it deliver?" long after the card is gone.

A WatchFire (ids and kinds only)
{
  "fireId": "0195...-fire-uuid",
  "watchId": "0193...-watch-uuid",
  "fireTx": 1751500000000000,
  "window": [1751496400000000, 1751500000000000],
  "events": [
    {"kind": "fact_changed", "changedRefType": "node",
     "changedRefId": "0193aaaa-...-000000000001", "changeTx": 1751499000000000}
  ],
  "truncated": false,
  "deliveryTargetKind": "chat_person",
  "status": "delivered",
  "createdAt": 1751500000000000
}

Fires are pruned oldest-first beyond fireKeepPerWatch; the audit sink keeps the complete record. A fire's status moves fired -> delivered or fired -> skipped (with a reason) or suppressed.

Webhooks

Webhook targets are operator-registered (an admin-audience concern), never a tenant-supplied URL. A watch references a webhook by id.

  • Egress follows the generation stance: https://, or http:// only to a loopback host.
  • Payloads are HMAC-signed. The receiver verifies x-telha-signature: hmac-blake3-v1=<hex> over the raw body.
  • Delivery retries (default 3 attempts) then dead-letters, which suspends the watch with an audit event.
Webhook payload (ids and kinds only)
{
  "watchId": "0193...-watch-uuid",
  "window": {"fromTx": 1751496400000000, "toTx": 1751500000000000},
  "events": [{"kind": "new_source_version", "sourceId": "0192...", "changeTx": 1751499000000000}],
  "truncated": false
}

Permissions

Permissions apply twice, and every recipient is an RBAC-checkable identity.

  1. At create, the caller's scope must be able to read the watched target.
  2. At fire time, per person, delivery re-checks the bound person's access. A recipient who can no longer see the target is skipped with a reason code. After suspendAfterGateDenials (default 5) consecutive denials, the watch auto-suspends with an audit event.

No memory content reaches a chat channel in v1. Even entity names and counts are content; a shared watch fans out to authorised members' DMs instead. Payloads carry ids and kinds only, so rendering fetches exactly what the recipient may see.

Configuration

[watch] in the config file:

Key Default Meaning
enabled false master switch (ships dark)
eval_interval_secs 60 evaluator ticker period
max_watches_per_tenant 1000 cap (409 WATCH_LIMIT)
max_rows_per_tick 50000 bounded sweep; backlog resumes next tick
debounce_default_secs 3600 default coalescing window
debounce_floor_secs 60 minimum window
max_events_per_fire 100 events per fire, then truncated
query_watch_max_results 1000 tracked ids per query watch
suspend_after_gate_denials 5 auto-suspend threshold
webhook_timeout_secs 10 webhook request timeout
webhook_max_attempts 3 webhook retries before dead-letter
fire_keep_per_watch 1000 WatchFire pruning ceiling