Temporal write & read paths¶

Architecture

Every write to a node or edge appends one immutable version and, if a prior version was open, closes it, atomically. Every read resolves a winner across two time axes without ever mutating or deleting a row. This page is the mechanics of both paths: the write mutex, the single WriteBatch, the covering indexes, and the bitemporal winner rule.

Overview & purpose¶

The tritemporal model (Concepts › The tritemporal model) promises that history is append-only and that any past belief is reconstructable. That promise is only as good as the code that writes and reads versions. Two modules carry the whole guarantee:

Edge versions follow the same shape in the sibling edge-ops path (not covered here); this page is scoped to the node write/read path, which the edge path mirrors key-for-key.

Design¶

Why atomicity matters. A version write is never a single put. Creating a node's second version must, in the reader's eyes, happen in one instant: the prior version's tx_time closes at exactly the moment the new version's tx_time opens. If those were two separate writes, a reader landing between them would see either two "current" versions or zero, and the global temporal scan (which walks the covering indexes independently of the primary CF) would disagree with a direct point lookup. node_ops.rs puts the closure of the prior row and the insertion of the new row, plus both rows' covering-index entries, in one WriteOp batch handed to StorageEngine::batch_write, which RocksDB commits as a single WriteBatch (core-engine/src/storage/rocks.rs). Either all of it becomes visible or none of it does.

Why a write mutex. update_node and upsert_node are read-modify-write: they first find the current open version, then compute the new version's inherited valid_time and the prior version's closure, then write. Two concurrent updates to the same node reading the same "current" version before either writes would both think they are closing the same prior row and would race to open the next one, corrupting the "exactly one open version" invariant. NodeOps holds a single std::sync::Mutex<()> (write_lock) and serializes every create-with-id, update, and upsert call through it. The code comments this as a deliberate v1 simplification:

// v1 write serialization: updates read-modify-write the prior
// version, so concurrent updates to one node must not interleave.
// Coarse by design; shard per logical id if PR-029 shows contention.
write_lock: Mutex<()>,

create_node (fresh UUIDv7, no existing row possible) is the one path that skips the lock; every other entry point acquires it before touching storage.

Why covering indexes. The primary node_versions CF is keyed [scope][logical_id][inv(valid_start)][inv(tx_start)], so a point lookup or per-entity history scan is cheap, but there is no way to ask "which versions were valid as of date X, across every entity in the tenant" without a second axis. valid_time_index and tx_time_index invert that: they lead with time instead of identity, and their stored value is the exact primary key bytes of the version they describe. indexing.rs states the rule plainly: every version write, with no exceptions, emits both index entries in the same batch as the primary put. That is what makes the global temporal scan a set of point lookups against node_versions rather than a full scan with a filter.

Data model¶

TxClock¶

TxClock is a process-wide hybrid clock: one AtomicU64 holding the last issued timestamp. now() returns max(wall_clock_µs, previous + 1), so it is strictly increasing across every thread in the process even when the wall clock stalls, steps backward (NTP correction), or two calls land in the same microsecond:

pub struct TxClock {
    last: AtomicU64,
}

impl TxClock {
    pub fn now(&self) -> Ts {
        let wall = SystemTime::now()
            .duration_since(UNIX_EPOCH)
            .map(|d| d.as_micros() as u64)
            .unwrap_or(0);
        let mut prev = self.last.load(Ordering::Relaxed);
        loop {
            let next = wall.max(prev + 1);
            match self.last.compare_exchange_weak(prev, next, Ordering::AcqRel, Ordering::Relaxed) {
                Ok(_) => return next,
                Err(observed) => prev = observed,
            }
        }
    }
}

Ts is a plain u64 (microseconds since the Unix epoch). One TxClock instance is shared by the whole engine, so the per-logical-id total order the PRD requires falls out of the process-wide total order for free: no two writes, anywhere, are ever stamped with the same tx_time.

WriteBatch contents¶

There is no dedicated WriteBatch struct in the temporal module; the batch is a Vec<WriteOp> built up by node_ops.rs and handed to StorageEngine::batch_write, which RocksDB turns into a native rocksdb::WriteBatch and commits with one db.write_opt call. WriteOp is the storage layer's atomic unit:

pub enum WriteOp {
    Put { cf: &'static str, key: Key, value: Vec<u8> },
    Delete { cf: &'static str, key: Key },
}

For an update_node call, the batch built by write_next_version contains, in order:

create_node (first version, no prior row) emits only rows 4 to 6 plus any schema/bitmap ops, since there is nothing to close. All of this ships in the one call to self.engine.batch_write(batch).

Cursor format¶

get_history paginates by returning an opaque cursor string. The cursor encodes the exact key coordinates of the last row returned so the next page can resume immediately after it:

/// Cursor = hex of (validTimeStart BE, txTimeStart BE). Opaque to
/// callers; only meaningful with the same (scope, logical_id).
fn encode_cursor(valid_start: Ts, tx_start: Ts) -> String {
    let mut bytes = [0u8; 16];
    bytes[..8].copy_from_slice(&valid_start.to_be_bytes());
    bytes[8..].copy_from_slice(&tx_start.to_be_bytes());
    bytes.iter().map(|b| format!("{b:02x}")).collect()
}

The result is a 32-character lowercase hex string (16 bytes x 2 hex digits). Decoding (decode_cursor) rejects anything that is not exactly 32 hex characters, returning NodeReadError::BadCursor; it never panics on attacker-controlled input. The cursor is scoped implicitly: it is only meaningful when replayed against the same (scope, logical_id) that produced it; nothing in the cursor bytes themselves records that scope, so the caller is responsible for calling get_history with the same identifiers again. ScanRange::node_history_after turns the decoded (valid_start, tx_start) back into a scan range that starts strictly after that key: it reconstructs the exact primary key and appends a single 0x00 byte, which is the smallest byte string greater than a fixed-width key and smaller than any later key of the same width.

Algorithms & invariants¶

The write path, step by step¶

create_node and update_node share the same shape once a request is normalized; the difference is only whether a prior version exists.

create_node / create_node_with_id:

1. Allocate (or validate) the logical id. create_node mints a fresh UUIDv7, so no existence check is needed. create_node_with_id acquires write_lock first and scans node_history for any existing row (open, closed, or tombstoned) under that id; any hit is NodeOpsError::AlreadyExists, because logical ids are never reused.
2. Draw tx = clock.now().
3. Build the NodeVersion: sort and dedup labels, default valid_time to Interval::open(tx) unless the caller supplied one, stamp tx_time = Interval::open(tx), tombstone = false.
4. Emit the version's primary put plus its two covering-index puts (version_index_ops), plus schema and bitmap ops.
5. engine.batch_write(batch); on success, confirm the schema cache entries (schema-inference spec: cache confirmation only happens after the batch is durable).

update_node / upsert_node:

1. Acquire write_lock (held for the whole call).
2. latest_version: scan node_history in latest-first order and return the first row whose tx_time.is_open(). No open row means NotFound. upsert_node additionally treats NotFound as "go create instead," and a tombstoned latest row as TombstonedSkip / update_node's hard Tombstoned error, since a dead entity is never silently resurrected.
3. Draw one new tx = clock.now() for the whole operation.
4. Build the new version, inheriting the prior version's valid_time unless the request overrides it.
5. Cap the prior version's tx_time.end = tx. This is a re-put of the same primary key the prior row already lives at (stored.key_valid_start, stored.key_tx_start); the row's coordinates never change, only its stored value.
6. Emit six index/version puts (three for the closure, three for the new row), then schema and bitmap ops.
7. engine.batch_write(batch) commits closure + insertion + all index maintenance as one unit; then confirm schema cache entries.

The read path: winner resolution¶

NodeReads exposes three entry points, all built on the same scan:

pub fn get_current(&self, scope: &TenantScope, logical_id: Uuid)
    -> Result<Option<NodeVersion>, NodeReadError>;

pub fn get_as_of(&self, scope: &TenantScope, logical_id: Uuid, valid_t: Ts, tx_t: Ts)
    -> Result<Option<NodeVersion>, NodeReadError>;

pub fn get_history(&self, scope: &TenantScope, logical_id: Uuid, limit: usize, cursor: Option<&str>)
    -> Result<HistoryPage, NodeReadError>;

get_as_of(scope, id, valid_t, tx_t) is the primitive the other two build on:

1. Scan every version of logical_id (ScanRange::node_history, which returns rows latest-valid-time-then-latest-tx-time first, but the scan here is exhaustive: it does not stop early).
2. For each version, keep it only if both version.valid_time.contains(valid_t) and version.tx_time.contains(tx_t) hold. Interval::contains is a plain half-open check: t >= start && t < end.
3. Among survivors, the winner is the one with the highest tx_time.start, i.e. the most recently recorded belief that was in force at tx_t.
4. If the winner is a tombstone (tombstone == true), return None instead of the version: a tombstone winning means the entity reads as deleted at that coordinate.

get_current(scope, id) is exactly get_as_of(scope, id, now, now) using wall-clock microseconds for both coordinates.

get_history(scope, id, limit, cursor) does not resolve a winner; it streams every stored version, latest-first, unfiltered by tombstone status, paginated:

1. Pick the scan range: ScanRange::node_history for the first page, or ScanRange::node_history_after(scope, id, valid_start, tx_start) (decoded from the cursor) to resume.
2. Pull rows until limit is reached; if a row remains after that, set more = true and stop (the extra row is not returned, only used to know a next page exists).
3. If more remain, encode a cursor from the last returned row's key coordinates; otherwise next_cursor = None.

The write batch¶

sequenceDiagram
    autonumber
    participant Caller
    participant NodeOps
    participant Lock as write_lock (Mutex)
    participant Clock as TxClock
    participant Engine as StorageEngine
    participant Rocks as RocksDB WriteBatch

    Caller->>NodeOps: update_node(scope, id, request)
    NodeOps->>Lock: acquire
    activate Lock
    NodeOps->>Engine: scan_prefix(node_versions, node_history(id))
    Engine-->>NodeOps: rows, latest first
    NodeOps->>NodeOps: latest_version = first row with open tx_time
    NodeOps->>Clock: now()
    Clock-->>NodeOps: tx (strictly increasing)
    NodeOps->>NodeOps: cap prior.tx_time.end = tx
    NodeOps->>NodeOps: build new version (inherit valid_time, tx_time open)
    NodeOps->>NodeOps: version_index_ops for prior key (closure)
    NodeOps->>NodeOps: version_index_ops for new key
    NodeOps->>NodeOps: schema_ops + bitmap_ops
    NodeOps->>Engine: batch_write([prior Put, prior vIdx, prior txIdx,<br/>new Put, new vIdx, new txIdx, schema..., bitmap...])
    Engine->>Rocks: build WriteBatch, db.write_opt(batch)
    Rocks-->>Engine: committed atomically (all-or-nothing)
    Engine-->>NodeOps: Ok(())
    NodeOps->>NodeOps: confirm_schema (cache, post-commit)
    NodeOps->>Lock: release
    deactivate Lock
    NodeOps-->>Caller: Ok(new NodeVersion)

Configuration¶

There is no configuration for the write mutex (it is unconditional, coarse, process-wide per NodeOps instance) and none for TxClock (one instance, no tunables). get_history's limit is caller-supplied per call, clamped to a minimum of 1 in code; the version-record spec does not define a global page-size ceiling for this path (the global temporal scan has its own max_page_size, which is a different code path).

As-built notes¶

Section 14 of the spec records one changelog entry relevant to this path:

• 2026-07-05, v0.2: Provenance gained clarification: Option<Uuid> (PR-060; temporal-clarification spec §5), set on a pending best-guess version at ingestion and on the corrected version a bound answer produces. This is additive, skip-if-none: it does not change the write batch shape, the closure mechanism, or the winner rule described above. The Provenance struct constructed in node_ops.rs test fixtures already carries this field.

Two details in the spec's prose are worth flagging as narrower than what the code actually does:

• The spec's Data Models section (§5) does not itself describe the write mutex, the exact WriteBatch op ordering, or the cursor byte layout; those are implementation choices this page verified directly against node_ops.rs, indexing.rs, and node_reads.rs rather than against spec prose.
• The spec's API Contracts section (§6) states plainly that "closing a prior version writes a new record with a capped interval" and that "there is no in-place mutation API," both of which match the code exactly: the closure is WriteOp::Put at the unchanged key, never WriteOp::Delete or a partial-value update.

Related¶

• The version record model
• Storage engine & key layout
• Global temporal scan
• Concepts › The tritemporal model