Spec: Version Record Model & Value Serialization¶

File: .ai/specs/core-engine/2026-07-02-version-record-model.md Status: Draft Owners: Storage lane Related: PR-013; composite-key-layout spec; tombstone-cascade spec; F9 (rmp-serde ratified)

1. Overview¶

Defines the normative structure of NodeVersion and EdgeVersion — the values stored in the node_versions and edge_versions CFs — their MessagePack serialization, the value-version byte that makes stored data forward-compatible, and the interval/tombstone/provenance semantics every reader depends on.

2. Problem Statement¶

Keys give us ordering; values give us truth. Without a fixed record model: temporal end-boundary validation has no defined field to validate; tombstones have no canonical representation; provenance (which claim-verification depends on in Phase 3) has no home; and any serialization change silently corrupts old data. All four must be pinned before the first write path lands (PR-014).

3. Proposed Solution¶

Two record structs serialized with rmp-serde, prefixed by a single value-version byte (0x01 for this spec). Intervals are half-open [start, end) with end = u64::MAX meaning open-ended. Tombstones are ordinary versions with tombstone: true — never a special key form. Provenance is a mandatory struct on every version. Unknown MessagePack map keys are ignored on read (additive evolution); removing or retyping a field requires a new value-version byte and a reader for both.

4. Architecture¶

Writers (temporal engine, cascade, ingestion submit) construct records → codec::encode(record) -> [version_byte][msgpack] → StorageEngine::put. Readers do the inverse; the version byte dispatches to the matching decoder. Golden-file fixtures pin the exact bytes in CI.

5. Data Models¶

struct NodeVersion {
    logical_id: Uuid,                 // duplicated from key for self-describing values
    labels: Vec<String>,              // sorted, deduped at write time
    properties: BTreeMap<String, Value>, // Value = Null|Bool|Int(i64)|Float(f64)|Str|Bytes|Array|Object|Datetime(u64 µs)
    valid_time: Interval,             // { start: u64, end: u64 }  half-open; end==u64::MAX => open
    tx_time: Interval,                //   same semantics
    source_record_time: Option<u64>,
    tombstone: bool,
    provenance: Provenance,
}
struct EdgeVersion {
    edge_logical_id: Uuid,
    from: Uuid, to: Uuid,
    edge_type: String,                // hash lives in the key; string lives here (collision registry check on write)
    properties: BTreeMap<String, Value>,
    valid_time: Interval, tx_time: Interval,
    source_record_time: Option<u64>,
    tombstone: bool,
    cascade_of: Option<Uuid>,         // node-tombstone id when created by a cascade
    provenance: Provenance,
}
struct Provenance { origin: Origin /* Api|Ingestion{source_id, job_id}|System|Cascade */, actor: Option<String>, request_id: Option<Uuid> }

Invariants: start < end always; BTreeMap guarantees deterministic serialization (golden files stay stable); labels sorted; edge_type string must round-trip the key's xxh3 hash or the write is rejected.

6. API Contracts¶

codec::encode_node/decode_node, codec::encode_edge/decode_edge; decode errors are typed (UnknownVersionByte, Malformed, HashMismatch) and never panic. "Closing" a prior version writes a new record with a capped interval (append-only; see tombstone-cascade spec) — there is no in-place mutation API.

7. UI/UX¶

Internal-only. Forensics: telha debug decode-value <cf> <hex> pretty-prints any stored value with its version byte (ships in PR-013).

8. Configuration¶

None. Record shape is not configurable; changes require a new value-version byte plus a spec amendment and dual-reader support.

9. Alternatives Considered¶

FlatBuffers (rejected for v1, F9: premature zero-copy; doubles codec surface). Storing edge_type only as hash (rejected: forensics and export need the string; 8B hash in key + string in value is the cheap dual). Protobuf for storage values (rejected: schema registry overhead for what is an internal format; proto stays at the RPC boundary). Closed intervals (rejected: half-open composes cleanly with end-boundary validation and avoids ±1µs fencepost bugs).

10. Implementation Approach¶

PR-013 commits: this spec → structs + Value enum → rmp-serde codecs with version byte → golden-file tests (byte-exact fixtures committed; CI fails on any drift).

11. Migration Path¶

Greenfield. Evolution posture: additive fields freely (readers ignore unknown keys); breaking changes bump the version byte, ship dual readers, and background-rewrite is optional, never required (old bytes remain readable forever).

12. Success Metrics¶

Golden-file suite green (encode determinism across platforms); property test decode(encode(x))==x over randomized records; HashMismatch rejection test; a v0x02 dual-reader dry-run test proving the evolution mechanism before it is ever needed.

13. Open Questions¶

• OQ-1: Should properties support a Decimal type for financial data, or is app-layer string-encoding acceptable for v1? Stance: string-encode in v1; revisit with a schema-inference lattice extension if demand appears.

14. Changelog¶

• 2026-07-02 — v0.1: initial draft for peer review.
• 2026-07-05 — v0.2: Provenance gains clarification: Option<Uuid> (PR-060; temporal-clarification spec §5) — set on a pending best-guess version at ingestion and on the corrected version a bound answer produces. Additive named field with skip-if-none, exactly the PR-032 spans pattern: stored bytes and golden fixtures are byte-stable when absent; no version-byte bump.