Field encryption¶

Architecture

Classified fields (PII, financial data, secrets) are encrypted in the application layer, before a record ever reaches Telha core. Core stores an opaque, magic-prefixed ciphertext envelope and never holds a decryption key or plaintext for a classified value.

Overview¶

Companion to Concepts › Tenancy & security, which states the product-level guarantee: classified fields are encrypted before core ever sees them, and the ciphertext is bound to the record it was created for. This page is the engineering treatment underneath that claim: the exact envelope bytes, the key hierarchy that produces them, the integration point that makes encryption unavoidable on the sanctioned write path, and the rotation and fail-closed rules that keep the guarantee honest over time.

Three problems the spec (§2) calls out by name: nonce management at volume (the original v7 draft specified AES-GCM, whose 96-bit random nonces demand rotation discipline as volume grows), which fields count as sensitive (tribal knowledge is a breach postmortem waiting to happen), and how keys rotate without rewriting history.

Design¶

Cipher and the AES-GCM to XChaCha20-Poly1305 deviation¶

The spec's v7 predecessor said "AES-GCM encrypts sensitive fields" and stopped there. Ratified decision F2, recorded in §9 as a deliberate deviation from that wording, replaces it with XChaCha20-Poly1305 and random 192-bit nonces: at 96 bits, GCM's nonce space forces a rotation policy purely to manage birthday-bound collision risk at high write volume. XChaCha's 192-bit nonce space makes collision probability negligible at any realistic volume, so nonce hygiene stops being a reason to rotate; rotation remains, but only for compromise response and policy (default 180 days, a reminder, not enforced, see Configuration).

Nonces come exclusively from the platform CSPRNG, libsodium randombytes_buf (security condition 3). Every random draw and every AEAD call in the package goes through one shared handle, app-layer/packages/enterprise/src/field-crypto/sodium.ts, so there is a single place a counter- or derived-nonce scheme would have to be substituted, and it is not.

Classification is declarative, not tribal knowledge¶

Each module ships a classification.ts mapping its Telha labels to a list of classified fields, each tagged with a reason:

export type ClassificationReason = "pii" | "financial" | "secret";

export interface ClassifiedField {
  field: string;
  reason: ClassificationReason;
}

export interface LabelClassification {
  /** Explicitly empty = "we decided nothing here is classified". */
  classified: ClassifiedField[];
}

export type ModuleClassification = Record<string, LabelClassification>;

ClassificationRegistry.register() (classification.ts) refuses a second module registering the same label with a different classification (ClassificationConflictError, one owner per label), and requireClassifiedFields(label) throws ClassificationMissingError when a label has no entry at all. Absence of classification must be an explicit classified: []. There is no default, and no way to write a record under a label nobody has made a decision about.

No side door: encryption lives on the transport, not on trust¶

The spec's §6(c) security condition calls for a "no side door" guard so a module cannot bypass encryption by calling the SDK directly. As built (§14 changelog item 4), this is implemented one level down from a runtime check: registerFieldCryptoModule (module.ts) re-registers the TELHA DI token itself with a TelhaConnector whose session transports are wrapped in EncryptingTransport. Every TelhaClient a module can obtain from the container already encrypts on write and decrypts on read; there is no unwrapped client to reach for. This subsumes the refusal condition rather than merely satisfying it: plaintext cannot cross the sanctioned path at all, and labels with no classification entry are still refused outright by the registry regardless.

Data model¶

Envelope format¶

The wire form is two magic bytes followed by a msgpack-encoded map. The magic is the detection contract: it is what the core-side opacity tests and the vector-storage exclusion function key on, both in raw bytes and behind one layer of base64.

export const ENVELOPE_MAGIC = Uint8Array.from([0x7e, 0x1a]);
export const ENVELOPE_VERSION = 1;
export const ENVELOPE_ALG = "xchacha20poly1305"; // not configurable (see Configuration)
export const NONCE_BYTES = 24;
export const TAG_BYTES = 16;

export interface Envelope {
  v: typeof ENVELOPE_VERSION;
  alg: typeof ENVELOPE_ALG;
  keyId: string;
  nonce: Uint8Array;   // 24 bytes
  ct: Uint8Array;      // ciphertext, variable length
  tag: Uint8Array;     // 16 bytes, Poly1305 MAC
}

encodeEnvelope() (envelope.ts) writes the msgpack body with field names v, alg, key_id, nonce, ct, tag (the on-wire msgpack map key is key_id, snake_case; the in-memory TypeScript field is keyId), prefixed by the two magic bytes:

[ 0x7E 0x1A ][ msgpack({ v, alg, key_id, nonce(24), ct, tag(16) }) ]
  2 bytes        variable length, self-describing

decodeEnvelope() is strict, not lenient: it rejects any v other than 1, any alg other than "xchacha20poly1305", a missing or empty key_id, a nonce that is not exactly 24 bytes, or a tag that is not exactly 16 bytes, throwing EnvelopeFormatError in every case. Magic-prefixed bytes are always treated as an envelope and never passed through as opaque data; a shape violation is a loud failure, not a silent pass.

The {"$bytes": ...} wire convention¶

Plain JSON cannot express raw bytes, and the opacity contract requires the bytes stored in core to be the magic-prefixed envelope, not a base64 rendering of it as text. The as-built wire convention (§14 changelog item 2, a normative addition recorded in the grpc-contracts spec) is:

• Write: a classified property is sent as {"$bytes": "<base64>"} at the top level of that property. Core's write DTOs turn this into Value::Bytes, so the stored bytes are the raw envelope, magic first.
• Read: Value::Bytes renders back as a plain base64 string (an existing convention predating this spec).

propertyToEnvelopeBytes() (envelope.ts) auto-detects both shapes on read: a bare base64 string, or the {$bytes: base64} object. A value is only treated as a candidate if it is valid base64 of at least 4 characters; it is only treated as an actual envelope if the decoded bytes start with the magic. Anything that looks like an envelope but fails the strict decodeEnvelope() checks throws rather than passing through, so ciphertext-shaped garbage is never silently written back as data.

AAD construction¶

Every encrypt and decrypt call binds an AAD (additional authenticated data) built from four parts, each length-prefixed so no concatenation of distinct tuples can collide:

AAD = u32LE(len(tenant_id)) ‖ tenant_id
    ‖ u32LE(len(label))     ‖ label
    ‖ u32LE(len(logical_id))‖ logical_id
    ‖ u32LE(len(field))     ‖ field

FieldCryptoService.buildAad() (service.ts) implements this exactly: four UTF-8 parts (tenantId, a label, the record's logicalId, and the field name), each preceded by its 4-byte little-endian length.

The label component is the lexicographically smallest label on the record, not "the classifying label" (confirmed directly in service.ts: [...labels].sort()[0] ?? "", used identically on both the encrypt and decrypt paths). The spec's §14 changelog records this as a deliberate choice: the label is re-derivable on read regardless of later registry changes, so a label being reclassified afterward does not change how an existing envelope authenticates. Records are single-labeled in practice, so this rarely resolves more than one candidate, but the rule is defined for the general case.

The security review (v0.2, §14) added the logical_id component as a condition, not part of the original draft. Without it, an envelope authenticates on tenant, label, and field alone, so an attacker with write access could cut a ciphertext value from one record and paste it into another record sharing the same tenant, label, and field, and it would still decrypt successfully (a classic cut-and-paste gap). Binding the record's own identity into the AAD closes this: a transplanted envelope's AAD no longer matches, and crypto_aead_xchacha20poly1305_ietf_decrypt_detached fails, surfaced as EnvelopeAuthError. A direct consequence is that a classified-label record must have a logical id before the first encrypt call; encryptRecordInput() generates a UUIDv7 client-side when the caller did not supply one (input.id ?? uuidv7()), which required the PR-043 wire addition of client-supplied ids on record creates (recorded normatively in the grpc-contracts spec).

flowchart LR
    T["tenant_id"] --> AAD
    L["label<br/>(lexicographically smallest on the record)"] --> AAD
    I["logical_id<br/>(UUIDv7, app-generated at encrypt time)"] --> AAD
    F["field name"] --> AAD
    AAD["AAD = length-prefixed concat"] --> ENC["XChaCha20-Poly1305 AEAD"]

Key hierarchy: KEK wraps per-tenant DEKs¶

flowchart TB
    KMS["KmsProvider (KEK holder)<br/>v1: EnvKekProvider, FIELD_CRYPTO_KEK env"]
    DEK1["Tenant A active DEK<br/>wrapped_dek (PostgreSQL row)"]
    DEK2["Tenant B active DEK<br/>wrapped_dek (PostgreSQL row)"]
    CACHE["DekCache: decrypted DEKs,<br/>TTL + bounded + zeroize on drop"]
    VAL["Field value plaintext"]
    ENV["Ciphertext envelope"]

    KMS -->|wrapDek / unwrapDek, AAD = tenant_id| DEK1
    KMS -->|wrapDek / unwrapDek, AAD = tenant_id| DEK2
    DEK1 -->|unwrap, cached| CACHE
    CACHE -->|XChaCha20-Poly1305, AAD = tenant‖label‖logical_id‖field| VAL
    VAL --> ENV

Two independent AEAD operations, both XChaCha20-Poly1305, at two different levels:

• Field-level encrypt/decrypt: DEK encrypts the field value; AAD is the four-part construction above.
• DEK wrap/unwrap: the KEK encrypts the DEK itself. EnvKekProvider (keys.ts) uses the tenant id as AAD on the wrap operation (`telha-dek-wrap:${tenantId}`), so a wrapped DEK row cannot be replayed under a different tenant's row even if the bytes were copied between rows.

DekRecord (keys.ts): {keyId, tenantId, wrappedDek, created, state}, where state is "active" | "retiring" | "destroyed". Persisted via the DekStore interface; the enterprise-schema MikroOrmDekStore (entities.ts) maps it to a PostgreSQL table field_crypto_deks with columns key_id (uuid, primary key), tenant_id (uuid, indexed), wrapped_dek (blob), created, state (string, length 16). A MemoryDekStore exists for tests and single-process demos. Only the wrapped DEK is ever persisted; plaintext key material never touches the database.

EnvKekProvider is the v1 KmsProvider implementation: a 32-byte KEK read from FIELD_CRYPTO_KEK (hex if it matches ^[0-9a-fA-F]{64}$, base64 otherwise). Its wrapped-DEK wire format is nonce(24) ‖ ct‖tag (combined mode). Per the spec (§14 changelog item 5) and the code, KmsProvider is an interface; pointing production at a real KMS is a deployment configuration change, not a code change.

Decrypted DEKs are cached in DekCache (keys.ts), keyed by key_id, with a TTL and a maximum entry count (see Configuration). The cache is one of the security-review conditions and has specific, non-obvious hygiene built in:

• Defensive copies both directions. get() returns new Uint8Array(entry.dek), not the stored buffer; put() stores new Uint8Array(dek), not the caller's buffer. The module doc comment records why: a naive shared-buffer zeroize corrupted in-flight encryptions under concurrency, caught by a test. The cache owns its own copy and only ever zeroes that copy.
• Bounded, LRU-ish eviction. put() evicts the oldest entry (by Map insertion order, refreshed on get()) once maxEntries is reached.
• Zeroize on every removal path. drop() calls entry.dek.fill(0) before deleting the map entry, on expiry, on explicit eviction, and on invalidation.
• Immediate invalidation on rotation. invalidateTenant(tenantId) drops every cache entry for that tenant. rotate() calls this unconditionally, so a compromised-then-rotated key cannot keep serving decryptions from cache until its TTL expires.

Algorithms & invariants¶

Encrypt path¶

sequenceDiagram
    participant Mod as Module (via TelhaClient)
    participant ET as EncryptingTransport
    participant Svc as FieldCryptoService
    participant Reg as ClassificationRegistry
    participant KMS as KmsProvider / DekCache
    participant Core as Telha core

    Mod->>ET: POST /v1/records
    ET->>Svc: encryptRecordInput(tenantId, record)
    Svc->>Reg: requireClassifiedFields(label) per label
    Reg-->>Svc: classified field names (throws if label unregistered)
    Svc->>Svc: id = input.id ?? uuidv7()
    Svc->>KMS: dekForWrite(tenantId) [provision-once if no active DEK]
    KMS-->>Svc: active DEK (unwrapped, cached)
    Svc->>Svc: nonce = randombytes_buf(24), AAD = tenant + label + logical_id + field
    Svc->>Svc: XChaCha20-Poly1305 encrypt_detached
    Svc-->>ET: record with {"$bytes": base64(envelope)} properties
    ET->>Core: POST with encrypted properties
    Core-->>ET: stored RecordVersion (opaque bytes)
    ET-->>Mod: decrypted response (see Decrypt path)

encryptRecordInput() (service.ts) unions the classified fields declared across every label on the record, and returns the input unchanged if that union is empty. For each classified field present on the record, a value that is already envelope-shaped is left alone (idempotent for rotation rewrites and retries); anything else is encrypted and replaced with the {"$bytes": ...} wire shape.

Decrypt path and auto-detection¶

decryptRecord() (service.ts) walks every property on a record, independent of what the classification registry currently says about that field: it detects envelope shape via propertyToEnvelopeBytes(), and for each hit, decodes the envelope, resolves the DEK for its key_id, rebuilds the same AAD from the record's own labels/id/field, and calls crypto_aead_xchacha20poly1305_ietf_decrypt_detached. This auto-detection by shape (not by current policy) is deliberate: a field that was declassified since it was written still decrypts correctly on read, and a field reclassified into a different reason tag doesn't change how its existing ciphertext authenticates, since the AAD only ever depended on the label set and identity, not the reason tag.

Any AEAD failure, wrong key, tampered ciphertext, or an AAD mismatch from a transplanted envelope, raises EnvelopeAuthError naming only the label, field, and key_id, never plaintext or ciphertext bytes.

EncryptingTransport: the integration point¶

EncryptingTransport (transport.ts) implements the SDK's Transport interface and wraps whichever transport a session actually uses (RestTransport or GrpcTransport; both speak the same TransportRequest shape), intercepting on request():

Because registerFieldCryptoModule re-registers the TELHA container token with a connector whose transportWrap always installs EncryptingTransport, every session client obtained through the sanctioned module path gets this interception automatically; there is no separate opt-in step per call site.

Index exclusion: classified fields are invisible to core¶

Core never receives plaintext for a classified field, only the envelope bytes, so it structurally cannot index or embed the plaintext, there is nothing on the core side to exclude by policy because nothing readable ever arrives. Two places in core-engine additionally treat envelope-shaped values as ciphertext defensively, verified directly in the code:

• Embedding assembly (core-engine/src/vector/embed.rs, contains_envelope()): before any node text is sent to an embedding provider, every property value is checked for the envelope magic, in raw Value::Bytes form and in base64-encoded string form (base64_prefix_is_envelope decodes just the leading characters and checks for the 0x7E 0x1A prefix). A value that trips either check is dropped from the assembled text before the provider ever sees it, enforced in one function per the vector-storage spec's "ONE function, tested" requirement, not scattered checks.
• GDPR masking/scanning (core-engine/src/erasure/mask.rs, core-engine/src/erasure/scan.rs): both skip values carrying the same 0x7E 0x1A magic, so masking and PII-scan sweeps never attempt to rewrite or pattern-match ciphertext.

The product consequence, stated plainly in the tenancy & security concept page: classified fields are not searchable by plaintext query or semantic similarity. That is the accepted tradeoff for keeping them out of core entirely, not an undisclosed limitation. The spec's Alternatives Considered (§9) explicitly rejects deterministic/searchable encryption for the same reason: it leaks equality patterns, which defeats the point of classifying the field at all.

Key rotation and the destruction gate¶

sequenceDiagram
    participant Job as BullMQ rotation job
    participant Svc as FieldCryptoService
    participant Raw as Raw (non-encrypting) client
    participant Enc as Encrypting client

    Job->>Svc: rotate(tenantId)
    Svc->>Svc: new active DEK, prior active becomes retiring
    Svc->>Svc: cache.invalidateTenant(tenantId)
    Svc-->>Job: {keyId, retiredKeyId}
    loop each classified label, each org session
        Job->>Raw: page through records (raw read, sees stored key_ids)
        Raw-->>Job: RecordVersion with envelope key_ids
        Job->>Svc: decryptRecord(tenantId, record) [locally, retiring key still decrypt-only valid]
        Job->>Enc: records.update(id, {labels, properties: plaintext})
        Enc-->>Job: re-encrypted under the new active key
    end
    Job->>Raw: re-scan: count remaining non-active key_ids
    Raw-->>Job: remainingOldKeyRefs

rotate(tenantId) (service.ts) mints a new active DEK, flips the prior active DEK's state to "retiring" (decrypt-only from that point forward, per dekForRead still accepting non-destroyed states), and invalidates the tenant's cache entries immediately.

The sweep (rotateTenantAndReEncrypt(), rotation.ts) then re-encrypts forward only, consistent with Telha's append-only write model: it reads existing records through a raw (non-encrypting) client so it can see the stored envelope key_ids, decrypts them locally, and writes a full snapshot back through the encrypting client (records.update), which re-keys the classified fields under the now-active DEK. Historical Telha versions keep their original ciphertext untouched and remain readable under the retiring key; only new versions are written.

A field that was declassified since it was written (no longer present in the registry's classified list for its label) comes out of the sweep as plaintext. The code comment is explicit that this is the intended declassification path, not a bug.

The destruction gate: RotationSweepResult.remainingOldKeyRefs counts envelope references still pointing at non-active keys after the sweep completes, computed by a second raw pass over every classified label and session. Per the spec (§6) and the rotation module's own comment, this should reach zero; a non-zero count means a concurrent writer raced the sweep, and the caller should re-run the sweep before considering destruction. destroy(keyId, order) (service.ts) is the explicit crypto-shred operation: it flips a key's state to "destroyed" and invalidates that tenant's cache, and a destroyed key can never decrypt again (dekForRead explicitly rejects state === "destroyed" with KeyUnavailableError). The spec frames destruction as gated on either zero remaining references from a sweep or an explicit crypto-shred order tied to GDPR erasure; the order parameter ({authorizedBy, reason}) is accepted as audit context for the caller's trail, but the code notes the store row itself keeps only the state flip, no free text, in the key inventory.

Rotation runs as a BullMQ queue named field-crypto-rotation (ROTATION_QUEUE in rotation.ts), one job per tenant, carrying the tenant id and the org sessions to sweep. Re-running a sweep is idempotent: records already under the active key are skipped (keyIds.delete(keyId); if (keyIds.size === 0) continue;).

Fail-closed behavior¶

Every failure mode in this package resolves to an explicit error, never a silent plaintext write or a silent ciphertext passthrough:

The pre-crypto tenant check is its own security condition (§6(a)): the DEK record's tenant is checked against the caller's scope before any unwrap or AEAD call happens, in both the cache-hit and cache-miss branches of dekForRead(). The AAD's tenant component would also fail authentication for a cross-tenant key_id, but failing earlier, before any cryptographic operation runs, avoids turning the decrypt call into a timing or error-shape oracle against key material that does not belong to the caller.

Logging hygiene is enforced by construction, not by convention: every error class in errors.ts accepts only label, field, and key_id in its constructor, so there is no code path by which a careless console.error(err) could leak plaintext or envelope bytes, because those values are simply never passed into an error object in the first place.

Configuration¶

As-built notes¶

From the spec's §14 changelog, reconciled against app-layer/packages/enterprise/src/field-crypto/:

• Core wire additions (v0.3): client-supplied id on record creates (recorded normatively in the grpc-contracts spec) exists because the AAD needs the logical id at encrypt time; and a full-snapshot record-update surface (PUT /v1/records/:id, SDK records.update) was added because the original §4 "re-encrypt-forward" design had no way to write new versions of an existing record until this landed. EncryptingTransport's PUT branch and rotation.ts's use of encrypted.records.update both confirm this is the path rotation actually uses.
• {"$bytes": base64} write convention: confirmed in envelope.ts (envelopeToWireProperty) and propertyToEnvelopeBytes, which auto-detects both the write shape and the plain-base64 read shape.
• AAD label = lexicographically smallest label on the record: confirmed verbatim in service.ts (buildAad, [...labels].sort()[0] ?? ""), used identically in decryptRecord's error path. This is a v0.3 clarification of the original wording ("the classifying label"), chosen so envelopes stay re-derivable and authenticate the same way even after a later registry change.
• Transport-level integration (v0.3): TelhaConnector's transportWrap and registerFieldCryptoModule's re-registration of TELHA are both present exactly as the changelog describes, and this is confirmed to subsume the §6(c) no-side-door refusal: there is no unwrapped client reachable from the DI container once the module is registered.
• KMS binding as an interface: KmsProvider (keys.ts) is exactly the interface described, with EnvKekProvider as the v1 implementation, 32-byte KEK from FIELD_CRYPTO_KEK, DEK-wrap using XChaCha20-Poly1305 with the tenant id as AAD.
• DEK cache defensive copies: the changelog's account of a naive-shared-buffer zeroize bug caught by a concurrency test matches the cache implementation's copy-in/copy-out design in keys.ts exactly; the "first-DEK provisioning is serialized per tenant in-process" claim matches FieldCryptoService's provisioning map (provisionDekOnce). The changelog's suggestion that the PostgreSQL store add a partial unique index on (tenant_id) WHERE state='active' is noted in the code as a recommendation; this page did not verify whether that index migration has actually been applied to the schema.
• Rotation as described: the BullMQ queue name (field-crypto-rotation), the raw-read-then-encrypting-write sweep shape, remainingOldKeyRefs as the destruction gate, and destroy(keyId, order) as the explicit crypto-shred operation all match the code in rotation.ts and service.ts exactly.
• Nonce statistical test: the spec's §14 describes a 64-bit-prefix uniqueness check plus a byte-uniformity band, env-scaled (TELHA_NONCE_TEST_N, default 200k, spec run at 10,000,000). This test file was not read for this page; its existence and parameters are taken from the changelog, not independently verified against a test file.
• Token signing keys (OQ-1, resolved at security review): join the same key inventory and rotation audit trail as a distinct non-tenant key class, but token verification does not route through the KMS; they stay on the existing in-process keyset rotation mechanism. This is an inventory/reporting-level claim about the admin view, not something this page verified in the field-crypto source itself; see Operators › Key & secret management for the full key inventory including token keys.

Related¶

• Concepts › Tenancy & security: the product-level guarantee this page implements. Classified fields are encrypted before core ever sees them, bound to the record they were created for.
• Operators › Key & secret management: the full secret inventory, rotation runbooks, and what breaks if the KEK is lost.
• gRPC contracts: the client-supplied-id and record-update wire additions this feature required.
• Data-split routing: how module writes are routed to the sanctioned Telha client that EncryptingTransport wraps.
• GDPR erasure: crypto-shred as an explicit destruction order, and the envelope-magic exclusion in masking and PII scans.