Skip to content

Spec: JSON Query Language — Grammar & Semantics

File: .ai/specs/core-engine/2026-07-02-query-language.md Status: Draft Owners: Storage lane + API lane (joint) Related: PR-021/022; traversal-semantics spec; confidence-decay spec; global-temporal-scan spec; SDK contract tests (PR-027)

1. Overview

Defines the complete v1 grammar of the JSON query DSL, its semantics, error taxonomy, cursor rules, and the shared fixture corpus that binds core, SDKs, and MCP tools to identical behavior. Serde-first (ratified F10): the AST is the serde type definitions; there is no separate parser.

2. Problem Statement

The DSL is the contract three consumers depend on (REST clients, TS/Python SDKs, MCP tools). Without a normative grammar: operators accrete ad-hoc; error messages diverge between core and SDK validation; cursors leak implementation detail or, worse, tenant data; and "the query language" becomes whatever the executor happens to accept.

3. Proposed Solution

One versioned grammar (v:1 field, optional, defaults 1). Strict deserialization: unknown fields and unknown operators are rejected with pointered errors (where.severity.$gtee: unknown operator). A committed fixture corpus (valid + invalid + expected errors) is the single source of truth, run against core in CI by both SDKs.

4. Architecture

JSON → serde(deny_unknown_fields) → typed AST → semantic validation (depth caps, operator/type compatibility, cursor integrity) → logical plan (TemporalResolve → Scan|IndexScan → Filter → Expand → VectorScore → Paginate → Trace).

5. Data Models

{
  "v": 1,
  "find": "RISK",                          // required; label
  "where": {                                // optional; property predicates
    "severity": { "$gte": 7 },              // $eq $ne $gt $gte $lt $lte $in $exists
    "$or": [ { "status": { "$eq": "open" } },
             { "$and": [ ... ] } ]          // nesting depth <= 8
  },
  "atValidTime": "2025-05-01T00:00:00Z",    // RFC3339 or unix-µs int; default = now
  "atTxTime":    "2025-06-01T00:00:00Z",    // default = now
  "expand": [ { "type": "OWNED_BY", "direction": "out", "depth": 1 } ],  // depth<=3; shorthand: "OWNED_BY"
  "vector": { "text": "supply chain delays", "model": "m1", "k": 20, "minScore": 0.5 },  // or "near": [f32...]
  "limit": 100,                             // 1..=1000
  "cursor": "opaque",                       // from previous page
  "trace": false
}

Semantics: where evaluates on the version selected at the bitemporal coordinate; comparisons are type-strict against the Value enum (Int vs Float coerce; Str vs Int is a validation error when schema knows the type, else runtime type-mismatch skips the row and counts in trace stats). $in max 256 elements. expand semantics per traversal spec; expanded neighbors are returned in a related section, not merged into primary results. vector semantics and score fusion per confidence-decay spec. Ordering: primary results by (score desc if vector present, else key order); stable across identical inputs.

6. API Contracts

Cursor: base64(msgpack{v, scope_hash, last_key, plan_hash}) — HMAC-signed with a server key; resume validates scope_hash (tenant-checked) and plan_hash (query must be identical except cursor). Error envelope: {code: "QUERY_INVALID", pointer: "/where/severity/$gtee", message, request_id}; the fixture corpus pins exact codes+pointers. Same AST serves REST /v1/query, gRPC Query, and MCP findRecords.

7. UI/UX

SDK query builders mirror this grammar with typed generics (PR-027); MCP findRecords accepts a constrained subset shaped for LLM emission (no cursor, lower caps) — defined in mcp-tools spec, referencing this one.

8. Configuration

Server ceilings: query.max_limit (1000), query.max_where_depth (8), query.max_in_elements (256), query.max_expand entries (4). Grammar itself is not configurable.

9. Alternatives Considered

Custom text query language / Cypher subset (rejected: parser sinkhole, F10; JSON composes with every client for free). GraphQL (rejected: resolver model fights bitemporal coordinates; temporal args on every field is ergonomic poison). Unsigned cursors (rejected: opaque-but-forgeable cursors are a tenant-probing vector; HMAC closes it). Permissive unknown-field handling (rejected: silent typo acceptance — $gtee matching nothing — is a correctness trap; strictness with pointered errors is kinder).

10. Implementation Approach

PR-021: this spec → serde AST + strict deser → semantic validation → plan builder → fixture corpus (≥60 valid, ≥40 invalid cases with exact errors). PR-022 consumes the plan. Corpus file is shared verbatim with SDK repos' contract tests.

11. Migration Path

Greenfield. Grammar evolution: additive fields under v:1; breaking changes introduce v:2 with both versions served for ≥1 minor release; fixture corpus is versioned alongside.

11b. Vector activation notes (PR-040, 2026-07-03)

vector gained optional alpha ([0,1], fusion override — decay spec §6). vector.text resolves to near in the async handler layer via the embedding provider (501 UNIMPLEMENTED without a configured endpoint). Cursors do not compose with vector ranking: supplying one is QUERY_INVALID at /cursor, and vector responses never emit nextCursor (results are k-bounded, similarity-ordered). Scores surface as score per record/related node; gRPC carries primary_scores + RelatedNode.score (additive proto fields).

12. Success Metrics

Fixture corpus: 100% pass in core, TS SDK, and Python SDK CI against a live binary. Cursor security test: tampered/foreign-tenant cursors rejected. Determinism: identical query + dataset → byte-identical response (modulo request_id). Round-trip: AST → JSON → AST identity property test.

13. Open Questions

  • OQ-1: Property path predicates on nested objects ("data.owner.name")? Stance: v1 supports one-level dotted paths into Object values; deeper paths deferred until schema inference tracks nested shapes (schema-inference OQ territory).

14. Changelog

  • 2026-07-02 — v0.1: initial draft for peer review.