[coderdan]

Summary

EQL v4 stores an encrypted JSON document in a public.eql_v4_json_search column. Until now a filter such as "some emergency contact is a Sibling and has more than five calls" could only be written as two separate tests, an equality (@>) and a range (an ordering over eql_v4.json_tree_path_query), and two different contacts could satisfy them. So the query returned more rows than the plaintext jsonpath exists($.contacts[*] ? (@.relationship == "Sibling" && @.calls > 5)) does.

This PR adds eql_v4.json_tree_element_match(col, $1::eql_v4.query_json_elements), which is true only when one array element satisfies every part of the predicate, and the Stack Encrypt query operator element_match that derives its one parameter. No stored format changes: existing documents answer the new query as they are.

Changes

EQL (packages/eql)

• src/v3/json_tree/functions.sql: new eql_v4.json_tree_subtree_indexed(e, tokens, budget DEFAULT 64) RETURNS TABLE (k integer, entry public.eql_v4_json_entry). It is the existing subtree read body, now returning k, the 1-based place in tokens of the token whose node each row sits in (0 for the sealed-root fallback past the budget). json_tree_subtree is now a projection of it, so there is one implementation.
• src/v3/json_tree/types.sql: new domain eql_v4.query_json_elements ({"m": [{"t": [token, …], "gt"|"gte": leaf, "lt"|"lte": leaf}, …]}), with a CHECK that validates each member's tokens and its bounds as query_json_leaf operands.
• src/v3/json_tree/elements.sql (new, v4 only): eql_v4.json_tree_element_match. It opens the row's document once, finds the node each member's tokens name by its self tag (the node json_tree_subtree_indexed returns at d = 0, and nothing beneath it), pairs the members' nodes by k, and compares with the existing entry-against-leaf operator functions. Each bound is checked by the same validator the query_json_leaf CHECK now calls (eql_v4_internal.json_tree_is_valid_leaf_query_payload). It raises invalid_parameter_value (SQLSTATE 22023) when a probe token finds a node (see "How the probe is detected" below).
• Catalog: eql_domains::JSON_TREE gains the ElementsQuery role (JsonSearchElementsQuery, eql_v4.query_json_elements). Generated from it: the eql-bindings v4 inventory and the Go type eql.JSONSearchElementsQuery in languages/golang/encrypt/eql/eql_gen.go. Hand-written: the JsonTreeElementsQuery binding struct, and Target::elements_query_sql_domain().
• Docs: a new "One array element in EQL v4" section in docs/reference/eql-functions.md (the function, the operand, the folded-equality caveat, the probe error, the planner cost, the hand-written EXISTS … JOIN … ON c.k = r.k form), and a bullet in json-tree-leakage.md on what the query reveals. packages/eql/AGENTS.md lists the new file and domain.
• Changeset: @cipherstash/eql minor.

Engine (packages/stack-encrypt)

• dynamic/json_query.rs: a new operator, element_match. Its input is {"path", "wildcard_bound"?, "predicates": [{"member", "op", "json" | "low"+"high"}]}, and it derives the eql_v4.query_json_elements operand. It reuses the subtree token derivation for path + member, and the leaf-operand derivation gt_at/between_at use. eq is one leaf operand used as both gte and lte. There is no new cryptographic derivation. The module's operator table has the new row and a section on element predicates.
• dynamic/query.rs, target.rs, mod.rs: OpSpec::ElementMatch (input path_predicates), TargetDescriptor::with_elements_query_sql_domain, and four new refusal reasons: json_path_no_wildcard, json_no_predicates, json_wildcard_bound_zero and json_unknown_predicate_op. OpSpec::fidelity's rustdoc says the orderings and element_match are exact on the ordering terms, so element_match's eq is an equality of folded values.
• Each member's tokens are the subtree key of its full path (json_subtree_key_now on, say, $.contacts[*].relationship), derived as subtree derives them, so the k-th token of every member names a field of one element and each ends with the probe. json/derive.rs is unchanged.
• tests/heap_hygiene.rs: the spy searches for the data key where its lock holds it, and records the key in place, rather than copying it onto the stack. See the review notes.

Verification

Everything ran locally on macOS against a private PostgreSQL 17 container (port 7541), after mise run build in packages/eql.

• mise x --env test -- cargo nextest run -p stack-encrypt --all-features: 1008 passed. That includes 8 new element_match tests in tests/json_operators.rs:

  • each op derives the ordering's leaf operand, and eq has lo = hi;
  • each member's tokens are the subtree read of its full path, and every member ends with the same probe;
  • the default bound is used when none is given;
  • a Rust model of the SQL function over sealed documents: two halves are not a match, one element with both is, and an array longer than the bound is a probe hit;
  • a proptest oracle (48 cases) against plaintext jsonpath semantics;
  • every refusal by its reason, including wildcard_bound: 0, an unknown op, and a member path error whose offset lands in the member's own text;
  • the operator is listed only with an ordered path and a named domain.
    The heap-hygiene test now covers the operator too.

• Root workspace: cargo clippy --workspace --all-targets -D warnings under default features and under --all-features, mise run wasm:wasi-check, mise run wasm:no-http-test, mise run wasm:guest:test (76 passed), cargo fmt --all --check, doctests (91) and mise run doc:stack-encrypt: clean.

• cargo-mutants, cargo mutants -p stack-encrypt --in-diff <diff against fix/eql-v4-json-function-blockers>: 35 mutants, 26 caught, 9 unviable, 0 missed (re-run after the rebase onto the new docs(eql): say what native jsonb functions do on the v4 JSON tree domains #1444 head). The unviable ones are Default::default() replacements of types that have no Default.

• SQLx lint_tests (12), build_validation_tests (5), selector_literal_guard (10) and v3_public_surface_tests (7): passed.

• EQL_TEST_DATABASE_URL=… mise run test:encryption:postgres on a fresh database: all suites passed, json_tree 28: the 14 existing json_tree PostgreSQL tests (which run json_tree_subtree unchanged), the 8 function-blocker tests docs(eql): say what native jsonb functions do on the v4 JSON tree domains #1444 had then, and six new tests in tests/json_tree/elements.rs. docs(eql): say what native jsonb functions do on the v4 JSON tree domains #1444 has since replaced those 8 tests with 4 native-function tests. After the rebase, mise run test:encryption:postgres at the top of the stack passed every suite on PostgreSQL 15 (json_tree 32). Every one seals real documents with Stack Encrypt under the fake key source and derives its operands through a data plan over eql-bindings' target table; EQL tests must run over real encrypted data, and the SQLx suite has no way to produce a JSON tree document, so the SQL's contract is pinned here rather than in a SQLx file with hand-built documents. Five of them cover:

  • the operand domain's CHECK, member by member, over a derived operand and its parts (20 refused shapes, 4 accepted);
  • one element must satisfy every member while the component orderings return the superset, under the sealed root and the open one;
  • each bound (gt, gte, lt, lte, between, eq, and two bounds on one member) compares as the entry operators do, and a field with no ordering term or no field satisfies none;
  • a probe hit raises: an array longer than the bound, and a token whose node was moved to the element's selector;
  • json_tree_subtree equals the indexed read without k under budgets 0, 1, 2, 3 and 64, k is the place of the token whose node a row sits in, and the sealed root falls back to k = 0.

  The sixth, postgres_an_element_predicate_holds_of_one_element, checks:

  • two halves are returned by @> plus json_tree_path_query, and not by json_tree_element_match;
  • one contact with both halves is returned by both queries;
  • "sibling" ties with "Sibling" under the element predicate but not under exact @>;
  • a bound of 1 raises 22023;
  • the subtree read equals the indexed read without k, and a k join pairs the fields of one contact;
  • an unordered member is refused by the engine.

• mise run test:crates (498), test:encryption (541), types:check, typescript:check and codegen:parity: passed.

• docs:validate:required-tags, docs:validate:coverage, docs:validate:documented-sql (against my own container), test:doc-anchors and test:docs_v3_grep: passed.

• Go: wasm:guest:build, wasm:auth-guest:build, wasm:guest:build:deterministic, wasm:guest:build:eql and wasm:guest:build:eql:deterministic built, then mise run go:test passed for every package.

Not run:

• v3_v4_coexistence_tests, v3_privilege_tests and v3_uninstall_tests fail to compile here: they read the credential-generated fixtures/eql_v3_integer.sql and v3_ste_vec.sql, and this environment has no CipherStash credentials. They test v3 and integer surfaces. This PR does not change the v3 bundle, because every file it touches is -- ONLY EQL VERSION: 4.
• pnpm run code:check: no JS or TS changed.
• go:test:postgres: not run.

Related

Closes #1440

Stacked on #1444. Refs #1441 (this operand becomes one leaf kind of that predicate tree) and #1442 (the Go call that wraps it).

Review notes

• Operand shape. One parameter, as preferred: {"m": [member, …]}, each member {"t": [tokens…], "gt"|"gte": leaf, "lt"|"lte": leaf}.
  • I used bound keys named by their comparison instead of plain lo/hi, because gt and lt are strict. A plain lo/hi pair could not express that without extra flags.
  • A domain CHECK can validate nested leaf operands through a PL/pgSQL validator, as query_json's group validator already does, so the single operand works.
  • The leaf check is restated inside the new validator rather than shared with query_json_leaf's inline CHECK. Sharing it would mean a new SQL helper, which pin_search_path then pins, or an eql-inline-critical allowlist entry.
• How the probe is detected. By selector, not by position. Every non-probe token of a member names the field ($.contacts[i].relationship), whose selector is the one the member's bounds carry. The probe names the element itself ($.contacts[N]), at the array's selector. So a d = 0 row whose selector differs from its member's raises.
  • This also covers nested wildcards, where probes sit in the middle of the list.
  • It raises even if an element within the bound matched. A true would be correct in that case, but the error stays deterministic per row rather than depending on which element matched first.
• Refusals Go SDK: no way to test an equality and a range on the same array element of a JsonSearch field — add an element-scoped Match on top of #1440 #1442 mirrors: json_path_no_wildcard, json_wildcard_bound_zero, json_no_predicates, json_unknown_predicate_op, json_input (keys that do not fit the op), json_path_wildcard (a wildcard in a member), and the index's JsonOrderingAtPosition, JsonPathNotOrdered, JsonLeafNotOrdered, JsonPath (at an offset into the member's own text) and JsonTooDeep. subtree still accepts wildcard_bound: 0: a probe-only read still answers whether an array is empty, where a probe-only predicate can never be true.
• The unordered-member refusal does not print the path. It is the same TermError::JsonPathNotOrdered the orderings give. The crate keeps path text out of errors, because a key name can be customer data (see JsonPathError), so the error names neither the path nor which predicate failed. Go SDK: no way to test an equality and a range on the same array element of a JsonSearch field — add an element-scoped Match on top of #1440 #1442's stashgen check runs at generate time, where the path is schema, and that is the place to name it. If you want the engine error to carry the predicate's index, it is a small follow-up.
• The Go guest is deliberately not wired. The operator is listed only when the target names the elements domain. The Go guest (languages/golang/encrypt/guest/src/targets.rs) does not name it yet, so the Go SDK sees no new operator and its pinned operator lists stay unchanged. Go SDK: no way to test an equality and a range on the same array element of a JsonSearch field — add an element-scoped Match on top of #1440 #1442 needs:
  • one with_elements_query_sql_domain(target.elements_query_sql_domain()) line in that guest;
  • element_match added to parseOperatorName (encrypt/checker.go) and to stashgen's jsonOperators.
    Without those, a catalog listing it is refused as an unknown operator.
• The heap-hygiene failure was the test's own copy of the key, now fixed; pending.rs is unchanged. Adding element_match to tests/heap_hygiene.rs made its json_open (subtree) watch fail (macOS arm64, debug): load_keysets' 256-byte future box was freed holding a data key. An earlier version of this PR boxed the keyset loads as a workaround. The cause was the spy itself:
  • What copied it: on every free during a watch, holds_key copied the recorded key out of its lock onto the stack to compare (KEY.lock().map(|key| *key)). The copy stayed in the spy's dead frame, and load_keysets' future, built at that depth moments later, carried it into its box through bytes it never writes (the unused tail of a JoinAll).
  • How it was shown: with the recorded key XOR'd with 0x5a, a value the engine never handles, the block held the XOR'd value; with a spy that never copies the key, the watch passes. Type layouts from -Zprint-type-sizes place the bytes in the JoinAll at offsets 24..112 and their copy at 160..248.
  • The fix: holds_key searches for the key where the lock holds it, and Recorded::record writes it in place. The workaround is gone. Heap hygiene passes 14 of 14 at the top of the stack.
  • Found on the way, and filed separately: the engine itself leaves copies of a data key on the dead stack, from handing keys out by value. No watch shows them reaching the heap. That, the long-lived keys not held in vitaminc's Locked, and a batch variant of Locked for per-row keys are stack-encrypt: data keys leave copies on the stack, and long-lived keys are not in Locked — hold every key in one place, by reference #1458.
• Equality is folded: text is case- and accent-folded, and integers past 2⁵³ can tie, as json_tree: an equality and a range cannot be tied to one array element — return the token index from json_tree_subtree and derive per-element reads in the engine #1440 states. Exact per-element equality would need a stored-format addition and is out of scope.
• Input values are JSON text, as in every other JSON operator: "json": "\"Sibling\"", not "json": "Sibling".

https://claude.ai/code/session_01MrxG724qYKcAWJdcABBFNY

[changeset-bot]

🦋 Changeset detected

Latest commit: b7fa5a6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@cipherstash/eql	Minor
stash	Patch
@cipherstash/stack-prisma	Patch
@cipherstash/basic-example	Patch
@cipherstash/e2e	Patch
@cipherstash/prisma-example	Patch
@cipherstash/stack-drizzle	Patch
@cipherstash/stack-supabase	Patch
@cipherstash/stack	Patch
@cipherstash/wizard	Patch
@cipherstash/bench	Patch
@cipherstash/test-kit	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI; Enforced overrides: Organization
• Review profile: CHILL
• Plan: Advanced
• Run ID: 90d42b32-9daf-4088-a190-d89411103f5a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

[github-actions]

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

caught	missed	unviable	timeout
26	0	9	0

Every mutant in the changed lines was caught by a test.

[coderdan]

Rebased onto #1444's new head (0883fb1); new head 15e9c56.

Why:

• docs(eql): say what native jsonb functions do on the v4 JSON tree domains #1444 took its review feedback: its blocker tests moved from a SQLx file with a hand-written envelope to the encryption test crate, over a document Stack Encrypt seals (EQL tests must use real encrypted data, and the SQLx suite cannot produce a JSON tree document), plus upgrade note U-004 and doc and changeset fixes.
• This PR's SQLx file v4_json_tree_element_match_tests.rs hand-built its documents the same way, so its five tests moved into packages/eql/tests/encryption/tests/json_tree/elements.rs, over documents Stack Encrypt seals and operands element_match derives (the SQL commit no longer adds the SQLx file; the test commit carries them).
• The clippy failure in fmt, clippy, nextest, doctests, rustdoc and WASI check, guests and Go (Linux): JsonSubtreeKey::below and its test now carry #[cfg(feature = "dynamic")], the feature of its only caller, folded into the stack-encrypt commit. Clippy passes under default features and --all-features, and mise run wasm:wasi-check passes.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T10:45:08.159755Z	15e9c56	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-30T10:46:18.480273Z	15e9c56	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[cipherstash-agent]

Review attestation: code-review (medium) + cross-check against existing feedback

Reviewed commit: 15e9c56 (base fix/eql-v4-json-function-blockers @ 0883fb1)
Result: no critical or blocking issues. No new findings.

Read in full:

• elements.sql
• the types.sql and functions.sql diffs
• the json_query.rs operator, input and operand derivation
• JsonSubtreeKey::below in derive.rs
• the query.rs and pending.rs diffs
• the catalog, binding and changeset changes

Not read in depth: the large new test files (elements.rs, json_operators.rs). I did not run the test suites; I relied on the PR's stated results.

Checked:

• k pairing and the probe.
  • The count-per-k pairing is sound. The CHECK requires equal token counts, so one k is one element, including with nested wildcards.
  • Probe detection by selector holds. A non-probe token's node carries the member's field selector, and the probe's carries the array's.
  • A 22023 raise on a probe hit is deterministic per row.
• json_tree_subtree as a projection of _indexed. The wrapper drops k and has one implementation, so the two reads cannot disagree.
• Leaf-validator reuse. query_json_leaf and the new elements CHECK share json_tree_is_valid_leaf_query_payload.
• Existing domain on upgrade. The IF NOT EXISTS guard keeps an existing query_json_leaf domain on its old CHECK. The validation is equivalent, so I found no behavioural difference.
• Rust refusals and offsets. The refusal ordering is consistent. member_path offset shifting is correct and saturating. below keeps probes as they are and refuses wildcards.
• Fidelity. ElementMatch fidelity matches the orderings.
• Skills and changesets.
  • No skills/*/SKILL.md references EQL v4 json_tree or its operands, so there is no skill drift.
  • The @cipherstash/eql minor changeset is present.
  • The stack-encrypt and eql-bindings crates are published through release-plz, so they need no changeset.

Cross-check: all 9 existing review threads (coderdan's efficiency, fidelity, refusal, offset, reuse, double-sort, per-member re-derivation and Box::pin items) are resolved and addressed at head, and I did not re-raise any of them.

Open for the author (already flagged in the PR notes, not a new finding): the pending.rs Box::pin is a workaround with an unidentified mechanism and adds one allocation per dispatch. The author asked for a decision. A future layout change could re-expose the heap-hygiene watch failure.

Still running: the Codex code and security reviews on this commit.

---

Generated by Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 15e9c56039

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderdan]

The typed Rust interface for the operator this PR adds is #1459: today it is reachable only through stack_encrypt::dynamic, with no compile-time checks.