Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions content/docs/reference/eql/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "The model behind every EQL page: domain variants that declare capa
type: reference
components: [eql]
verifiedAgainst:
eql: "3.0.0"
eql: "3.0.3"
---

<EqlVersion />
Expand All @@ -26,11 +26,11 @@ For any scalar type `<T>`, the family looks like this:
| `public.eql_v3_<T>_ord` | Comparisons (`<` … `>=`), `BETWEEN`, `ORDER BY`, `MIN` / `MAX` — plus equality. |
| `public.eql_v3_<T>_ord_ope` | The byte-identical twin of `_ord`, with OPE pinned. See [SEM specifiers](#sem-specifiers). |
| `public.eql_v3_<T>_ord_ore` | As `_ord`, with block-ORE pinned. |
| `public.eql_v3_text_match` (text only) | Free-text token containment: `@>` / `<@`. |
| `public.eql_v3_text_search` (text only) | Equality + ordering + token containment. |
| `public.eql_v3_text_match` (text only) | Free-text fuzzy match: `@@`. |
| `public.eql_v3_text_search` (text only) | Equality + ordering + fuzzy match. |
| `public.eql_v3_text_search_ore` (text only) | As `text_search`, with block-ORE pinned. |

Every public domain name carries the `eql_v3_` prefix. It keeps EQL's types from shadowing built-in Postgres type names such as `text` and `json`, and gives each EQL version its own column-type namespace so two versions can coexist. Query-operand domains live in the versioned `eql_v3` schema already, so they are unprefixed: `eql_v3.query_text_eq`, `eql_v3.query_jsonb`.
Every public domain name carries the `eql_v3_` prefix. It keeps EQL's types from shadowing built-in Postgres type names such as `text` and `json`, and gives each EQL version its own column-type namespace so two versions can coexist. Query-operand domains live in the versioned `eql_v3` schema already, so they are unprefixed: `eql_v3.query_text_eq`, `eql_v3.query_json`.

Two things worth calling out:

Expand Down Expand Up @@ -70,7 +70,7 @@ CREATE TABLE users (
);
```

Every scalar type — `int2`, `int4`, `int8`, `numeric`, `float4`, `float8`, `date`, `timestamp`, `text`, and `bool` in EQL 3.0.0 — ships some subset of this family. The per-category pages list exactly which variants each type has and how to choose between them: [Numbers](/reference/eql/numbers), [Dates & times](/reference/eql/dates-and-times), [Text](/reference/eql/text), and [Booleans](/reference/eql/booleans). Encrypted JSON documents use a separate domain, `public.eql_v3_json`, with its own operator surface see [JSON](/reference/eql/json).
Every scalar type — `int2`, `int4`, `int8`, `numeric`, `float4`, `float8`, `date`, `timestamp`, `text`, and `bool` in EQL 3.0.3 — ships some subset of this family. The per-category pages list exactly which variants each type has and how to choose between them: [Numbers](/reference/eql/numbers), [Dates & times](/reference/eql/dates-and-times), [Text](/reference/eql/text), and [Booleans](/reference/eql/booleans). Encrypted JSON documents use separate domains — `public.eql_v3_json_search` for searchable documents, `public.eql_v3_json` for storage-only ones — with their own operator surface; see [JSON](/reference/eql/json).

## The three schemas

Expand Down Expand Up @@ -117,9 +117,9 @@ Alongside the envelope, a payload carries the index terms for its column's capab
| Key | SEM type | Wire shape | Enables | Reveals |
| --- | --- | --- | --- | --- |
| `hm` | `eql_v3_internal.hmac_256` (domain over `text`) | Hex string (HMAC-SHA-256) | `=`, `<>` on `_eq` and `text_search` domains | Whether two values are equal — nothing else |
| `op` | `eql_v3_internal.ope_cllw` (domain over `bytea`) | Hex-encoded CLLW OPE ciphertext | `<`, `<=`, `>`, `>=`, `ORDER BY` on `_ord` / `_ord_ope` domains and `text_search`, and on String / Number leaves of `public.eql_v3_json` | The relative order of two values |
| `op` | `eql_v3_internal.ope_cllw` (domain over `bytea`) | Hex-encoded CLLW OPE ciphertext | `<`, `<=`, `>`, `>=`, `ORDER BY` on `_ord` / `_ord_ope` domains and `text_search`, and on String / Number leaves of `public.eql_v3_json_search` | The relative order of two values |
| `ob` | `eql_v3_internal.ore_block_256` (composite: array of `bytea` block terms) | Array of hex-encoded ORE blocks (block count varies by scalar width) | The same comparisons on the pinned `_ord_ore` and `text_search_ore` domains — and `=` / `<>`, since ORE comparison collapses to equality | The relative order of two values |
| `bf` | `eql_v3_internal.bloom_filter` (domain over `smallint[]`) | Array of set bit positions (**signed** 16-bit — large filters emit negative positions) | `@>` / `<@` token containment on `_match` domains | Probabilistic token overlap between values |
| `bf` | `eql_v3_internal.bloom_filter` (domain over `smallint[]`) | Array of set bit positions (**signed** 16-bit — large filters emit negative positions) | `@@` fuzzy match on `_match` and `text_search` domains | Probabilistic token overlap between values |

The capability is encoded as **required keys**: the payload for a `public.eql_v3_text_eq` column must carry `hm`; a `public.eql_v3_integer_ord` payload must carry `op` (and only `op`); a `text_match` payload must carry `bf`; a `text_search` payload carries `hm`, `op`, and `bf`. A payload missing its term key fails the domain `CHECK` — and fails to deserialize in the client bindings.

Expand All @@ -139,9 +139,9 @@ A scalar payload for a `public.eql_v3_text_search` column (lookup + ordering + f
- `v`, `i`, `c` — the envelope
- `hm` — equality term: `WHERE email = $1` compares this
- `ob` — ordering term: `ORDER BY` and range comparisons walk these blocks
- `bf` — bloom-filter term: `@>` token containment tests these bit positions
- `bf` — bloom-filter term: `@@` fuzzy match tests these bit positions

Encrypted JSON documents use a different payload shape — an `sv` array with one encrypted entry per path in the document instead of a root ciphertext — defined in [JSON](/reference/eql/json).
Encrypted JSON documents use a different payload shape — a document-level key header `h` and an `sv` array with one encrypted entry per path, instead of a root ciphertext — defined in [JSON](/reference/eql/json).

### Machine-readable schemas

Expand All @@ -154,12 +154,16 @@ The [EQL repository](https://github.com/cipherstash/encrypt-query-language) publ

The `eql_v3` domains are backed by `jsonb`. When an operand has no known type — a bare string literal, an untyped parameter — PostgreSQL reduces the domain to its `jsonb` base type and resolves the **native `jsonb` operator** instead of the encrypted one. The query doesn't fail; it silently returns native `jsonb` semantics, which are meaningless for encrypted payloads.

<BadExample label="Silently wrong — untyped parameter resolves the native jsonb operator">

```sql
-- ❌ Wrong: untyped parameter. PostgreSQL falls back to the native jsonb `=`,
-- which compares raw payloads — syntactically valid, semantically meaningless.
SELECT * FROM users WHERE email = $1;
```

-- ✅ Right: typed operand — the encrypted `=` resolves.
</BadExample>

```sql
-- Typed operand — the encrypted `=` resolves.
SELECT * FROM users WHERE email = $1::public.eql_v3_text_eq;
```

Expand All @@ -179,7 +183,7 @@ SELECT * FROM users WHERE salary > $1::public.eql_v3_bigint_eq;

A `NULL` operand still raises — the blockers are deliberately not `STRICT`, so PostgreSQL can't skip the check. (A SQL `NULL` column value is not encrypted, so `IS NULL` / `IS NOT NULL` themselves always work, on every variant.)

`LIKE` and `ILIKE` are blocked on **every** encrypted variant — pattern matching is meaningless on ciphertext. Encrypted text matching is bloom-filter token containment instead; [Text](/reference/eql/text) covers it.
`LIKE` and `ILIKE` are blocked on **every** encrypted variant — pattern matching is meaningless on ciphertext. Encrypted text matching is bloom-filter fuzzy match (`@@`) instead; [Text](/reference/eql/text) covers it.

One equality subtlety follows from the term table above, and it splits on whether the column is text.

Expand Down
27 changes: 13 additions & 14 deletions content/docs/reference/eql/filtering.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "WHERE-clause patterns on encrypted columns: equality, IN lists, ra
type: reference
components: [eql]
verifiedAgainst:
eql: "3.0.0"
eql: "3.0.3"
---

<EqlVersion />
Expand Down Expand Up @@ -60,33 +60,32 @@ WHERE occurred_at >= $1::public.eql_v3_timestamp_ord
AND occurred_at < $2::public.eql_v3_timestamp_ord;
```

## Text token matching: `@>`
## Text fuzzy matching: `@@`

There is no `LIKE` on encrypted columns — encrypted free-text matching is bloom-filter token containment via `@>` on a `text_match` or `text_search` column:
There is no `LIKE` on encrypted columns — encrypted free-text matching is bloom-filter fuzzy match via `@@` on a `text_match` or `text_search` column:

```sql
SELECT * FROM users WHERE name @> $1::public.eql_v3_text_match;
SELECT * FROM users WHERE name @@ $1::public.eql_v3_text_match;
```

The client encrypts the search term into a bloom-filter query value; matching is probabilistic (false positives possible, false negatives not). For the full no-`LIKE` story and match-term tuning, see [Text](/reference/eql/text).
The client encrypts the search term into a bloom-filter query value; matching is probabilistic (false positives possible, false negatives not). `@>` / `<@` raise on these domains — they were the old spelling of this match and are now blockers. For the full no-`LIKE` story, short-term behaviour, and match-term tuning, see [Text](/reference/eql/text).

## JSON containment and path filters

Encrypted JSON documents (`public.eql_v3_json`) filter by containment and path existence:
Encrypted JSON documents (`public.eql_v3_json_search`) filter by containment and path existence:

```sql
-- Does the document contain this (encrypted) structure?
SELECT * FROM orders WHERE metadata @> $1::eql_v3.query_jsonb;
SELECT * FROM orders WHERE metadata @> $1::eql_v3.query_json;

-- Does this path exist in the document?
SELECT * FROM orders WHERE eql_v3.jsonb_path_exists(metadata, 'region_selector');

-- Equality on an extracted leaf
SELECT * FROM orders
WHERE metadata -> 'email_selector'::text = $1::public.eql_v3_jsonb_entry;
-- Exact match on a field: containment on a value selector, built by the client
SELECT * FROM orders WHERE metadata @> $1::eql_v3.query_json;
```

Field access is by selector hash, not plaintext path. The full JSON surface — containment, field access, path queries, and range filters on extracted leaves — is in [JSON](/reference/eql/json).
Field access is by selector hash, not plaintext path — and exact field matching is containment on a **value selector**, not `=` on an extracted leaf, which raises. The full JSON surface — containment, field access, path queries, and range filters on extracted leaves — is in [JSON](/reference/eql/json).

## Combining predicates

Expand All @@ -97,7 +96,7 @@ SELECT * FROM users
WHERE status = 'active' -- plaintext column, native operator
AND created_at >= $1::public.eql_v3_timestamp_ord -- encrypted range
AND (email = $2::public.eql_v3_text_eq -- encrypted equality
OR name @> $3::public.eql_v3_text_match); -- encrypted token match
OR name @@ $3::public.eql_v3_text_match); -- encrypted fuzzy match
```

The planner treats each encrypted predicate independently, so it can combine an index on a plaintext column with a functional index on an encrypted one (bitmap-AND, or whichever plan is cheapest).
Expand All @@ -119,8 +118,8 @@ Don't confuse this with a JSON `null` *inside* an encrypted document, which is a
| --- | --- | --- | --- |
| Equality | `=` `<>` `IN` | `_eq`, all `_ord` variants, `text_search` variants | hash (or btree) on `eql_v3.eq_term` — btree on `eql_v3.ord_term` for the non-text `_ord` variants |
| Range | `<` `<=` `>` `>=` `BETWEEN` | all `_ord` variants, `text_search` variants | btree on `eql_v3.ord_term` (`eql_v3.ord_term_ore` on `_ord_ore`) |
| Text token match | `@>` `<@` | `text_match`, `text_search` variants | GIN on `eql_v3.match_term` |
| JSON containment | `@>` `<@` | `public.eql_v3_json` | GIN on `eql_v3.to_ste_vec_query(col)::jsonb` |
| Text fuzzy match | `@@` | `text_match`, `text_search` variants | GIN on `eql_v3.match_term` |
| JSON containment | `@>` `<@` | `public.eql_v3_json_search` | GIN on `eql_v3.to_ste_vec_query(col)::jsonb` |
| Null check | `IS NULL` / `IS NOT NULL` | every variant | — |

Every one of these has a full index recipe — which method, which extractor, and how to confirm the index engages with `EXPLAIN` — in [Indexes](/reference/eql/indexes).
Loading