Skip to content

Commit 62746dc

Browse files
committed
fix(skills): correct index-recipe SQL surfaced by EQL docs audit
An audit of the EQL repo's documentation against the shipped @cipherstash/eql@3.0.2 SQL surfaced four defects that had been copied into the new indexing guidance: - The JSON containment GIN recipe was a PostgreSQL syntax error: a cast over a function call is a general expression and needs its own parens in an index_elem — ((eql_v3.to_ste_vec_query(col)::jsonb) jsonb_path_ops). Verified both forms against Postgres 17. Fixed in stash-indexing, stash-supabase, and stash-prisma-next (the Drizzle helper already emitted the correct form). - The troubleshooting operand-typing example cast a query parameter to the column domain (public.eql_v3_text_eq), whose CHECK requires the ciphertext key 'c' that query payloads deliberately omit — the correct cast is the term-only eql_v3.query_text_eq domain. - The term probe checked hm/ob/bf but not op, the CLLW-OPE term the same page names as the equality-and-range term for _ord domains. - GROUP BY guidance now says which extractor to group on per domain: eq_term only exists on hm-carrying domains; numeric/date/timestamp *Ord/*OrdOre group on their injective ordering term. Also documented a trap the audit exposed: the EQL install SQL begins with DROP SCHEMA eql_v3 CASCADE, so stash eql upgrade / reinstall silently cascade-drops every functional index built on the extractors. stash-indexing and stash-cli (eql upgrade) now say to re-run index migrations + ANALYZE afterwards. Claude-Session: https://claude.ai/code/session_01BkEpKJC3975NHsKgMrCT8R
1 parent 0f16174 commit 62746dc

4 files changed

Lines changed: 10 additions & 5 deletions

File tree

skills/stash-cli/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -394,7 +394,7 @@ After writing the migration, `--drizzle` sweeps the output directory for sibling
394394

395395
#### `eql upgrade`
396396

397-
The install SQL is idempotent. `upgrade` checks the current version, re-runs it, and reports the new one. If EQL isn't installed it points you at `eql install`. Same `--supabase`, `--exclude-operator-family`, `--eql-version`, `--latest`, `--dry-run`, `--database-url` flags.
397+
The install SQL is safe to re-run — columns and data survive — but it is not fully idempotent: it begins with `DROP SCHEMA IF EXISTS eql_v3 CASCADE`, which cascade-drops any **functional indexes** built on the `eql_v3` extractors (see `stash-indexing`). After an upgrade, re-run your index migrations and `ANALYZE`. `upgrade` checks the current version, re-runs the install SQL, and reports the new one. If EQL isn't installed it points you at `eql install`. Same `--supabase`, `--exclude-operator-family`, `--eql-version`, `--latest`, `--dry-run`, `--database-url` flags.
398398

399399
#### `eql status`
400400

skills/stash-indexing/SKILL.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ For `public.eql_v3_json_search` (`types.Json`) document containment (`@>`):
9797

9898
```sql
9999
CREATE INDEX orders_data_gin
100-
ON orders USING gin (eql_v3.to_ste_vec_query(data_encrypted)::jsonb jsonb_path_ops);
100+
ON orders USING gin ((eql_v3.to_ste_vec_query(data_encrypted)::jsonb) jsonb_path_ops);
101101
ANALYZE orders;
102102

103103
SELECT * FROM orders WHERE data_encrypted @> $1::eql_v3.query_json;
@@ -177,6 +177,8 @@ SELECT eql_v3.eq_term(encrypted_email), count(*)
177177

178178
The term is small and deterministic, so `HashAggregate` fits in `work_mem` with no tuning. If an ORM insists on grouping the raw column, raising `work_mem` is the rescue knob — but the extractor form is the design.
179179

180+
Pick the extractor the domain actually has: `eq_term` on the `hm`-carrying domains (`types.*Eq`, `types.TextOrd*`, `types.TextSearch`). The numeric/date/timestamp `types.*Ord` / `*OrdOre` domains have **no** `eq_term` — group on `eql_v3.ord_term(col)` (or `ord_term_ore(col)`); their ordering term is injective, so it is an exact grouping key, and the ordering btree covers it.
181+
180182
## Building Indexes at Scale
181183

182184
Query performance and *build* performance are separate axes; on large encrypted tables the build is the one that bites.
@@ -232,12 +234,13 @@ Index not being used:
232234

233235
```sql
234236
SELECT encrypted_email::jsonb ? 'hm' AS has_hmac,
237+
encrypted_email::jsonb ? 'op' AS has_ope,
235238
encrypted_email::jsonb ? 'ob' AS has_ore_block,
236239
encrypted_email::jsonb ? 'bf' AS has_bloom
237240
FROM users LIMIT 1;
238241
```
239242

240-
2. **Verify the operand is typed** (`$1` or `$1::public.eql_v3_text_eq` — not `$1::jsonb`).
243+
2. **Verify the operand is typed** (`$1` or `$1::eql_v3.query_text_eq` — not `$1::jsonb`, and not the column domain `public.eql_v3_text_eq`: query payloads are term-only, and the column domains' CHECK requires the ciphertext key `c` that query payloads deliberately omit).
241244
3. **Recreate the index** if the column's term composition changed after it was built.
242245
4. **Run `ANALYZE`.** Also note: on very small tables a `Seq Scan` is the *correct* plan — don't chase it below a few thousand rows.
243246

@@ -259,6 +262,8 @@ Index not being used:
259262
- **Fresh encrypted column (new table or new field):** ship the `CREATE INDEX` in the **same migration** that adds the column. Every value written carries its terms from day one, so the index is correct from the first row.
260263
- **Encrypting an existing column** (the `stash encrypt` lifecycle): create the indexes **after `stash encrypt backfill` completes and before switching reads** to the encrypted column. Building after backfill is one bulk pass instead of per-row index maintenance across the whole backfill, and the reads you cut over to engage an index from the first query. Remember `ANALYZE` after the build. See `stash-encryption` § "Rolling Encryption Out to Production" for the full lifecycle.
261264

265+
**These indexes do not survive an EQL reinstall or upgrade.** The install SQL begins with `DROP SCHEMA IF EXISTS eql_v3 CASCADE`, and every functional index on an extractor depends on that schema — so `stash eql upgrade` (or `eql install --force`, or re-applying the bundle by hand) cascade-drops all of them. Columns and data are untouched (the `public.eql_v3_*` domains deliberately don't depend on the `eql_v3` schema); only the indexes vanish, and queries fall back to sequential scans without erroring. After any EQL upgrade or reinstall, re-run your index migrations and `ANALYZE`, and confirm with the `EXPLAIN` checklist above.
266+
262267
## Reference
263268

264269
- `stash-encryption` — the `types.*` domain catalog, wire-format operators and ordering, and the rollout/cutover lifecycle.

skills/stash-prisma-next/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ CREATE INDEX users_created_at_ord ON users USING btree (eql_v3.ord_term(created_
169169
CREATE INDEX users_bio_match ON users USING gin (eql_v3.match_term(bio));
170170
-- cipherstash.Json: containment
171171
CREATE INDEX users_profile_json
172-
ON users USING gin (eql_v3.to_ste_vec_query(profile)::jsonb jsonb_path_ops);
172+
ON users USING gin ((eql_v3.to_ste_vec_query(profile)::jsonb) jsonb_path_ops);
173173

174174
ANALYZE users;
175175
```

skills/stash-supabase/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ CREATE INDEX users_created_at_ord ON users USING btree (eql_v3.ord_term(created_
9797
CREATE INDEX users_bio_match ON users USING gin (eql_v3.match_term(bio));
9898
-- eql_v3_json_search: containment
9999
CREATE INDEX users_profile_json
100-
ON users USING gin (eql_v3.to_ste_vec_query(profile)::jsonb jsonb_path_ops);
100+
ON users USING gin ((eql_v3.to_ste_vec_query(profile)::jsonb) jsonb_path_ops);
101101

102102
ANALYZE users;
103103
```

0 commit comments

Comments
 (0)