Hi — really enjoyed the agent-swarm model-economics post and minisqlite as its artifact. I've been building a parallel thing from a different direction: a pure-Rust SQLite-compatible engine driven by agent orchestration. I am opening this as a constructive conformance note rather than a bug report — feel free to close if it's not useful.
The observation
The post's headline is 100% on the held-out sqllogictest suite. That's a real milestone — but from a lot of time in the SQLite-conformance trenches, I'd gently push back on treating it as "SQLite parity," because sqllogictest and SQLite's own TCL test suite are wildly different difficulty bars, and the gap is easy to underestimate.
- sqllogictest almost entirely asserts result rows for well-formed queries.
- SQLite's TCL suite asserts SQLite's documented behavior: exact error-message text,
PRAGMA output, sqlite_master/schema details, trigger timing and statement legality, DML edge semantics, the on-disk file format, the C-API, collations, and the "documentation-evidence" tests (e_select, e_insert, e_update, …) that verify each individually-testable statement in the SQL language docs.
The second surface is categorically larger and stricter, and it's where the genuine long tail of conformance work lives.
Concrete flavor
A representative sample of things the TCL suite pins down that a result-row harness never exercises:
- Emitting SQLite's exact diagnostics, e.g.
aggregate functions are not allowed in the GROUP BY clause, or a JOIN clause is required before ON (not just an error, the right error text).
- Rejecting
ORDER BY / LIMIT inside a trigger body with SQLite's precise near "..." wording.
REINDEX behavior observed through a user-defined collation.- Integer-affinity edge cases on
LIMIT/OFFSET, HAVING-only aggregates that must not leak into the projection, etc.
Hard to say which is the better benchmark but "cloning sqlite" is much harder than "passing sqllogictest".
Hi — really enjoyed the agent-swarm model-economics post and minisqlite as its artifact. I've been building a parallel thing from a different direction: a pure-Rust SQLite-compatible engine driven by agent orchestration. I am opening this as a constructive conformance note rather than a bug report — feel free to close if it's not useful.
The observation
The post's headline is 100% on the held-out sqllogictest suite. That's a real milestone — but from a lot of time in the SQLite-conformance trenches, I'd gently push back on treating it as "SQLite parity," because sqllogictest and SQLite's own TCL test suite are wildly different difficulty bars, and the gap is easy to underestimate.
PRAGMAoutput,sqlite_master/schema details, trigger timing and statement legality, DML edge semantics, the on-disk file format, the C-API, collations, and the "documentation-evidence" tests (e_select,e_insert,e_update, …) that verify each individually-testable statement in the SQL language docs.The second surface is categorically larger and stricter, and it's where the genuine long tail of conformance work lives.
Concrete flavor
A representative sample of things the TCL suite pins down that a result-row harness never exercises:
aggregate functions are not allowed in the GROUP BY clause, ora JOIN clause is required before ON(not just an error, the right error text).ORDER BY/LIMITinside a trigger body with SQLite's precisenear "..."wording.REINDEXbehavior observed through a user-defined collation.LIMIT/OFFSET, HAVING-only aggregates that must not leak into the projection, etc.Hard to say which is the better benchmark but "cloning sqlite" is much harder than "passing sqllogictest".