perf: sqlite prepare cached

[francesco-stacks]

Description

Profiling a mainnet follower during catch-up (on top of #7300) showed the SQLite SQL parser (resolveExprStep, sqlite3RunParser, yy_reduce) as the largest remaining CPU sink: ~5.3% of chains-coordinator self-time. We re-compile the same SQL strings millions of times (the Clarity side store on every data read, the MARF on every trie-cache miss, and the query_* helpers in util_lib/db.rs for every caller).

rusqlite ships a per-connection prepared-statement cache, prepare_cached, which compiles once and reuses the bytecode. We just don't use it on these paths, and its default capacity (16) would LRU-thrash under the node's breadth of statements.

This PR routes the hot read paths through the cache and raises the per-connection cache capacity to 200 in sqlite_open.

Measured impact - A/B on a mainnet follower (identical chainstate copies, same start height, 10-minute windows):

	baseline	prepare_cached
catch-up throughput	357 blk/min	417 blk/min (+16.8%)
SQL-parser self-time (coordinator)	~5.3%	~1.9%

I also run a first iteration (that wasn't caching everything this one is caching) with stacks-bench in cylewitruk#5 (comment) that measured it ~4% faster during the benchmark.

Applicable issues

• fixes #

Additional info (benefits, drawbacks, caveats)

Checklist

• Test coverage for new or modified code paths
• For new Clarity features or consensus changes, add property tests (see docs/property-testing.md)
• Changelog fragment(s) or "no changelog" label added (see changelog.d/README.md)
• Required documentation changes (e.g., rpc/openapi.yaml for RPC endpoints, event-dispatcher.md for new events)
• New clarity functions have corresponding PR in clarity-benchmarking repo

[Copilot]

Pull request overview

This PR improves SQLite query performance by switching hot-path queries to use rusqlite’s prepared-statement cache (prepare_cached) and increasing per-connection statement-cache capacity to reduce repeated SQL parsing.

Changes:

• Use prepare_cached in shared SQLite query helpers and several frequently-called read paths (Clarity VM headers queries, MARF squashed-block lookups, SortDB tip lookups).
• Increase prepared-statement cache capacity on long-lived connections (stackslib: 200; Clarity side-store: 32).
• Add a changelog fragment documenting the performance improvement.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
stackslib/src/util_lib/db.rs	Switch generic query_* helpers and a MARF lookup to prepare_cached; bump per-connection statement-cache capacity to 200 in sqlite_open().
stackslib/src/clarity_vm/database/mod.rs	Cache dynamically-built header-table SELECT statements via prepare_cached for repeated access patterns.
stackslib/src/chainstate/stacks/index/trie_sql.rs	Use prepare_cached for MARF squashed-block and block-id/hash lookup queries.
stackslib/src/chainstate/burn/db/sortdb.rs	Use prepare_cached for canonical tip lookup queries in SortDB.
clarity/src/vm/database/sqlite.rs	Use prepare_cached for side-store reads/writes; bump statement-cache capacity to 32.
changelog.d/sqlite-prepare-cached.changed	Document statement caching + cache-capacity increase.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coveralls]

Coverage Report for CI Build 29476362671

Coverage increased (+0.8%) to 86.538%

Details

• Coverage increased (+0.8%) from the base build.
• Patch coverage: 1 uncovered change across 1 file (69 of 70 lines covered, 98.57%).
• 12003 coverage regressions across 165 files.

Uncovered Changes

File	Changed	Covered	%
stackslib/src/chainstate/stacks/index/trie_sql.rs	24	23	95.83%
Total (5 files)	70	69	98.57%

Coverage Regressions

12003 previously-covered lines in 165 files lost coverage.

Top 10 Files by Coverage Loss	Lines Losing Coverage	Coverage
stackslib/src/chainstate/burn/db/sortdb.rs	638	90.44%
stackslib/src/chainstate/stacks/db/blocks.rs	528	89.93%
stackslib/src/chainstate/nakamoto/mod.rs	488	84.83%
stackslib/src/config/mod.rs	324	78.53%
stackslib/src/net/relay.rs	316	74.81%
stackslib/src/net/mod.rs	312	78.18%
stackslib/src/chainstate/stacks/index/storage.rs	277	82.41%
clarity/src/vm/database/clarity_db.rs	273	82.02%
stackslib/src/chainstate/stacks/miner.rs	270	82.96%
stackslib/src/chainstate/stacks/db/transactions.rs	269	97.13%

---

Coverage Stats

Relevant Lines:	231314
Covered Lines:	200174
Line Coverage:	86.54%
Coverage Strength:	19763703.01 hits per line

---

💛 - Coveralls

[benjamin-stacks]

🚀