feat(core): optional distributed object cache for query results#1378
Conversation
Add an opt-in read-through cache that sits beneath the per-request cache and above the database, so content and chrome (settings, menus, taxonomies) reads can be served from a fast key/value store instead of hitting D1/SQLite on every request. - New ObjectCache abstraction (interface + descriptor + virtual module + per-isolate backend), mirroring the storage adapter pattern. Off by default: when unconfigured, cachedQuery is a transparent passthrough. - Backends: in-isolate memory (emdash/object-cache/memory via memoryCache() from emdash/astro) and Cloudflare KV (@emdash-cms/cloudflare/cache/kv via kvCache()). - JSON codec preserves Date instances; content entries are snapshotted (dropping the .edit proxy, capturing the CURSOR_RAW_VALUES symbol) and rebuilt on read. - Epoch-based invalidation at the repository chokepoint (content, seo, byline, taxonomy, menu) and settings; content reads fold in shared bylines/taxonomies epochs so author/term renames invalidate correctly. - Auth/preview/edit-mode and isolated DBs always bypass. Existing sites are unaffected until they opt in.
A KV read that stalls without resolving or rejecting (cold cross-region read, or one queued behind the Workers connection limit) could hang the isolate: getEpoch cached the never-settling promise and every later cached read on that namespace reused it, poisoning the isolate until it recycled. - Race every backend read against a timeout (default 2000ms, configurable via the `timeout` option on kvCache/objectCache, 0 disables). A timed-out read degrades to a cache miss; the database stays the source of truth. - Apply the timeout in the KV backend (get/set/delete) and in the core read path (getEpoch + cachedQuery value read), so any backend that stalls self-heals once the bounded read settles. - Also switch the cache debug-log gate from process.env to import.meta.env.DEV (repo convention). Adds regression tests: a never-settling backend resolves via load() instead of hanging, the namespace self-heals afterward, and the KV backend rejects a stalled get/set.
Public renders that read an entry's terms still hit D1 on every request even with the object cache on: getEmDashEntry caches the entry (and bakes in byline/term hydration), but templates that call getEntryTerms / getTermsForEntries / getTerm directly fell through to D1, because only the taxonomy *definitions* (getTaxonomyDefs) and full term *lists* (getTaxonomyTerms) were wrapped. On a warm content-cache hit, hydration (which used to prime the request cache for getEntryTerms) doesn't run, so those direct calls query the database — a cache-busted load that should be served entirely from KV still pays D1 round-trips. Wrap the per-entry/term taxonomy reads in cachedQuery: - getEntryTerms, getTermsForEntries — namespaced under [content:<collection>, taxonomies]; assignments bump taxonomies and content writes bump content:<collection>, so they invalidate correctly. getEntryTerms keeps its requestCached wrapper so hydration priming still short-circuits within a request. - getTerm — namespaced under taxonomies (count is TTL-bounded). - getTermsForEntries returns a Map (not JSON-serializable): cache it as an array of [entryId, terms] pairs and rebuild the Map on read. Large id batches (which come from collection hydration, already served by the content cache) bypass the object cache to stay under KV's key-size limit. getEntriesByTerm already delegates to the cached getEmDashCollection, and getAllTermsForEntries only runs behind a content-cache miss, so neither needs separate wrapping. Test: with a configured backend, getEntryTerms and getTermsForEntries serve the second read from KV with D1 made unavailable, and the Map round-trips correctly.
…-trip
The read path did two sequential KV round-trips per cached query: read the
namespace epoch(s) to build the key, then read the value. On a cold isolate
(epochs not yet cached in-memory) a page making several cached reads paid
that doubled latency on each one.
Make the value key epoch-independent and store the namespace epochs inside
the value envelope ({ e: epochs, v: value }). A read now fetches the value
and all epochs concurrently (Promise.all) and treats it as a HIT only when
every stored epoch still matches the current one — one round-trip instead of
two. Invalidation is unchanged from the caller's view (bump the epoch; the
next read sees a mismatch and reloads), but a stale value is now overwritten
in place under its stable key rather than orphaned under a dead epoch-keyed
name — so KV no longer accumulates orphaned generations between TTL sweeps.
Note this parallelizes the epoch/value reads *within* each cached query;
ordering across a template's awaits is still the template's concern (use
Promise.all for independent reads).
Existing object-cache, content, taxonomy, and edge-cache tests pass
unchanged (behavior is identical: hit after first load, reload after
invalidation, multi-namespace busting, timeout-to-miss).
…ache These were the last per-request D1 reads on a public post render. The <Comments> component server-renders two reads on every page — even with content/taxonomy reads already served from KV: - getCollectionInfo (the commentsEnabled / supports / fields lookup), and - getComments (approved comments), when comments are enabled. Wrap both in cachedQuery: - getCollectionInfo → `schema` namespace, busted by invalidateUrlPatternCache (every schema-mutation path already routes through it, so editing a collection's settings/fields invalidates it). - getComments → `comments` namespace, busted by any CommentRepository write (create / status change / delete), so a new or moderated comment shows without waiting for TTL. With this, a warm-isolate logged-out post render makes no D1 query — the whole render is served from KV. Tests: getCollectionInfo and getComments serve the second read with D1 unavailable, and reload after a schema change / comment write respectively.
🦋 Changeset detectedLatest commit: a1eecf1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Scope checkThis PR changes 2,636 lines across 39 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
docs | a1eecf1 | Jun 22 2026, 05:34 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | a1eecf1 | Jun 22 2026, 05:35 PM |
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/auth-atproto
@emdash-cms/blocks
@emdash-cms/cloudflare
@emdash-cms/contentful-to-portable-text
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/plugin-cli
@emdash-cms/plugin-types
@emdash-cms/registry-client
@emdash-cms/registry-lexicons
@emdash-cms/sandbox-workerd
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-field-kit
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-cache | a1eecf1 | Jun 22 2026, 05:37 PM |
Overlapping PRsThis PR modifies files that are also changed by other open PRs:
This may cause merge conflicts or duplicated work. A maintainer will coordinate. |
# Conflicts: # packages/core/src/astro/integration/runtime.ts # packages/core/src/database/repositories/content.ts # packages/core/src/settings/index.ts # packages/core/src/taxonomies/index.ts
…alidation An object-cache epoch read started before `invalidateObjectCache` could resolve afterwards and unconditionally write the pre-bump backend value over the freshly-bumped local epoch, resurrecting values the invalidation had just orphaned (stale content served until the value TTL, default 1h). Epochs are monotonic, so the resolved read now merges with the current cached epoch via Math.max and never lowers it.
…path When the value read errored or timed out, `cachedQuery` left `currentEpochs` empty and re-read the epochs in the deferred write — *after* `load()`. A write that invalidated the namespace mid-load would then be picked up by that re-read and stamp the stale value under the new epoch, so a later read served it as a HIT. The value and epoch reads now run concurrently but are awaited separately (getEpoch never rejects), so the pre-load epochs are always captured, even when the value read fails.
A scheduled entry becomes visible on a future clock tick, not on a write, so an object-cache snapshot taken before its go-live time kept it hidden past that time — until the publish sweep bumped the epoch or the value TTL (default 1h) lapsed. getEmDashEntry now marks a resolution time-sensitive when it sees a scheduled, not-yet-due candidate and skips caching that result.
Creating, updating, or deleting a field changes the collection's columns, but the field handlers invalidated nothing — cached content snapshots (which embed field values) kept serving the old shape to anonymous visitors until a content write or the value TTL. The field create/update/delete handlers now bump the collection's content cache. Reorder is untouched: field order isn't part of any cached read.
The object-cache codec rehydrated any object with a `$$emdashDate` string key into a Date, so a user value that happened to carry that key alongside others lost every other field. encode() only ever emits the tag as an object's sole key, so decode now requires exactly one key before treating it as a tagged Date.
The memory backend computes expiry from `Date.now()`, not `performance.now()`.
github-actions
Bot
added
the
review/needs-rereview
![emdashbot[bot]](https://avatars.githubusercontent.com/in/3255304?s=60&v=4){kind=small}
There was a problem hiding this comment.
Approach
This adds an optional, opt-in distributed read-through object cache (KV or in-isolate memory) sitting beneath requestCached and above the DB, with epoch-based invalidation. The approach is sound and fits EmDash well: epoch-stamped namespaces are the right invalidation primitive for KV (which has no prefix delete), the value+epoch parallel fetch and pre-load epoch capture correctly avoid masking mid-load writes, backend reads are timeout-bounded so a stalled KV read can't hang a render, and edit/preview/isolated-DB requests bypass. It's off by default and additive, so existing sites are unaffected — consistent with the pre-1.0 stability rule. The Discussion link is left as a placeholder in the PR body, but that's a process gap for the maintainer, not a code problem.
What I checked
- Invalidation coverage: traced every write path —
ContentRepository(create/update/delete/restore/permanentDelete/schedule/publish/unpublish/setDraftRevision/discardDraft), byline create/update/delete/setContentBylines/copyContentBylines, taxonomy create/update/delete/attach/detach/setTerms/clear/copy, menu + menu-item mutations, comment create/status/bulk/delete, SEO upsert/delete, schema field create/update/delete, settings, and the collection create/update/delete routes (which route throughinvalidateUrlPatternCache→invalidateSchemaObjectCache). Content reads foldbylines+taxonomiesepochs into their key, so author/term renames invalidate affected entries. All paths I checked are covered. - Codec/snapshot:
Datepreservation and the$$emdashDatesingle-key guard are correct; the contententrySnapshot/reviveEntrycorrectly drop+rebuild theeditproxy and the non-enumerableCURSOR_RAW_VALUESsymbol, andsliceCollectionResultre-encodes the cursor from the raw date string so bucketed-limit pagination stays precise. - Scheduled content:
getEmDashEntrymarks a not-yet-due scheduled resolutiontimeSensitiveand skips caching it, andcacheableis evaluated afterload()— verified the closure variable can't leak across concurrent calls. Tested. - Concurrency: the in-flight epoch dedup, the
Math.maxguard preventing a stale backend epoch from reverting a concurrent bump, and thependingBumpstick-coalescing all hold up under the test scenarios. - Backend wiring: virtual module,
getBackendfail-open (a missing KV binding silently disables the cache, dev-only warn), KV TTL clamping, and timeout.dbIsIsolatedis actually set by the playground/preview-DO middleware, so the bypass isn't a dead branch.
Conclusion
The implementation is strong and well-tested. I found one real correctness bug and one doc-accuracy concern.
The bug: getComments builds its cache key from only collection/contentId/threaded, omitting reactions and sort — both of which change the returned value. Calls that differ only in those options collide, so the second caller is served the first caller's ordering/reaction state. The built-in Comments.astro forwards both options, so this is reachable. This violates the AGENTS.md rule that a cache key must include every argument that changes the result.
The concern: the freshness/bypass wording in the runtime doc comment (mirrored in the PR description, changeset, and docs) overstates what the KV backend delivers — shouldBypass doesn't check authentication, and KV's eventual consistency means cross-isolate invalidation can lag well past revalidate's 1s.
| return getCommentsWithDb(db, options); | ||
| return cachedQuery({ | ||
| namespace: CacheNamespace.COMMENTS, | ||
| key: `comments:${options.collection}:${options.contentId}:${options.threaded ? "t" : "f"}`, |
There was a problem hiding this comment.
[needs fixing] The cache key for getComments only folds in collection, contentId, and threaded, but getCommentsWithDb also branches on options.reactions (attaches reaction counts) and options.sort ("best" reorders top-level comments, implies reactions). Two calls that differ only in those options collide on the same key, so whichever call populates the entry first wins and the other is served its result. Concretely: a page rendering getComments({ collection, contentId, threaded: true }) (oldest, no reactions) and one rendering getComments({ collection, contentId, threaded: true, sort: "best" }) (or reactions: true) share comments:<c>:<id>:t; the second receives the first's oldest-ordered, reaction-less snapshot. The built-in components/Comments.astro forwards reactions/sort straight through, so this is reachable in practice. AGENTS.md is explicit that the cache key must include every argument that changes the result.
| * Authenticated, preview, and visual-edit requests always bypass the cache, | ||
| * so editors see live content immediately. Anonymous visitors may see | ||
| * content up to `revalidate` ms stale after an edit (default 1s). |
There was a problem hiding this comment.
[suggestion] This comment (and the matching wording in the PR description, changeset, and docs/.../object-cache.mdx) makes two claims the code doesn't fully deliver for the KV backend:
-
"Authenticated ... requests always bypass the cache."
shouldBypass(object-cache/index.ts) checks onlyeditMode,preview, anddbIsIsolated— it never inspects authentication. An authenticated editor browsing the public site outside edit/preview mode hits the cache like anyone else. That's not a correctness bug (the cache only stores published content), but the claim is inaccurate. -
"Anonymous visitors may see content up to
revalidatems stale after an edit (default 1s)."revalidateonly bounds how long an isolate reuses its local copy of a namespace epoch. For the KV backend, cross-isolate invalidation depends on other isolates reading the freshly-bumped epoch from KV, and KV reads use the default edge cache (eventual consistency, up to ~60s) — the KV backend'sgetpasses nocacheTtl. So the real cross-isolate staleness window is dominated by KV propagation, notrevalidate, and can be far larger than 1s. Worth either documenting the KV propagation lag or passing a lowcacheTtlon epoch reads.
github-actions
Bot
added
review/awaiting-author
getCommentsWithDb branches on `reactions` and `sort` ("best" reorders and
implies reactions), but the object-cache key folded in only collection,
contentId, and threaded. Two reads differing solely in those options collided,
so the second was served the first's snapshot (e.g. a `sort: "best"` render
getting oldest-ordered, reaction-less comments). The key now includes both,
normalizing the best-implies-reactions rule so identical results still share an
entry.
Reaction counts and `best` ordering are folded into cached getComments reads, but handleReactionToggle never invalidated — so a toggled reaction left stale counts (and stale "best" order) until a comment write or the value TTL. The toggle handler now bumps the comments cache.
Two claims were inaccurate: `shouldBypass` checks only edit/preview/isolated mode, not authentication, so authenticated browsing outside edit mode is served from the cache (safe — it only stores published content); and `revalidate` bounds only an isolate's local epoch reuse, not total cross-isolate staleness. On KV the real window is dominated by KV's edge-cache propagation (eventually consistent, up to ~60s) plus `revalidate`. Updated the runtime JSDoc, the revalidate type doc, and the deployment guide to say so.
github-actions
Bot
added
review/needs-rereview
3 participants
What does this PR do?
Adds an optional, opt-in distributed object cache for query results. Content reads (
getEmDashCollection,getEmDashEntry,resolveEmDashPath) and chrome reads (site settings, menus, taxonomies, per-entry terms, collection info, public comments) can be served from a fast key/value store instead of hitting the database on every request. It sits beneath the per-request cache and above the database, dramatically reducing read pressure on D1/SQLite — especially on Cloudflare, where KV absorbs far more requests than D1.The cache is off by default and fully opt-in. Configure a backend in
astro.config.mjs:Invalidation is epoch-based and automatic: content, byline, taxonomy, menu, and settings writes bump a per-namespace version, instantly orphaning stale entries (no key enumeration). Value and epoch are fetched in one parallel round-trip via a stable-key envelope, so there are no orphaned keys to accumulate. Preview and visual-edit requests bypass the cache, so editors previewing see live content; other reads are served from the cache, which only ever stores published content. After an edit, anonymous visitors may see stale content until isolates pick up the bumped epoch — immediate on the memory backend, and on KV bounded by KV's edge-cache propagation (eventually consistent, up to ~60s) plus the
revalidatewindow (default 1s, configurable). Backend reads are bounded by a timeout so a slow/unavailable cache can never hang a render.Existing sites are unaffected until they opt in.
Closes #
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change) — object-cache unit/content/comments/schema/entry-terms + cloudflare kv-timeout suitespnpm formathas been runAI-generated code disclosure
Screenshots / test output
Try this PR
Open a fresh playground →
A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.
Tracks
feat/object-cache. Updated automatically when the playground redeploys.