fix(core): add offset pagination to getEmDashCollection for numbered archives (#1364)

[marcusbellamyshaw-cell]

What does this PR do?

getEmDashCollection only supported cursor/keyset pagination, so numbered archive routes (/page/2, /tag/security/page/3) had no efficient option — they had to either walk cursors page-by-page or over-fetch from the start and slice (fetching up to 1,000 rows to render 20 on /page/50).

This adds an additive offset option to the public CollectionFilter (and the underlying loader filter), so numbered archives become a one-liner:

const perPage = 20;
const { entries, hasMore } = await getEmDashCollection("posts", {
  limit: perPage,
  offset: (page - 1) * perPage,
  orderBy: { published_at: "desc" },
});

Details:

• offset is applied as a SQL OFFSET, cross-dialect: SQLite only accepts OFFSET with a LIMIT, so a bare offset uses LIMIT -1 (unbounded); Postgres takes a standalone OFFSET.
• offset and cursor are mutually exclusive — cursor (keyset) wins when both are set, so the two don't stack into a double skip. Only a positive integer applies (0 / negative / fractional are no-ops).
• New hasMore on CollectionResult, set whenever limit is provided, computed from the existing over-fetch-by-one — so archive routes can render a "next page" link without an extra count query.
• Offset calls bypass limit bucketing (their limit is part of the pagination contract), and offset is part of the request-cache key so distinct pages don't collide.

Closes #1364

Type of change

• Bug fix

Checklist

• I have read CONTRIBUTING.md
• pnpm typecheck passes
• pnpm lint passes
• pnpm test passes (or targeted tests for my change)
• pnpm format has been run
• I have added/updated tests for my changes (if applicable)
• User-visible strings in the admin UI are wrapped for translation — n/a, no admin UI strings (site-side query API + docs)
• I have added a changeset
• New features link to an approved Discussion — n/a, additive API closing a triaged bug report

AI-generated code disclosure

• This PR includes AI-generated code — model/tool: Claude Opus 4.8 ultracode

Screenshots / test output

Test Files  5 passed (5)
     Tests  31 passed (31)

New tests: loader-offset-pagination.test.ts (skip/disjoint pages/past-end/cursor-wins), query-collection-offset.test.ts (offset forwarding, hasMore semantics, cache-key separation), plus a bucketing-exemption case in query-collection-bucketing.test.ts.

[changeset-bot]

🦋 Changeset detected

Latest commit: 5464f24

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
emdash	Minor
@emdash-cms/cloudflare	Minor
@emdash-cms/sandbox-workerd	Patch
@emdash-cms/fixture-perf-site	Patch
@emdash-cms/perf-demo-site	Patch
@emdash-cms/cache-demo-site	Patch
@emdash-cms/do-demo-site	Patch
@emdash-cms/do-solo-demo-site	Patch
@emdash-cms/admin	Minor
@emdash-cms/auth	Minor
@emdash-cms/blocks	Minor
@emdash-cms/gutenberg-to-portable-text	Minor
@emdash-cms/x402	Minor
create-emdash	Minor
@emdash-cms/auth-atproto	Patch
@emdash-cms/plugin-embeds	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[emdashbot]

Could not push formatting changes to this fork. The contributor may have "Allow edits by maintainers" disabled.

Please run the formatter locally:

pnpm format

[emdashbot]

Reviewed the PR adding offset pagination to getEmDashCollection and surfacing hasMore on CollectionResult.

What I checked:

• Traced the offset path through loader.ts (rawOffset/offset resolution, the LIMIT/OFFSET clause construction, and the SQLite LIMIT -1 OFFSET N vs Postgres standalone OFFSET N split) and through query.ts (over-fetch hasMore detection, hasMoreResult, bucketFilter bypass, sliceCollectionResult, and the cache key).
• Verified correctness across edge cases by hand: offset near the end, exactly-at-boundary row counts, empty page past the end, offset: 0 (treated as no-op), fractional/negative offsets (no-ops), cursor-wins-over-offset, and the bucketed+sliced small-limit path. hasMore is correct in every case — including the new unconditional hasMore: true in sliceCollectionResult, which is safe because slicing only occurs when cached.entries.length > limit.
• SQL safety: offset is interpolated via Kysely's tagged template, so it's parameterized (no injection). Field/identifier handling unchanged.
• Conventions: locale filter still applies to the offset query (no new locale bug); no admin UI/Lingui/RTL surface; no route files changed so no query-count snapshot update is needed; changeset is present, leads with Adds, describes the observable effect, and uses a minor bump appropriate for an additive feature; docs updated in api.mdx.

Conclusion: The implementation is correct and well-tested. The loader tests cover skip/disjoint-pages/empty-page/cursor-wins, and the query tests cover offset forwarding, hasMore true/false/undefined, and cache-key distinctness. The only finding is a non-blocking suggestion: one test name states the opposite of what the code does.

[emdashbot]

[suggestion] The test name says it "keeps offset out of the request-cache key," but the implementation does the opposite: collectionCacheKey in query.ts adds filter.offset ?? "" to the key, and the changeset explicitly states "the offset is part of the request-cache key so distinct pages don't collide." The test body is correct (it asserts two distinct loader calls for offsets 0 and 20, proving the pages don't collide), but the name reads as the inverse of the intended behavior. A future maintainer taking the name literally could drop offset from the cache key and silently break numbered-archive pagination. Suggested rename to match what the test actually verifies:

Suggested change

	it("keeps offset out of the request-cache key so different pages don't collide", async () => {
	it("includes offset in the request-cache key so distinct pages don't collide", async () => {

[marcusbellamyshaw-cell]

Renamed in f3e1ff9 to "includes offset in the request-cache key so distinct pages don't collide", matching what the test asserts.

[pkg-pr-new]

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@1577

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@1577

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@1577

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@1577

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@1577

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@1577

emdash

npm i https://pkg.pr.new/emdash@1577

create-emdash

npm i https://pkg.pr.new/create-emdash@1577

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@1577

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@1577

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@1577

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@1577

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@1577

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@1577

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@1577

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@1577

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@1577

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@1577

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@1577

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@1577

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@1577

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@1577

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@1577

commit: 5464f24

[ascorbic]

Instead of hving two mutually exclusive, optional properties on this, could you change it to a union type. Use a base type, with two different subtypes for the offset and cursor variants and export CollectionFilter as the union of these. This will mean it will show up as a type error if a user passes both.

[marcusbellamyshaw-cell]

Done in b57ac46. Split into a shared CollectionFilterBase plus CursorCollectionFilter (offset?: never) and OffsetCollectionFilter (cursor?: never) variants, with CollectionFilter exported as their union — applied to both the public query filter and the loader filter, so supplying both is now a compile-time error. The internal query→loader mapping spreads only the active key (cursor wins) so it still satisfies the union, and there's a @ts-expect-error test locking in the both-at-once rejection.

[ascorbic]

@marcusbellamyshaw-cell Can you run pnpm format

[emdashbot]

This is a sensible, additive fix for numbered archive routes (#1364). Offset pagination fits the existing loader/query split: the loader applies a SQL OFFSET, the query layer reuses its over-fetch logic to compute hasMore, and offset joins the request-cache key so distinct pages don't collide. The approach is the right one for /page/N style archives, and the changeset correctly documents it as a user-facing additive change.

For this re-review I walked through:

• SQL construction in loader.ts: offset is validated to a positive integer before use and interpolated through Kysely's tagged-template parameters, so there is no SQL injection risk. SQLite's LIMIT -1 OFFSET N path and Postgres's standalone OFFSET N are faithful to the dialects.
• Locale handling: offset is applied after the existing locale WHERE clause; no new locale filter bug.
• hasMore semantics: the no-bucket path computes entries.length > requestedLimit correctly, and sliceCollectionResult only sets hasMore: true after the early cached.entries.length <= limit return, so small buckets with fewer total entries than the request still return hasMore: false.
• Bucketing bypass: bucketFilter treats offset-bearing filters like cursor-bearing ones, preserving the pagination contract.
• Cache key: filter.offset is part of collectionCacheKey, so pages 0 and 20 don't share snapshots.
• Type safety: the CursorCollectionFilter | OffsetCollectionFilter union makes cursor and offset mutually exclusive at compile time while still allowing neither.
• Tests: the new unit tests cover skip behavior, disjoint pages, past-the-end pages, cursor-wins, query forwarding, hasMore true/false/undefined, cache-key separation, and bucketing exemption.

The previous review's only note was a misleading test name; on re-reading the current branch, all test names match the assertions, so that appears to have been addressed.

No blocking issues. LGTM.

[marcusbellamyshaw-cell]

Branch is already formatted — pnpm format produces no changes, and pnpm format:check (oxfmt + prettier) passes locally on 5464f241; the green Format (pull_request) check confirms the same in CI.

The red check is Auto Format (pull_request_target), which only runs oxfmt and pushes a style: format commit. It's failing at the checkout step, not on a format diff — actions/checkout@v7 now refuses to check out fork code under pull_request_target (the "pwn request" guard), so it will fail on any fork PR until the workflow is updated (e.g. allow-unsafe-pr-checkout: true or run oxfmt against the base ref). Happy to send a small PR for that if useful.