Skip to content

Fetch an entire discussion page in one D1 query - #67

Merged
boomzero merged 1 commit into
masterfrom
perf/getpost-single-query
Jul 27, 2026
Merged

Fetch an entire discussion page in one D1 query#67
boomzero merged 1 commit into
masterfrom
perf/getpost-single-query

Conversation

@boomzero

@boomzero boomzero commented Jul 27, 2026

Copy link
Copy Markdown
Member

Problem

GetPost issued five awaited queries in series: the post row, a COUNT over bbs_reply, the board name, the lock row, and finally the page of replies. On Workers each of those is a separate round trip to D1, so opening a discussion cost five times the network latency regardless of how little data came back.

Change

They collapse into a single statement built from two CTEs — post (the post row left-joined to bbs_board and bbs_lock) and page (the fifteen replies for the requested page) — cross-joined so every returned row carries the post metadata alongside one reply, with the total reply count as a scalar subquery.

The page's LIMIT/OFFSET has to be bound before PageCount is known, so the offset is clamped at zero and the rows are discarded if the range check then rejects the page. Responses are byte-for-byte unchanged, including the out-of-range and empty-discussion cases.

Verification

Built a SQLite database from migrations/ and ran the query against it for four cases: page 1 of a locked 20-reply post (15 rows, correct lock and board fields), page 2 (5 rows, correct offset), a post with zero replies (1 row of nulls, reply_count 0), and a nonexistent post (0 rows).

Checked the plan against the production index set (idx_bbs_reply ON bbs_reply(post_id)) — every access is a seek, no table scans, and the count is served from a covering index:

QUERY PLAN
|--MATERIALIZE page
|  |--SEARCH bbs_reply USING INDEX idx_bbs_reply (post_id=?)
|  `--USE TEMP B-TREE FOR ORDER BY
|--SEARCH p USING INTEGER PRIMARY KEY (rowid=?)
|--SEARCH b USING INTEGER PRIMARY KEY (rowid=?) LEFT-JOIN
|--SEARCH l USING INTEGER PRIMARY KEY (rowid=?) LEFT-JOIN
|--SCAN page LEFT-JOIN
`--SCALAR SUBQUERY 3
   `--SEARCH bbs_reply USING COVERING INDEX idx_bbs_reply (post_id=?)

Added 9 GetPost tests covering those cases plus bind arguments, the legacy-domain rewrite, offset clamping, and mention clearing. One asserts exactly one prepare call, so a regression back to N+1 fails the suite. npm test: 57 pass, 0 fail.

Follow-ups, not in this PR

  • The page CTE still sorts the post's full reply set in memory to take 15 of them (the USE TEMP B-TREE above). CREATE INDEX idx_bbs_reply_post_time ON bbs_reply(post_id, reply_time) removes it — verified. Invisible on small posts, real on large ones.
  • GetPosts runs ROW_NUMBER() OVER (PARTITION BY post_id ...) across all of bbs_reply to find each post's last reply, which no index can help. That is a full scan and sort on every list page load, and is probably a larger cost than anything here.

🤖 Generated with Claude Code


Summary by cubic

Fetch an entire discussion page in GetPost with one D1 query, replacing five sequential calls to cut round trips and speed up discussion loads. The response shape and edge-case behavior stay the same.

  • Refactors
    • Collapse five queries into one SQL using post and page CTEs; returns post metadata, board name, lock state, per-page replies, and reply_count in one round trip.
    • Bind LIMIT/OFFSET up front with a zero-clamped offset; compute PageCount from reply_count and reject out-of-range pages; empty discussions still return PageCount: 0.
    • Add 9 tests to enforce a single prepare call, correct pagination binding, lock reporting, legacy domain rewrite, nonexistent/empty post handling, range checks, negative offset clamp, and mention clearing.

Written for commit dc05c2a. Summary will update on new commits.

Review in cubic

Summary by Sourcery

Fetch full discussion metadata and replies for GetPost using a single SQL query instead of multiple round trips while preserving existing response semantics.

Enhancements:

  • Refactor GetPost to retrieve post metadata, board name, lock state, reply page, and reply count via a single CTE-based query with pagination offset clamping.
  • Inline lock and board lookups into the main GetPost query and adjust reply processing to use the new projected column names while maintaining legacy behavior for empty discussions.

Tests:

  • Add a suite of GetPost tests covering single-query execution, pagination binding, lock reporting, legacy-domain rewriting, nonexistent and empty discussions, out-of-range pages, negative page clamping, and mention clearing behavior.

GetPost issued five awaited queries in series - the post row, a COUNT
over bbs_reply, the board name, the lock row, and finally the page of
replies. On Workers each is a separate round trip to D1, so opening a
discussion cost five times the network latency no matter how little data
came back.

Collapse them into a single statement built from two CTEs: `post` (the
post row left-joined to bbs_board and bbs_lock) and `page` (the fifteen
replies for the requested page), cross-joined so every row carries the
post metadata alongside one reply, with the total reply count as a
scalar subquery.

The page's LIMIT/OFFSET has to be bound before PageCount is known, so
the offset is clamped at zero and the rows are discarded when the range
check rejects the page. Responses are unchanged, including the
out-of-range and empty-discussion cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="Source/Process.ts">

<violation number="1" location="Source/Process.ts:679">
P2: A D1 failure now becomes the generic `服务器运行错误` response instead of the existing database-query failure Result. Preserve the endpoint's error contract by catching this direct `all()` failure and throwing/returning the same `数据库查询失败` Result shape.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread Source/Process.ts
// offset is clamped here and the rows are simply discarded when the
// range check below rejects the page.
const Offset = Math.max(0, (Data["Page"] - 1) * 15);
const Rows: Array<Record<string, any>> = (await this.RawDatabase.prepare(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A D1 failure now becomes the generic 服务器运行错误 response instead of the existing database-query failure Result. Preserve the endpoint's error contract by catching this direct all() failure and throwing/returning the same 数据库查询失败 Result shape.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At Source/Process.ts, line 679:

<comment>A D1 failure now becomes the generic `服务器运行错误` response instead of the existing database-query failure Result. Preserve the endpoint's error contract by catching this direct `all()` failure and throwing/returning the same `数据库查询失败` Result shape.</comment>

<file context>
@@ -669,13 +669,38 @@ export class Process {
+      // offset is clamped here and the rows are simply discarded when the
+      // range check below rejects the page.
+      const Offset = Math.max(0, (Data["Page"] - 1) * 15);
+      const Rows: Array<Record<string, any>> = (await this.RawDatabase.prepare(
+        "WITH post AS (" +
+        "  SELECT p.user_id AS post_user_id, p.problem_id AS problem_id, p.title AS title, " +
</file context>

@boomzero
boomzero merged commit 35f0bc9 into master Jul 27, 2026
6 checks passed
@boomzero
boomzero deleted the perf/getpost-single-query branch July 27, 2026 06:10
@sourcery-ai

sourcery-ai Bot commented Jul 27, 2026

Copy link
Copy Markdown

🧙 Sourcery is reviewing your pull request!


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant