Fix N+1 query bottleneck in discussion list (GetPosts) - #66
Merged
Conversation
GetPosts previously issued 4 extra sequential DB round trips per post (reply count, last reply, lock status, board name) on top of the page query, i.e. ~60+ round trips for a single 15-post page. Since D1 queries go over the network per call, this dominates latency regardless of indexing. Replaced the per-post loop with a single query using correlated subqueries and joins.
WhereClause is guaranteed empty at the first check (nothing between declaration and use could change it), so CodeFactor correctly flagged it as always-true dead code.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a529fdeb80
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The page-count query went through this.XMOJDatabase's own D1DatabaseSession while the page data query used this.RawDatabase's session. With D1 read replication, sequential consistency is only guaranteed within a session, so the two reads could observe different snapshots and corrupt pagination. Both queries now run as raw SQL through this.RawDatabase. Per Codex review on PR #66.
Contributor
There was a problem hiding this comment.
All reported issues were addressed
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Replaces two separate correlated subqueries (one for last_reply_user_id, one for last_reply_time) that each scanned bbs_reply per row, with a single LEFT JOIN against a ROW_NUMBER()-based derived table. Per cubic-dev-ai review on PR #66.
|
🧙 Sourcery has finished reviewing your pull request! Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The dynamically built WHERE clause and inline SQL string in GetPosts are getting fairly complex; consider extracting the query construction into a helper function or constant to improve readability and reduce the chance of subtle bugs when filters change.
- The new RawDatabase queries access results via array indices and property names (e.g.,
['results'][0]['count']); adding minimal guards or helper accessors for empty result sets and type coercion would make this pagination path more robust.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The dynamically built WHERE clause and inline SQL string in GetPosts are getting fairly complex; consider extracting the query construction into a helper function or constant to improve readability and reduce the chance of subtle bugs when filters change.
- The new RawDatabase queries access results via array indices and property names (e.g., `['results'][0]['count']`); adding minimal guards or helper accessors for empty result sets and type coercion would make this pagination path more robust.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The discussion list endpoint (
GetPostsinSource/Process.ts) was slow because of an N+1 query pattern, not missing indexes. For every one of the 15 posts returned on a page, it made 4 additional sequential, awaited DB round trips:GetTableSizeonbbs_replyfor the reply countSelectonbbs_replyfor the last replySelectonbbs_lockfor lock statusSelectonbbs_boardfor the board nameThat's up to ~62 sequential round trips for a single page load. Since D1 queries go over the network per call, this latency is dominated by round-trip count, not per-query execution time — which is why adding indexes didn't help.
This PR replaces the per-post loop with a single SQL query using correlated subqueries and
LEFT JOINs againstbbs_boardandbbs_lock, cutting the endpoint down to 2 total DB round trips (page count + page data) regardless of page size.Also preserved a subtle existing behavior/bug:
Database.Select/GetTableSizebuild malformed SQL when passed an empty (but defined) condition object, so the page-count query now explicitly passesundefinedwhen no filters are set, matching the original code's behavior for the "all boards / all problems" view.Test plan
bbs_post/bbs_reply/bbs_board/bbs_lockschema from the migrations.npm installwas too slow in this sandbox to verify before pushing — deferring to CI)Generated by Claude Code
Summary by cubic
Fixed the N+1 query in GetPosts and made the count and page queries run on the same D1 session. This drops DB round trips from ~62 to 2 per page and keeps pagination consistent.
this.RawDatabasein one session to avoid snapshot mismatches.objectinstead ofObjectfor types.Written for commit a73e46a. Summary will update on new commits.
Summary by Sourcery
Optimize the discussion list endpoint to remove an N+1 query pattern and reduce database round trips while keeping pagination and behavior consistent.
Enhancements:
objectinstead ofObjectand updating various response collections to be typed as arrays of objects.