Skip to content

notifications (2/5): notification_history keyset listing#453

Merged
illegalprime merged 1 commit into
mainfrom
eden/notifications-2-history-store
Jun 16, 2026
Merged

notifications (2/5): notification_history keyset listing#453
illegalprime merged 1 commit into
mainfrom
eden/notifications-2-history-store

Conversation

@illegalprime

@illegalprime illegalprime commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Stack 2/5 — base: eden/notifications-1-proto (#452)

1. Summary

Adds the read/list path for notification history. Background already on main: an Alertmanager webhook receiver persists each fired-alert notification into the notification_history table. This PR adds the ability to page through that history newest-first, so an operator can review what alerts fired for their organization. It delivers the DB/query/store layer only — a ListNotificationHistory SQL query, a supporting index, and a List store method using keyset pagination. The gRPC handler that calls this arrives in #455.

2. How it works

The flow is a single-page read keyed off the caller's organization and an optional cursor:

  1. A caller (the HistoryService handler landing in notifications (4/5): Grafana-backed service, handlers, and server wiring #455) invokes store.List(ctx, organizationID, beforeID, limit). beforeID is nil for the first page; for subsequent pages the caller passes the id of the last row it already received.
  2. List maps the Go arguments into sqlc query params (organizationIDNullInt64, beforeID → nullable cursor, limit → page size) and calls the generated ListNotificationHistory.
  3. The SQL selects from notification_history filtered to the organization, with id < before_id applied only when a cursor is present, ordered id DESC, capped by LIMIT. It LEFT JOINs the device and discovered_device tables to resolve device_id into a human-readable device name and MAC at read time (falling back to '' when the device is unknown or soft-deleted).
  4. Postgres serves the ordering and seek from the new (organization_id, id DESC) index rather than scanning.
  5. List converts each generated row into a StoredNotification (the persisted Notification plus row identity ID, ReceivedAt, and the read-time DeviceName/DeviceMAC) and returns the slice.

Keyset (cursor) pagination: instead of OFFSET, each next page seeks past the last seen id (id < before_id) and reads the next LIMIT rows in id DESC order. Because notification_history is insert-only, ids are monotonic and stable, so pages never shift or duplicate as new rows arrive. The handler in #455 owns page-size policy (default 50, max 200); this layer just honors the limit it is given.

3. Diagrams

Component / data flow (boundaries this PR touches are bold in prose; the handler is in #455):

flowchart LR
    H["HistoryService handler (#455)"] -->|"List(orgID, beforeID, limit)"| S["SQLNotificationHistoryStore.List"]
    S -->|"ListNotificationHistoryParams"| Q["sqlc: ListNotificationHistory (generated)"]
    Q -->|"SELECT ... WHERE org_id=? AND id&lt;? ORDER BY id DESC LIMIT ?"| PG[("Postgres: notification_history")]
    PG -. "index scan" .-> IDX["idx_notification_history_org_id (organization_id, id DESC)"]
    Q -.->|"LEFT JOIN"| DEV[("device / discovered_device")]
    PG -->|"rows"| S
    S -->|"[]StoredNotification"| H
Loading

Keyset pagination over an insert-only table (ordering by id DESC, seeking past the cursor):

sequenceDiagram
    participant C as Caller (#455)
    participant L as Store.List
    participant DB as Postgres
    C->>L: List(org, beforeID=nil, limit=50)
    L->>DB: WHERE org_id=org ORDER BY id DESC LIMIT 50
    DB-->>L: rows [id=900 .. 851]
    L-->>C: page 1 (last id = 851)
    C->>L: List(org, beforeID=851, limit=50)
    L->>DB: WHERE org_id=org AND id < 851 ORDER BY id DESC LIMIT 50
    DB-->>L: rows [id=850 .. 801]
    L-->>C: page 2 (last id = 801)
    Note over C,DB: New inserts get higher ids, already-paged rows never shift.
Loading

4. Areas of the code involved

Area / package / file What changed Why it matters for review
server/sqlc/queries/notification_history.sql New query ListNotificationHistory: org filter, optional before_id cursor, id DESC, LIMIT, device-enrichment joins Source of truth for the read path — review the SQL, cursor predicate, join correctness, and '' fallbacks here
server/internal/domain/notificationhistory/notificationhistory.go New StoredNotification read-back type and Lister interface (List with beforeID *int64 cursor) Defines the domain contract the handler (#455) codes against
server/internal/domain/stores/sqlstores/notification_history.go New List method on the SQL store; asserts Lister is implemented; maps params and rows Mapping layer between domain and sqlc; check cursor passthrough and row→StoredNotification conversion (helpers ptrToNullInt64/nullInt64ToPtr/nullTimeToPtr are pre-existing)
server/migrations/000087_notification_history_keyset_index.up.sql New migration: CREATE INDEX CONCURRENTLY (organization_id, id DESC) Backs the access pattern; review the CONCURRENTLY rationale and recovery note
server/migrations/000087_notification_history_keyset_index.down.sql New DROP INDEX CONCURRENTLY IF EXISTS Matching online rollback
server/generated/sqlc/notification_history.sql.go Generated ListNotificationHistory query, params, row, scanner Generated — skip
server/generated/sqlc/db.go Generated prepared-statement registration/teardown for the new query Generated — skip

5. Key technical decisions & trade-offs

  • Keyset pagination (id < before_id, id DESC) over OFFSET/LIMIT — stable on an insert-only table (no page drift/skew as rows arrive) and index-friendly; cost is the caller must thread the last id as a cursor rather than a page number.
  • CREATE INDEX CONCURRENTLY over a plain CREATE INDEX — a plain build takes ACCESS EXCLUSIVE on the live table and blocks webhook inserts for the build duration; CONCURRENTLY builds online (matches the convention in migration 000074). Trade-off: must be the sole statement and cannot run in a transaction, and a failed build can leave a dirty migration + an INVALID index (recovery noted inline in the migration).
  • Migration numbered 000087 so it merges before the 000088 permission seed in notifications (3/5): notification:read and notification:manage permissions #454 — guarantees schema_migrations advances in numeric = merge order, avoiding golang-migrate skipping a lower-numbered migration on already-upgraded DBs.
  • Device name/MAC resolved at read time via LEFT JOIN rather than denormalized onto notification_history at insert — names stay current and deleted/unknown devices degrade to ''; cost is a join per page (served by existing device keys).
  • Lister as a separate interface from the existing Store (insert) interface — read and write capabilities are segregated so the notifications (4/5): Grafana-backed service, handlers, and server wiring #455 handler can depend only on the read surface.

6. Testing & validation

  • just gen re-run so server/generated/sqlc matches the committed query (the generated-code CI check enforces a clean tree).
  • The SQL compiles under sqlc; the store satisfies both notificationhistory.Store and notificationhistory.Lister via compile-time var _ = (*...) assertions.
  • Not covered in this PR: no unit/integration test exercises List against a live DB, and there is no handler/end-to-end coverage — the gRPC HistoryService that calls List (and enforces the page-size clamp of default 50 / max 200) lands in notifications (4/5): Grafana-backed service, handlers, and server wiring #455, where request-level validation and tests belong. The CONCURRENTLY migration's online behavior is asserted by convention (matching 000074), not by an automated deploy test.

🤖 Generated with Claude Code

@illegalprime illegalprime requested a review from a team as a code owner June 15, 2026 15:53
@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated security-focused code review generated by Codex.
It should be used as a supplementary check alongside human review.
False positives are possible - use your judgment.

Scope summary

  • Reviewed pull request diff only (591590e960533ecb778b9d03032f5c9dd16f1fba...6f51c39065b22e590ba4fc5effa6f3b0efcea698, exact PR three-dot diff)
  • Model: gpt-5.5

💡 Click "edited" above to see previous reviews for this PR.


Review Summary

Overall Risk: MEDIUM

Findings

[MEDIUM] Unbounded notification-history page limit reaches SQL

  • Category: Reliability
  • Location: server/internal/domain/stores/sqlstores/notification_history.go:64
  • Description: The new List method accepts limit int32 and passes it directly into ListNotificationHistory as PageLimit; the SQL query then uses it directly as LIMIT page_limit. This new store boundary does not apply a default, reject negative values, or cap oversized values, unlike nearby pagination paths such as activity, diagnostics, fleetmanagement, collections, and curtailment.
  • Impact: If this is wired directly to ListNotificationsRequest.page_size, an authenticated caller can request a very large page from a growing notification_history table, forcing a large index scan and causing sqlc to materialize all returned rows into memory before the store maps them. That can degrade DB/server availability or cause memory pressure.
  • Recommendation: Normalize page size before calling SQL, preferably at the domain/store boundary as defense in depth: default <= 0 to a small value, cap to a fixed max such as 100 or 1000, and fetch pageSize + 1 only after applying the cap when computing has_more. Add tests for zero, negative, and oversized limits.

Notes

The authoritative diff only changed notification-history sqlc/store/query/index files. I did not see changed auth handlers, Connect-RPC wiring, frontend rendering, plugin logic, command execution, network discovery, pool configuration, or protobuf field definitions in .git/codex-review.diff.

No SQL injection, command injection, cryptostealing/pool hijack, or cross-org data exposure issue was evident in the changed hunks. This was a static review; I did not run tests.


Generated by Codex Security Review |
Triggered by: @illegalprime |
Review workflow run

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 923f5e3745

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread server/migrations/000088_notification_history_keyset_index.up.sql Outdated
Comment thread server/migrations/000088_notification_history_keyset_index.up.sql Outdated
illegalprime added a commit that referenced this pull request Jun 15, 2026
CREATE INDEX took ACCESS EXCLUSIVE on the live notification_history table,
blocking webhook-driven inserts for the duration of the build. Switch to
CREATE INDEX CONCURRENTLY (and a matching DROP INDEX CONCURRENTLY in the down
migration) so the build runs online, matching the convention established in
000074. golang-migrate's postgres driver runs statements without an implicit
transaction, so CONCURRENTLY is safe.

Addresses Codex security review finding (LOW) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added javascript Pull requests that update javascript code client shared labels Jun 15, 2026
@illegalprime

Copy link
Copy Markdown
Contributor Author

Codex findings on this PR:

  • [LOW] CREATE INDEX blocks writes — fixed here (commit d9857a9): now CREATE INDEX CONCURRENTLY (+ DROP ... CONCURRENTLY in the down migration), matching the 000074 convention. golang-migrate's postgres driver runs statements without an implicit transaction, so this is safe.
  • [MEDIUM] unbounded page limit — resolved in notifications (4/5): Grafana-backed service, handlers, and server wiring #455: the HistoryService handler clamps pageSize (historyDefaultPageSize=50, historyMaxPageSize=200) before calling the store, which is the only caller. No change needed at the store layer.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9857a968f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread server/generated/sqlc/notification_history.sql.go Outdated
illegalprime added a commit that referenced this pull request Jun 15, 2026
CREATE INDEX took ACCESS EXCLUSIVE on the live notification_history table,
blocking webhook-driven inserts for the duration of the build. Switch to
CREATE INDEX CONCURRENTLY (and a matching DROP INDEX CONCURRENTLY in the down
migration) so the build runs online, matching the convention established in
000074. golang-migrate's postgres driver runs statements without an implicit
transaction, so CONCURRENTLY is safe.

Addresses Codex security review finding (LOW) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@illegalprime illegalprime force-pushed the eden/notifications-2-history-store branch from d9857a9 to c49d919 Compare June 15, 2026 17:44
@github-actions github-actions Bot removed javascript Pull requests that update javascript code client shared labels Jun 15, 2026
illegalprime added a commit that referenced this pull request Jun 15, 2026
Two review fixes:
- The keyset-index migration was numbered 000088 while the permission-seed
  migration in the next stack PR is 000087. Since this PR merges first,
  deploying it would advance schema_migrations to 88 and golang-migrate would
  then skip the lower-numbered 000087 seed on already-upgraded DBs. Renumber
  the index to 000087 (this PR merges first); the seed becomes 000088 in the
  authz PR so the numeric order matches the merge order.
- Regenerate sqlc: the committed ListNotificationHistory query comment had been
  shortened but generated/sqlc was stale, so 'just gen' + the generated-code CI
  check would have failed on a dirty tree.

Addresses Codex connector findings (P1/P2) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
illegalprime added a commit that referenced this pull request Jun 15, 2026
…to the mount PR

Two review fixes:
- Renumber the permission-seed migration 000087 -> 000088 so it sits above the
  keyset-index migration (now 000087) that merges first in the stack, keeping
  numeric order aligned with merge order (see #453).
- Move the notification entries out of registeredServices (the RBAC contract
  fixture): TestRPCContract_RegisteredServicesMatchMainMux compares it against
  the handlers mounted in main.go, and the mounts land in the next stack PR, so
  listing them here failed the test standalone. The ProcedurePermissions policy
  map (and its catalog keys) stay here; only the mount-coupled registration
  moves to the server PR.

Addresses Codex connector findings (P1/P2) on #454.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@illegalprime

Copy link
Copy Markdown
Contributor Author

codex-connector triage (all fixed):

  • [P1] Use the next sequential migration number — the "missing 000087" was a split artifact (it lived in the next PR), but the connector was right that the ordering was wrong: the index was 000088 while the seed was 000087 in a later-merging PR. Renumbered the index to 000087 here (merges first); seed becomes 000088 in notifications (3/5): notification:read and notification:manage permissions #454.
  • [P2] Build the index concurrently — fixed: CREATE INDEX CONCURRENTLY (+ concurrent down).
  • [P2] Regenerate sqlc — fixed: re-ran sqlc generate; the stale ListNotificationHistory comment is now in sync.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 607669536a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread server/migrations/000087_notification_history_keyset_index.up.sql Outdated
@illegalprime illegalprime force-pushed the eden/notifications-1-proto branch from d032a2d to 7a2ceb2 Compare June 15, 2026 18:35
illegalprime added a commit that referenced this pull request Jun 15, 2026
CREATE INDEX took ACCESS EXCLUSIVE on the live notification_history table,
blocking webhook-driven inserts for the duration of the build. Switch to
CREATE INDEX CONCURRENTLY (and a matching DROP INDEX CONCURRENTLY in the down
migration) so the build runs online, matching the convention established in
000074. golang-migrate's postgres driver runs statements without an implicit
transaction, so CONCURRENTLY is safe.

Addresses Codex security review finding (LOW) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
illegalprime added a commit that referenced this pull request Jun 15, 2026
Two review fixes:
- The keyset-index migration was numbered 000088 while the permission-seed
  migration in the next stack PR is 000087. Since this PR merges first,
  deploying it would advance schema_migrations to 88 and golang-migrate would
  then skip the lower-numbered 000087 seed on already-upgraded DBs. Renumber
  the index to 000087 (this PR merges first); the seed becomes 000088 in the
  authz PR so the numeric order matches the merge order.
- Regenerate sqlc: the committed ListNotificationHistory query comment had been
  shortened but generated/sqlc was stale, so 'just gen' + the generated-code CI
  check would have failed on a dirty tree.

Addresses Codex connector findings (P1/P2) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@illegalprime illegalprime force-pushed the eden/notifications-2-history-store branch from 6076695 to 03bb87c Compare June 15, 2026 18:35
illegalprime added a commit that referenced this pull request Jun 15, 2026
…to the mount PR

Two review fixes:
- Renumber the permission-seed migration 000087 -> 000088 so it sits above the
  keyset-index migration (now 000087) that merges first in the stack, keeping
  numeric order aligned with merge order (see #453).
- Move the notification entries out of registeredServices (the RBAC contract
  fixture): TestRPCContract_RegisteredServicesMatchMainMux compares it against
  the handlers mounted in main.go, and the mounts land in the next stack PR, so
  listing them here failed the test standalone. The ProcedurePermissions policy
  map (and its catalog keys) stay here; only the mount-coupled registration
  moves to the server PR.

Addresses Codex connector findings (P1/P2) on #454.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@illegalprime illegalprime force-pushed the eden/notifications-1-proto branch from 7a2ceb2 to cd175c1 Compare June 15, 2026 19:24
illegalprime added a commit that referenced this pull request Jun 15, 2026
CREATE INDEX took ACCESS EXCLUSIVE on the live notification_history table,
blocking webhook-driven inserts for the duration of the build. Switch to
CREATE INDEX CONCURRENTLY (and a matching DROP INDEX CONCURRENTLY in the down
migration) so the build runs online, matching the convention established in
000074. golang-migrate's postgres driver runs statements without an implicit
transaction, so CONCURRENTLY is safe.

Addresses Codex security review finding (LOW) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
illegalprime added a commit that referenced this pull request Jun 15, 2026
Two review fixes:
- The keyset-index migration was numbered 000088 while the permission-seed
  migration in the next stack PR is 000087. Since this PR merges first,
  deploying it would advance schema_migrations to 88 and golang-migrate would
  then skip the lower-numbered 000087 seed on already-upgraded DBs. Renumber
  the index to 000087 (this PR merges first); the seed becomes 000088 in the
  authz PR so the numeric order matches the merge order.
- Regenerate sqlc: the committed ListNotificationHistory query comment had been
  shortened but generated/sqlc was stale, so 'just gen' + the generated-code CI
  check would have failed on a dirty tree.

Addresses Codex connector findings (P1/P2) on #453.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@illegalprime illegalprime force-pushed the eden/notifications-2-history-store branch from 03bb87c to b89405f Compare June 15, 2026 19:24
illegalprime added a commit that referenced this pull request Jun 15, 2026
…to the mount PR

Two review fixes:
- Renumber the permission-seed migration 000087 -> 000088 so it sits above the
  keyset-index migration (now 000087) that merges first in the stack, keeping
  numeric order aligned with merge order (see #453).
- Move the notification entries out of registeredServices (the RBAC contract
  fixture): TestRPCContract_RegisteredServicesMatchMainMux compares it against
  the handlers mounted in main.go, and the mounts land in the next stack PR, so
  listing them here failed the test standalone. The ProcedurePermissions policy
  map (and its catalog keys) stay here; only the mount-coupled registration
  moves to the server PR.

Addresses Codex connector findings (P1/P2) on #454.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Base automatically changed from eden/notifications-1-proto to main June 16, 2026 14:38
@illegalprime illegalprime force-pushed the eden/notifications-2-history-store branch from b89405f to c8602d1 Compare June 16, 2026 14:43
Add the notification_history keyset Lister/Store, the sqlc query, and the
CONCURRENTLY-built (organization_id, id DESC) keyset index (migration 000087).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@illegalprime illegalprime force-pushed the eden/notifications-2-history-store branch from c8602d1 to 6f51c39 Compare June 16, 2026 14:55

@ankitgoswami ankitgoswami left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving based on the PR description

@illegalprime illegalprime merged commit a5ccb30 into main Jun 16, 2026
74 checks passed
@illegalprime illegalprime deleted the eden/notifications-2-history-store branch June 16, 2026 18:22
illegalprime added a commit that referenced this pull request Jun 17, 2026
Completes the notifications feature with the read-only delivery history
surface, backed by Proto Fleet's own notification_history table (#453)
rather than Grafana.

Server: the HistoryService (ListNotifications) with page-size clamping
and exact has_more via keyset pagination, plus its registration in the
mux, RPC contract test, and session-only interceptor list.

Client: the History section + table with load-more pagination, the
dashboard "Active notifications" card (gated on notification:read,
deriving firing alerts client-side from the first history page), and the
history parts of the API module, store, and view-model types.

With this slice the mux mounts all four notifications.v1 services and the
settings page renders all four sections.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants