Skip to content

feat(webhooks): implement adaptive polling in WebhookProcessorService#157

Open
Vswaroop04 wants to merge 2 commits into
BetterDB-inc:masterfrom
Vswaroop04:feat/webhook-adaptive-polling
Open

feat(webhooks): implement adaptive polling in WebhookProcessorService#157
Vswaroop04 wants to merge 2 commits into
BetterDB-inc:masterfrom
Vswaroop04:feat/webhook-adaptive-polling

Conversation

@Vswaroop04
Copy link
Copy Markdown
Contributor

@Vswaroop04 Vswaroop04 commented May 7, 2026

Summary

  • Replaces the fixed setInterval(10s) with a setTimeout-based self-scheduling loop
  • Updates processRetries() to return a boolean indicating whether any deliveries were found
  • Introduces adaptive polling:
    • 2s delay when retries are present (fast path)
    • 10s delay when the queue is empty (base path, unchanged behavior)
  • Adds an isShuttingDown flag to prevent re-scheduling after onModuleDestroy

Why

Webhook retries follow an exponential backoff pattern with short initial delays (1s, 2s, 4s). With the previous fixed 10s polling interval, retries could sit idle for up to 10 seconds before being processed.

This adaptive approach reduces that delay during active retry periods while maintaining the same database load during idle periods.

Test Plan

  • Run pnpm test and confirm existing webhook tests pass
  • Manually trigger a failing webhook delivery and verify retries are picked up in ~2s instead of ~10s
  • Confirm that no polling occurs after the service shuts down

@Vswaroop04
Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

Replaces the fixed 10s setInterval with a setTimeout loop that reschedules
itself based on whether the previous cycle found any pending retries:

- BASE_RETRY_CHECK_INTERVAL_MS (10s) when the queue is empty — avoids
  unnecessary database load during quiet periods.
- FAST_RETRY_CHECK_INTERVAL_MS (2s) immediately after a cycle that found
  retries — picks up subsequent backoff windows quickly without a full 10s
  wait.

processRetries() now returns boolean so the scheduler can read the result.
Shutdown guard (isShuttingDown) prevents re-scheduling after destroy.
@Vswaroop04 Vswaroop04 force-pushed the feat/webhook-adaptive-polling branch from 18d5768 to 85bb896 Compare May 13, 2026 19:12
Copy link
Copy Markdown
Member

@KIvanow KIvanow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Vswaroop04 - good idea and the implementation logic is sound, but I can't merge this without test coverage on the new scheduling behaviour. A few things to address:

  1. Tests for the adaptive loop. The schedule closure, the isShuttingDown guard, and the error-path fallback to BASE are all untested. Jest fake timers + a spy on processRetries returning true/false/rejecting would cover the three cases without spinning real timers.
  2. Check off the test plan. Please confirm pnpm test is green and the manual retry paths were verified before marking ready for review.
  3. Minor cleanup. The initial run and the schedule inner function have identical .then()/.catch() chains written out twice. Consider calling schedule(0) for the initial kick-off instead - one code path, fully testable.

- Replace duplicated initial-run chain with schedule(0)
- Add fake-timer tests covering FAST/BASE scheduling, isShuttingDown guard,
  and error-path fallback to BASE interval
@Vswaroop04
Copy link
Copy Markdown
Contributor Author

Hey @Vswaroop04 - good idea and the implementation logic is sound, but I can't merge this without test coverage on the new scheduling behaviour. A few things to address:

1. Tests for the adaptive loop. The schedule closure, the isShuttingDown guard, and the error-path fallback to BASE are all untested. Jest fake timers + a spy on processRetries returning true/false/rejecting would cover the three cases without spinning real timers.

2. Check off the test plan. Please confirm pnpm test is green and the manual retry paths were verified before marking ready for review.

3. Minor cleanup. The initial run and the schedule inner function have identical .then()/.catch() chains written out twice. Consider calling schedule(0) for the initial kick-off instead - one code path, fully testable.

Done

@Vswaroop04 Vswaroop04 force-pushed the feat/webhook-adaptive-polling branch from 1a4c3d0 to 339af3b Compare May 14, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants