Skip to content

**Title:** feat(rate-limit): cap per-provider queue depth with reject-on-overflow#588

Open
SHEROSE0 wants to merge 2 commits into
Talenttrust:mainfrom
SHEROSE0:enhancement/rate-limit-queue-cap
Open

**Title:** feat(rate-limit): cap per-provider queue depth with reject-on-overflow#588
SHEROSE0 wants to merge 2 commits into
Talenttrust:mainfrom
SHEROSE0:enhancement/rate-limit-queue-cap

Conversation

@SHEROSE0

Copy link
Copy Markdown

#closes #409
Description:
This PR implements a configurable queue depth cap to prevent unbounded memory accumulation when upstream providers are persistently slower than the refill rate.

Changes:

  • Queue Capping: Integrated maxQueueDepth validation in src/rateLimit.ts. Requests exceeding this limit now immediately reject with RATE_LIMIT_QUEUE_CAP_EXCEEDED instead of queueing indefinitely.
  • Metrics: Integrated with src/webhookMetrics.ts to track rejections for better observability.
  • Backpressure Policy: Defined deterministic behavior: requests rejected under load can now be explicitly routed to a Dead Letter Queue (DLQ) by the caller.
  • Testing: Added src/rateLimit.test.ts using fake timers to verify FIFO processing and hard rejection limits.

Security & Stability:

  • DoS Protection: Mitigates memory-exhaustion (OOM) scenarios where thousands of closures are captured in an ever-growing queue.
  • Fail-Fast Semantics: Allows upstream services to detect backpressure immediately.

Acceptance Criteria Checklist:

  • Configurable maxQueueDepth applied per provider.
  • Typed error rejection on overflow.
  • Metrics recorded for QUEUE_CAP_EXCEEDED.
  • FIFO order preserved for queued items.
  • Test coverage ≥ 95%.
    #closes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound the token-bucket queue depth to prevent unbounded waiter accumulation

2 participants