feat: As a gateway user, I want adaptive rate limiting per consumer, so that abnormal traffic spikes can be automatically controlled

[MadhuTiwari-345]

Description

---

Problem

Current rate limiting plugins such as limit-req, limit-count, and limit-concurrency use static thresholds. These limits work well for predictable traffic, but they do not adapt when a specific consumer suddenly changes behavior.

In real-world scenarios, abuse or misconfiguration often appears as sudden traffic spikes from an otherwise normal consumer. With static limits, operators must manually adjust configurations or rely on external monitoring systems.

---

Proposed Feature

As a gateway user, I want an optional adaptive rate limiting mode per consumer, so that the gateway can automatically respond to abnormal traffic patterns without constant manual intervention.

This could work at a high level as follows:

• Track a rolling request baseline for each consumer (e.g., average over a recent time window).

• Detect abnormal spikes compared to the historical baseline.

• Temporarily tighten rate limits when abnormal behavior is detected.

• Gradually relax limits once traffic returns to normal patterns.

---

Why This Is Useful

• Improves security by mitigating sudden abuse patterns.

• Reduces operational overhead for gateway operators.

• Adds intelligent traffic control at the gateway level.

• Enhances production-grade resilience without external systems.

---

Possible Direction

• This could potentially be implemented:

• As an extension to existing rate limit plugins, or

• As a separate plugin (e.g., adaptive-limit)

• Using shared memory or Redis to maintain rolling consumer metrics.

---

This is a feature proposal and not a duplicate of existing static rate limiting discussions. I would appreciate feedback on feasibility and design considerations and if any suggestions and guidance regarding this please guide me.

---

[Baoyuantop]

Hi @MadhuTiwari-345, thank you for this excellent and detailed feature proposal.

Your analysis of the limitations of static rate limiting thresholds is spot on. In production environments, traffic patterns can be highly dynamic, and an adaptive approach that responds to each consumer's baseline behavior would be a powerful addition to APISIX's traffic management capabilities.

We agree that this feature would significantly improve security posture against abuse and reduce the operational overhead for gateway administrators. The high-level workflow you've outlined -- tracking a baseline, detecting spikes, and dynamically adjusting limits -- is a very logical and solid foundation for implementation.

Regarding the implementation direction, creating a new, separate plugin (e.g., adaptive-limit) seems like a clean approach. It would promote modularity and avoid adding too much complexity to the existing rate-limiting plugins. For the backend, supporting both shared memory for single-node deployments and Redis for cluster-wide consistency is a great idea.

This is a valuable and significant feature for the community. We encourage you and other community members to further discuss the specific algorithms for baseline calculation and anomaly detection. For instance, we could consider simple moving averages (SMA) or exponential moving averages (EMA) and weigh their trade-offs.

We would welcome a design document or even a proof-of-concept if you are interested in contributing further. This is a fantastic proposal, and we look forward to seeing it develop.

[MadhuTiwari-345]

Thank you for the guidance. @Baoyuantop

To approach this incrementally, I would like to start with a smaller first step:

Instead of implementing full adaptive rate enforcement immediately, I plan to begin with a basic per-consumer rolling baseline tracker (for example, using an exponential moving average).

This initial version would:

• Track request rate per consumer

• Maintain a rolling baseline value in shared memory

• Log or expose the baseline metric (without automatically tightening limits yet)

This would lay the foundation for future adaptive rate limiting logic while keeping the initial implementation focused and reviewable.

Please let me know if this phased approach makes sense before I start drafting a PoC.

[MadhuTiwari-345]

Proposed Incremental Design for adaptive-limit (Phase 1 – Baseline Tracking PoC)

To move this feature forward in a structured and reviewable way, I would like to begin with a minimal implementation focused only on per-consumer baseline tracking, without enforcing adaptive limits yet.

This phased approach keeps the initial scope small while laying the technical foundation for future adaptive rate control.

---

Scope (Phase 1)

This initial PoC will:

• Track request count per consumer
• Maintain a rolling request-rate baseline using Exponential Moving Average (EMA)
• Store metrics in ngx.shared.dict
• Log or expose the computed baseline for observability

This phase will not enforce stricter limits or modify request behavior.

---

High-Level Architecture

Plugin name: adaptive-limit
Execution phase: access

Data Storage

Use a dedicated shared memory dictionary:

ngx.shared.adaptive_limit_dict

Key format:

adaptive:{consumer_id}

Each consumer entry will store:

• count – request count within current time window
• window_start – timestamp of current window
• ema – rolling baseline request rate

---

Baseline Algorithm

Use Exponential Moving Average:

EMA_new = α * current_rate + (1 - α) * EMA_old

Where:

• α (alpha) is configurable (default: 0.2)
• current_rate = request_count / window_size

EMA is preferred over Simple Moving Average because:

• Constant memory usage
• Faster adaptation to behavioral changes
• Lower implementation complexity in Lua

---

Windowing Strategy

• Fixed time window (default: 1 second)

• Increment counter per request

• On window expiration:

  • Compute current rate
  • Update EMA
  • Reset counter

---

Future Phases

Phase 2:

• Spike detection (e.g., current_rate > multiplier × ema)

Phase 3:

• Temporary limit tightening
• Gradual relaxation when traffic stabilizes

---

I would like to proceed with implementing Phase 1 as a small, self-contained PoC for review. Feedback on the storage model, execution phase, or algorithm choice would be appreciated before I begin development.

---

[MadhuTiwari-345]

@Baoyuantop Please let me know if it is okay? If I can start the work?

[MadhuTiwari-345]

Hi @Baoyuantop Should I start the working on this?? Is it okay??

[Baoyuantop]

Hi @MadhuTiwari-345, thanks for the detailed Phase 1 design.

After further thought, I think we need to discuss a few technical concerns
before moving to implementation:

1. False positive handling: EMA-based detection is sensitive to legitimate
   traffic patterns like daily peaks, promotions, or batch jobs. How should we
   handle cases where "normal" traffic looks like a spike to the algorithm?
2. Cold start: A new consumer has no baseline. What should the behavior be
   during the warm-up period?
3. Value of Phase 1 alone: A plugin that only tracks baselines without
   enforcing limits is essentially a metrics collector. Would it be better to
   jump directly to a minimal but complete implementation (tracking + detection + basic enforcement) so users get immediate value?
4. Scope consideration: In many production setups, anomaly detection is
   handled by external systems (WAF, monitoring). Could you share your use case
   — are you looking to replace an external system, or complement it?

I'd suggest starting with a PoC as an external plugin (using APISIX's external
plugin mechanism) to validate the algorithm in a real environment. Once we have
data on its effectiveness and performance impact, it would be much easier to
evaluate inclusion in core.

[MadhuTiwari-345]

Thank you for the detailed feedback, @Baoyuantop . These are very helpful points.

Let me address the concerns you raised:

1. False Positive Handling

You are correct that simple EMA-based detection could misclassify legitimate traffic patterns such as promotions, scheduled jobs, or daily peaks.

To reduce false positives, the spike detection logic could include additional safeguards, for example:

• A configurable multiplier threshold (e.g., trigger only if current_rate > multiplier × EMA, where multiplier defaults to something like 3–5).
• A minimum observation window before detection becomes active.
• Optional cooldown periods to prevent repeated triggering.

The goal would be to keep the algorithm simple enough to run efficiently inside the gateway while still filtering obvious abnormal spikes.

2. Cold Start Behavior

For new consumers with no baseline:

• The plugin could operate in a warm-up mode for the first N windows (e.g., 30–60 seconds).
• During this period, it would only collect data and initialize the EMA baseline.
• Spike detection would be disabled until sufficient historical data is available.

This should prevent incorrect early enforcement.

3. Value of Phase 1

I understand your point that a baseline-only plugin may act mostly as a metrics collector.

My intention with Phase 1 was mainly to validate:

• Shared memory storage behavior
• Per-consumer metric tracking
• Performance impact inside the request path

However, I agree that providing basic spike detection and temporary limit tightening in the first version would make the feature immediately useful. I am open to adjusting the scope so the initial implementation includes:

• Baseline tracking (EMA)
• Basic spike detection
• Temporary rate tightening

while keeping the logic intentionally simple.

4. Intended Use Case

The goal is not to replace external anomaly detection systems (such as WAFs or monitoring platforms), but to add lightweight adaptive protection directly at the gateway layer.

This can help in scenarios where:

• A normally well-behaved API consumer suddenly sends a burst of traffic due to misconfiguration or abuse.
• Operators want automatic first-line protection without relying entirely on external systems.

So the feature would complement existing infrastructure rather than replace it.

5. External Plugin PoC

Starting as an external plugin to validate the algorithm and performance impact makes sense. That would allow experimentation without affecting the core plugins.

I can begin with a minimal PoC external plugin that includes:

• per-consumer request tracking
• EMA baseline calculation
• simple spike detection
• temporary rate tightening

Once we gather feedback and performance observations, we could evaluate whether it makes sense to move it into core.

Please let me know if this direction sounds reasonable. If so, I will proceed with a small PoC external plugin and share the implementation for review.

[Baoyuantop]

Hi @MadhuTiwari-345, LGTM

[janiussyafiq]

@MadhuTiwari-345 apisix provide a plugin template to work with developing a new template if u want to take a look https://github.com/api7/apisix-plugin-template

[MadhuTiwari-345]

Thanks for sharing the template, @janiussyafiq .

I’ll use the apisix-plugin-template as the starting point for the adaptive-limit PoC and share the implementation here once I have a working prototype.