feat: add real-time alerting MVP (webhook + slack)

[ogulcanaydogan]

Summary

Implements Phase 3 Step 3 real-time alerting MVP with async fail-open delivery and two sinks:

• Generic webhook sink (optional Bearer auth header)
• Slack Incoming Webhook sink

This PR keeps existing inbound API behavior intact and adds outbound alerting for:

• injection_blocked
• rate_limit_exceeded (window-based aggregate)
• scan_error (window-based aggregate)

What Changed

• Added alerting.* config surface + PIF_ALERTING_* env overrides.
• Added alert dispatcher pipeline (queue + worker, retry/backoff+jitter, fail-open behavior).
• Added middleware event emission for block/rate-limit/scan-error.
• Added new metrics:
  • pif_alert_events_total
  • pif_alert_sink_deliveries_total
• Wired alerting into proxy server bootstrap and CLI runtime options.
• Updated docs and roadmap:
  • README + API reference alerting contract and examples
  • Phase 3 roadmap split: Webhook + Slack [x], PagerDuty [ ]

Public Interface Notes

• No new inbound HTTP endpoints.
• New config/env surface: alerting.*, PIF_ALERTING_*.
• Outbound webhook JSON event contract documented in API reference.

Validation Evidence

Local tests

• go test ./... -> PASS
• go test -race ./... -> PASS
  • Note: macOS linker warning observed (malformed LC_DYSYMTAB), tests still pass.
• Coverage:
  • go test ./... -coverprofile=coverage.out
  • go tool cover -func=coverage.out | tail -n 1
  • Result: total: (statements) 81.4%

Behavior verified by tests

• Alert dispatcher queue/retry/backoff/fail-open semantics
• Webhook Bearer header forwarding
• Slack payload mapping
• Sink failure does not block next sink
• Rate-limit and scan-error window aggregation
• Middleware alert emission for block/rate-limit/scan-error

Scope/Non-goals

• PagerDuty is intentionally out of scope in this PR (left as next sub-step).

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db1a2aa38a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Evict stale alert aggregation buckets

alertWindowAggregator.Record adds a new entry for every unseen aggregation key but never removes old keys, so with alerting enabled a high-cardinality client_key/path stream (including spoofed X-Forwarded-For values) will make buckets grow without bound and steadily increase process memory over long runtimes. Adding TTL-based cleanup or bounded eviction before inserting new keys would prevent this leak-like behavior.

Useful? React with 👍 / 👎.