feat(appsec): implementation and tests #63
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add AppSec validation to SPOA request handler - Integrate AppSec component with global URL/API key config - Add AppSec collections and acquisition setup for docker-compose - Update HAProxy configs with http-buffer-request option - Add Vagrantfile for testing environment - Update bouncer config to support AppSec per-host configuration
LaurenceJJones
force-pushed
the
feature/lj-appsec
branch
from
b30cd88 to
8867d32
Compare
- Fix readHeaders to properly handle CRLF line endings from HAProxy req.hdrs - Extract User-Agent from parsed headers instead of separate SPOE message arg - Only parse HTTP data when necessary (not for Allow/Ban unless AppSec always_send) - Move debug logging in createAppSecRequest to show actual headers being sent - Remove excessive logging from parseHTTPData function - Clean up verbose debug logs for cookie clearing and redirects
- Check io.Copy return value (errcheck) - Remove unused logger parameter from parseHTTPData (unparam) - Remove duplicate User-Agent logging from debug log - Access req fields directly in debug log instead of reading from headers
Track metrics in SPOA handler where we already have the IP as netip.Addr type, avoiding unnecessary string-to-IP parsing. This is more efficient and cleaner than parsing the IP from string in the AppSec component.
LaurenceJJones
changed the title
Closed
Copilot finished reviewing on behalf of
LaurenceJJones
November 23, 2025 07:16
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive AppSec (Web Application Firewall) integration to the CrowdSec SPOA bouncer, enabling additional layer of protection beyond IP-based remediation. The implementation follows a fail-open pattern, optimizes for performance with connection pooling, and supports both global and per-host AppSec configuration.
Key Changes
- AppSec Client Implementation: New HTTP client with optimized connection pooling (keep-alive, 5s timeout) that validates requests against CrowdSec's AppSec engine via custom X-Crowdsec-Appsec-* headers
- SPOA Handler Integration: AppSec validation integrated into remediation flow with conditional execution (skips if already banned/captcha'd unless
always_send: true), reuses parsed HTTP data from captcha handling - Infrastructure Setup: Complete test environments added via Vagrantfile and Docker Compose with AppSec collections, acquisition configs, and HAProxy body buffering enabled
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
internal/appsec/root.go |
New AppSec client with request validation, response processing, and HTTP transport configuration with connection pooling |
pkg/spoa/root.go |
AppSec integration into HTTP request handling flow with data reuse, improved header parsing with HTTP request line detection, and metrics tracking |
pkg/host/root.go |
Global AppSec configuration management with per-host override support |
pkg/cfg/config.go |
Global appsec_url and api_key configuration fields |
cmd/root.go |
Global AppSec configuration initialization in host manager |
docker-compose.yaml, docker-compose.proxy-test.yaml |
AppSec collections installation and acquisition configuration mounting |
docker/crowdsec/acquisitions/appsec.yaml |
AppSec component acquisition config listening on port 7422 |
docker/crowdsec/README.md |
Documentation for AppSec collections and manual installation |
config/haproxy.cfg, config/haproxy-upstreamproxy.cfg |
Added http-buffer-request option and removed non-existent SPOA server |
Vagrantfile |
Complete test environment provisioning with CrowdSec, AppSec, HAProxy, and Nginx |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix metrics call to use WithLabelValues instead of prometheus.Labels - Improve HTTP request line detection with method validation - Clarify remediation condition (>= Captcha instead of > Unknown) - Add body length check before sending to AppSec - Add debug logging for malformed headers - Fix port mismatch in Vagrantfile (4241 -> 7422) - Remove HTTP client timeout, rely on context timeout
Remove HTTP request line detection logic since HAProxy only sends headers, not the initial HTTP request line
…obal AppSec from host manager - Simplified validateWithAppSec to return remediation instead of modifying pointer - Removed req parameter - ban injection handled at call sites - Removed hoststring parameter - extracted from message in extractAppSecData - Removed httpData parameter - always parsed from message - Created logger with host context for better logging - Removed global AppSec config from host manager - Hosts now only use explicit AppSec overrides - Global AppSec handled entirely at SPOA level
- Fixed validateWithAppSec to return the more restrictive remediation - Prevents security downgrade when AppSec returns Allow but current remediation is Ban/Captcha - Now correctly compares and returns max(currentRemediation, appSecRemediation) - Matches function documentation behavior
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 15 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Move API key validation before cloning headers (fail-fast optimization) - Fix error message capitalization to follow Go conventions - Improve HTTP version conversion with explicit switch for edge cases - Simplify response body handling - just discard without size checks
LaurenceJJones
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add AppSec (WAF) Integration Support
This PR adds integration with CrowdSec's AppSec engine, enabling Web Application Firewall (WAF) functionality alongside existing remediation mechanisms (ban, captcha).
Feature Overview
The bouncer now forwards HTTP requests to the CrowdSec AppSec engine for additional validation. AppSec can override decisions made by LAPI, providing more granular WAF-based protection. The integration supports both global and per-host AppSec configuration.
Key Changes
AppSec Component (
internal/appsec/root.go)AppSecClientwith optimized HTTP transport (keep-alive, connection pooling)ValidateRequest()forwards requests to AppSec engine with full request contextappsec_urlandapi_keyconfiguration (with global fallback)Note: We should consider changing the underlying SPOE implementation since AppSec increases processing time and we hit timeouts quite easily.
SPOA Handler Integration (
pkg/spoa/root.go)validateWithAppSec()function signature (removed unnecessary parameters)always_send: true)\r\n, skips malformed lines gracefully)Host Manager (
pkg/host/root.go)Configuration (
pkg/cfg/config.go)appsec_urlconfiguration optionapi_keyas LAPI by default (can be overridden per-host)Infrastructure
http-buffer-requestoption for body capture, increased SPOA timeout to 6sSecurity Fixes
validateWithAppSec()now correctly returns the more restrictive remediation, preventing security bypasses when AppSec returns Allow but current remediation is Ban/CaptchaBug Fixes & Improvements
\r\nline endings correctly)maps.Clone()Configuration Example
Testing
The Vagrantfile provides a complete test environment with:
Current hub results