fix(sandbox): validate always-blocked IPs at load time, enrich denial logs, and filter un-fixable proposals (#814) (#815)

Policies with allowed_ips entries targeting loopback, link-local, or
unspecified ranges now fail at connection time instead of being silently
blocked at runtime. The shorthand log format for DENIED events includes
a [reason:...] suffix so operators can distinguish 'allowlist miss' from
'structurally un-allowable'. The mechanistic mapper skips proposals for
always-blocked destinations, preventing the infinite TUI notification
loop. The gateway validates proposed rules on approval as defense-in-depth.

- Extract shared IP helpers (is_always_blocked_ip, is_always_blocked_net,
  is_internal_ip) to openshell_core::net
- Reject always-blocked entries in parse_allowed_ips with hard error
- Skip implicit allowed_ips synthesis for always-blocked literal IP hosts
- Add status_detail to HttpActivityBuilder for denial reason propagation
- Enrich NET and HTTP shorthand with [reason:...] for DENIED events
- Add engine: tag to HTTP shorthand (consistency with NET shorthand)
- Filter always-blocked proposals in mechanistic mapper generate_proposals
- Add validate_rule_not_always_blocked server-side defense-in-depth
- Update architecture docs, published docs, and E2E test assertions

Author: johntmyers

Cargo.lock

@@ -53,16 +53,24 @@ The `mechanistic_mapper` module (`crates/openshell-sandbox/src/mechanistic_mappe
 1. Groups denial summaries by `(host, port, binary)` — one proposal per unique triple
 2. For each group, generates a `NetworkPolicyRule` allowing that endpoint for that binary
 3. Generates idempotent rule names via `generate_rule_name(host, port)` producing deterministic names like `allow_httpbin_org_443` — DB-level dedup handles uniqueness, no collision checking needed
-4. Resolves each host via DNS; if any resolved IP is private (RFC 1918, loopback, link-local), populates `allowed_ips` in the proposed endpoint for the SSRF override
-5. Computes confidence scores based on:
+4. **Filters always-blocked destinations.** Before DNS resolution, `is_always_blocked_destination(host)` checks if the host is a literal always-blocked IP (loopback, link-local, unspecified) or the hostname `localhost`. If so, the proposal is skipped with an info log and `continue`. This prevents an infinite TUI notification loop: the proxy denies these destinations regardless of policy, so they re-trigger denials every flush cycle without any possible fix. The helper lives in the mapper module and delegates to `openshell_core::net::is_always_blocked_ip` for literal IP addresses.
+5. Resolves each host via DNS; if any resolved IP is private (RFC 1918, loopback, link-local), populates `allowed_ips` in the proposed endpoint for the SSRF override. The `resolve_allowed_ips_if_private` function filters out always-blocked IPs from the resolved address list before populating `allowed_ips` — only RFC 1918/ULA addresses survive. If *all* resolved IPs are always-blocked (e.g., a host that resolves solely to `127.0.0.1`), the function returns an empty vec.
+6. Computes confidence scores based on:
    - Denial count (higher count = higher confidence)
    - Port recognition (well-known ports like 443, 5432 get a boost)
    - SSRF origin (SSRF denials get lower confidence)
-6. Generates security notes for private IPs, database ports, and ephemeral port ranges
-7. If L7 request samples are present, generates specific L7 rules (method + path) with `protocol: rest` (TLS termination is automatic — no `tls` field needed). Plumbed but not yet fed data — see issue #205.
+7. Generates security notes for private IPs, database ports, and ephemeral port ranges
+8. If L7 request samples are present, generates specific L7 rules (method + path) with `protocol: rest` (TLS termination is automatic — no `tls` field needed). Plumbed but not yet fed data — see issue #205.
 
 The mapper runs in `flush_proposals_to_gateway` after the aggregator drains. It produces `PolicyChunk` protos that are sent alongside the raw `DenialSummary` protos to the gateway.
 
+#### Shared IP Classification Helpers
+
+IP classification functions (`is_always_blocked_ip`, `is_always_blocked_net`, `is_internal_ip`) live in `openshell_core::net` (`crates/openshell-core/src/net.rs`). They are shared across the sandbox proxy (runtime SSRF enforcement), the mechanistic mapper (proposal filtering), and the gateway server (defense-in-depth validation on approval). The distinction between the two tiers:
+
+- **Always-blocked** (`is_always_blocked_ip`): loopback (`127.0.0.0/8`), link-local (`169.254.0.0/16`, `fe80::/10`), unspecified (`0.0.0.0`, `::`), and IPv4-mapped IPv6 equivalents. These are blocked unconditionally — no policy can override them.
+- **Internal** (`is_internal_ip`): a superset that adds RFC 1918 (`10/8`, `172.16/12`, `192.168/16`) and IPv6 ULA (`fc00::/7`). These are blocked by default but can be allowed via `allowed_ips` in policy rules.
+
 ### Gateway: Validate and Persist
 
 The gateway's `SubmitPolicyAnalysis` handler (`crates/openshell-server/src/grpc.rs`) is deliberately thin:
@@ -75,6 +83,16 @@ The gateway's `SubmitPolicyAnalysis` handler (`crates/openshell-server/src/grpc.
 
 The gateway does not store denial summaries (they are included in the request for future audit trail use but not persisted today). It does not run the mapper or any analysis.
 
+#### Always-Blocked Validation on Approval
+
+`merge_chunk_into_policy` (`crates/openshell-server/src/grpc/policy.rs`) validates proposed rules before merging them into the active policy. The `validate_rule_not_always_blocked` function runs as a defense-in-depth gate, catching rules that the sandbox mapper should have filtered but didn't (e.g., proposals from an older sandbox version):
+
+- Rejects endpoint hosts that parse as always-blocked IPs (loopback, link-local, unspecified)
+- Rejects the literal hostname `localhost` (case-insensitive, with or without trailing dot)
+- Rejects `allowed_ips` entries that parse as always-blocked networks via `is_always_blocked_net`
+
+On failure, the function returns `Status::invalid_argument` with a message explaining that the proxy will deny traffic to the destination regardless of policy. This uses the same `openshell_core::net` helpers as the sandbox-side filtering.
+
 ### Persistence
 
 Draft chunks are stored in the gateway database:
@@ -201,6 +219,20 @@ Keybindings are state-aware:
 | `OPENSHELL_DENIAL_FLUSH_INTERVAL_SECS` | `10` | How often the aggregator flushes and submits proposals |
 | `OPENSHELL_POLICY_POLL_INTERVAL_SECS` | `10` | How often the sandbox polls for policy updates |
 
+## Known Behavior
+
+### Always-Blocked Destinations
+
+Destinations classified as always-blocked (loopback, link-local, unspecified, `localhost`) are filtered at three layers:
+
+1. **Sandbox mapper** — `generate_proposals` skips them before building a `PolicyChunk`
+2. **Sandbox mapper** — `resolve_allowed_ips_if_private` strips always-blocked IPs from `allowed_ips`, returning empty if none survive
+3. **Gateway approval** — `merge_chunk_into_policy` rejects them with `INVALID_ARGUMENT`
+
+If a sandbox process repeatedly attempts connections to these addresses, the proxy denies them every time and the denial aggregator accumulates counts. The mapper discards these summaries silently rather than forwarding un-fixable proposals. Before issue #814, these proposals would reach the TUI and reappear every flush cycle (default 10 seconds) since approving them would have no effect — the proxy blocks them regardless of policy.
+
+Existing `pending` rows for always-blocked destinations that were persisted before this filtering was added remain in the database. They are inert: attempting to approve them now fails at the gateway's `validate_rule_not_always_blocked` check. They can be rejected manually via `openshell rule reject` or left to age out.
+
 ## Future Work (Issue #205)
 
 The LLM PolicyAdvisor agent will run sandbox-side via `inference.local`:

architecture/policy-advisor.md

@@ -803,6 +803,8 @@ Every CONNECT request to a non-`inference.local` target produces an `info!()` lo
 
 After OPA allows a connection, the proxy resolves DNS and rejects any host that resolves to an internal IP address (loopback, RFC 1918 private, link-local, or IPv4-mapped IPv6 equivalents). This defense-in-depth measure prevents SSRF attacks where an allowed hostname is pointed at internal infrastructure. The check is implemented by `resolve_and_reject_internal()` which calls `tokio::net::lookup_host()` and validates every resolved address via `is_internal_ip()`. If any resolved IP is internal, the connection receives a `403 Forbidden` response and a warning is logged. See [SSRF Protection](security-policy.md#ssrf-protection-internal-ip-rejection) for the full list of blocked ranges.
 
+IP classification helpers (`is_always_blocked_ip`, `is_always_blocked_net`, `is_internal_ip`) are shared from `openshell_core::net`. The `parse_allowed_ips` function rejects entries overlapping always-blocked ranges (loopback, link-local, unspecified) at load time with a hard error, and `implicit_allowed_ips_for_ip_host` skips synthesis for always-blocked literal IP hosts. The mechanistic mapper filters proposals for always-blocked destinations to prevent infinite TUI notification loops.
+
 ### Inference interception
 
 When a CONNECT target is `inference.local`, the proxy TLS-terminates the client side and inspects the HTTP traffic to detect inference API calls. Matched requests are executed locally via the `openshell-router` crate. The function `handle_inference_interception()` implements this path and returns an `InferenceOutcome`: