diff --git a/docs/decisions/17685-network-handler-behavior.md b/docs/decisions/17685-network-handler-behavior.md new file mode 100644 index 0000000000000..8bf190c784cf7 --- /dev/null +++ b/docs/decisions/17685-network-handler-behavior.md @@ -0,0 +1,220 @@ +# 17685. The user directly controls network handler behavior and event disposition + +- Status: Proposed +- Discussion: [#17685](https://github.com/SeleniumHQ/selenium/pull/17685) + +## Context + +A user can register more than one handler for the same network phase, and matching handlers +can disagree: the company framework always adds a test header, the local suite stubs all calls +to a domain, and one test aborts a single call. Selenium must resolve that and provide a single +response to the browser in a consistent and obvious way. + +The bindings diverge today: each grew its handler API independently, so dispatch order, +multi-handler resolution, error handling, and what an event exposes are all inconsistent. + +| Binding | Current behavior | +|------------|------------------| +| Java | Only the first matching handler runs; disposition is always continue; a throwing handler propagates and leaves the request blocked; return-value driven; no response handler or managed body collection. | +| Python | An explicit `continue` in a handler fires immediately and wins; otherwise staged outcomes reconcile by `fail` > `provide_response` > `continue`; response handlers have no `fail`; dispatch is FIFO; a throwing handler's staged mutations are still sent; only the mutated event is visible; body is not collected behind the handler. | +| Ruby | Handlers run in parallel threads, so multi-handler disposition races; exceptions are logged; dispatch is FIFO with no default-continue; only the mutated event is visible; body collection is user-managed. | +| .NET | No request or response handler API. | +| JavaScript | No request or response handler API. | + +## Decision + +A network handler is registered either to **observe** or to **intercept**, and the event object it +receives enforces the difference (behavior 1). For intercept handlers, Selenium consults them one at +a time and lets each dispose of the event as it runs: the first handler to specify a disposition +(fail, respond, or submit) resolves the event and stops the chain, while a handler that only stages +mutations passes the event to the next one. Selenium does not gather every handler's outcome and +reconcile it at the end. The behaviors below apply to request and response handlers, but not +authentication handlers, since authentication should not use a callable. + +Note that there are multiple ways to implement these behaviors; the code examples are one +option in one language, and represent user-facing code. + +1. **A handler is added to observe or to intercept, and the event object it receives enforces + which.** There is one method to add handlers, and which mode the handler operates under is decided + at creation; intercepting is the default and observing is opt-in. An observing handler receives a + read-only event object: it can read the event but has no methods to mutate or settle it, and it + does not pause + network traffic. An intercepting handler receives a mutable event object: it can stage changes and + settle the event, and network traffic is paused until handling resolves it. Because the object's + type carries the difference — a read-only object simply has no mutate-or-settle methods — nothing + has to introspect the callable to tell the modes apart. + * How a binding lets the user pick the mode — a keyword argument, an options object, an overload — + is its own idiom; what is fixed is that it is the same method, not a separate observe one. + +```ruby +# Same method, two modes; the event object handed to the block differs +network.add_request_handler { |r| r.fail if something } # intercept: mutable, blocking +network.add_request_handler(observe: true) { |r| log(r.url) } # observe: read-only, non-blocking + +# An observed event object has no mutation methods, so trying to mutate raises +network.add_request_handler(observe: true) { |r| r.fail } # raises: observed events are read-only +``` + +2. **An intercept handler can specify event disposition.** Allow the user to specify how the event is disposed of + by acting on the object provided to the callable. + * Playwright only intercepts requests and requires an explicit disposition: continue (stop + processing other handlers), fulfill (respond with a mock), abort (respond with an error), + fallback (process other handlers, if any). + * Selenium supports: + * Request: `fail` (Playwright's `abort`, BiDi's `FailRequest`), `respond` (Playwright's `fulfill`, BiDi's `ProvideResponse`), and `submit` (Playwright's `continue`, BiDi's `ContinueRequest`). + * Response: `fail` (BiDi's `FailRequest`), and `submit`: note that since we don't need to prevent a round trip from a request, whether this is a BiDi `ContinueResponse` or `ProvideResponse` can be an implementation detail based on whether a replacement body value is provided. + +```ruby +# Specifics of parameters and names can match spec details +network.add_request_handler { |r| r.fail if something } +network.add_request_handler { |r| r.respond(content: mocked_response) if something } +network.add_request_handler { |r| r.add_header("X-Test", true) && r.submit if something } + +network.add_response_handler { |r| r.fail if something } +network.add_response_handler { |r| r.submit(content: mocked_response) if something } +network.add_response_handler { |r| r.add_header("X-Test", true) && r.submit if something } +``` + +3. **Default disposition is to process other handlers.** If a handler does not specify the + disposition, the original event and any staged mutations pass to the next handler. If no + handler ever specifies one, the event proceeds with the staged mutations. + * In Playwright request interception there is no default; the user must specify fallback if + that is the intent. + +```ruby +# All of these stage a change and pass to the next handler +network.add_request_handler { |r| r.add_header("X-Test", true) } +network.add_request_handler { |r| r.remove_header("upgrade-insecure-requests") } +network.add_request_handler { |r| r.content = r.request.content.gsub("a", "b") } +``` + +4. **Later-registered handlers are consulted first.** Registering an additional handler can + mutate the state used by previously registered ones. + * Matches Playwright's Last-In-First-Out (LIFO) behavior. + * Allows users to locally override handlers set by a shared library or suite. + * The alternative is being stuck with the top-level behavior everywhere, or not being able to + set top-level defaults at all. + +```ruby +# Header will be there because removal is attempted before it is added +network.add_request_handler { |r| r.add_header("X-Test", true) } +network.add_request_handler { |r| r.remove_header("X-Test") } +``` + +5. **An uncaught exception discards the handler's staged changes; it propagates for an intercept + handler and is logged for an observe handler.** Either way the event keeps flowing as if that + handler had not run, so one broken handler cannot corrupt live traffic or stall the page. The + difference is visibility: an intercept handler expresses the test's intent, so a failure in it + surfaces to the user; an observe handler is passive monitoring, so an incidental failure (a + third-party beacon, an analytics call) is logged and never fails the test. + * In Playwright every uncaught exception ends the session; routing by mode keeps interception + strict without coupling the test to errors from the open internet it never meant to assert on. + +```ruby +# Intercept: the error surfaces; the header addition from the other handler still applies +network.add_request_handler { |r| r.add_header("X-Test", true) } +network.add_request_handler { |r| raise Exception } + +# Observe: the error is logged, the test is unaffected +network.add_request_handler(observe: true) { |r| raise Exception } +``` + +6. **Return values within the callables are ignored.** No meaning will ever be applied to + anything a user explicitly or implicitly returns within the callable. + * Playwright also does this, as does Selenium's current Python implementation. + +```ruby +# Ruby: this implicit return value is ignored +network.add_request_handler { |r| r.add_header("X-Test", true); "this value is ignored" } +``` + +```python +# Python: this explicit return value is ignored +def handler(r): + r.add_header("X-Test", True) + return "this value is ignored" + +driver.network.add_request_handler(handler) +``` + +7. **A handler has access to the original event value.** It may see the changes staged by + handlers already executed, but can also read the unmodified event value. + * Even when intercepting and mutating, a conditional can be evaluated against the original value + rather than the version a prior handler changed. + * Playwright uses a completely separate mechanism to differentiate observation from mutation, + so it does not need to address this. + +```ruby +# Nothing gets raised +network.add_request_handler { |r| raise unless r.headers.include?("X-Test") } +network.add_request_handler { |r| raise if r.request.headers.include?("X-Test") } +network.add_request_handler { |r| r.add_header("X-Test", true) } +``` + +8. **Body data is collected only when the handler opts in at registration.** A body is not + available by default; the handler declares that it needs the body when it is registered — not + from inside the callback, since the collector must be in place before the event — and Selenium + then owns the collector's lifecycle, size cap, and browser-support quirks. The body is readable + on the event inside that handler. + * The user never calls `addDataCollector` / `getData` or tears a collector down. + * There is no way to collect or read body data outside a handler; collection happens only + through `add_x_handler`. + * Playwright exposes bodies through its response object without a user-managed collector; + Selenium does the same, owning the collector behind the handler. + +```ruby +# Declare body collection at registration; the body is then available on the event +network.add_response_handler(collect_body: true) { |r| log(r.body) } +``` + +## Considered options + +- **Modes (behavior 1).** + - We could give observation its own method, separate from interception, but both modes share the + same registration shape (URL patterns, body opt-in, the removal handle), so a separate method + duplicates the whole surface; and the read-only contract cannot be enforced by the method + anyway — we will not introspect the callable — so the event object's type carries it and the + method stays the same. + - We could make every handler an interception and add read-only observation later, but routing + observation through interception pauses traffic and perturbs what it records (cache behavior, + timing), so observation-only is worth providing as its own option now, not deferred. +- **Reconciliation (behaviors 2 & 3).** + - We could run every handler and reconcile by a fixed priority (fail > stub > continue). But + this prevents a user from exercising the `continueRequest` behavior from a specific + handler, so all mutations from all handlers would be applied by default. + - We could run every handler but have `continueRequest` override failures and stubs (current + Python behavior), but it is not obvious why that command should have precedence. +- **Verb names (behavior 2).** + - We could follow Playwright's (abort / fulfill / continue / fallback). + - We could follow BiDi's more explicitly (failRequest / provideResponse / continueRequest / continueResponse). +- **Explicit disposition (3).** + - We could require the user to specify fallback explicitly like Playwright does. +- **Ordering (behavior 4).** + - We could run in order of handler registration, but this prevents users from overriding global settings locally. +- **Failure (5).** + - We could propagate the uncaught exception to end the session like Playwright does, but this + puts a larger burden on the users to manage network issues and bugs that aren't part of a + test. This is likely to be a bigger issue if we intercept every event by default. + - We could log every handler's exception regardless of mode, but an intercept handler's failure + is the test's own bug and should not be swallowed. +- **Return values (6).** + - We could have the return values set state for the event or handler rather than storing it in + the event wrapper object we provide, but this is not as straightforward in all languages + and adds additional complications. +- **Original access (7).** + - We could only expose the modified event, or only the original, instead of both. + - We could provide a separate observation API like Playwright, but even when mutating it could + make sense to evaluate a conditional from the original event rather than the mutated one. +- **Data collection (8).** + - We could collect every body always, but bodies are large and most handlers never read them, + so collection is opt-in at registration instead. + - We could require the user to manage the data collector directly through the low-level + commands, but collection has no meaning outside a handler and would push lifecycle, + size-cap, and browser-support bookkeeping onto the user. + +## Consequences + +- Client code can override shared handlers locally and resolve a request its own way, a broken + handler stays contained, and the original event remains readable. +- This changes handler behavior that several bindings already ship, so it is not backwards + compatible.