WIP: Document hash-based routing

hash-based-routing.html.md.erb

@@ -0,0 +1,49 @@
+---
+title: Hash-Based Routing
+owner: CF for VMs Networking
+---
+
+## <a id="purpose"></a> Purpose
+
+Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value,  e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance.
+
+## <a id="key-features"></a> Key Features
+
+- **Configurable via Per-Route Options**: Easily set up Hash-Based Routing using application route options
+- **Consistent Hashing**: Implements the Maglev consistent hashing algorithm as outlined in the paper "Maglev: A Fast and Reliable Software Network Load Balancer" (https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf)
+- **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing changes to hash assignments when application instances are added or removed
+- **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property
+- **Session Affinity Precedence**: Prioritizes session affinity, or sticky sessions, over hash-based routing when available
+- **Handling imbalanced loads**: Implements detection and mitigation of higher request loads on single instances due to different usage patterns for specific hashes. This avoids overloading a single instance while keeping instances for a particular hash at a minimum.
+- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` will be ignored when using Hash-Based Routing
+
+Hash-Based Routing implements a clear precedence hierarchy:
+
+1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available
+2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance
+3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for hash-based routing is absent in the request
+
+
+## <a id="hash-based-vs-session-affinity"></a> Hash-Based Routing vs. Session Affinity
+
+Using Session Affinity, requests from a particular client are always routed to the same app instance. This allows apps to store session data specific to a user session (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation).
+
+While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses.
+
+In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below).
+
+This makes it especially suitable for applications requiring high scalability and performance, such as microservices architectures or multi-tenant applications, when dealing with limited or memory-intensive resources in backend services.
+
+
+## <a id="handling-imbalance-loads"></a> Handling imbalanced loads
+
+The implementation of the hash-based routing includes mechanisms to detect imbalanced loads across application instances. Imbalanced loads can occur when certain hashes receive more traffic, such as when a particular tenant generates a large number of requests, resulting in heavier use of their mapped instances compared to others. Moreover, multiple high-traffic targets could be assigned to the same instance.
+
+To prevent overloading specific instances while others remain under-utilized, the acceptable threshold for load imbalance can be configured using the `hash_balance_factor` property.
+This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all application instances, measured by the number of in-flight requests. For instance, with a balance factor of 1.25, no single instance should exceed 125% of the average number of in-flight requests across all instances managed by the current router. Consequently, the router redirected the next requests to other predetermined instances that are not overloaded.
+
+This ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded.
+
+## <a id="retries-in-hash-based"></a> Retries in Hash-Based Routing
+
+For idempotent requests, the retry mechanism is supported in Hash-Based Routing. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different predetermined application instance. The next entry in the predefined Maglev lookup table determines this instance. This is the same approach used in handling imbalanced loads. This ensures that the retry mechanism adheres to the principles of Hash-Based Routing while providing resilience against transient failures.