support partial anonymous authentication in the impersonation proxy

[cfryanr]

The impersonation proxy is how the Pinniped Concierge provides authentication on clusters which the Kubernetes signing keys are not available. These are usually could provider distributions such as GKE, EKS, and AKS. The impersonation proxy is not used at all on clusters where the signing key is available, such as a Kind cluster, TKG cluster, or VKS cluster. You can check if the impersonation proxy is being used on your cluster by looking at the status of the CredentialIssuer resource.

One feature of the impersonation proxy is that it tries to mimic the cluster's Kubernetes API server's anonymous authentication behavior. If the cluster allows anonymous authentication, then the impersonation proxy also allowed anonymous authentication. (Of course, it still enforces authorization (RBAC) checks for every API.) If the cluster did not, then the impersonation proxy also did not.

Previously, anonymous authentication in Kubernetes was either on or off for the entire API surface. The impersonation proxy would auto-detect this by making an anonymous request to the k8s API server's /healthz endpoint. If the response was Unauthorized, then this indicated that anonymous authentication was disabled. If the response was Forbidden, then this indicated that anonymous authentication was allowed and that the request was rejected due to RBAC configuration. If the response was success, then this also indicated that anonymous authentication was allowed. Based on this heuristic, the impersonation proxy would either allow or disallow anonymous authentication for the whole API surface.

However, more recent versions of Kubernetes have configuration options to allow anonymous authentication to be partially enabled for only selected endpoints. The heuristic described above cannot determine these configurations. The most common configurations now seem to be:

• Anonymous authentication is disabled for all endpoints, such as in AKS clusters.
• Anonymous authentication is enabled for all endpoints, such as in EKS clusters before version 1.32 and GKE clusters before 1.35.
• Anonymous authentication is enabled for the health endpoints /healthz, /readyz, and /livez, but is disabled for all other endpoints, such as on EKS clusters 1.32 and newer and by default on GKE clusters 1.35 and newer. However, note that on newer GKE clusters there is an option that allows the user to enable anonymous authentication for all APIs at cluster creation time.

This implies that the impersonation proxy is no longer accurately mimicking the cluster's anonymous authentication configuration in all cases. When the /healthz endpoint allows anonymous authentication, the impersonation proxy incorrectly assumes that the whole API surface should allow anonymous authentication. Of course, authorization checks are still performed for every API call, including anonymous calls. But since the goal is to accurately mimic the behavior of the API server, the impersonation proxy needs a behavior change.

This PR updates that heuristic to instead make two probe requests. The /healthz probe is still made as described above, but its response is only interpreted to be representative of the three health endpoints. The /api/v1/nodes API is also called, and its response is interpreted to be representative of all other API endpoints. Then the impersonation proxy allows anonymous requests to the three health check endpoints only if the cluster allowed the anonymous request to the /healthz endpoint. And the impersonation proxy allows anonymous requests to the rest of the API's endpoints only if the cluster allowed the anonymous request to the /api/v1/nodes endpoint.

This new heuristic cannot detect all possible configurations of anonymous authentication, but can capture the three most common configurations as described above. There is no practical way for Pinniped to auto-detect the exact configuration of which endpoints allow anonymous authentication, so the only alternative would be to allow more configuration of Pinniped's impersonation proxy. Adding more configuration is possible, but puts more burden on the user to configure it correctly. For now, automatically supporting the three most common configurations seems like a sufficient improvement, but the maintainers are always open to feedback from the community.

Release note:

The heuristic used by the impersonation proxy to decide if the cluster supports anonymous
authentication, and therefore decides whether the impersonation proxy should allow
anonymous authentication on k8s API requests, was updated to better support recent
AKS and GKE clusters which allow anonymous authentication only at the health endpoints.

[netlify]

✅ Deploy Preview for pinniped-dev canceled.

Name	Link
🔨 Latest commit	736e247
🔍 Latest deploy log	https://app.netlify.com/projects/pinniped-dev/deploys/69780111e9bf900007214a90