docs(hail): troubleshoot dev deploy "Unauthorized" failures#182
Open
violetbrina wants to merge 3 commits into
Open
docs(hail): troubleshoot dev deploy "Unauthorized" failures#182violetbrina wants to merge 3 commits into
violetbrina wants to merge 3 commits into
Conversation
Add a troubleshooting subsection to the Developer deploy section explaining that a 401 "You must be logged in to the server (Unauthorized)" in dev deploy jobs (first seen in create_test_database_server_config, recurring in create_ssl_config_hail_root) is caused by GKE invalidating the legacy admin-token service-account token. Includes local read-only diagnosis steps and the fix (delete + recreate the admin-token secret). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the duplicated error blocks, merge the root-cause explanation into a single paragraph, and drop the redundant read-only diagnostic checks so the section leads straight to the confirm-and-fix steps. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MD040 requires fenced code blocks to declare a language. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a Troubleshooting: dev deploy fails with
Unauthorizedsubsection to the Developer deploy section ofhail.md.Why
Dev deploys can fail with:
This surfaces first in the
create_test_database_server_configjob (the earliest step to runkubectl) and recurs in laterkubectlsteps such ascreate_ssl_config_hail_root(error: failed to create secret Unauthorized). The failing step is a red herring — the real cause is that GKE has invalidated the legacyadmin-tokenservice-account token in the developer's namespace, so everykubectlcall in the deploy gets a 401. Re-running doesn't help becausedefault_ns'skubectl applywon't regenerate an existing secret's token.Contents of the new section
LegacyServiceAccountTokenCleanUp)kubectl auth whoami --token=...against the stored token)admin-tokensecret so the token controller mints a fresh tokenVerified end-to-end against a live namespace: the diagnosis reproduces the 401, and the fix restores
system:serviceaccount:<ns>:adminauth.🤖 Generated with Claude Code