feat(runs): serve external OAuth2 authorization-server metadata by pingsutw · Pull Request #7507 · flyteorg/flyte

pingsutw · 2026-06-11T22:12:50Z

Tracking issue

Related to #6998

Why are the changes needed?

The v2 runs service exposes flyteidl2.auth.AuthMetadataService, but
GetOAuth2Metadata is an unimplemented stub (returns Unimplemented/501), and
there is no /.well-known/oauth-authorization-server HTTP route. As a result,
OAuth2/OIDC discovery clients (flyte-sdk) and upstream proxies cannot
learn which authorization server to use from a v2 deployment.

This is needed when a deployment fronts its API with an external identity
provider (e.g. Okta) — for example when an ALB validates Okta-issued JWTs in
front of the API. Clients that discover auth at the deployment host must be
pointed at the external IdP so they obtain tokens the upstream actually accepts.

What changes were proposed in this pull request?

Implement OAuth2 authorization-server metadata in the runs service by proxying
an external authorization server's metadata document:

runs/service/auth_metadata_service.go — implement GetOAuth2Metadata:
when an external auth server is configured, fetch its
.well-known/oauth-authorization-server (with retry) and return it. Unknown
fields are discarded (protojson with DiscardUnknown) so real servers like
Okta — which return many extra metadata fields — parse cleanly. When not
configured, it returns Unimplemented. Also adds OAuth2MetadataHTTPHandler,
which serves the document at the RFC 8414 well-known path as proto3 JSON.
runs/setup.go — pass the new config into NewAuthMetadataService and
mount GET /.well-known/oauth-authorization-server on the mux.
runs/config/config.go — add AuthMetadataConfig
(externalAuthServerBaseUrl, externalMetadataUrl, retryAttempts,
retryDelay) under the runs config section.

The external-fetch/unmarshal/retry logic is adapted from #6998; this PR ports
just the metadata-serving slice (no token issuance / cookie / middleware stack).

How was this patch tested?

Unit tests in runs/service/auth_metadata_service_test.go:

not-configured → Unimplemented (RPC) / 501 (HTTP)
external proxy returns the upstream document, including snake_case + extra
fields (Okta-shaped) via DiscardUnknown
custom metadata path resolution
upstream 5xx → Unavailable
HTTP handler returns 200 with camelCase proto3 JSON

go build ./runs/..., go vet, and gofmt clean.

Labels

added

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Implement GetOAuth2Metadata in the runs AuthMetadataService by proxying an external authorization server's (e.g. Okta) metadata document, and mount it at the RFC 8414 /.well-known/oauth-authorization-server path. Adds runs config (authMetadata.externalAuthServerBaseUrl, ...) and tests. External-fetch logic adapted from #6998. Signed-off-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org>

…adata-external

Port flyteadmin's OAuth2MetadataProvider.GetPublicClientConfig (auth/authzserver/metadata_provider.go): serve the public CLI/SDK OAuth2 client settings (clientId, redirectUri, scopes, audience, authorizationMetadataKey) from runs.authMetadata config instead of the previous stub that only returned dataplaneDomain. Empty authorizationMetadataKey defaults to the standard "authorization" header so upstream JWT validators (e.g. ALB) see the token. Signed-off-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org>

Copilot

Pull request overview

This PR enables OAuth2/OIDC discovery for the v2 runs service by implementing flyteidl2.auth.AuthMetadataService.GetOAuth2Metadata and exposing the RFC 8414 well-known endpoint (/.well-known/oauth-authorization-server) that proxies an externally configured authorization server’s metadata document.

Changes:

Implements external OAuth2 authorization-server metadata proxying with retry and protojson parsing (discarding unknown fields).
Adds an HTTP handler for /.well-known/oauth-authorization-server serving proto3 JSON.
Introduces runs config for auth metadata proxying + public client settings, and adds unit tests covering the new behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
runs/setup.go	Wires new AuthMetadataService config and mounts the RFC 8414 well-known HTTP route.
runs/service/auth_metadata_service.go	Implements GetOAuth2Metadata proxying + HTTP handler and supporting helpers.
runs/service/auth_metadata_service_test.go	Adds unit tests for configured/unconfigured behavior and JSON shape handling.
runs/config/config.go	Adds `runs.authMetadata` configuration (external metadata proxy + public client settings).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Signed-off-by: Kevin Su <pingsutw@apache.org>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

…ew suggestion Signed-off-by: Kevin Su <pingsutw@apache.org>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

…tadataService Drop the service-local ExternalAuthServerConfig/PublicClientConfig mirror structs and pass cfg.AuthMetadata wholesale, removing the field-by-field copy in setup.go. Signed-off-by: Kevin Su <pingsutw@apache.org>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

+		retryDelay = time.Second
+	}
+
+	client := &http.Client{Timeout: 10 * time.Second}


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

- Reuse a single http.Client (10s timeout) instead of per-request construction - Bound the upstream metadata read to 1 MiB - Cache successfully fetched metadata for 5 minutes (the endpoint is public and the document is effectively static; avoids an outbound fetch with retries per discovery call) - Classify non-retryable upstream 4xx as Internal and propagate pre-classified connect codes instead of blanket Unavailable - Fix discovery-clients wording in setup.go Signed-off-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org>

pingsutw · 2026-06-12T20:29:11Z

Triage of the Copilot review comments as of 84ec9d46c:

Fixed in 84ec9d4

Shared http.Client (10s timeout) instead of per-request construction
Upstream metadata read bounded to 1 MiB
Successful metadata fetches cached for 5 min (public endpoint; avoids an outbound fetch + retries per discovery call)
Non-retryable upstream 4xx now classified Internal (was blanket Unavailable); pre-classified connect codes propagate

Signed-off-by: Kevin Su <pingsutw@apache.org>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

go generate ./runs/config was failing on types the pflags generator does not support, so config_flags.go had gone stale (it also predated the triggerScheduler fields). Unblock generation and regenerate: - RetryDelay: time.Duration -> flytestdlib config.Duration (idiomatic, keeps the flag) - Domains, triggerScheduler.resyncInterval, triggerScheduler.executionQps: pflag:"-" (slice-of-struct / raw duration / float64 are unsupported by the generator; config-file only) Generated flags now include runs.authMetadata.* (incl. flyteClient.*) and the previously missing triggerScheduler.* entries. Signed-off-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 3 comments.

Files not reviewed (2)

runs/config/config_flags.go: Generated file
runs/config/config_flags_test.go: Generated file

Signed-off-by: Kevin Su <pingsutw@apache.org>

…adata-external

EngHabu · 2026-06-15T20:30:15Z

 ) (*connect.Response[auth.GetPublicClientConfigResponse], error) {
+	authMetadataKey := s.cfg.AuthorizationMetadataKey
+	if authMetadataKey == "" {
+		authMetadataKey = "authorization"


Move to const.
Shouldn't it be Authorization (capitalized)?

it's the same as sdk. https://github.com/flyteorg/flyte-sdk/blob/d2623981ef88a389bea521fc689f90dd58105d68/src/flyte/remote/_client/auth/_client_config.py#L23

EngHabu · 2026-06-15T20:31:00Z

+	// Serve from cache while fresh; only successful fetches are cached.
+	s.mu.Lock()
+	if s.cachedMetadata != nil && time.Now().Before(s.cacheExpiry) {
+		cached := proto.Clone(s.cachedMetadata).(*auth.GetOAuth2MetadataResponse)
+		s.mu.Unlock()
+		return connect.NewResponse(cached), nil
+	}
+	s.mu.Unlock()


Make it an RWLock instead so multiple readers don't block

EngHabu · 2026-06-15T20:32:41Z

+	// so OAuth2/OIDC discovery clients (flyte-sdk) can find it. When
+	// runs.authMetadata.externalAuthServerBaseUrl is set, this proxies the
+	// external IdP's (e.g. Okta) metadata document.
+	sc.Mux.Handle("/.well-known/oauth-authorization-server", service.OAuth2MetadataHTTPHandler(authMetadataSvc))


Do we also need to serve /.well-known/openid-configuration? maybe not

I don't use that so far, I'll add it later if needed

popojk · 2026-06-16T01:54:52Z

+// GetPublicClientConfig returns the public (CLI/SDK) OAuth2 client settings.
 func (s *AuthMetadataService) GetPublicClientConfig(
 	ctx context.Context,
 	req *connect.Request[auth.GetPublicClientConfigRequest],


nit: Can we fix the "ctx" and "req" variable name to "_" as they are unused?

popojk · 2026-06-16T02:19:50Z

+	switch connectErr.Code() {
+	case connect.CodeUnimplemented:
+		return http.StatusNotImplemented
+	case connect.CodeInvalidArgument:


nit: Should we remove this case as we don't return connect.CodeInvalidArgument error in GetOAuth2Metadata function?

I'll keep it. we may return that in the future

Signed-off-by: Kevin Su <pingsutw@apache.org>

Copilot

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 3 comments.

Files not reviewed (2)

runs/config/config_flags.go: Generated file
runs/config/config_flags_test.go: Generated file

Signed-off-by: Kevin Su <pingsutw@apache.org>

pingsutw self-assigned this Jun 11, 2026

pingsutw marked this pull request as draft June 11, 2026 22:13

pingsutw changed the base branch from v2 to main June 11, 2026 22:13

pingsutw added 2 commits June 11, 2026 15:37

Merge branch 'main' of github.com:flyteorg/flyte into runs-oauth2-met…

835bd7b

…adata-external

Copilot AI review requested due to automatic review settings June 11, 2026 23:55

github-actions Bot added the flyte2 label Jun 11, 2026

Copilot started reviewing on behalf of pingsutw June 11, 2026 23:56 View session

Copilot AI reviewed Jun 11, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/config/config.go

Comment thread runs/service/auth_metadata_service.go

pingsutw and others added 2 commits June 12, 2026 12:41

nit

fe87589

Signed-off-by: Kevin Su <pingsutw@apache.org>

Apply suggestions from code review

06c25e3

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot AI review requested due to automatic review settings June 12, 2026 19:42

Copilot started reviewing on behalf of pingsutw June 12, 2026 19:42 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/config/config.go

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/setup.go

pingsutw marked this pull request as ready for review June 12, 2026 19:49

pingsutw and others added 2 commits June 12, 2026 12:52

fix(runs): restore AuthMetadataConfig struct declaration lost in revi…

77ba23b

…ew suggestion Signed-off-by: Kevin Su <pingsutw@apache.org>

Potential fix for pull request finding

33c56c0

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot AI review requested due to automatic review settings June 12, 2026 19:53

Copilot started reviewing on behalf of pingsutw June 12, 2026 19:54 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/service/auth_metadata_service.go Outdated

Comment thread runs/service/auth_metadata_service.go Outdated

pingsutw and others added 2 commits June 12, 2026 13:00

Potential fix for pull request finding

cd992d8

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot AI review requested due to automatic review settings June 12, 2026 20:05

Copilot started reviewing on behalf of pingsutw June 12, 2026 20:06 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/service/auth_metadata_service.go Outdated

retryDelay = time.Second

}

client := &http.Client{Timeout: 10 * time.Second}

Potential fix for pull request finding

aefcca8

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot AI review requested due to automatic review settings June 12, 2026 20:17

Potential fix for pull request finding

5b21be1

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot started reviewing on behalf of pingsutw June 12, 2026 20:17 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/setup.go

Comment thread runs/service/auth_metadata_service.go Outdated

pingsutw added this to the V2 GA milestone Jun 12, 2026

nit

9cf415f

Signed-off-by: Kevin Su <pingsutw@apache.org>

Copilot AI review requested due to automatic review settings June 12, 2026 20:36

Copilot started reviewing on behalf of pingsutw June 12, 2026 20:36 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/config/config.go

pingsutw and others added 2 commits June 12, 2026 14:14

Potential fix for pull request finding

40d582a

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com>

Copilot AI review requested due to automatic review settings June 12, 2026 21:27

Copilot started reviewing on behalf of pingsutw June 12, 2026 21:27 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/config/config_flags_test.go

Comment thread runs/config/config_flags_test.go

pingsutw added 2 commits June 12, 2026 14:34

nit

efa8428

Signed-off-by: Kevin Su <pingsutw@apache.org>

Merge branch 'main' of github.com:flyteorg/flyte into runs-oauth2-met…

ccdb2ea

…adata-external

EngHabu reviewed Jun 15, 2026

View reviewed changes

popojk reviewed Jun 16, 2026

View reviewed changes

nit

66d99f8

Signed-off-by: Kevin Su <pingsutw@apache.org>

Copilot AI review requested due to automatic review settings June 17, 2026 01:20

Copilot started reviewing on behalf of pingsutw June 17, 2026 01:21 View session

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/service/auth_metadata_service.go

Comment thread runs/config/config_flags_test.go

rwlock

4a387c1

Signed-off-by: Kevin Su <pingsutw@apache.org>

machichima approved these changes Jun 17, 2026

View reviewed changes

pingsutw merged commit d3e6958 into main Jun 17, 2026
21 checks passed

pingsutw deleted the runs-oauth2-metadata-external branch June 17, 2026 05:59

Conversation

pingsutw commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Check all the applicable boxes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pingsutw commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pingsutw commented Jun 11, 2026 •

edited

Loading

pingsutw commented Jun 12, 2026 •

edited

Loading