[Task]: [Bottom Up Deployment] Support "other" and "none" upstream auth types for LLM Providers in GW

[Thushani-Jayasekera]

Please select the area the issue is related to

Gateway

Please select the aspect the issue is related to

Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI)

Description

Relates to: #2686

$subject

[thivindu]

Update

Currently the upstream.auth can be empty (only set the upstream.url) or upstream.auth can be of api-key type from the gateway controller management REST APIs.
We do not support other or none as upstream auth types. -

api-platform/gateway/gateway-controller/pkg/config/llm_validator.go

Lines 478 to 491 in 467b5a9

	if upstream.Auth != nil {
	auth := upstream.Auth
	// Validate 'type'
	if auth.Type == "" {
	errors = append(errors, ValidationError{
	Field: fmt.Sprintf("%s.auth.type", fieldPrefix),
	Message: "Auth type is required",
	})
	} else if auth.Type != "api-key" {
	errors = append(errors, ValidationError{
	Field: fmt.Sprintf("%s.auth.type", fieldPrefix),
	Message: "Auth type must be 'api-key'",
	})
	}

If we try to create a LLM provider with a auth.type other than api-key we get this error :

{"message":"provider validation failed with 1 error(s): 1. spec.upstream.auth.type: Auth type must be 'api-key'","status":"error"}

We propose to support none, other, api-key as valid spec.upstream.auth.type values. If the spec.upstream.auth is not defined the type will set to none. When the spec.upstream.auth.type is not api-key the spec.upstream.auth.header and spec.upstream.auth.value fields will be neglected.
So when a LLM provider is deployed from the Platform API (CP->DP) the spec.upstream.auth block will be set accordingly based on the user provided values and defaults.
When LLM provider is created from the gateway controller management REST APIs (DP->CP) spec.upstream.auth block will be set accordingly based on the user provided values and pushed to the Platform API as is. The platform API will store the gateway runtime artifact as is and set default values to fields that are not set. For example, if spec.upstream.auth block is empty, set the type to none from the Platform API not the gateway. The AI Workspace will have handling empty value to default logic as well.

If the upstream.auth is empty from the AI Workspace it defaults to api-key auth even if the GET llm-providers/{llmProviderID} response does not contain upstream.main.auth. We are fixing this to fallback spec.upstream.auth.type to none if it is empty and show that in the AI Workspace. Only if the spec.upstream.auth.type is set to other we will show the type as other and the message No credentials are stored for this provider. Use a policy to configure authentication. in the AI Workspace.

To summarize:

• CP to DP flow
  • if the spec.upstream.auth.type is not set or none -

Gateway deployed artifact:

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: none

The AI Workspace will show Authentication in the Connection tab as None.

• if the spec.upstream.auth.type is other -

Gateway deployed artifact:

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: other

The AI Workspace will show Authentication in the Connection tab as Other and display the No credentials are stored for this provider. Use a policy to configure authentication. message.

• if the spec.upstream.auth.type is api-key -

Gateway deployed artifact:

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: api-key
      header: Authorization
      value: <api-key>

The AI Workspace will show Authentication in the Connection tab as api-key along with the Authentication header and masked credentials (existing behaviour)

• DP to CP flow
  • if the spec.upstream.auth.type is not set or none -

 upstream:
   url: https://api.openai.com/v1

or

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: none

The AI Workspace will show Authentication in the Connection tab as None.

• if the spec.upstream.auth.type is other -

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: other

The AI Workspace will show Authentication in the Connection tab as Other and display the No credentials are stored for this provider. Use a policy to configure authentication. message.

• if the spec.upstream.auth.type is api-key -

  upstream:
    url: https://api.openai.com/v1
    auth:
      type: api-key
      header: Authorization
      value: <api-key>

The AI Workspace will show Authentication in the Connection tab as api-key along with the Authentication header and masked credentials (existing behaviour)

[thivindu]

Furthermore, we will add none to the Authentication drop down in LLM provider creation form.

This will extend to LLM Proxies as well.