-
Notifications
You must be signed in to change notification settings - Fork 106
Description
Describe the bug
The OpenAI backend's validate() method fails when connecting to OpenAI-compatible API endpoints that require authentication on their health check endpoints. The validation request to the /health endpoint does not include the Authorization header, causing a 401 Unauthorized error even when a valid API key is provided.
Expected behavior
When using --backend-type openai_http with a valid API key, the backend validation should successfully authenticate against the remote server's health endpoint.
Actual behavior
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://my-server/health'
RuntimeError: Backend validation request failed. Could not connect to the server or validate the backend
Environment
Include all relevant environment information:
guidellm version: 0.4.0
Python 3.12.12
To Reproduce
Exact steps to reproduce the behavior:
- Set environment variables for the key and base url (GUIDELLM__OPENAI__API_KEY, BASE_URL)
- Run the benchmark
guidellm benchmark run \
--target "$BASE_URL" \
--backend-type openai_http \
--model "model" \
--rate-type constant --rate 1 \
--max-requests 10 \
--data 'prompt_tokens=256,output_tokens=128'
Additional Context
Refer to #491