Feature request: Consider exposing extra API headers as argument

[parmsam-pfizer]

Similar to ellmer::chat_openai() in R, please consider adding an api_headers argument that allows for arbitrary extra headers appended to every chat API call. This would make it easier to integrate internal APIs that use the OpenAI format and a Bearer Token approach in the API header with chatlas in Python.

[cpsievert]

Hi @parmsam-pfizer, this should be possible through HTTP parameters, which gives you a ton of control over the HTTP client configuration (i.e., the OpenAI() HTTP client).

I believe the equivalent of ellmer's api_headers would be default_headers:

from chatlas import ChatOpenAI

chat = ChatOpenAI(
    system_prompt="You are a helpful assistant.",
    kwargs={
        "default_headers": {
            "Authorization": "Bearer YOUR_API_KEY"
        }
    }
)

[parmsam-pfizer]

Oh, perfect. Thanks for sharing that example! I missed that. I'll try it out.

[parmsam-pfizer]

Thank you so much! It worked! I was able to get past the API key error (OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable) by setting api_key to "".

chat = ChatOpenAI(
    system_prompt="You are a helpful assistant.",
    api_key = "",
    kwargs={
        "default_headers": {
            "Authorization": "Bearer YOUR_API_KEY"
        }
    }
)

[cpsievert]

Huh, interesting. Is the api_key thing needed if you have the latest (chatlas v0.13.2)? Seems to work ok for me without the api_key

[parmsam-pfizer]

Yes, I'm using 0.13.2.dev6+gdf9873c02 and it gives me that error if I don't use it. Did you also remove your OPENAI_API_KEY environment variable?

[cpsievert]

Ah yea, ok (I had set to an empty string, not removed it entirely) 👍

[parmsam-pfizer]

Sorry, just want to confirm one more thing. Can I update the default_headers after chat creation? Our bearer token expires after a certain time, and I want to see if I can trigger an update to still keep the chat object alive. For example, this way via the set_model_params method and the kwargs parameter:

chat = ChatOpenAI(
    system_prompt="You are a helpful assistant.",
    api_key = "",
    kwargs={
        "default_headers": {
            "Authorization": "Bearer ORIGINAL_AUTH_TOKEN"
        }
    }
)
 refreshed_header = {
    "default_headers": {
                 "Authorization": "Bearer REFRESHED_AUTH_TOKEN"
 }}
chat.set_model_params(kwargs = refreshed_header)

I did a check using chat.provider._client._custom_headers to print the custom HTTP headers and the approach above doesn't seem to work for me. I understand that I could create a new chat and use set_turns to add in the conversation history. However, I worry that would disrupt Shiny apps built around a chatlas object. It would be nice to update the headers when needed and keep a shinychat UI active. Similar issue raised for ellmer: tidyverse/ellmer#807

[cpsievert]

I think you could achieve a hack if you're motivated to do so (by doing something like chat.provider._client = OpenAI(...), but I'll likely do the equivalent of tidyverse/ellmer#799 in the coming weeks