Skip to content

[FEA]: Allow fully qualified reranking endpoint URLs #2229

@nv-alaiacano

Description

@nv-alaiacano

Is this a new feature, an improvement, or a change to existing functionality?

Change

How would you describe the priority of this feature request

Currently preventing usage

Please provide a clear description of problem this feature solves

We have an internally hosted ranking endpoint that ends with /v1/rerank. The way that NRL parses the RERANKER_INVOKE_URL is with the following logic:

When ``rerank_invoke_url`` is set the actor/function calls a vLLM (>=0.14) or NIM
server that exposes the NIM ranking REST API. The helper accepts
either a fully qualified ``.../reranking`` URL or a base URL and appends
``/v1/ranking`` automatically::

This means that we end up in the following scenarios:

If RERANKER_INVOKE_URL=https://endpoint.com/v1/rerank, then the library calls https://endpoint.com/v1/rerank/v1/ranking

If RERANKER_INVOKE_URL=https://endpoint.com, then the library calls https://endpoint.com/v1/ranking

Describe the feature, and optionally a solution or implementation and any alternatives

It would be nice to be able to provide a custom URL that does not get manipulated by the library. We cannot currently use this feature of NRL because of this logic.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions