Nvidia Ranker URL issue

[2-ulfhednar-2]

Describe the bug
The URL finalization for the NvidiaRanker in the NIM-Backend function rank is broken.
It doesnt fully create the link.

To Reproduce
Just run the example code provided in the ranker.py for nvidia

Describe your environment (please complete the following information):

• OS: ubuntu 22.04
• Haystack version: Haystack 2.10.0
• Integration version: Nvidia-Haystack 0.1.6

To Fix:
Change lines 173/174 in the nim_backend.py to properly complete the URL
from:
def rank(self, query_text: str, document_texts: List[str]) -> List[Dict[str, Any]]:
url = self.api_url

To:
def rank(self, query_text: str, document_texts: List[str]) -> List[Dict[str, Any]]:
url = f"{self.api_url}/ranking"

right now the ranker.py takes the provided URL adds v1 to it and then passes it to the def rank. To properly access the endpoint usually /ranking is needed.

[julian-risch]

Hello @2-ulfhednar-2 thanks for opening this issue. I wasn't able to reproduce it with the latest version of the integration. Here is the code I used:

from haystack_integrations.components.rankers.nvidia import NvidiaRanker
from haystack import Document
from haystack.utils import Secret

ranker = NvidiaRanker(
    model="nvidia/nv-rerankqa-mistral-4b-v3",
    api_key=Secret.from_env_var("NVIDIA_API_KEY"),
)
ranker.warm_up()

query = "What is the capital of Germany?"
documents = [
    Document(content="Berlin is the capital of Germany."),
    Document(content="The capital of Germany is Berlin."),
    Document(content="Germany's capital is Berlin."),
]

result = ranker.run(query, documents, top_k=2)
print(result["documents"])

When I run this code after installing nvidia-haystack 0.1.6 with pip install nvidia-haystack, I get the expected response
[Document(id=ded27a743b8980b58b13311c6f640dfbdccf0b2e523055c5845ee772ef473869, content: 'The capital of Germany is Berlin.', score: 27.171875), Document(id=f780ec15ed0011c1af18da28519f9205d5144de3ee84d7c613cf6bc0ad3f5da9, content: 'Germany's capital is Berlin.', score: 24.46875)]

When I run this code after installing nvidia-haystack with pip install -e . on the branch that includes your suggested fix nvidia-ranker-api-url (draft PR here), I get the an error because the URL that is being used is then https://ai.api.nvidia.com/v1/retrieval/nvidia/nv-rerankqa-mistral-4b-v3/reranking/ranking.

Error when calling NIM ranking endpoint: Error - 404 page not found

Traceback (most recent call last):
  File "/Users/julian/deepset/dev/haystack-core-integrations/integrations/nvidia/src/haystack_integrations/utils/nvidia/nim_backend.py", line 187, in rank
    res.raise_for_status()
  File "/Users/julian/miniforge3/envs/py310-nvidia/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://ai.api.nvidia.com/v1/retrieval/nvidia/nv-rerankqa-mistral-4b-v3/reranking/ranking

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "haystack-core-integrations/integrations/nvidia/src/haystack_integrations/components/rankers/nvidia/ranker.py", line 244, in run
    sorted_indexes_and_scores = self.backend.rank(query_text=query_text, document_texts=document_texts)
  File "haystack-core-integrations/integrations/nvidia/src/haystack_integrations/utils/nvidia/nim_backend.py", line 191, in rank
    raise ValueError(msg) from e
ValueError: Failed to rank endpoint: Error - 404 page not found

What version of the integration are you using?

[julian-risch]

The issue occurs with self-hosted models only and that's why my code examples above couldn't reproduce the issue.