Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional parameters for openai client #385

Merged

Conversation

touseefahmed96
Copy link
Contributor

fix #381

@aymeric-roucher
Copy link
Collaborator

@touseefahmed96 normally you shouldn't need a PR fort these parameters to work, they should work as kwargs. Tell me if it's not the case!

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aymeric-roucher I was just looking at the base code because I thought the same.

And I think the kwargs are not passed to the client instantiation, but to the client call only.

  • Client instantiation is not passed the self.kwargs:

    self.client = openai.OpenAI(
    base_url=api_base,
    api_key=api_key,
    )

  • The self.kwargs are only passed to the client call (through the completion_kwargs):

    completion_kwargs = self._prepare_completion_kwargs(
    messages=messages,
    stop_sequences=stop_sequences,
    grammar=grammar,
    tools_to_call_from=tools_to_call_from,
    model=self.model_id,
    custom_role_conversions=self.custom_role_conversions,
    convert_images_to_image_urls=True,
    **kwargs,
    )
    response = self.client.chat.completions.create(**completion_kwargs)

@touseefahmed96
Copy link
Contributor Author

touseefahmed96 commented Jan 28, 2025

So, @albertvillanova in order to use the organization or project parameter isn't it necessary to pass kwargs to client ?
like:

 self.client = openai.OpenAI( 
     base_url=api_base, 
     api_key=api_key, 
    **kwargs,
 ) 

@aymeric-roucher
Copy link
Collaborator

Thanks for looking into it @albertvillanova !
Then indeed this PR makes sense!
Maybe if going forward we see many additional (and unpredictable) args needed for model initialization rather than model completion, we could add something like an initialization_args dict that users can use to pass init args to the underlying model? Not sure what would be best.

@albertvillanova
Copy link
Member

I agree @aymeric-roucher. If in the future we see more client args are requested by users, maybe we could define a client_kwargs parameter.

Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @touseefahmed96.

@albertvillanova albertvillanova changed the title Additional parameters for openai Additional parameters for openai client Jan 28, 2025
@albertvillanova albertvillanova merged commit 8178eec into huggingface:main Jan 28, 2025
3 checks passed
@touseefahmed96
Copy link
Contributor Author

I agree @aymeric-roucher. If in the future we see more client args are requested by users, maybe we could define a client_kwargs parameter.

I will look into it and let's see if I can find any better solution. The client_kwargs parameter makes sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"Error in generating tool call with model: 'NoneType' object is not subscriptable" when using FinalAnswerTool
3 participants