refactor: ChatAgent #1424
refactor: ChatAgent #1424
Conversation
…into refactor/chatagent
Co-authored-by: Xiaotian Jin <jinxiaotian_sal@outlook.com>
There was a problem hiding this comment.
Thanks @WHALEEYE 's great work!
- Seems the prompt based tool calling is not support in this PR
- Some test need to be fixed.
- Currently external tools are also executed?
Other comments are minor issues, let's fix all these in this PR, we also need to update the doc and cookbook to align with our change in another PR
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def generate_tool_prompt(tool_schema_list: List[Dict[str, Any]]) -> str: |
There was a problem hiding this comment.
where is this function used?
There was a problem hiding this comment.
Not yet used, they are supposed to be used by models that does not natively support tool callings, but we havent added support for them yet
| # Empty -> use no tools | ||
| elif not tools: | ||
| tools = None | ||
| return self._run(messages, response_format, tools) |
There was a problem hiding this comment.
for models doesn't support native tool calling or structured output, these 2 parameters would still be passed to _run, causing error, we need to check and add logic to handle this in model level further
There was a problem hiding this comment.
Since we are going to implement _run() for every model backend so we will handle these inside each model backend. For now if our model backend doesn't support these two options, they will be simply ignored.
| return None | ||
|
|
||
|
|
||
| def extract_tool_call( |
There was a problem hiding this comment.
Why the argument is self and do we use it anywhere?
There was a problem hiding this comment.
I moved this from inside ChatAgent and forgot to remove this. I will update this one in the follow up PRs
|
Thanks @WHALEEYE! It is a bit cofusing for me when is |
| `ChatCompletion` in the non-stream mode, or | ||
| `AsyncStream[ChatCompletionChunk]` in the stream mode. | ||
| """ | ||
| response = await self._async_client.chat.completions.create( |
There was a problem hiding this comment.
Do we need here?
request_config = self._prepare_request(
messages, response_format, tools
)
Description
Thoroughly refactored
ChatAgent.Motivation and Context
ChatAgentThis will close #1334.
Important Changes
For the convenience of the reviewers, the important changes to the code are listed here:
ainstead of being postfixed withasync, for example,step_async->astep, to make it more concise._utils.pyand_types.py.ModelResponseto represent the responsefrom models, which will only be used in internalChatAgent.Nonevalue in configs afteras_dict().