-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] pydantic validation errors for ChatCompletion #3637
Comments
I'm also seeing this in my PR #2798, when the input is ![]() |
@shuaills Seeing the exact same issue, any tool call will fail on qwen25 model (I'm using Qwen2.5-72B-Instruct, apparently deepseek model have the same problem). Can someone help look into this? This is blocking and I had to fallback to vLLM beacuse of this :( I appreciate any insights to this problem. I can confirm my LangGraph script is correct because the same script when using GPT4o API works, but fails when using SGLang. The sglang server errors when a toolcall result (ToolMessage) is added to the context: (flow works like following snippet output from LangGraph). It is the same flow as the issue author @xxll88 , SGLang can call tools properly, but it cannot receive the tool results. Original Query -> Model Choose Tool -> Tool Response -> Model (Fails here)
Actual OpenAI SDK sent request:
Only messages part in json: {
"messages": [
{
"content": "CALL A Search TOOL NOW, on how to cook a lobster",
"role": "user"
},
{
"content": null,
"role": "assistant",
"tool_calls": [
{
"type": "function",
"id": "0",
"function": {
"name": "duckduckgo_results_json",
"arguments": "{\"query\": \"how to cook a lobster\"}"
}
}
]
},
{
"content": "[{\"snippet\": \"From that point, boil the lobsters for 7 to 14 minutes or longer, depending on the size of the lobster. 7 to 10 minutes for a 1-pound lobster, 8 to 12 minutes for a 1 1/4-pound lobster, and 10 to 14 minutes for a 1 1/2-pound lobster.\", \"title\": \"How to Boil and Eat Lobster - Simply Recipes\", \"link\": \"https://www.simplyrecipes.com/recipes/how_to_boil_and_eat_lobster/\"}, {\"snippet\": \"Learn how to choose, prepare, and cook lobster in different methods, such as boiling, steaming, and baking. Find out how to break down, serve, and explore the versatility of this culinary treasure.\", \"title\": \"How to Cook Lobster: Foolproof Recipe for Perfect Results Every Time\", \"link\": \"https://oliviaoven.com/how-to-cook-lobster/\"}, {\"snippet\": \"For humane cooking, place the lobster in the freezer for 15-20 minutes to gently put it to sleep. 3. Master Boiling. Boiling is one of the simplest ways to cook a lobster. Bring a large pot of salted water to a rolling boil, then add your lobster headfirst. Cook for about 7-8 minutes per pound. The shell should turn bright red when it's done. 4.\", \"title\": \"Ways to Cook a Lobster: 10 Beginner Tips for Fresh Results\", \"link\": \"https://www.wholesalemainelobster.com/blog/top-10-fresh-lobster-cooking-tips-for-beginners/\"}, {\"snippet\": \"Learn how to cook lobster at home with various methods, such as boiling, steaming, and grilling. Find out how to choose the best type of lobster, what tools to use, and how to serve it for a restaurant-quality experience.\", \"title\": \"How to Cook Lobster: Step-by-Step Guide for Perfect Lobster at Home\", \"link\": \"https://globalseafoods.com/blogs/recipes/how-to-cook-a-lobster-a-step-by-step-guide\"}]",
"role": "tool",
"tool_call_id": "0"
}
]
}
Error message from SGLang side:
|
Thanks, we are looking into this. |
@Superskyyy Currently, the content field of a message can't be null. If there's no message, simply set it as an empty string (like this). {
"content": "",
"role": "assistant",
"tool_calls": [
{
"type": "function",
"id": "0",
"function": {
"name": "duckduckgo_results_json",
"arguments": "{\"query\": \"how to cook a lobster\"}"
}
}
]
} |
Hello @xxll88. Could you please provide the messages in json? It seems that the content field in one of your messages happens to be
|
@Superskyyy I'm not using tool calling, just running a tester https://github.com/sgl-project/sglang/blob/main/test/srt/test_vision_openai_server.py |
I can confirm that changing to an empty string works as a workaround. |
So we should PR to make it convert None to an empty string? |
I think so, it can avoid unexpected complications. |
Checklist
Describe the bug
when use autogen with qwen2.5
messages=[SystemMessage(content='You are a helpful AI assistant. Solve tasks using your tools. Reply with TERMINATE when the task has been completed.', type='SystemMessage'), UserMessage(content='shanghai weather', source='user', type='UserMessage')],
client.chat.completions.create(
messages=messages,
stream=False,
tools=converted_tools,
**create_args,
)
when using the SGLang inference framework, it can normally infer and call the function named "get_weather." However, if the conversation continues and carries the history of previous chats, there will be an issue.
messages=[SystemMessage(content='You are a helpful AI assistant. Solve tasks using your tools. Reply with TERMINATE when the task has been completed.', type='SystemMessage'), UserMessage(content='shanghai weather', source='user', type='UserMessage'), AssistantMessage(content=[FunctionCall(id='0', arguments='{"city": "shanghai"}', name='get_weather')], source='weather_agent', type='AssistantMessage'), FunctionExecutionResultMessage(content=[FunctionExecutionResult(content='The weather in shanghai is 73 degrees and Sunny.', call_id='0', is_error=False)], type='FunctionExecutionResultMessage'), UserMessage(content='beijing weather', source='user', type='UserMessage')],
pydantic_core._pydantic_core.ValidationError: 5 validation errors for ChatCompletionRequest
messages.2.ChatCompletionMessageGenericParam.content.str
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type
messages.2.ChatCompletionMessageGenericParam.content.list[ChatCompletionMessageContentTextPart]
Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/list_type
messages.2.ChatCompletionMessageUserParam.role
Input should be 'user' [type=literal_error, input_value='assistant', input_type=str]
For further information visit https://errors.pydantic.dev/2.10/v/literal_error
messages.2.ChatCompletionMessageUserParam.content.str
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/string_type
messages.2.ChatCompletionMessageUserParam.content.list[union[ChatCompletionMessageContentTextPart,ChatCompletionMessageContentImagePart]]
Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/list_type
Reproduction
python3 -m sglang.launch_server --model-path /mnt/vdb1/model/qwen/Qwen2___5-7B-Instruct-GPTQ-Int4/ --served-model-name qwen2-72b --host 0.0.0.0 --port 8001 --trust-remote-code --mem-fraction-static 0.2 --chunked-prefill-size 8192 --schedule-conservativeness 0.3 --tool-call-parser qwen25
Environment
ubuntu
autogen 0.4.6
The text was updated successfully, but these errors were encountered: