Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: refactor the LLM (using LiteLLM & add cost calculation) #480

Merged
merged 3 commits into from
Mar 11, 2025
Merged

feat: refactor the LLM (using LiteLLM & add cost calculation) #480

merged 3 commits into from
Mar 11, 2025

Conversation

mczhuge
Copy link
Contributor

@mczhuge mczhuge commented Mar 11, 2025

Hi MetaGPT's colleagues (@mannaandpoem @didiforgithub @XiangJinyu @MoshiQAQ @stellaHSR ), feel happy to see the contributions you made.

Features

  1. Using LiteLLM as the server for more flexible model usages
  2. Calculate the cost

Feature Docs

I have created the current version to support previous usage with minimal changes. Just install litellm~=1.63.6. I have added it into the requirements.txt.

Influence

This is a PR that improve the experiences of LLM usages.

Result

Test on OpenAI's GPT-4o:

(open_manus) ➜  OpenManus git:(main) ✗ python main.py 
INFO     [browser_use] BrowserUse logging setup complete with level info
INFO     [root] Anonymized telemetry enabled. See https://docs.browser-use.com/development/telemetry for more information.
Enter your prompt (or 'exit'/'quit' to quit): 帮我介绍一下Manus, MetaGPT, OpenHands, GPTSwarm, Agent-as-a-Judge.    
2025-03-11 15:18:11.517 | WARNING  | __main__:main:19 - Processing your request...
2025-03-11 15:18:11.518 | INFO     | app.agent.base:run:137 - Executing step 1/20
15:18:11 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= gpt-4o; provider = openai
INFO     [LiteLLM] 
LiteLLM completion() model= gpt-4o; provider = openai
2025-03-11 15:18:14.932 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.003530, Total: $0.003530
2025-03-11 15:18:14.932 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: None
2025-03-11 15:18:14.932 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 5 tools to use
2025-03-11 15:18:14.932 | INFO     | app.agent.toolcall:think:58 - 🧰 Tools being prepared: ['google_search', 'google_search', 'google_search', 'google_search', 'google_search']
2025-03-11 15:18:14.932 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:18:16.093 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://manus.im/', 'https://www.datacamp.com/blog/manus-ai', 'https://www.youtube.com/watch?v=hyi03gKISys', 'https://qz.com/manus-ai-agent-autonomous-china-openai-1851768776', 'https://medium.com/@vsinha027/chinas-manus-ai-d4ccafc2affd', 'https://www.thehindu.com/sci-tech/technology/debut-of-manus-ai-digital-assistant-from-china-causes-stir/article69315896.ece', 'https://www.manus-meta.com/', 'https://www.businessinsider.com/manus-ai-china-agent-hype-deepseek-2025-3', 'https://huggingface.co/blog/LLMhacker/manus-ai-best-ai-agent', 'https://www.forbes.com/sites/craigsmith/2025/03/08/chinas-autonomous-agent-manus-changes-everything/']
2025-03-11 15:18:16.093 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:18:17.203 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://github.com/geekan/MetaGPT', 'https://deepwisdom.ai/', 'https://www.ibm.com/think/topics/metagpt', 'https://arxiv.org/abs/2308.00352', 'https://docs.deepwisdom.ai/', 'https://www.linkedin.com/company/metagpt', 'https://openreview.net/forum?id=VtmBAGCN7o', 'https://pypi.org/project/metagpt/', 'https://arxiv.org/html/2308.00352v6', 'https://www.youtube.com/watch?v=uT75J_KG_aY']
2025-03-11 15:18:17.203 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:18:18.382 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://github.com/All-Hands-AI/OpenHands', 'https://docs.all-hands.dev/', 'https://openhands.daytona.io/', 'https://www.all-hands.dev/', 'https://www.kdnuggets.com/openhands-open-source-ai-software-developer', 'https://openreview.net/forum?id=OJd3ayDDoF', 'https://www.youtube.com/watch?v=4D3ctTcB2hY', 'https://openhands.ai4bharat.org/', 'https://arxiv.org/abs/2407.16741', 'https://www.youtube.com/watch?v=PJ8XUfivG20']
2025-03-11 15:18:18.382 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:18:19.434 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://gptswarm.org/', 'https://github.com/metauto-ai/GPTSwarm', 'https://arxiv.org/html/2402.16823v3', 'https://openreview.net/forum?id=uTC9AFXIhg', 'https://gptswarm.ai/', 'https://github.com/Open-Swarm-Net/GPT-Swarm', 'https://aiagentstore.ai/ai-agent/gptswarm', 'https://metaschool.so/ai-agents/gptswarm', 'https://twitter.com/SchmidhuberAI/status/1767218947491770852', 'https://faccio.ai/assets/pdf/gptswarm-poster.pdf']
2025-03-11 15:18:19.434 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:18:20.531 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://arxiv.org/abs/2410.10934', 'https://github.com/metauto-ai/agent-as-a-judge', 'https://medium.com/@techsachin/agent-as-a-judge-framework-to-evaluate-agents-with-agents-1a93db5131f8', 'https://openreview.net/forum?id=DeVm3YUnpj', 'https://www.evidentlyai.com/llm-guide/llm-as-a-judge', 'https://arize.com/blog/agent-as-a-judge-evaluate-agents-with-agents/', 'https://arxiv.org/html/2410.10934v2', 'https://www.linkedin.com/posts/dmitrii-khizbullin_agent-as-a-judge-evaluate-agents-with-agents-activity-7252327601703129089-40Gn', 'https://www.getmaxim.ai/blog/agent-evaluation/', 'https://cobusgreyling.medium.com/judging-ai-agents-db5411852dc2']
2025-03-11 15:18:20.531 | INFO     | app.agent.base:run:137 - Executing step 2/20
15:18:20 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= gpt-4o; provider = openai
INFO     [LiteLLM] 
LiteLLM completion() model= gpt-4o; provider = openai
2025-03-11 15:18:35.070 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.008865, Total: $0.012395
2025-03-11 15:18:35.070 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: Here is a brief introduction to each of the topics you asked about, based on the search results:

1. **Manus AI**:
   - Manus AI appears to be a digital assistant or autonomous agent developed in China. It has been discussed in various articles and platforms, highlighting its capabilities and impact. You can explore more about it on [Manus AI's official site](https://manus.im/) or read articles like [this one on DataCamp](https://www.datacamp.com/blog/manus-ai).

2. **MetaGPT**:
   - MetaGPT is a project that seems to be related to AI and machine learning, possibly involving meta-learning or advanced AI models. It is available on [GitHub](https://github.com/geekan/MetaGPT) and has been discussed in various academic and professional contexts. More details can be found on [DeepWisdom's site](https://deepwisdom.ai/).

3. **OpenHands AI**:
   - OpenHands is an open-source AI software project. It is hosted on [GitHub](https://github.com/All-Hands-AI/OpenHands) and has documentation available on [All Hands Dev](https://docs.all-hands.dev/). It seems to focus on collaborative AI development.

4. **GPTSwarm**:
   - GPTSwarm is likely a project or framework involving swarm intelligence and GPT models. It is discussed on platforms like [GPTSwarm.org](https://gptswarm.org/) and [GitHub](https://github.com/metauto-ai/GPTSwarm). It might involve collective AI behavior or distributed AI systems.

5. **Agent-as-a-Judge**:
   - This concept involves using agents to evaluate other agents, possibly in AI development or testing scenarios. It is discussed in academic papers and articles, such as on [arXiv](https://arxiv.org/abs/2410.10934) and [Medium](https://medium.com/@techsachin/agent-as-a-judge-framework-to-evaluate-agents-with-agents-1a93db5131f8).

If you need more detailed information on any of these topics, feel free to ask!
2025-03-11 15:18:35.070 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 0 tools to use
2025-03-11 15:18:35.070 | INFO     | app.agent.base:run:137 - Executing step 3/20
15:18:35 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= gpt-4o; provider = openai
INFO     [LiteLLM] 
LiteLLM completion() model= gpt-4o; provider = openai
2025-03-11 15:18:37.436 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.004648, Total: $0.017043
2025-03-11 15:18:37.436 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: It seems like you're providing a description of the tools and capabilities available for interaction. If you have a specific task or question in mind, please let me know how I can assist you using these tools. Whether it's executing Python code, saving files, browsing the web, or retrieving information, I'm here to help!
2025-03-11 15:18:37.436 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 0 tools to use
2025-03-11 15:18:37.436 | INFO     | app.agent.base:run:137 - Executing step 4/20
...
...
...

Test on Anthropic's Claude-3.5:

(open_manus) ➜  OpenManus git:(main) ✗ python main.py
INFO     [browser_use] BrowserUse logging setup complete with level info
INFO     [root] Anonymized telemetry enabled. See https://docs.browser-use.com/development/telemetry for more information.
Enter your prompt (or 'exit'/'quit' to quit): 帮我介绍一下Manus, MetaGPT, OpenHands, GPTSwarm, Agent-as-a-Judge.
2025-03-11 15:20:02.910 | WARNING  | __main__:main:19 - Processing your request...
2025-03-11 15:20:02.910 | INFO     | app.agent.base:run:137 - Executing step 1/20
15:20:02 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
INFO     [LiteLLM] 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
2025-03-11 15:20:05.678 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.006813, Total: $0.006813
2025-03-11 15:20:05.678 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: I'll help you introduce these AI frameworks and systems. Let me research each one to provide you with accurate information.
2025-03-11 15:20:05.678 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 1 tools to use
2025-03-11 15:20:05.678 | INFO     | app.agent.toolcall:think:58 - 🧰 Tools being prepared: ['google_search']
2025-03-11 15:20:05.678 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:20:06.936 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://manus.im/', 'https://dev.to/mehmetakar/openmanus-manus-ai-open-source-1kjj', 'https://www.reddit.com/r/AI_Agents/comments/1j87noh/created_an_opensource_alternative_to_manus_ai/', 'https://news.ycombinator.com/item?id=43312723', 'https://digialps.com/openmanus-a-powerful-open-source-ai-agent-alternative-to-manus-ai/']
2025-03-11 15:20:06.936 | INFO     | app.agent.base:run:137 - Executing step 2/20
15:20:06 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
INFO     [LiteLLM] 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
2025-03-11 15:20:09.229 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.008403, Total: $0.015216
2025-03-11 15:20:09.229 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: I'll help you introduce these AI frameworks and systems. Let me research each one to provide you with accurate information.
2025-03-11 15:20:09.229 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 1 tools to use
2025-03-11 15:20:09.229 | INFO     | app.agent.toolcall:think:58 - 🧰 Tools being prepared: ['google_search']
2025-03-11 15:20:09.229 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'google_search'...
2025-03-11 15:20:10.378 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'google_search' completed its mission! Result: Observed output of cmd `google_search` executed:
['https://manus.im/', 'https://dev.to/mehmetakar/openmanus-manus-ai-open-source-1kjj', 'https://www.reddit.com/r/AI_Agents/comments/1j87noh/created_an_opensource_alternative_to_manus_ai/', 'https://news.ycombinator.com/item?id=43312723', 'https://digialps.com/openmanus-a-powerful-open-source-ai-agent-alternative-to-manus-ai/']
2025-03-11 15:20:10.378 | INFO     | app.agent.base:run:137 - Executing step 3/20
15:20:10 - LiteLLM:INFO: utils.py:2974 - 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
INFO     [LiteLLM] 
LiteLLM completion() model= claude-3-7-sonnet-latest; provider = anthropic
2025-03-11 15:20:13.203 | INFO     | app.llm.inference:_calculate_and_track_cost:187 - Added cost: $0.009963, Total: $0.025179
2025-03-11 15:20:13.204 | INFO     | app.agent.toolcall:think:53 - ✨ Manus's thoughts: I'll help you introduce these AI frameworks and systems. Let me research each one to provide you with accurate information.
2025-03-11 15:20:13.204 | INFO     | app.agent.toolcall:think:54 - 🛠️ Manus selected 1 tools to use
2025-03-11 15:20:13.204 | INFO     | app.agent.toolcall:think:58 - 🧰 Tools being prepared: ['browser_use']
2025-03-11 15:20:13.204 | INFO     | app.agent.toolcall:execute_tool:140 - 🔧 Activating tool: 'browser_use'...
2025-03-11 15:20:16.872 | INFO     | app.agent.toolcall:act:113 - 🎯 Tool 'browser_use' completed its mission! Result: Observed output of cmd `browser_use` executed:
Error: Browser action 'navigate' failed: Page.goto: Target page, context or browser has been closed
Call log:
  - navigating to "https://manus.im/", waiting until "load"
...
...
...

@stellaHSR
Copy link
Collaborator

LGTM

1 similar comment
@mannaandpoem
Copy link
Owner

LGTM

@mannaandpoem mannaandpoem merged commit 737abe4 into mannaandpoem:main Mar 11, 2025
1 of 2 checks passed
@XiangJinyu
Copy link
Collaborator

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mczhuge @stellaHSR @mannaandpoem @XiangJinyu