Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and test MemoryStep #432

Merged
merged 4 commits into from
Jan 30, 2025

Conversation

albertvillanova
Copy link
Member

@albertvillanova albertvillanova commented Jan 30, 2025

Fix and test MemoryStep.

Note:

  • .dict method was raising: TypeError: asdict() should be called on dataclass instances
  • .raw attribute is never used

TODO:

  • Decide about the .raw attribute

@aymeric-roucher
Copy link
Collaborator

aymeric-roucher commented Jan 30, 2025

@albertvillanova the .raw attribute was added after discussions with @matthewcarbone who highlighted the need to access full raw model outputs for inspection of logs ex-post.

@matthewcarbone
Copy link

I am definitely a strong proponent of keeping all raw metadata. I think this was added in a recent PR though right? Through the logging? @aymeric-roucher

I think this is used somewhere and should be kept, but I could be wrong. There were lots of fast changes in that discussion.

@aymeric-roucher
Copy link
Collaborator

Yes it should be kept, even if not used anywhere else in the code base it can be useful for later inspection!

@albertvillanova
Copy link
Member Author

albertvillanova commented Jan 30, 2025

Currently, MemoryStep.raw is never used in the code: no value is assigned to this variable.

  • Before this PR, it was a class attribute (the same shared value for all instances)!
  • And never used: neither assigned, nor accessed.

What we use in the code is ChatMessage.raw instead:

class ChatMessage:
role: str
content: Optional[str] = None
tool_calls: Optional[List[ChatMessageToolCall]] = None
raw: Optional[Any] = None # Stores the raw output from the API

What is the difference between these two?
CC: @aymeric-roucher @matthewcarbone

@albertvillanova
Copy link
Member Author

Note that I commented the .raw attribute, because otherwise, it is a required argument for all the MemoryStep subclasses:

FAILED tests/test_memory.py::TestAgentMemory::test_initialization - TypeError: SystemPromptStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::TestMemoryStep::test_initialization - TypeError: MemoryStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::TestMemoryStep::test_dict - TypeError: MemoryStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::TestMemoryStep::test_to_messages - TypeError: MemoryStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::test_action_step_to_messages - TypeError: ActionStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::test_planning_step_to_messages - TypeError: PlanningStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::test_task_step_to_messages - TypeError: TaskStep.__init__() missing 1 required positional argument: 'raw'
FAILED tests/test_memory.py::test_system_prompt_step_to_messages - TypeError: SystemPromptStep.__init__() missing 1 required positional argument: 'raw'

@aymeric-roucher
Copy link
Collaborator

Oh, sorry, I misunderstood your question : then indeed, MemoryStep.raw is useless and should be removed, the .raw attribute is only useful for ChatMessages!

@albertvillanova albertvillanova marked this pull request as ready for review January 30, 2025 18:15
@albertvillanova
Copy link
Member Author

Removed.

@aymeric-roucher aymeric-roucher merged commit 181a500 into huggingface:main Jan 30, 2025
3 checks passed
@aymeric-roucher
Copy link
Collaborator

Thank you @albertvillanova !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants