Refactor prompts

[aymeric-roucher]

No description provided.

[aymeric-roucher]

@albertvillanova do you have pointers about that failed test? It's linked to the MagicMock, I don't really have experience with these.
https://github.com/huggingface/smolagents/actions/runs/13163295123/job/36737183697?pr=502

[albertvillanova]

Let me have a look! 😉

[albertvillanova]

This will fix the MagicMock issue in the test.

[albertvillanova]

However there is another issue:

• MultiStepAgent does not implement initialize_system_prompt
• Therefore: self.system_prompt = None
• self.system_prompt.strip() will raise an error

AttributeError: 'NoneType' object has no attribute 'strip'

[aymeric-roucher]

Oh and btw @albertvillanova explanations about the removal of single_step option: this option was only introduced for a benchmark, not really useful because single-step agents struggle to solve real tasks, and it introduced lots of overhead with a dedicated prompt: thus the removal!

[aymeric-roucher]

@albertvillanova regarding you command above on system_prompt being None: I've removed the .strip() from the incriminated line, WDYT?

[albertvillanova]

@albertvillanova regarding you command above on system_prompt being None: I've removed the .strip() from the incriminated line, WDYT?

Does it make sense to return "text": None?

[Message(role=MessageRole.SYSTEM, content=[{"type": "text", "text": None}])]

[albertvillanova]

To fix the failing test.

[albertvillanova]

You removed managed_agent_prompt.

[aymeric-roucher]

@albertvillanova I've created this new logic with final_answer_checks to force the agent to keep running while the validation does not pass. It's quite handy for some validation logic like "output should be a list" or "let a LLM verify your output according to a pre-defined validation prompt and parse this LLM's output to find if the test passes".

[aymeric-roucher]

Example of such a check for a task that involves making a plot:

def check_reasoning_and_plot(final_answer, agent_memory):
    final_answer
    multimodal_model = OpenAIServerModel("gpt-4o")
    filepath = "saved_map.png"
    assert os.path.exists(filepath), "Make sure to save the plot under saved_map.png!"
    image = Image.open(filepath)
    prompt = (
        f"Here is a user-given task and the agent steps: {agent_memory.get_succinct_steps()}. Now here is the plot that was made."
        "Please check that the reasoning process and plot are correct: do they correctly answer the given task?"
        "First list 3 reasons why yes/no, then write your final decision: PASS in caps lock if it is satisfactory, FAIL if it is not."
        "Don't be harsh: if the plot mostly solves the task, it should pass."
        "But if any data was hallucinated/invented, you should refuse it."
    )
    messages = [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": prompt,
                },
                {"type": "image_url", "image_url": {"url": make_image_url(encode_image_base64(image))}},
            ],
        }
    ]
    output = multimodal_model(messages).content
    print("Feedback: ", output)
    if "FAIL" in output:
        raise Exception(output)
    return True

[albertvillanova]

Thanks! Let's start sharing agents!