Question of fine tuning

[ioky31]

Thank you for your excellent work. I am planning to fine-tune show-o for a specific downstream task, I used medical image text dataset for stage3 fine-tuning, but I found that the generated result is not satisfactory, and the mode collapse, how can I solve this problem?

[ioky31]

I wonder if it's caused by the dataset being too homogeneous in terms of data

[Sierkinhane]

Can you show some examples and what's the dataset size you use to finetune?

[ioky31]

Approximately 90k cases of data

[Sierkinhane]

The cfg is disabled when online inference during training. You'd try offline inference and enable the cfg.

Besides, the image on the first row is predicted in one step with a high mask ratio (0.87), which significantly degrades the generation quality.