Possible to Train non-COCO?

[afiaka87]

I see this only supports the COCO dataset. Any plans to support something a little more generic like an ImageTextFolder or WebDataset?

I helped write this implementation (and there's also support for webdataset in that repo):

https://github.com/lucidrains/DALLE-pytorch/blob/main/dalle_pytorch/loader.py

[afiaka87]

This would work with COCO btw - which stores images and text files in this fashion.

[dzryk]

Definitely. I'll look into this, probably next week. Thanks!

[afiaka87]

Definitely. I'll look into this, probably next week. Thanks!

No - thank you!

[dzryk]

TextImageDataset is now the default. I have only tested on coco so far. If you try other datasets, please let me know if you encounter any errors. I'll look into supporting WebDataset later on.

[dzryk]

I've begun working on webdataset support here: https://github.com/dzryk/cliptalk

That project will be able to handle more generic image->text settings and uses GPT models instead of BART.

I'll add webdataset here for completion once I've confirmed things work over there