Adding arbitrary custom models

[anoadragon453]

I saw the following paragraph in the README:

How can I get my own models interactable on the WebUI?

As long as we open sourced the WebUI you can implement your own UIs, but for now you can contribute to this carefree-creator repo and then ask me to do the UI jobs for you (yes, you can be my boss 😆).

Would it be possible to implement a generic way to import 1 or more custom models server-side, then communicate that to the webUI and allow the user to choose the model to use when using image generation/manipulation features? Much how AUTOMATIC1111/stable-diffusion-webui allows placing any *.ckpt in a models/ directory.

[carefree0910]

Yes indeed!

The handy version

If you need a handy version (e.g. placing any *.ckpt in some directory and get it working), it is currently not supported (it's on my TODO though), but we have:

The advanced version

I haven't documented these stuffs yet, but here are some brief guides:

1. The local APIs are exposed from here on.

↑ You can ignore this if you just want to change the existing models, instead of introducing new models / endpoints / features!

2. The APIs are implemented in txt2img.py and img2img.py.

3. I'm currently using my own library (carefree-learn) to implement the APIs, but you can re-implement the APIs with whatever you want! Take the basic text2img feature as an example:

   a. Rewrite the initialize method, where you can initialize your models.
   b. Rewrite the run method, where you need to generate the output (image) based on the input (the Txt2ImgSDModel, which contains almost all the necessary arguments)

Once all the modifications are done (on your own fork / a PR to a new branch of this project), you can modify the Install carefree-creator section in the Google Colab, and change this line:

!git clone https://github.com/carefree0910/carefree-creator.git

into the corresponding git-clone-url, so the Colab will install your own customized version and serve it!

Please feel free to comment if you encountered any trouble! 😆

[anoadragon453]

Thank you so much! This is definitely enough to get me started.

I think I'll close this issue as you've answered my question - if you want to go for a less advanced version that should probably be designed in a new issue.

I'll make a new issue if I run in to any problems as well. Thanks!

[carefree0910]

In case your are still interested: now we have a more handy way to use your own checkpoints!

I've updated the README (here) but I'm not sure whether I've made a clear instruction... So I wish I can get some feedback from you. 🤣

BTW: we also supported Textual Inversion now as well!