✭ MAGNETRON ™ ✭: This is a Google Colab/Jupyter Notebook for developing an IMAGINATION (G2) PROXIA when working with ARTIFICIAL INTELLIGENCE 2.0 ™ (ARTIFICIAL INTELLIGENCE 2.0™ is part of MAGNETRON ™ TECHNOLOGY).
🤖 THE ABC 123 GROUP ™ 🤖
🌐 GENERAL CONSULTING ABC 123 BY OSAROPRIME ™.
🌐 ABC 123 USA ™
🌐 ABC 123 DESYGN ™
🌐 ABC 123 FILMS ™
=============================================================
🌐 MAGENTRON ™ 🌐
🌐 ARTIFICIAL INTELLIGENCE 2.0 ™ : FOR MAKING IMAGINATION PROXIA G-2
*️⃣📶🤖
REQUIREMENTS:
[*] Software Requirements: Python
[*] HARDWARE REQUIREMENTS: fast GPU (Graphics Processing Unit)
[*] DEPENDENCIES:
-
LIBRARIES: torch, transformers, diffusers, numpy, PIL, tqdm, difflib
-
Weights and Biases account: https://wandb.ai/site
This repository contains 2 Google Colab NOTEBOOKS that will guide you on one possible scheme to create an IMAGINATION PROXIA in the ARTIFICIAL INTELLIGENCE 2.0™ FRAMEWORK/DOCUMENTATION. This NOTEBOOK will guide you one generating an IMAGE from text and then fine tuning/editing IMAGE through subsequent/additional text prompts (using Cross Attention Control). You can adapt this to your needs.
This PROXIA will endow the ROBOT with ability to EDIT/fine tune its IMAGINATION by making changes to the IMAGE via text.
EXAMPLE USAGE:
e.g On an ASTRAL MINDCLOUD this PROXIA can be used to process INFORMATION sent to it from an INSTINCTIVE MIND PROXIA/MINDCLOUD (OBJECT DETECTION). So for example if the ROBOT or a SWARM/HIVE/PHALANX of ROBOTS encounters an intersting object they can use their eye cameras to IMAGES of the subject which can be used as INPUT for this IMAGINATION PROXIA. This can help ROBOTS better understand the subject or environment of the IMAGE (as well as how humans view it). [SEE EXAMPLE IMAGES]
e.g DREAMING: IMAGINATION PROXIAS CAN BE USED BY THE ROBOTS TO "DREAM". BY DREAMING I MEAN WHEN THE ROBOT IS IN HIBERNATION/SLEEP MODE IT CAN STILL PROCESS INFORMATION ABOUT THE OUTSIDE WORLD ON A LIMITED BASIS FROM PERIODIC TEXT PROMPTS (e.g from news, police reports).
=============================================================
🌐 MAGNETRON ™ : ARTIFICIAL INTELLIGENCE 2.0 ™ 🌐
CLICK ON THE FOLLOWING LINKS FOR MORE JUPYTER NOTEBOOKS ON MAKING IMAGINATION PROXIA:
https://github.com/GCABC123/magnetron.artificial-intelligence-2.0.mincloud.proxia--IMAGINATION-A1
https://github.com/GCABC123/magnetron.artificial-intelligence-2.0.mincloud.proxia--IMAGINATION-B
https://github.com/GCABC123/magnetron.artificial-intelligence-2.0.mincloud.proxia--IMAGINATION-C
https://github.com/GCABC123/magnetron.artificial-intelligence-2.0.mincloud.proxia--IMAGINATION-D
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Prerequisite reading:
🌐 ARTIFICIAL INTELLIGENCE PRIMER ™: https://www.facebook.com/artificialintelligenceprimer
🌐 ARTIFICIAL INTELLIGENCE 2.0 ™ DOCUMENTATION: https://www.facebook.com/aibyabc123/
🌐 MEMBER'S CLUB ™ DOCUMENTATION - https://www.facebook.com/abc123membersclub/
👑 INCLUDED STICKERS/SIGN:
FIND STICKERS HERE: https://bit.ly/3B8D3lE
PROMOTIONAL MATERIAL FOR 𝗠𝗔𝗚𝗡𝗘𝗧𝗥𝗢𝗡 𝗧𝗘𝗖𝗛𝗡𝗢𝗟𝗢𝗚𝗬 ™. (CUSTOM GRAPHICS BY 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗗𝗘𝗦𝗬𝗚𝗡 ™/𝗢𝗦𝗔𝗥𝗢 𝗛𝗔𝗥𝗥𝗜𝗢𝗧𝗧). THE 𝗠𝗔𝗚𝗡𝗘𝗧𝗥𝗢𝗡 𝗧𝗘𝗖𝗛𝗡𝗢𝗟𝗢𝗚𝗬 ™ SYMBOL/LOGO IS A TRADEMARK OF 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™ FOR 𝗠𝗔𝗚𝗡𝗘𝗧𝗥𝗢𝗡 𝗧𝗘𝗖𝗛𝗡𝗢𝗟𝗢𝗚𝗬 ™. 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™ SYMBOL/LOGO IS A TRADEMARK OF 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™. *️⃣📶🤖
PROMOTIONAL MATERIAL FOR 𝗔𝗥𝗧𝗜𝗙𝗜𝗖𝗜𝗔𝗟 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘 𝟮.𝟬 ™. (CUSTOM GRAPHICS BY 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗗𝗘𝗦𝗬𝗚𝗡 ™/𝗢𝗦𝗔𝗥𝗢 𝗛𝗔𝗥𝗥𝗜𝗢𝗧𝗧) THE 𝗗𝗥𝗔𝗚𝗢𝗡 & 𝗖𝗥𝗢𝗪𝗡 👑 SYMBOL/LOGO IS A TRADEMARK OF 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™ ASSOCIATED WITH TECHNOLOGY. 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™ SYMBOL/LOGO IS A TRADEMARK OF 𝗧𝗛𝗘 𝗔𝗕𝗖 𝟭𝟮𝟯 𝗚𝗥𝗢𝗨𝗣 ™. You must display the included stickers/signs (so that it is clearly visible) if you are working with MAGNETRON ™ TECHNOLOGY for the purposes of determining whether you want to purchase a technology license or not. This includes but is not limited to public technology displays, trade shows, technology expos, media appearances, Investor events, Computers (exterior), MINDCLOUD STORAGE (e.g server room doors, render farm room doors) etc.
.
🌐 NOTE: IMAGINATION PROXIA A IS DESCRIBED IN THE 𝗔𝗥𝗧𝗜𝗙𝗜𝗖𝗜𝗔𝗟 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘 𝟮.𝟬 ™ DOCUMENTATION.
🌐 NOTE: 𝗔𝗥𝗧𝗜𝗙𝗜𝗖𝗜𝗔𝗟 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘 𝟮.𝟬 ™ is part of MAGNETRON ™ TECHNOLOGY.
🌐 NOTE: REMEMBER 𝗔𝗥𝗧𝗜𝗙𝗜𝗖𝗜𝗔𝗟 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘 𝟮.𝟬 ™ ROBOTS WORK WELL TOGETHER (e.g HIVES, PHALANX, SWARM) MAKING GATHERING IMAGES FOR THIS KIND OF IMAGE SYNTHESIS EASY.
🌐 NOTE: REMEMBER 1 FRAME OF VIDEO IS EUIVALENT TO AN IMAGE THAT CAN BE USED AS INPUT FOR THIS IMAGINATION PROXIA.
Large-scale language-image models (eg. Stable Diffusion) are usually hard to control just with editing the prompts alone and can be very unpredictable and unintuitive for users. Most existing methods require the user to input a mask which is cumbersome and might not yield good results if the mask has an inadequate shape. Cross Attention Control allows much finer control of the prompt by modifying the internal attention maps of the diffusion model during inference without the need for the user to input a mask and does so with minimal performance penalities (compared to clip guidance) and no additional training or fine-tuning of the diffusion model.
This notebook uses the following libraries: torch transformers diffusers numpy PIL tqdm difflib
Simply install the required libraries using pip and run the jupyter notebook, some examples are given inside.
A description of the parameters are given at the end of the readme.
All images shown below are generated using the same seed. The initial and target images must be generated with the same seed for cross attention control to work.
Top left prompt: [a cat] sitting on a car
Clockwise: a smiling dog..., a hamster..., a tiger...
Note: different strength values for prompt_edit_spatial_start were used, clockwise: 0.7, 0.5, 1.0
Top left prompt: a fantasy landscape with a maple forest
Clockwise: a watercolor painting of..., a van gogh painting of..., a charcoal pencil sketch of...
Top left prompt: a fantasy landscape with a pine forest
Clockwise: ..., autumn, ..., winter, ..., spring, green
Left image prompt: a fantasy landscape with a pine forest
Right image prompt: a winter fantasy landscape with a pine forest
Middle image: Cross attention enabled prompt editing (left image -> right image)
Left image prompt: a fantasy landscape with a pine forest
Right image prompt: a watercolor painting of a landscape with a pine forest
Middle image: Cross attention enabled prompt editing (left image -> right image)
Left image prompt: a fantasy landscape with a pine forest
Right image prompt: a fantasy landscape with a pine forest and a river
Middle image: Cross attention enabled prompt editing (left image -> right image)
Left image prompt: a fantasy landscape with a pine forest
Towards the right: -fantasy
Left image prompt: a fantasy landscape with a pine forest
Towards the right: +fantasy and +forest
Left image prompt: a fantasy landscape with a pine forest
Towards the right: -fog
Left image: from previous example
Towards the right: -rocks
Let's compare our results above where we removed fog and rocks from our fantasy landscape using cross attention maps against what people usually do, by editing the prompt alone.
We can first try adding "without fog and without rocks" to our prompt.
Image prompt: A fantasy landscape with a pine forest without fog and without rocks
However, we still see fog and rocks.
We can try adding words like dry, sunny and grass.
Image prompt: A fantasy landscape with a pine forest without fog and rocks, dry sunny day, grass
There are less rocks and fog, but the image's composition and style is completely different from before and we still haven't obtained our desired fog and rock-free image...
Two functions are included, stablediffusion(...) which generates images and prompt_token(...) that is used to help the user find the token index for words in the prompt, which is used to tweak token weights in prompt_edit_token_weights.
Parameters of stabledifusion(...):
| Name = Default Value | Description | Example |
|---|---|---|
prompt="" |
the prompt as a string | "a cat riding a bicycle" |
prompt_edit=None |
the second prompt as a string, used to edit the first prompt using cross attention, set None to disable |
"a dog riding a bicycle" |
prompt_edit_token_weights=[] |
values to scale the importance of the tokens in cross attention layers, as a list of tuples representing (token id, strength), this is used to increase or decrease the importance of a word in the prompt, it is applied to prompt_edit when possible (if prompt_edit is None, weights are applied to prompt) |
[(2, 2.5), (6, -5.0)] |
prompt_edit_tokens_start=0.0 |
how strict is the generation with respect to the initial prompt, increasing this will let the network be more creative for smaller details/textures, should be smaller than prompt_edit_tokens_end |
0.0 |
prompt_edit_tokens_end=1.0 |
how strict is the generation with respect to the initial prompt, decreasing this will let the network be more creative for larger features/general scene composition, should be bigger than prompt_edit_tokens_start |
1.0 |
prompt_edit_spatial_start=0.0 |
how strict is the generation with respect to the initial image (generated from the first prompt, not from img2img), increasing this will let the network be more creative for smaller details/textures, should be smaller than prompt_edit_spatial_end |
0.0 |
prompt_edit_spatial_end=1.0 |
how strict is the generation with respect to the initial image (generated from the first prompt, not from img2img), decreasing this will let the network be more creative for larger features/general scene composition, should be bigger than prompt_edit_spatial_start |
1.0 |
guidance_scale=7.5 |
standard classifier-free guidance strength for stable diffusion | 7.5 |
steps=50 |
number of diffusion steps as an integer, higher usually produces better images but is slower | 50 |
seed=None |
random seed as an integer, set None to use a random seed |
126794873 |
width=512 |
image width | 512 |
height=512 |
image height | 512 |
init_image=None |
init image for image to image generation, as a PIL image, it will be resized to width x height |
PIL.Image() |
init_image_strength=0.5 |
strength of the noise added for image to image generation, higher will make the generation care less about the initial image | 0.5 |