This guide is a loose collection of prompt guidelines, that generally apply to the most Stable Diffusion Checkpoints/Models from 1.4 to 2.1. Keep in mind, they might have different effects on your models and that there is NO strict rules for your prompting generally – so keep on playing 🙂
basic prompt guidelines
I – use representational words
It is crutial to use representational words and vocabulary in the prompt, as the models can hardly deal with idioms like „Better late than never“. It is best to simply stick to describe in simple words what should be seen in the image like: „a vintage drawing of a tree in the middle of a foggy forest“.
II – order of words by importance
The order in which the words are written is essential to getting the desired output. The words at the beginning of your prompt or section are weighted more than the following. Each prompt can be comma seprarated in sections -> see prompt matrix examples
III – prompt matrix
The best way to compose a proper prompt is to run through a small checklist of aspects reflected in the prompt. They can be separated in SUBJECT, MEDIUM, STYLE, COLOR&LIGHTING – This is no strict rule, as you can change the order or importance of each component at any given time.
IV – weightening of words/strings
When writing a prompt each word you use has a continous shrinking relevance to the rendering outcome. With custom weightening you can highlight or minimize specific aspect within your prompt by putting it into braces with a simple numerical indicator like: ( a huge house:1.3 ) to increase or ( a huge house:0.8 ) to decrease
V – negative prompting
A negative prompt allows the user to specify what avoid rendering, without any extra input. It is a parameter that tells the model what not to include in the generated image. The negative prompt comes with the same form as the positive prompt but is often pretty model specific. In newer cases, that negative prompt cloud is already attached as textual inversion.
VI – rendering ratio and resolution
Although the ratio is no part of the prompt, they need to be mentioned, as they have a huge influence on the outcome and the behaviour of the render enginge. Especially formats besides square ratio create much interesting compositions. This does also depend on the subject that needs to be visualized – experiment with that. The follwoing example is done with the identical seed and prompts to pinpoint the difference in composition.
prompt matrix examples
This section is an applied row of rendering example to demonstrate the effects of prompt engineering by example. We run with this basic setup:
engine: Stable Diffusion / A1111
seed: 3095911944
hires-fix: enabled
sampler: Euler A
checkpoint: Cyber Realistic 3.2
SUBJECT
The subject is the core element of the prompt, as it pinpoints the main element you will see in the generated image. It works best, when literally describing the picture with what elements to see, how they relate to each other, what the environment is like and so on. This can be very diverse, so there is a lot of need to experiment with that!
MEDIUM
The medium sets the kind of way how the subject is interpreted material wise. This can be any media from drawing to 3D rendering or a mixture of it. Keep in mind, that the chosen medium has a huge impact how your motif is realized. You might also modify your subject prompt in relation to the chosen medium. A common mistake is often not depicting the subject in sufficient detail.
STYLE
The style can be seen as a more detailed description how the motif should be executed. This can range from hyper realistic to naive abstract. This can be also refer to epochs like renaissance or baroque or design styles like Bauhaus or Minimalism.
COLORS & LIGHTING
Pinpointing the light situation and a suitable color palette is another design addition in the prompt cloud. This can, again, have a huge influence on your overall composition.
sources and references
*(copied from the prompt guide by Umberto Grando at Medium)
https://www.fotor.com/blog/stable-diffusion-prompts/
Stable Diffusion – Prompt 101 by Not that Complicated – Youtube