Stable Diffusion Prompt Guide

Prompts make or break your Stable Diffusion visuals, and those are crucial for engagement. But crafting a prompt design that captures your concepts flawlessly? That's an art in itself.

This guide breaks down the process step-by-step, so you can master prompt engineering from the ground up. We'll cover Stable Diffusion's "language" and using advanced techniques like prompt mixing, and more. Get ready to start producing top-notch AI-generated art that separates you from the pack. Let's begin!

Prompt Building Principles

Clear your prompt structure must be, descriptive and specific.
a Jedi prompt construction wisdom

There are no formal rules or a strict syntax for Stable Diffusion prompts. Rather, these techniques enhance prompt writing for better control over the text-to-image model.

Stable Diffusion is fairly flexible in interpreting natural language prompts. However, employing these principles allows you to communicate your wishes more clearly to the AI.

Specifically, those principles are:

• Using clear, specific and descriptive language.
• Incorporating art styles, artist names, and technique descriptions.
• Combining multiple prompts and using negative prompts to refine results.
• Using weight modifiers to prioritize prompt components.

These are not hard-coded rules, but more like guidelines based on how the model interprets and processes your inputs. Following them helps you grasp how to construct prompts that accurately steer the AI towards the intended output.

Incorporating Art Styles and Techniques

Weaving in art styles and technique descriptions in your prompt takes some practice. Here are some tips:

For art styles:

• Use descriptive adjectives like "impressionistic", "cubist", "surrealist", "abstract expressionist"
• Reference famous art movements like "Art Nouveau", "Baroque", or artists like "Alphonse Mucha"
• Try combining styles like "photorealistic with elements of pop art"

For techniques:

• "Oil painting", "watercolor", "charcoal sketch", "pen and ink drawing"
• Textures like "impasto", "stippled", "crosshatched"
• Brushwork styles such as "broad brushstrokes", "delicate brushwork"
• Lighting effects like "Bokeh effect", "dramatic lighting"

Try to be as specific as possible, e.g. "finely detailed Renaissance oil painting" or "loose impressionistic watercolor." And don't be afraid to experiment and stack multiple styles/techniques into one prompt, like: "Renaissance-style oil painting of a surreal, impressionistic landscape with fantastical flora, dramatic chiaroscuro lighting, and baroque-art nouveau aesthetic"

Multi Prompt In Stable Diffusion

Mixing seemingly unrelated concepts in a prompt can produce surreal and imaginative results. Here's how you could approach mixing a green apple with an orange giraffe:

The basic formula is: [object 1] + [object 2] + [descriptors/modifiers]

For example "A green apple with the long neck and spotted pattern of an orange giraffe, surreal dreamlike rendering, intricate details"

Let's break it down:

1. "A green apple" - this establishes the base object/concept
2. "with the long neck and spotted pattern of an orange giraffe" - this starts blending in attributes from the second object
3. "surreal dreamlike rendering, intricate details" - additional descriptors and modifiers

You can get even more creative by chaining multiple object blends: "Lush, impressionistic meadow with a towering, mechanical giraffe made of gears and clockwork, emitting steam and sparks"

The key is using connectors like "with..." to merge the attributes you want from each object into the base prompt.

You can also use styles/techniques to push it further: “Minimalist style painting of dreamlike forest glade with towering mechanical tree, bioluminescent branches, origami rabbits”

Don't be afraid to go wild and unexpected with your combinations! Stable Diffusion excels at interpreting and visualizing even bizarre prompts in creative ways. Start with a solid base object, then get liberal with blending attributes from other prompts using connectors.

Writing Negative Prompts

Negative prompts filter unwanted objects, styles, or concepts from the text-to-image generation process. It gives you a way to be very specific about what you don't want, while the regular prompt specifies what you do want to see. Using them together provides control over both inclusion and exclusion criteria.

One way to think about negative prompts is that using them is essentially saying "but under no circumstances show/include..." followed by the things you want to exclude from the generated image.

• Prompt: "A fantasy landscape with a medieval castle"
• Negative Prompt: "-modern, -futuristic"
• Result: "Make a fantasy medieval castle scene, but under no circumstances show anything modern or futuristic.”

Here's a table of useful negative prompts for Stable Diffusion, categorized by type:

You can negate entire sections by prepending with "no" as well:

• -no humans, -no animals, -no nature, -no objects

Remember that this is a powerful tool and when the conflicts arise between your prompt and negative prompt the latter one prevails. For example:

• Positive Prompt: "A group of clowns performing at a circus"
• Negative Prompt: "no clowns, no circuses"
• Result: The model might generate an image of awkward people in regular clothes, as it tries to avoid clowns but still capture the essence of a prompt.
• Positive Prompt: "A majestic dragon breathing fire"
• Negative Prompt: "no dragons, no fire"
• Result: You could get an image of a large lizard yawning, as the model tries to create something dragon-like without actual dragons or flames.

Negative prompts are useful, but their application warrants caution. Overly broad or contradictory prompts confuse the model and may result in unintended or awkward outputs.

How To Emphasize Prompt In Stable Diffusion

Weight modifiers provide an additional level of control when feeding your prompts to SD. They allow you to emphasize or de-emphasize the importance of specific words or concepts within your prompt.

To use a weight modifier, enclose the word in parentheses and follow it with a colon and a numerical weight value. For example:

• (sunset:1.5). Values above 1.0 increase the word's importance, while values below 1.0 decrease it.

Alternatively, you can use a shortcut to easily adjust the numerical weight value. Just highlight the word in your prompt, such as by double-clicking it and then scroll your mouse wheel up or down.

This technique is useful for ensuring the generated image accurately reflects your priorities. For instance:

• (Baby:1.5) bunny is holding a (delicate:1.5) bouquet of (vibrant:1.4) flowers in the rain

Weight modifiers can also be applied to negative prompts to fine-tune what gets excluded. Using a negative prompt of "(rendering artifacts:2.0), (blur:1.5)" strongly avoids artifacts while moderately reducing blur. However, an overly complex approach may lead to poor outputs, so we recommend nuanced adjustments.

Tuning CFG Scale

CFG scale stands for "Classifier-Free Guidance” scale. It’s a setting that controls the balance between following the text prompt and allowing the AI model to be deliriously imaginative.

• When the CFG scale is low (e.g., around 2-4), the AI model sticks more closely to the text prompt you provide. It tries to generate an image that accurately represents the prompt, but with less room for unexpected elements.
• A middle range CFG (around 5-8) aims to strike a balance between following the prompt and allowing some creative freedom for the AI model.
• When the scale is high (around 10-20), the AI model has more freedom to be unpredictable in its depiction of the text prompt. It can introduce elements or interpretations that lead to more imaginative and unique results.

Adjusting the CFG scale allows you to control how much room the AI has to generate unexpected results.

Explore And Experiment: Ignite Your Creativity

With the principles of prompt engineering now under your belt, you're ready to explore the full potential of your creativity! Experiment with your new skills to craft your unique AI Image generation masterpieces. Embrace the power of negative prompts and weight modifiers to refine your results to perfection. The future of visual storytelling awaits - may the Force of Stable Diffusion be with you!