5. Prompts for Images

Description

The quality of a generated image depends more on the prompt than on the model. You can have access to DALL-E 3, Stable Diffusion XL and Midjourney, but if your prompt says "a cat", you'll get a generic cat, with no personality, no style, no context. The difference between a mediocre result and a professional one is not the model — it's the prompt engineering.

Why it matters: In production, image prompts aren't written by a human looking at the screen. Your pipeline generates them: an LLM that summarizes a document and requests an illustration, an e-commerce system that needs consistent product photos, a marketing tool that produces variations of the same visual concept. If you don't master the anatomy of an image prompt, your pipelines will produce inconsistent, generic or outright unusable results.

Connection with the module: In capsules 02 and 03 you learned to call the DALL-E and Stable Diffusion APIs. In this capsule you'll master what to send them. The prompting techniques you learn here are used directly in capsule 07 (pipelines) and in the final project (capsule 08), where the quality of the prompt determines whether the system generates useful images or expensive garbage.

Anatomy of an Effective Image Prompt

A professional image prompt isn't a random sentence. It's a visual specification with clear components.

The 7 components

Minimal prompt vs. professional prompt

❌ Minimal prompt:
"A cat"

✅ Professional prompt:
"An elegant Siamese cat sitting in a blue velvet armchair,
portrait photography with rembrandt lighting, blurred bokeh background,
warm colors, centered composition, 4K high resolution"

Building Prompts by Components

Modular implementation

from dataclasses import dataclass, field


@dataclass
class ImagePrompt:
    subject: str
    style: str = "realistic photography"
    medium: str = ""
    lighting: str = ""
    composition: str = ""
    mood: str = ""
    technical: str = "high resolution"
    extra_details: list[str] = field(default_factory=list)

    def build(self) -> str:
        parts = [self.subject]
        for attr in [self.style, self.medium, self.lighting,
                     self.composition, self.mood, self.technical]:
            if attr:
                parts.append(attr)
        parts.extend(self.extra_details)
        return ", ".join(parts)


prompt = ImagePrompt(
    subject="a professional workspace with dual monitors showing code",
    style="modern minimalist photography",
    lighting="soft natural light from large windows",
    composition="wide angle shot",
    mood="clean and productive atmosphere",
    technical="4K, sharp focus",
    extra_details=["plants on desk", "coffee cup"]
)
print(prompt.build())

Templates by use case

PROMPT_TEMPLATES = {
    "product_photo": ImagePrompt(
        subject="{product_description}",
        style="commercial product photography",
        lighting="studio lighting, soft shadows",
        composition="centered, white background",
        technical="high resolution, sharp focus, 4K",
        extra_details=["clean background", "professional"]
    ),
    "illustration": ImagePrompt(
        subject="{scene_description}",
        style="digital illustration",
        medium="clean vector style",
        mood="friendly and approachable",
        technical="vibrant colors",
    ),
    "ui_mockup": ImagePrompt(
        subject="{ui_description}",
        style="modern UI design",
        medium="flat design, clean interface",
        composition="device mockup view",
        technical="crisp edges, readable text",
        extra_details=["minimalist", "professional color palette"]
    ),
    "marketing_banner": ImagePrompt(
        subject="{concept_description}",
        style="modern advertising photography",
        lighting="dramatic lighting",
        composition="wide format, text space on left",
        mood="bold and eye-catching",
        technical="16:9 aspect ratio, high contrast"
    ),
}


def build_from_template(template_name: str, **kwargs) -> str:
    if template_name not in PROMPT_TEMPLATES:
        raise ValueError(f"Template not found: {template_name}")

    template = PROMPT_TEMPLATES[template_name]
    raw = template.build()
    for key, value in kwargs.items():
        raw = raw.replace(f"{{{key}}}", value)
    return raw


prompt_product = build_from_template(
    "product_photo",
    product_description="wireless bluetooth headphones in matte black"
)
print(prompt_product)

Visual Styles: Reference Catalog

STYLE_CATALOG = {
    "photo": {
        "editorial": "editorial fashion photography, magazine quality, dramatic pose",
        "product": "commercial product photography, studio lighting, white background",
        "portrait": "portrait photography, shallow depth of field, rembrandt lighting",
        "landscape": "landscape photography, golden hour, wide angle, vivid colors",
        "food": "food photography, overhead shot, styled plating, warm tones",
    },
    "art": {
        "watercolor": "watercolor painting, soft edges, translucent washes, paper texture",
        "oil_painting": "oil painting, thick brushstrokes, rich colors, canvas texture",
        "vector": "clean vector illustration, flat design, bold colors, no gradients",
        "anime": "anime style illustration, vibrant colors, expressive eyes, clean lines",
        "concept_art": "concept art, detailed environment, cinematic composition, matte painting",
    },
    "render": {
        "3d_render": "3D render, octane render, global illumination, photorealistic",
        "isometric": "isometric 3D illustration, soft shadows, pastel colors, clean geometry",
        "clay_render": "clay render, matte material, soft studio lighting, no texture",
    },
}


def apply_style(subject: str, category: str, style_name: str) -> str:
    catalog = STYLE_CATALOG.get(category, {})
    style = catalog.get(style_name, "")
    if not style:
        raise ValueError(f"Style '{style_name}' not found in '{category}'")
    return f"{subject}, {style}"


prompt = apply_style("a modern coffee shop interior", "photo", "editorial")
print(prompt)

Negative Prompts (Stable Diffusion)

DALL-E 3 doesn't support negative prompts — it interprets the intent of the full prompt internally. Stable Diffusion, on the other hand, relies heavily on them to avoid artifacts.

A negative prompt tells the model what not to include. It isn't the opposite of the positive prompt — it's a list of defects the model should avoid.

Negative prompts by category

NEGATIVE_PROMPTS = {
    "universal": (
        "blurry, low quality, distorted, ugly, watermark, text overlay, "
        "signature, logo, cropped, out of frame, worst quality, low resolution, "
        "jpeg artifacts, compression artifacts"
    ),
    "portraits": (
        "bad anatomy, extra fingers, mutated hands, deformed face, "
        "extra limbs, missing fingers, fused fingers, too many fingers, "
        "cross-eyed, asymmetric eyes, long neck, disproportionate body"
    ),
    "landscapes": (
        "blurry, oversaturated, distorted perspective, unrealistic colors, "
        "floating objects, impossible architecture, tiling artifacts"
    ),
    "products": (
        "blurry, bad lighting, cluttered background, shadows on product, "
        "distorted shape, wrong proportions, unrealistic reflections, text on product"
    ),
}


def build_negative(categories: list[str]) -> str:
    parts = [NEGATIVE_PROMPTS[cat] for cat in categories if cat in NEGATIVE_PROMPTS]
    return ", ".join(parts)

Integration with Stable Diffusion

import replicate


def generate_sd_with_negatives(
    prompt: str,
    negative_categories: list[str] = None,
    custom_negative: str = "",
    width: int = 1024,
    height: int = 1024,
    guidance_scale: float = 7.5,
) -> str:
    if negative_categories is None:
        negative_categories = ["universal"]

    negative = build_negative(negative_categories)
    if custom_negative:
        negative = f"{negative}, {custom_negative}"

    output = replicate.run(
        "stability-ai/sdxl:39a52a2a03a4faf0651640ac8a542059c52f2d04fc26c8b83e22b0a957ffedd3",
        input={
            "prompt": prompt,
            "negative_prompt": negative,
            "width": width,
            "height": height,
            "guidance_scale": guidance_scale,
        }
    )
    return output[0] if isinstance(output, list) else str(output)


url = generate_sd_with_negatives(
    prompt="portrait of a young woman in a garden, soft natural light, film photography",
    negative_categories=["universal", "portraits"],
)
print(f"Image generated: {url}")

Consistency: Seeds and Style References

Seeds in Stable Diffusion

The seed is the number that initializes the random generator. Same seed + same prompt = same image.

def generate_consistent_set(
    variations: list[str],
    seed: int = 42,
    style_suffix: str = "professional product photography, white background, studio lighting"
) -> list[dict]:
    results = []
    for variation in variations:
        full_prompt = f"{variation}, {style_suffix}"
        output = replicate.run(
            "stability-ai/sdxl:39a52a2a03a4faf0651640ac8a542059c52f2d04fc26c8b83e22b0a957ffedd3",
            input={
                "prompt": full_prompt,
                "negative_prompt": NEGATIVE_PROMPTS["universal"],
                "seed": seed,
                "width": 1024,
                "height": 1024,
            }
        )
        url = output[0] if isinstance(output, list) else str(output)
        results.append({"variation": variation, "url": url, "seed": seed})
    return results


images = generate_consistent_set([
    "red wireless headphones from front angle",
    "red wireless headphones from side angle",
    "red wireless headphones from top angle",
], seed=12345)

for img in images:
    print(f"  {img['variation']}: {img['url']}")

Style reference with DALL-E 3

DALL-E 3 doesn't have a public seed, but you can achieve consistency with very detailed style descriptions used as a constant prefix:

from openai import OpenAI

client = OpenAI()

STYLE_REFERENCE = (
    "minimalist flat illustration style, limited color palette of blue "
    "and orange on white background, clean geometric shapes, no outlines, "
    "subtle shadows, modern corporate aesthetic"
)


def generate_consistent_dalle(subjects: list[str], style_ref: str = STYLE_REFERENCE) -> list[dict]:
    results = []
    for subject in subjects:
        response = client.images.generate(
            model="dall-e-3",
            prompt=f"{subject}. Style: {style_ref}",
            size="1024x1024",
            quality="standard",
            n=1,
        )
        results.append({
            "subject": subject,
            "url": response.data[0].url,
            "revised_prompt": response.data[0].revised_prompt,
        })
    return results

Note: DALL-E 3 rewrites your prompt internally. The revised_prompt field shows what it actually used. Including very specific style instructions reduces the variation across images.

Prompt Style Comparison: DALL-E vs Stable Diffusion

Prompt adapter across providers

def adapt_prompt_for_provider(description: str, target_provider: str) -> str:
    system_prompts = {
        "dalle": (
            "Convert the description into a prompt for DALL-E 3. "
            "Natural language in English, 2-3 descriptive sentences. No tag format."
        ),
        "sd": (
            "Convert the description into a prompt for Stable Diffusion XL. "
            "Comma-separated tags in English. Maximum 30 tags. No complete sentences."
        ),
    }

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system_prompts[target_provider]},
            {"role": "user", "content": description},
        ],
        max_tokens=200,
        temperature=0.3,
    )
    return response.choices[0].message.content.strip()


description = "A mountain landscape at sunset with a lake in the foreground"
print(f"DALL-E: {adapt_prompt_for_provider(description, 'dalle')}")
print(f"SD: {adapt_prompt_for_provider(description, 'sd')}")

Iterative Refinement

In practice, the first prompt rarely produces the ideal result. Iterative refinement uses Vision to improve the prompt automatically.

import json
from openai import OpenAI

client = OpenAI()


def iterative_refinement(
    initial_prompt: str,
    max_iterations: int = 3,
    quality_threshold: float = 0.8,
) -> dict:
    current_prompt = initial_prompt
    history = []

    for iteration in range(max_iterations):
        response = client.images.generate(
            model="dall-e-3",
            prompt=current_prompt,
            size="1024x1024",
            response_format="b64_json",
            n=1,
        )
        b64_image = response.data[0].b64_json

        evaluation = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": (
                            f"Evaluate this image against: '{initial_prompt}'\n"
                            'Respond with JSON: {{"score": 0-1, "issues": [...], "suggestions": [...]}}'
                        ),
                    },
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64_image}"}},
                ],
            }],
            max_tokens=300,
        )

        try:
            eval_data = json.loads(evaluation.choices[0].message.content)
        except json.JSONDecodeError:
            eval_data = {"score": 0.5, "issues": [], "suggestions": []}

        history.append({
            "iteration": iteration + 1,
            "prompt": current_prompt,
            "score": eval_data.get("score", 0),
            "issues": eval_data.get("issues", []),
        })

        if eval_data.get("score", 0) >= quality_threshold:
            break

        suggestions = eval_data.get("suggestions", [])
        if suggestions:
            refinement = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {"role": "system", "content": "Improve the prompt by incorporating the suggestions. Return only the prompt."},
                    {"role": "user", "content": f"Prompt: {current_prompt}\nSuggestions: {suggestions}"},
                ],
                max_tokens=200,
            )
            current_prompt = refinement.choices[0].message.content.strip()

    return {"final_prompt": current_prompt, "iterations": len(history), "history": history}


result = iterative_refinement(
    "Minimalist logo for an AI company, blue and white colors",
    max_iterations=3,
    quality_threshold=0.85,
)
print(f"Iterations: {result['iterations']}")
for step in result["history"]:
    print(f"  Iteration {step['iteration']}: score={step['score']}")

Troubleshooting

Vague or generic results

Symptom: The image is generic, it doesn't have the style or composition you asked for.

Cause: Prompt too short or ambiguous. The models fill the gaps with averages from the dataset.

def diagnose_prompt(prompt: str) -> dict:
    checks = {
        "subject": len(prompt.split()) >= 3,
        "style": any(k in prompt.lower() for k in ["photography", "illustration", "painting", "style", "estilo"]),
        "lighting": any(k in prompt.lower() for k in ["light", "lighting", "luz", "shadow", "golden hour"]),
        "composition": any(k in prompt.lower() for k in ["close-up", "wide angle", "aerial", "centered"]),
        "mood": any(k in prompt.lower() for k in ["dramatic", "serene", "vibrant", "nostalgic", "warm"]),
    }
    missing = [k for k, v in checks.items() if not v]
    return {
        "completeness": sum(checks.values()) / len(checks),
        "missing": missing,
        "recommendation": f"Add: {', '.join(missing)}" if missing else "Complete prompt",
    }

Incorrect style

Symptom: You asked for "watercolor" but it looks like digital photography.

Solution: Put the style at the beginning and reinforce it with related terms:

def reinforce_style(prompt: str, style: str) -> str:
    reinforcements = {
        "watercolor": "watercolor painting, wet on wet technique, paper texture, soft bleeding edges",
        "oil_painting": "oil painting, visible brushstrokes, thick impasto, canvas texture, rich pigments",
        "photography": "DSLR photography, photorealistic, sharp focus, natural lighting",
        "vector": "flat vector illustration, clean edges, solid colors, no gradients, SVG style",
    }
    return f"{reinforcements.get(style, style)}, {prompt}"

Content policy rejection (DALL-E)

Symptom: The API rejects the prompt with a content policy error.

POLICY_SAFE_REPLACEMENTS = {
    "blood": "red liquid", "weapon": "tool", "gun": "device",
    "knife": "utensil", "fight": "competition", "dead": "fallen",
    "naked": "minimal clothing", "war": "conflict", "explosion": "burst of energy",
}


def sanitize_prompt(prompt: str) -> str:
    sanitized = prompt.lower()
    for unsafe, safe in POLICY_SAFE_REPLACEMENTS.items():
        sanitized = sanitized.replace(unsafe, safe)
    return sanitized

Exercises

Exercise 1: Expand a prompt with an LLM

Given a 3-5 word concept, use an LLM to expand it into a professional prompt with the 7 components. Return JSON with each component separated and the combined prompt.

import json
from openai import OpenAI

client = OpenAI()


def expand_to_professional_prompt(concept: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": (
                    "You are an expert in image prompt engineering. "
                    "Given a short concept, expand it into a professional prompt.\n"
                    'Respond with JSON: {"subject": "...", "style": "...", "medium": "...", '
                    '"lighting": "...", "composition": "...", "mood": "...", '
                    '"technical": "...", "full_prompt": "..."}\n'
                    "full_prompt in English, 2-3 sentences, combining all the components."
                ),
            },
            {"role": "user", "content": concept},
        ],
        max_tokens=300,
        temperature=0.7,
    )

    result = json.loads(response.choices[0].message.content)
    components = ["subject", "style", "medium", "lighting", "composition", "mood", "technical"]
    filled = sum(1 for k in components if result.get(k))
    result["completeness"] = filled / len(components)
    return result


result = expand_to_professional_prompt("futuristic tech office")
print(f"Prompt: {result['full_prompt']}")
print(f"Completeness: {result['completeness']:.0%}")

Exercise 2: Dual generator with automatic adaptation

Create a function that takes a description in Spanish and generates the image with DALL-E and SD, adapting the prompt to each one's optimal format. Return both URLs and the adapted prompts.

from openai import OpenAI
import replicate

client = OpenAI()


def generate_dual(description_es: str) -> dict:
    dalle_prompt = adapt_prompt_for_provider(description_es, "dalle")
    sd_prompt = adapt_prompt_for_provider(description_es, "sd")

    dalle_result = {"url": None, "error": None, "prompt_used": dalle_prompt}
    sd_result = {"url": None, "error": None, "prompt_used": sd_prompt}

    try:
        resp = client.images.generate(model="dall-e-3", prompt=dalle_prompt, size="1024x1024", n=1)
        dalle_result["url"] = resp.data[0].url
    except Exception as e:
        dalle_result["error"] = str(e)

    try:
        output = replicate.run(
            "stability-ai/sdxl:39a52a2a03a4faf0651640ac8a542059c52f2d04fc26c8b83e22b0a957ffedd3",
            input={"prompt": sd_prompt, "negative_prompt": NEGATIVE_PROMPTS["universal"],
                   "width": 1024, "height": 1024},
        )
        sd_result["url"] = output[0] if isinstance(output, list) else str(output)
    except Exception as e:
        sd_result["error"] = str(e)

    return {"original": description_es, "dalle": dalle_result, "sd": sd_result}


result = generate_dual("A friendly robot teaching programming to kids in a colorful classroom")
print(f"DALL-E: {result['dalle']['prompt_used']}")
print(f"SD: {result['sd']['prompt_used']}")

Exercise 3: Dynamic negative prompts system

Create a system that analyzes the positive prompt with an LLM and automatically generates the most relevant negative prompts, detecting the image category.

import json
from openai import OpenAI

client = OpenAI()


def generate_smart_negatives(prompt: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": (
                    "Analyze the image prompt and determine:\n"
                    "1. Category (portrait, landscape, product, ui_design, abstract)\n"
                    "2. Negative prompts specific to that category\n"
                    "3. Negative prompts based on the specific content\n"
                    'Respond with JSON: {"category": "...", "category_negatives": "...", '
                    '"specific_negatives": "...", "combined": "..."}'
                ),
            },
            {"role": "user", "content": prompt},
        ],
        max_tokens=300,
        temperature=0.2,
    )

    result = json.loads(response.choices[0].message.content)
    result["final_negative"] = f"{NEGATIVE_PROMPTS['universal']}, {result.get('combined', '')}"
    return result


neg = generate_smart_negatives("professional headshot of a business executive")
print(f"Category: {neg['category']}")
print(f"Negatives: {neg['final_negative'][:100]}...")

Exercise 4: Prompt optimizer with A/B testing

Generate two variations of a prompt (concise and detailed), generate images with both, evaluate with the Vision API, and return the winning version.

import json
from openai import OpenAI

client = OpenAI()


def ab_test_prompts(concept: str) -> dict:
    variations_resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": (
                    "Generate two image prompt variations:\n"
                    "- version_a: concise, 10-15 words\n"
                    "- version_b: detailed, 30-50 words\n"
                    "Both in English for DALL-E 3.\n"
                    'Respond with JSON: {"version_a": "...", "version_b": "..."}'
                ),
            },
            {"role": "user", "content": concept},
        ],
        max_tokens=200,
    )

    versions = json.loads(variations_resp.choices[0].message.content)
    results = {}

    for name, prompt_text in versions.items():
        img = client.images.generate(
            model="dall-e-3", prompt=prompt_text, size="1024x1024",
            response_format="b64_json", n=1,
        )
        b64 = img.data[0].b64_json

        ev = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": f"Evaluate for '{concept}'. JSON: {{\"overall\": 0-1}}"},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
                ],
            }],
            max_tokens=50,
        )

        scores = json.loads(ev.choices[0].message.content)
        results[name] = {"prompt": prompt_text, "overall": scores.get("overall", 0)}

    winner = max(results, key=lambda k: results[k]["overall"])
    return {"winner": winner, "results": results}


test = ab_test_prompts("analytics dashboard for an AI startup")
print(f"Winner: {test['winner']}")
for name, data in test["results"].items():
    print(f"  {name}: overall={data['overall']}, prompt={data['prompt'][:60]}...")

Additional Resources

1. OpenAI Image Generation Guide — Official DALL-E 3 documentation
2. Stable Diffusion Prompt Guide — Complete prompt guide for SD
3. Lexica.art — Stable Diffusion prompt search engine with results
4. PromptHero — Prompt gallery for multiple models
5. DALL-E 3 Cookbook — OpenAI Cookbook