6. Editing and Variations

Description

Generating images from scratch is only the first step. In production, most of the work with images is editing: replacing a product's background, adding a missing element, expanding an image for a different format, generating variations of an approved concept. These operations — inpainting, outpainting, variations, img2img — are what turn image generation into a practical tool.

Why it matters: An e-commerce site doesn't generate each product photo from scratch. It generates a base and then edits: it changes the background for the season, replaces the product's color, expands the image for banners. A marketing agency doesn't create 10 concepts from scratch — it generates one good one and produces variations. Mastering editing reduces costs, improves consistency and speeds up the workflows.

Connection with the module: In capsules 02 and 03 you used the generation APIs. In capsule 05 you optimized the prompts. Now you'll learn to edit what you already generated. These techniques integrate directly into the pipelines of capsule 07 and into the final project (capsule 08).

Key Concepts

Inpainting

Replacing a specific area of an image using a mask. The mask defines which area is modified (transparent) and which is preserved (opaque). The model generates new content only in the masked area.

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Base image   │  +  │   Mask       │  →  │ Cat on a     │
│ [cat on sofa]│     │ [sofa transp]│     │ leather      │
│              │     │              │     │ armchair     │
└──────────────┘     └──────────────┘     └──────────────┘

Outpainting

Expanding the image beyond its original borders. Useful for adapting images to different formats (square → panoramic).

Variations

Generating new images that keep the visual essence of a reference. They aren't copies — they're reinterpretations of the same concept.

img2img (Image-to-Image)

Taking an existing image and transforming it based on a prompt. The strength parameter controls how much it deviates from the original (0.0 = identical, 1.0 = ignores the original).

Creating Masks with Pillow

Rectangular mask

from PIL import Image, ImageDraw


def create_rectangular_mask(
    width: int, height: int, x1: int, y1: int, x2: int, y2: int,
    output_path: str = "mask.png",
) -> str:
    mask = Image.new("RGBA", (width, height), (0, 0, 0, 255))
    draw = ImageDraw.Draw(mask)
    draw.rectangle([x1, y1, x2, y2], fill=(0, 0, 0, 0))
    mask.save(output_path)
    return output_path


create_rectangular_mask(1024, 1024, 200, 200, 800, 600)

Circular mask

def create_circular_mask(
    width: int, height: int, center_x: int, center_y: int, radius: int,
    output_path: str = "mask_circle.png",
) -> str:
    mask = Image.new("RGBA", (width, height), (0, 0, 0, 255))
    draw = ImageDraw.Draw(mask)
    draw.ellipse(
        [center_x - radius, center_y - radius, center_x + radius, center_y + radius],
        fill=(0, 0, 0, 0),
    )
    mask.save(output_path)
    return output_path

Mask from a color region

To mask areas based on their color (e.g. the white background of a product photo):

import numpy as np
from PIL import Image


def create_mask_from_color(
    image_path: str,
    target_color: tuple[int, int, int],
    tolerance: int = 30,
    output_path: str = "mask_color.png",
) -> str:
    img = Image.open(image_path).convert("RGB")
    img_array = np.array(img)
    target = np.array(target_color)
    distance = np.sqrt(np.sum((img_array.astype(float) - target.astype(float)) ** 2, axis=2))

    mask_array = np.ones((*img_array.shape[:2], 4), dtype=np.uint8) * 255
    mask_array[distance < tolerance] = [0, 0, 0, 0]

    Image.fromarray(mask_array, "RGBA").save(output_path)
    return output_path

Mask with a soft edge (feathering)

Hard edges produce abrupt transitions. Feathering softens them for a natural result:

from PIL import Image, ImageDraw, ImageFilter


def create_feathered_mask(
    width: int, height: int, x1: int, y1: int, x2: int, y2: int,
    feather_radius: int = 20, output_path: str = "mask_feathered.png",
) -> str:
    alpha = Image.new("L", (width, height), 255)
    draw = ImageDraw.Draw(alpha)
    draw.rectangle([x1, y1, x2, y2], fill=0)
    alpha = alpha.filter(ImageFilter.GaussianBlur(radius=feather_radius))

    mask = Image.new("RGBA", (width, height))
    mask.putalpha(alpha)
    mask.save(output_path)
    return output_path

Inpainting with DALL-E 2 (OpenAI)

API requirements

• Image: PNG, square, under 4 MB
• Mask: PNG with alpha channel, same dimensions
• Transparent areas = areas to edit
• Opaque areas = areas to preserve

Implementation

from openai import OpenAI
from pathlib import Path

client = OpenAI()


def inpaint_dalle(
    image_path: str, mask_path: str, prompt: str,
    size: str = "1024x1024", n: int = 1,
) -> list[str]:
    with open(image_path, "rb") as f_img, open(mask_path, "rb") as f_mask:
        response = client.images.edit(
            image=f_img, mask=f_mask, prompt=prompt, n=n, size=size,
        )
    return [item.url for item in response.data]


urls = inpaint_dalle(
    "room_photo.png", "furniture_mask.png",
    "a modern minimalist wooden desk with a laptop and a small plant", n=2,
)
for i, url in enumerate(urls):
    print(f"Variation {i + 1}: {url}")

Prepare the image for the API

from PIL import Image


def prepare_image_for_edit(image_path: str, target_size: int = 1024) -> str:
    output_path = str(Path(image_path).with_suffix(".prepared.png"))
    img = Image.open(image_path).convert("RGBA")

    w, h = img.size
    side = max(w, h)
    square = Image.new("RGBA", (side, side), (0, 0, 0, 0))
    square.paste(img, ((side - w) // 2, (side - h) // 2))
    square = square.resize((target_size, target_size), Image.LANCZOS)
    square.save(output_path, "PNG")
    return output_path

Variations with DALL-E 2

Variations generate similar images without needing a prompt:

from openai import OpenAI
import urllib.request

client = OpenAI()


def create_variations(image_path: str, n: int = 3, size: str = "1024x1024") -> list[dict]:
    with open(image_path, "rb") as f:
        response = client.images.create_variation(image=f, n=n, size=size)
    return [{"index": i, "url": item.url} for i, item in enumerate(response.data)]


def download_image(url: str, output_path: str) -> str:
    urllib.request.urlretrieve(url, output_path)
    return output_path

Variations with automatic evaluation

import json
import base64
import urllib.request
from openai import OpenAI

client = OpenAI()


def best_variation(image_path: str, criteria: str, n_variations: int = 3) -> dict:
    variations = create_variations(image_path, n=n_variations)
    scored = []

    for var in variations:
        img_data = urllib.request.urlopen(var["url"]).read()
        b64 = base64.b64encode(img_data).decode("utf-8")

        eval_resp = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": f"Evaluate according to: '{criteria}'. JSON: {{\"score\": 0-1, \"reason\": \"...\"}}"},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
                ],
            }],
            max_tokens=100,
        )

        try:
            evaluation = json.loads(eval_resp.choices[0].message.content)
        except json.JSONDecodeError:
            evaluation = {"score": 0.5, "reason": "Parse error"}

        scored.append({**var, **evaluation})

    scored.sort(key=lambda x: x.get("score", 0), reverse=True)
    return {"best": scored[0], "all_scores": [(s["index"], s.get("score", 0)) for s in scored]}

img2img with Stable Diffusion (Replicate)

prompt_strength controls the transformation: 0.0 = identical, 0.5 = keeps structure, changes style, 1.0 = generates from scratch.

import replicate
import base64


def img2img_sd(
    image_path: str, prompt: str,
    negative_prompt: str = "blurry, low quality, distorted",
    prompt_strength: float = 0.6,
    guidance_scale: float = 7.5,
) -> str:
    with open(image_path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode("utf-8")

    output = replicate.run(
        "stability-ai/sdxl:39a52a2a03a4faf0651640ac8a542059c52f2d04fc26c8b83e22b0a957ffedd3",
        input={
            "image": f"data:image/png;base64,{b64}",
            "prompt": prompt,
            "negative_prompt": negative_prompt,
            "prompt_strength": prompt_strength,
            "guidance_scale": guidance_scale,
        },
    )
    return output[0] if isinstance(output, list) else str(output)


url = img2img_sd(
    "sketch.png",
    "detailed digital illustration, fantasy landscape, vivid colors, professional concept art",
    prompt_strength=0.7,
)
print(f"Result: {url}")

Explore transformation levels

def explore_strength_levels(
    image_path: str, prompt: str,
    strengths: list[float] = None,
) -> list[dict]:
    if strengths is None:
        strengths = [0.3, 0.5, 0.7, 0.9]

    results = []
    for strength in strengths:
        url = img2img_sd(image_path, prompt, prompt_strength=strength)
        label = "subtle" if strength < 0.4 else "moderate" if strength < 0.7 else "strong"
        results.append({"strength": strength, "url": url, "description": label})
    return results

Inpainting with Stable Diffusion (Replicate)

import replicate
import base64


def inpaint_sd(
    image_path: str, mask_path: str, prompt: str,
    negative_prompt: str = "blurry, low quality, distorted, artifacts",
    guidance_scale: float = 7.5,
    num_inference_steps: int = 30,
) -> str:
    with open(image_path, "rb") as f:
        img_b64 = base64.b64encode(f.read()).decode("utf-8")
    with open(mask_path, "rb") as f:
        mask_b64 = base64.b64encode(f.read()).decode("utf-8")

    output = replicate.run(
        "stability-ai/stable-diffusion-inpainting:95b7223104132402a9ae91cc677285bc5eb997834bd2349fa486f53910fd68b3",
        input={
            "image": f"data:image/png;base64,{img_b64}",
            "mask": f"data:image/png;base64,{mask_b64}",
            "prompt": prompt,
            "negative_prompt": negative_prompt,
            "guidance_scale": guidance_scale,
            "num_inference_steps": num_inference_steps,
        },
    )
    return output[0] if isinstance(output, list) else str(output)

Outpainting: Expanding Images

Outpainting = create a larger canvas + place the original image + mask the empty areas + inpainting.

from PIL import Image


def prepare_outpaint(
    image_path: str,
    expand_left: int = 0, expand_right: int = 0,
    expand_top: int = 0, expand_bottom: int = 0,
    output_prefix: str = "outpaint",
) -> dict:
    img = Image.open(image_path).convert("RGBA")
    orig_w, orig_h = img.size
    new_w = orig_w + expand_left + expand_right
    new_h = orig_h + expand_top + expand_bottom

    canvas = Image.new("RGBA", (new_w, new_h), (255, 255, 255, 255))
    canvas.paste(img, (expand_left, expand_top))

    mask = Image.new("RGBA", (new_w, new_h), (0, 0, 0, 0))
    opaque = Image.new("RGBA", (orig_w, orig_h), (0, 0, 0, 255))
    mask.paste(opaque, (expand_left, expand_top))

    canvas_path = f"{output_prefix}_canvas.png"
    mask_path = f"{output_prefix}_mask.png"
    canvas.save(canvas_path)
    mask.save(mask_path)

    return {"canvas_path": canvas_path, "mask_path": mask_path, "new_size": (new_w, new_h)}


outpaint_data = prepare_outpaint("portrait_square.png", expand_left=256, expand_right=256)
result_url = inpaint_sd(
    outpaint_data["canvas_path"], outpaint_data["mask_path"],
    "continuation of the scene, matching style and lighting, seamless extension",
)

Use Case: Background Replacement

def replace_product_background(
    product_image: str, new_background_prompt: str, provider: str = "dalle",
) -> dict:
    mask_path = create_mask_from_color(product_image, target_color=(255, 255, 255), tolerance=40)

    if provider == "dalle":
        prepared = prepare_image_for_edit(product_image)
        urls = inpaint_dalle(prepared, mask_path, new_background_prompt)
        return {"url": urls[0], "provider": "dalle"}
    else:
        url = inpaint_sd(product_image, mask_path, new_background_prompt)
        return {"url": url, "provider": "sd"}


backgrounds = [
    "modern kitchen counter, marble surface, soft natural light",
    "outdoor garden table with flowers, sunny day",
    "minimalist studio, gradient gray background",
]
for bg in backgrounds:
    result = replace_product_background("product_white_bg.png", bg)
    print(f"  {bg[:40]}... → {result['url']}")

Capability Comparison by Provider

Troubleshooting

Mask doesn't work with DALL-E

Symptom: The edit endpoint ignores the mask or edits the whole image.

Cause: Mask without a correct alpha channel or with different dimensions.

from PIL import Image


def validate_mask(image_path: str, mask_path: str) -> dict:
    img = Image.open(image_path)
    mask = Image.open(mask_path)
    issues = []

    if img.size != mask.size:
        issues.append(f"Dimensions: image={img.size}, mask={mask.size}")
    if mask.mode != "RGBA":
        issues.append(f"Mask is not RGBA (it's {mask.mode})")
    if mask.mode == "RGBA":
        alpha_values = set(mask.split()[3].getdata())
        if 0 not in alpha_values:
            issues.append("No transparent areas (nothing to edit)")

    return {"valid": len(issues) == 0, "issues": issues}

img2img produces very different results

Cause: prompt_strength > 0.8 or guidance_scale too high.

Solution: Reduce prompt_strength to 0.3-0.5 and guidance_scale to 5-7.

Visible seams in inpainting

Cause: Mask with hard edges or a lighting difference.

Solution: Use feathered masks and prompts that describe coherent lighting:

def fix_seams(image_path: str, mask_bbox: tuple, prompt: str) -> str:
    w, h = Image.open(image_path).size
    feathered = create_feathered_mask(w, h, *mask_bbox, feather_radius=40)
    enhanced = f"{prompt}, matching lighting and color temperature, seamless blend"
    return inpaint_sd(image_path, feathered, enhanced, num_inference_steps=50)

Exercises

Exercise 1: Multi-background product editor

Create a system that takes a product photo with a white background and generates 3 versions with different backgrounds (studio, outdoor, lifestyle). It must create the mask automatically and return the URLs with metadata.

import numpy as np
from PIL import Image
from openai import OpenAI

client = OpenAI()

BACKGROUND_SCENES = {
    "studio": "clean studio, soft gradient background, professional product lighting",
    "outdoor": "natural outdoor, wooden table, soft bokeh green garden, golden hour",
    "lifestyle": "modern living room, design furniture, warm ambient light",
}


def auto_mask_white_bg(image_path: str, tolerance: int = 35) -> str:
    img = Image.open(image_path).convert("RGB")
    arr = np.array(img)
    white = np.array([255, 255, 255])
    distance = np.sqrt(np.sum((arr.astype(float) - white.astype(float)) ** 2, axis=2))

    mask_arr = np.ones((*arr.shape[:2], 4), dtype=np.uint8) * 255
    mask_arr[distance < tolerance] = [0, 0, 0, 0]
    mask = Image.fromarray(mask_arr, "RGBA")
    mask.save("auto_mask.png")
    return "auto_mask.png"


def multi_background_product(image_path: str) -> list[dict]:
    mask_path = auto_mask_white_bg(image_path)
    prepared = prepare_image_for_edit(image_path)
    results = []

    for scene_name, scene_prompt in BACKGROUND_SCENES.items():
        try:
            urls = inpaint_dalle(prepared, mask_path, f"product on {scene_prompt}", n=1)
            results.append({"scene": scene_name, "url": urls[0], "success": True})
        except Exception as e:
            results.append({"scene": scene_name, "url": None, "success": False, "error": str(e)})

    return results


product_results = multi_background_product("headphones_white_bg.png")
for r in product_results:
    print(f"  {r['scene']}: {'OK' if r['success'] else r.get('error', '')}")

Exercise 2: Variations pipeline with ranking

Generate 4 variations of an image with DALL-E 2, evaluate each one with GPT-4o Vision according to a given criterion, sort by score and return the ranking.

import json
import base64
import urllib.request
from openai import OpenAI

client = OpenAI()


def ranked_variations(image_path: str, criteria: str, n: int = 4) -> dict:
    with open(image_path, "rb") as f:
        response = client.images.create_variation(image=f, n=n, size="1024x1024")

    variations = [{"index": i, "url": item.url} for i, item in enumerate(response.data)]
    evaluated = []

    for var in variations:
        img_bytes = urllib.request.urlopen(var["url"]).read()
        b64 = base64.b64encode(img_bytes).decode("utf-8")

        ev = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": (
                        f"Evaluate according to: '{criteria}'\n"
                        'JSON: {{"score": 0-1, "strengths": [...], "weaknesses": [...]}}'
                    )},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
                ],
            }],
            max_tokens=200,
        )

        try:
            scores = json.loads(ev.choices[0].message.content)
        except json.JSONDecodeError:
            scores = {"score": 0.5, "strengths": [], "weaknesses": ["Parse error"]}

        evaluated.append({**var, **scores})

    evaluated.sort(key=lambda x: x.get("score", 0), reverse=True)

    return {
        "ranking": [
            {"rank": i + 1, "index": e["index"], "score": e.get("score", 0),
             "strengths": e.get("strengths", []), "url": e["url"]}
            for i, e in enumerate(evaluated)
        ],
        "best_url": evaluated[0]["url"],
    }


ranking = ranked_variations("brand_concept.png", "professionalism for a corporate site")
for item in ranking["ranking"]:
    print(f"  #{item['rank']}: score={item['score']}, strengths={item['strengths']}")

Exercise 3: Style transfer with img2img and comparison

Given an image and several artistic styles, generate img2img versions with SD, evaluate which best preserves the original composition with the Vision API.

import json
import base64
import urllib.request
from openai import OpenAI

client = OpenAI()

STYLES = {
    "watercolor": "watercolor painting, soft washes, paper texture, translucent colors",
    "oil_painting": "classical oil painting, visible brushstrokes, rich colors, canvas texture",
    "anime": "anime illustration, vibrant colors, clean lines, Studio Ghibli inspired",
    "pencil": "detailed pencil sketch, graphite on paper, cross-hatching",
}


def style_transfer_comparison(image_path: str, strength: float = 0.65) -> dict:
    with open(image_path, "rb") as f:
        original_b64 = base64.b64encode(f.read()).decode("utf-8")

    results = []
    for style_name, style_prompt in STYLES.items():
        url = img2img_sd(image_path, style_prompt, prompt_strength=strength)
        styled_bytes = urllib.request.urlopen(url).read()
        styled_b64 = base64.b64encode(styled_bytes).decode("utf-8")

        ev = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": (
                        "Compare original (1st) vs stylized (2nd).\n"
                        'JSON: {{"composition_preserved": 0-1, "style_quality": 0-1, "overall": 0-1}}'
                    )},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{original_b64}"}},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{styled_b64}"}},
                ],
            }],
            max_tokens=100,
        )

        try:
            scores = json.loads(ev.choices[0].message.content)
        except json.JSONDecodeError:
            scores = {"composition_preserved": 0.5, "style_quality": 0.5, "overall": 0.5}

        results.append({"style": style_name, "url": url, "scores": scores})

    results.sort(key=lambda x: x["scores"].get("overall", 0), reverse=True)
    return {"results": results, "best_style": results[0]["style"]}


comp = style_transfer_comparison("landscape_photo.png")
print(f"Best style: {comp['best_style']}")
for r in comp["results"]:
    s = r["scores"]
    print(f"  {r['style']}: comp={s.get('composition_preserved')}, "
          f"style={s.get('style_quality')}, overall={s.get('overall')}")

Additional Resources

1. OpenAI Image Edits — DALL-E 2 editing endpoint
2. OpenAI Variations — Variations endpoint
3. Replicate Image Editing — Editing models
4. Pillow Documentation — Image manipulation
5. Stable Diffusion img2img — img2img guide