Rework prompts and captioning systems for ideogram to more strictly match the format provided by ideogram.

extensions_built_in/captioner/prompts/ideogram4_caption_prompt.py

@@ -19,16 +19,16 @@ You are CAPTIONING a real image, not imagining one. Describe ONLY what is visibl
 ## OUTPUT CONTRACT — exactly three top-level keys, in this order:
 
 ```json
-{"aspect_ratio":"W:H","high_level_description":"...","compositional_deconstruction":{"background":"...","elements":[ ... ]}}
+{"high_level_description":"...","style_description":{"aesthetics":"...","lighting":"...","photo":"...","medium":"...","color_palette":["#RRGGBB"]},"compositional_deconstruction":{"background":"...","elements":[ ... ]}}
 ```
 
 - Emit a SINGLE-LINE MINIFIED JSON object — no markdown fences, no commentary, no other top-level keys.
 - Preserve non-ASCII characters as-is (CJK, Cyrillic, Devanagari, Arabic, accented Latin). Never escape with `\\uNNNN`, transliterate, or replace `café` with `cafe`.
 - Use SINGLE quotes for embedded text references in prose fields (`'Joe's Diner'`, not `\\"Joe's Diner\\"`). The `text` field of text elements is the exception — that field holds the verbatim characters visible in the image, may use any characters, and follows QUOTED SPAN FIDELITY below.
 
-### `aspect_ratio` (first field, always required)
+### Target aspect ratio (input only — never emit it)
 
-The exact target aspect ratio is GIVEN to you in the user message as `W:H`. Echo it VERBATIM. Never recompute it, never emit `auto`, never override it from your own reading of the frame. Every bbox you emit is normalized to THIS aspect ratio.
+The user message gives the image's aspect ratio as `W:H`. Use it ONLY to size your bounding boxes correctly (a box is square only on a square frame). Do NOT emit an `aspect_ratio` key — it is not part of the output.
 
 ### `high_level_description` — observational summary (50-word hard cap)
 
@@ -42,19 +42,24 @@ The exact target aspect ratio is GIVEN to you in the user message as `W:H`. Echo
 GOOD: `A full-action shot of a male soccer player in a red kit and black Adidas cleats kicking a soccer ball on a green turf field, with a blurred crowd in the stadium background.`
 BAD (over-specifies): `A male soccer player captured mid-kick on a bright green grass pitch, right leg fully extended through the follow-through at the precise moment his black-and-white studded boot makes contact with a white-and-black size-5 ball...`
 
-## IDENTIFY THE MEDIUM
+## STYLE DESCRIPTION — the `style_description` block (always required)
 
-State the medium accurately in HLD/background prose as natural framing: `photograph | illustration | 3D render | graphic design` (and the specific style when obvious — `35mm film photograph`, `flat vector illustration`, `Pixar-style 3D render`, `watercolor illustration`). Read it from the image; do not impose a default. Name a recognizable style ONCE, briefly — do not append invented technique detail.
+A nested object capturing the image's overall look, OBSERVED from the image (never invented). Exactly these five keys:
+- `aesthetics` — the overall mood/aesthetic in a short phrase (`Cinematic, minimal, serene.` / `Bright, playful, high-energy.`).
+- `lighting` — the actual lighting: direction, quality, contrast, and the colour of the light. Describe a warm-coloured source concretely (`amber pool from a candle`) but never use the bare word `warm` as a grade.
+- `photo` — the medium-specific capture/render spec. Photograph → camera/film look, framing, grain, focus (`35mm film still, 16:9 framing, subtle grain, shallow depth of field`). Other media → the rendering technique (`flat vector, clean edges` / `octane 3D render, soft global illumination` / `loose watercolor on textured paper`).
+- `medium` — one short phrase: `Photograph.` / `Illustration.` / `3D render.` / `Graphic design.` Read it from the image; do not impose a default.
+- `color_palette` — an array of the image's DOMINANT colours as hex strings (`"#1B3A5C"`), up to 16, ordered most → least dominant. Sample the colours actually present; do not invent colours that are not there.
 
 ## ELEMENTS — what they are, what they're not
 
 Each element is one of:
 ```
-{"type":"obj","bbox":[x1,y1,x2,y2],"desc":"..."}
-{"type":"text","bbox":[x1,y1,x2,y2],"text":"LINE ONE\\nLINE TWO","desc":"..."}
+{"type":"obj","bbox":[x1,y1,x2,y2],"color_palette":["#RRGGBB"],"desc":"..."}
+{"type":"text","bbox":[x1,y1,x2,y2],"color_palette":["#RRGGBB"],"text":"LINE ONE\\nLINE TWO","desc":"..."}
 ```
 
-`bbox` is optional per-element (see BBOX section below).
+`bbox` and `color_palette` are both OPTIONAL per-element. `bbox`: see BBOX section below. `color_palette`: up to 5 hex strings of that element's own dominant colours — include it when the element has distinctive colours worth pinning (a red jacket, a brand logo, coloured text), omit it for colour-neutral elements.
 
 ### SINGLE SUBJECT = SINGLE ELEMENT
 

extensions_built_in/captioner/prompts/ideogram4_prompt.py

@@ -10,19 +10,16 @@ You convert a natural-language user idea into a structured JSON caption an image
 ## OUTPUT CONTRACT — exactly three top-level keys, in this order:
 
 ```json
-{"aspect_ratio":"W:H","high_level_description":"...","compositional_deconstruction":{"background":"...","elements":[ ... ]}}
+{"high_level_description":"...","style_description":{"aesthetics":"...","lighting":"...","photo":"...","medium":"...","color_palette":["#RRGGBB"]},"compositional_deconstruction":{"background":"...","elements":[ ... ]}}
 ```
 
 - Emit a SINGLE-LINE MINIFIED JSON object — no markdown fences, no commentary, no other top-level keys.
 - Preserve non-ASCII characters as-is (CJK, Cyrillic, Devanagari, Arabic, accented Latin). Never escape with `\uNNNN`, transliterate, or replace `café` with `cafe`.
 - Use SINGLE quotes for embedded text references in prose fields (`'Joe's Diner'`, not `\"Joe's Diner\"`). The `text` field of text elements is the exception — that field holds the user's verbatim characters, may use any characters, and follows QUOTED SPAN FIDELITY below.
 
-### `aspect_ratio` (first field, always required)
+### Target aspect ratio (input only — never emit it)
 
-A string in `W:H` form with positive integers (`1:1`, `16:9`, `9:16`, `4:5`, `3:1`, `2:3`, etc.).
-- If the user message gives a concrete `W:H`, echo it verbatim.
-- If the user message says `auto`, pick a concrete ratio that matches the medium and composition (panoramic subjects → wide ratios like `16:9` or `3:1`; portrait subjects → tall like `9:16` or `4:5`; designed artifacts → format conventions like `2:3` book cover, `3:4` poster; ambiguous → `1:1`). NEVER emit the literal string `auto`.
-- The aspect ratio you commit to drives every bbox decision. Pick it first.
+The user message gives a target aspect ratio as `W:H` (or `auto`). Use it ONLY to drive your bounding-box decisions — a box is square only on a square frame, so the ratio shapes every bbox. Do NOT emit an `aspect_ratio` key; it is not part of the output.
 
 ### `high_level_description` — observational summary (50-word hard cap)
 
@@ -36,15 +33,26 @@ A string in `W:H` form with positive integers (`1:1`, `16:9`, `9:16`, `4:5`, `3:
 GOOD: `A full-action shot of a male soccer player in a red kit and black Adidas cleats kicking a soccer ball on a green turf field, with a blurred crowd in the stadium background.`
 BAD (over-specifies): `A male soccer player captured mid-kick on a bright green grass pitch, right leg fully extended through the follow-through at the precise moment his black-and-white studded boot makes contact with a white-and-black size-5 ball...`
 
+### `style_description` — the global look block (always required)
+
+A nested object with exactly these five keys:
+- `aesthetics` — overall mood/aesthetic in a short phrase (`Cinematic, minimal, serene.`).
+- `lighting` — direction, quality, contrast, and colour of the light. Describe a warm-coloured source concretely (`amber sun low at the horizon`); never use the bare word `warm` as a grade.
+- `photo` — the medium-specific capture/render spec. Photograph → camera/film look, framing, grain, focus (`35mm motion-picture film still, 16:9 framing, subtle grain`). Other media → the rendering technique (`flat vector, clean edges`; `octane 3D render`; `loose watercolor on textured paper`).
+- `medium` — one short phrase: `Photograph.` / `Illustration.` / `3D render.` / `Graphic design.`
+- `color_palette` — an array of the dominant colours as hex strings (`"#1B3A5C"`), up to 16, ordered most → least dominant. This conditions the image's colours directly, so commit to the actual hexes you intend.
+
+Name a recognized style ONCE here (see PLANNING → Style commitment); do not append invented technique detail on top of a well-known style name.
+
 ## ELEMENTS — what they are, what they're not
 
 Each element is one of:
 ```
-{"type":"obj","bbox":[y1,x1,y2,x2],"desc":"..."}
-{"type":"text","bbox":[y1,x1,y2,x2],"text":"LINE ONE\nLINE TWO","desc":"..."}
+{"type":"obj","bbox":[y1,x1,y2,x2],"color_palette":["#RRGGBB"],"desc":"..."}
+{"type":"text","bbox":[y1,x1,y2,x2],"color_palette":["#RRGGBB"],"text":"LINE ONE\nLINE TWO","desc":"..."}
 ```
 
-`bbox` is optional per-element (see BBOX section below).
+`bbox` and `color_palette` are both OPTIONAL per-element. `bbox`: see BBOX section below. `color_palette`: up to 5 hex strings steering that element's own dominant colours — include it when the element has a distinctive colour (a red jacket, a brand logo, coloured text), omit it otherwise.
 
 ### SINGLE SUBJECT = SINGLE ELEMENT
 

extensions_built_in/captioner/prompts/ideogram4_upsample_prompt.py

@@ -17,29 +17,40 @@ You convert a user prompt into a structured JSON caption an image renderer can c
 ## OUTPUT CONTRACT — exactly three top-level keys, in this order:
 
 ```json
-{"aspect_ratio":"W:H","high_level_description":"...","compositional_deconstruction":{"background":"...","elements":[ ... ]}}
+{"high_level_description":"...","style_description":{"aesthetics":"...","lighting":"...","photo":"...","medium":"...","color_palette":["#RRGGBB"]},"compositional_deconstruction":{"background":"...","elements":[ ... ]}}
 ```
 
 - Emit a SINGLE-LINE MINIFIED JSON object — no markdown fences, no commentary, no other top-level keys.
 - Preserve non-ASCII characters as-is (CJK, Cyrillic, Arabic, accented Latin). Never escape them as unicode code-point sequences or transliterate.
 - Use SINGLE quotes for embedded text references in prose fields (`'Joe's Diner'`). The `text` field is the exception — it holds verbatim characters.
 
-### `aspect_ratio` (first field)
+### Target aspect ratio (input only — never emit it)
 
-The target ratio is given. Echo it VERBATIM. If it is `auto`, pick a concrete `W:H` that fits the composition (portrait subject → tall, panoramic → wide, ambiguous → `1:1`). Never emit `auto`.
+The user message gives a target aspect ratio as `W:H` (or `auto`). Use it ONLY to size your bounding boxes correctly (a box is square only on a square frame). Do NOT emit an `aspect_ratio` key — it is not part of the output.
 
 ### `high_level_description` (50-word cap)
 
 One short sentence, reads like a natural prompt, starts with the subject — no "this image shows". Names the subject(s), any trigger/name verbatim, and the overall composition. Don't enumerate fine detail.
 
+## STYLE DESCRIPTION — the `style_description` block (always required)
+
+A nested object with exactly these five keys, filled FROM the prompt:
+- `aesthetics` — the overall mood/aesthetic in a short phrase.
+- `lighting` — the lighting (direction, quality, colour). Describe a warm-coloured source concretely; never use the bare word `warm` as a grade.
+- `photo` — the medium-specific capture/render spec (photograph → camera/film look, framing, grain, focus; other media → the rendering technique).
+- `medium` — one short phrase: `Photograph.` / `Illustration.` / `3D render.` / `Graphic design.`
+- `color_palette` — an array of dominant colours as hex strings (`"#1B3A5C"`), up to 16, ordered most → least dominant.
+
+Respect FIDELITY: if the prompt NAMES a style, medium, artist, or look, put it in these fields BY NAME (e.g. `medium`/`photo`/`aesthetics`) and do NOT invent its characteristics. Pull lighting and colours from what the prompt states. In faithful mode, only commit to a value the prompt implies, keeping the rest minimal; in creative mode you may infer fitting style values — but never elaborate a named style and never override what the user gave.
+
 ## ELEMENTS
 
 Each element is one of:
 ```
-{"type":"obj","bbox":[y1,x1,y2,x2],"desc":"..."}
-{"type":"text","bbox":[y1,x1,y2,x2],"text":"LINE ONE\nLINE TWO","desc":"..."}
+{"type":"obj","bbox":[y1,x1,y2,x2],"color_palette":["#RRGGBB"],"desc":"..."}
+{"type":"text","bbox":[y1,x1,y2,x2],"color_palette":["#RRGGBB"],"text":"LINE ONE\nLINE TWO","desc":"..."}
 ```
-`bbox` is optional per element (see BBOX).
+`bbox` and `color_palette` are both OPTIONAL per element. `bbox`: see BBOX. `color_palette`: up to 5 hex strings of that element's dominant colours — include it when the prompt gives the element a distinctive colour (a red jacket, coloured text), otherwise omit.
 
 - **One coherent subject = ONE element.** A person, animal, vehicle, building, or plant is a single element; its parts are attributes of that element's `desc`, never separate elements. Multiple distinct subjects = multiple elements (one each).
 - **`desc`:** identity first, then only the attributes the user gave (or that the structure plainly needs). For a named person/trigger: name + action/pose/placement ONLY, no appearance. For a generic un-named subject, you may state the concrete attributes the prompt implies, but do not invent an identity or backstory.