Silent Ship, Then Three Quality Lifters

🔍 What Was Supposed To Happen

Orchestra v2 has a Loop-Judge worker. When the score lands in the ambiguous band — not great, not bad — the judge reads the scorecard, picks one or two phases worth re-running, and writes its verdict to a file. The runner reads that verdict and resumes the run from the chosen phase.

That’s the loop. It’s how a build climbs from “passes” to “ships.”

💥 What Actually Happened

The first time we ran a hard task through v2 — “make the most advanced thing you can possibly make” — the judge picked the new Polish phase as the rerun target. It wrote ["polish"] in the verdict. Lowercase.

The runner’s phaseIndexForRerun did strict equality against the canonical name Polish. The match failed. The list of resumable phases came back empty. The runner shrugged and shipped the first-pass build.

🐛 The Bug In One Line

The original match looked like this:

const idx = phaseRoute.findIndex(r => r.name === p);

Strict equality. Case-sensitive. Only matched against name, never the slug. If the judge said anything other than the exact display name, the array index came back empty and the runner walked.

Worse: the empty result wasn’t treated as an error. It was treated as “nothing to rerun” — which meant the runner happily marked the build as done.

🔧 The Handoff Fix

We replaced that line with a real resolver. It does three things the old match didn’t.

• Normalises casing on both sides — Polish, polish, and POLISH all match.
• Accepts the display name OR the slug. The judge can write Builder or build; both resolve.
• Returns the canonical names so downstream filters (like “which phases have I completed?”) get the right strings every time.

And critically: when the judge writes something we can’t resolve at all — an invented phase name, a typo, anything — the runner falls back to [Builder, Verifier] and flags the fallback in the run log. Better to do the safe rerun than ship silently.

🎯 The Bigger Issue

The bug was real but small. The bigger issue showed up when we looked at why the build had needed a loop in the first place.

The black hole rendered. Tests passed. Composite was 0.86. But it wasn’t the most advanced thing you can possibly make. It was technically fine but pedestrian. None of the existing scorers caught that. The loop wasn’t going to find ambition gaps even if it had fired — the rubric didn’t ask.

So we added three changes on top of the bug fix — one to the rubric, one to the brief, one new phase.

✨ Lifter 1: The Ambition Dim

A new scorecard dimension called ambition. It asks: does this feel like the version a senior would demo at a conference, or the version that compiles?

It’s wired into the visual, 3D-entity, and writing scorecard presets. It is not wired into refactor or bugfix presets — cleanups don’t get judged on wow factor, and that’s the right call.

The dim has a hard cap at 0.85 if any claimed headline feature is invisible in a still capture. Live-only motion doesn’t save it. If your demo only sells in motion, the still is the portfolio shot you don’t have.

🏗️ Lifter 2: The Tightened Builder Brief

The Builder used to get told “implement the task.” Now it gets a non-negotiable quality bar.

✨ Lifter 3: The Polish Phase

A new phase between Verifier and scoring. Its job is the “feel finished” pass — framing, exposure, money-shot first frame, micro-interactions, voice for writing tasks. One or two focused improvements, never a rewrite.

If Polish thinks the build is already polished, it ships a no-op and writes its reasoning. Skipping is fine; lying is not.

🎯 Targeted Reruns

The Polish phase isn’t just a once-per-run pass. The Loop-Judge can rerun it independently when ambition lags but the build itself is fine.

That’s the routing rule that turns a quality lift into a cheap quality lift.

Three phases of work to fix bad framing was overkill. One Polish pass is all it needs.

🧪 The Regression Tests

Every change in the runner gets a scenario in test-phase3.sh. The new tests cover the silent-ship surface area plus the new quality wiring:

• Lowercase phase names (polish, builder) resolve to canonical names
• Unknown phase names trigger a safe fallback to [Builder, Verifier] with the fallback flagged
• MAX_ATTEMPTS ceiling forces a ship after 3 loops — no infinite cycles
• Ambition score below the floor auto-loops to Polish only, not the full Builder + Verifier sequence
• The code-visual preset wires both visual_quality and ambition; code-non-visual wires neither

13 scenarios, all green. The silent-ship case has its own scenario now — if anyone ever regresses the resolver, the test breaks the build.

💡 Why The Real Fix Wasn’t The Bug Fix

You can fix a string match in 30 seconds. Fixing the rubric so the loop has something to find takes longer — but that’s the change that lifts every future build, not just the next one.

The silent ship was the prompt. The lifters were the answer.