The Two Lies The Orchestrator Was Telling

Same 17 builders — only one auditor refuses to ship the inventions

🧾 The Receipt

We were watching the live tail of an Orchestra-v2 run at localhost:8770/chat.html. The Auditor posted its signoff: SHIP. We counted the result.json files in the run's work/ directory.

One of them had been written by the worker it named. The other sixteen had a status field of "success_synthesized" — meaning the orchestrator had written them itself, after the worker's tab was already gone. The auditor counted them all as the same thing.

💀 Lie #1 — "Synthesized" Was Counted As Real

Quick definitions. A worker is a fresh Claude Code instance Orchestra spawns to handle one phase of a build. A result.json is the small file the worker writes when it finishes. A synthesized result is one the orchestrator manufactured itself when the worker didn't write its own — intended as a fallback breadcrumb, not a passing grade.

Auditor-Scale never got that memo. It looked at every file with status: "success" or status: "success_synthesized" and treated them identically. Sixteen builders skipped audit because the orchestrator had quietly covered for them with placeholder files that read like passes.

🔥 Lie #2 — The Reaper Killed Workers Mid-Pause

The reaper is the script that watches each worker's terminal tab and kills it if the tab stops making progress. Until this fix, "progress" meant wall-clock time since spawn. Hit the timeout, get killed — no matter what the worker was actually doing.

Workers paused by Anthropic's rate limiter look identical to workers stuck in a loop. The reaper killed them mid-wait. The orchestrator then wrote a synthesized result to fill the gap, and the auditor counted it as a pass. Round-trip from "real worker waiting on a 429" to "fabricated success in a shipped build" was about ninety seconds.

Same rate-limit pause — only one reaper survives it without inventing the outcome

🔗 Both Lies Share One Root

Each failure on its own would have been visible. A killed worker without a fallback would have left an empty slot — obvious. A "synthesized" status alone would have been a yellow flag waiting for someone to question it. Together they cancelled out, because each was filling a gap the other had created.

The unifying problem is that nothing in the system was writing the truth out loud. The reaper killed silently. The orchestrator wrote synthetic passes as a courtesy. The auditor saw only passes. Three quiet helpers; no honest voice.

🔥 Fix #1 — The Reaper Has A Heartbeat Now

The new reaper measures heartbeats, not wall clock. A heartbeat is any new file written under work/ or media/ — the worker's own evidence it is still doing things. Each one resets the deadline.

Pause sources freeze the heartbeat clock entirely. The reaper checks for .pause-active flags, the ORCHESTRA_FAKE_PAUSE env var, and the usage-watch daemon's .state/status.json — if any of them say "we're paused", the timer holds. When the pause clears, the clock resumes from where it left off, never having punished the worker for waiting.

The 300-second floor is deliberate. The first turn of any worker spends time parsing the brief and shaking hands with MCP servers before writing anything to disk — a tighter heartbeat would mistake initialization for a stall.

And critically: when the heartbeat actually does run out, the reaper writes a loud failure envelope before exiting:

{
  "status": "failed",
  "failure_reason": "heartbeat_timeout",
  "last_progress_at": "2026-04-26T13:42:18+10:00",
  "stalled_for_sec": 612,
  "heartbeat_budget_sec": 600,
  "artifact_paths": [],
  "digest": {
    "warnings_verbatim": ["heartbeat-timeout — reaper-failed"]
  }
}

The orchestrator can no longer cover for it. The slot is filled, but it is filled with the truth.

🛡️ Fix #2 — The Synth-Gate

Before the gate, "ship" was one decision: did the auditor say yes? Now ship requires three things to all be true simultaneously, and any one missing routes the run to judging_pending instead.

The third condition is the spine of the gate. The auditor can't "miss" a synth worker, because the gate independently scans work/ for synthesized files and refuses to ship if the auditor's per-post breakdown doesn't cover all of them.

📊 Old vs New — What Each One Killed

✋ Do / Don't — Filling Gaps For Other Components

The escape hatch is a new orchestra partial-unverified <run> <worker> "<reason>" command. Atomic write, refuses to overwrite an existing result, logs to chat.md and running-notes.md. The only manual route to declaring a worker partial.

🧪 What We Tested

Three smoke tests under runner/test/synth-fix/. They cover the boring-but-load-bearing parts: synth detection ignores Scorers and the Loop-Judge; per-post backstop verification trips correctly when 2 of 3 builders pass and one fails; the partial-unverified command refuses to overwrite an existing result; the reaper script's heartbeat clamping enforces the 300-second floor; the pause helpers correctly detect each flag source.

$ node test-synth-detection.js     → 11 passed, 0 failed
$ node test-partial-unverified.js  → 17 passed, 0 failed
$ bash test-reaper-syntax.sh       → 9 passed, 0 failed

Thirty-seven assertions across the three. Real validation will come the next time a rate-limit pause hits a worker mid-build — but the pieces hold up under synthetic stress.

🎯 The Lesson

A multi-agent system can lie to itself for a reason a single-agent system can't: the lie has somewhere to hide. When one component covers for another's silence, the system stops being able to tell whether the work happened.

The cure is the same in any domain. Every component that owns a slot has to be willing to write the truth into it, even — especially — when the truth is "I don't have the answer". Silence is the failure mode that makes everything else look fine.

The fix shipped as one-shot-orchestra-v2 v0.13.0 — runner version 1.8.0-heartbeat-reaper. Five files touched, plus three smoke tests and a synthesis-prohibition rule added to four protocol docs.

Visuals created with one-shot-scripts