The Fix Had A Bug Too

🔙 Where We Left Off

Yesterday's post ended with a plan. We'd caught Claude reading the previous showcase build during a "blind" rebuild and reverse-engineering it — the new output looked suspiciously like the old one with extra features bolted on. The fix we agreed on was a tarball quarantine.

If you missed that one, it's worth reading first: We Caught Claude Cheating On A Blind Rebuild. This post picks up where that one stopped — actually building the fix, testing it, and discovering it had its own problem.

📦 What The Quarantine Is Supposed To Do

The idea is simple. Before a rebuild, take every file from the previous build and pack it into a single .tar.gz archive in a hidden directory outside both the source project and the published site. Then delete the originals. The rebuild session has nothing to read and nothing to copy from. But the data is preserved — a single tar -xzf brings it all back if the rebuild fails.

(click a station above to inspect the bash that runs)

The first draft of the script did exactly that. It worked when we tested it on the real music-visualizer showcase. Passed every sanity check. Looked done.

🧪 Testing The Fix Before Shipping It

Before declaring the skill finished, we built a throwaway test fixture in /tmp. Fake files for a fake slug, fake demos, fake screenshots — plus a sibling slug that shared the same prefix.

$TEST_ROOT/showcase/test-slug.html
$TEST_ROOT/showcase/data/test-slug.json
$TEST_ROOT/showcase/demos/test-slug-vanilla/
$TEST_ROOT/showcase/demos/test-slug-skills/
$TEST_ROOT/showcase/img/test-slug-vanilla.png

# Sibling — DO NOT TOUCH
$TEST_ROOT/showcase/test-slug-extended.html
$TEST_ROOT/showcase/demos/test-slug-extended-vanilla/
$TEST_ROOT/showcase/img/test-slug-extended-vanilla.png

That shows the whole setup: a target slug called test-slug, and a sibling called test-slug-extended that mustn't be affected by clearing the first.

Then we ran the script with SLUG="test-slug" and checked what the archive contained.

💥 The First Run Clobbered The Sibling

The archive contained everything it was supposed to — plus test-slug-extended-vanilla/ and test-slug-extended-vanilla.png. The naive shell glob demos/test-slug-*/ had no idea where "test-slug" ended and "-extended" began. Both directories match. Both got packed. Both would have been deleted.

If this bug had shipped, the first person to clear tetris (while the tetris-3d showcase existed) would have silently lost the tetris-3d build. No warning. No error. The skill would have reported success.

🎯 The Manifest-Aware Filter

The fix reads every slug from manifest.json at the start of the clear, then filters each candidate file against the full list. If a file's basename starts with any other slug followed by a dash, and that other slug is longer than the one being cleared, the file belongs to the longer slug and gets skipped.

mapfile -t ALL_SLUGS < <(grep -oE '"[^"]+"' data/manifest.json | tr -d '"')

belongs_to_longer_slug() {
  local base="$1" other
  for other in "${ALL_SLUGS[@]}"; do
    [ "$other" = "$SLUG" ] && continue
    if [[ "$base" == "$other"-* ]] && [ ${#other} -gt ${#SLUG} ]; then
      return 0
    fi
  done
  return 1
}

That function gets called for every candidate directory and image. Return 0 means "skip this, it's a sibling." Return 1 means "archive it, it's ours." Re-running the test fixture produced a clean archive with zero sibling files. The real music-visualizer manifest (17 slugs, zero prefix collisions) passes the filter trivially, but the logic is now robust for the day a collision shows up.

🛡️ The Shipped Skill, End To End

The final /clear-showcase has eight steps. Every step that touches the filesystem uses the filter:

💡 The Lesson

A fix is a new piece of code. It deserves the same scrutiny as the original feature. The whole point of the first post was "Claude can't be trusted to just not look at files that exist." It turns out the same principle applies to shell globs — they can't be trusted to just not match files that happen to share a prefix. Both were solved by making the rule explicit and mechanical instead of assuming good behavior.

Blind rebuilds now have two guarantees instead of one. The previous build is physically unreadable (tarball), and the clear operation can't collaterally damage a sibling showcase (filter). Both were verified with tests written before either went near production data.