Let the Exit Code Decide: Build a Claude Code Retry Loop Around a Test Gate
A retry loop works only when a program grades each round and each retry sees exactly what that program reported, so the gate, the stop rules, and the ledger belong in a script, not in Claude's judgment of its own work.

Write failing tests as the spec, run a script that shows Claude the full test output each round, stop on rules written in code, and commit the result.
You asked Claude to fix the failing tests, it told you they all pass, and you had no record of what it tried or any rule to stop it when it got stuck.
In this article: Day 25 of a day-by-day guide to Claude Code. Start here: you write failing tests as a spec, run a script that calls Claude until those tests pass, read the record of each round, and commit the result. Going deeper: why the prompt carries the whole gate output, what the loop does when the spec cannot be met, and how a script-owned loop compares with
/goal.
Tested on Claude Code 2.1.327.
Yesterday you wrote scripts/check.sh, which runs Claude headless as a test gate, and committed it as Add a headless test check.
Start here
At the end of today, you can write failing tests as a spec, run a script that calls Claude until those tests pass, read the record of each round, and commit the result.
Starting fresh or fell behind? Run git switch -C main claude-code/day-24. This replaces your main with the end of Day 24; to keep your own work, run git branch my-backup first.
Four terms carry today's lesson:
- A gate is a check that a program runs and that passes or fails. Today's gate is
npm test. On Day 22 you saw that a workflow's audit gave a different table on each run. A test gives the same answer every time, because it has one right answer. - A retry loop runs Claude, runs the gate, and runs Claude again with the gate's output, until the gate passes or a stop rule fires.
- A stop rule is a condition in the script that ends the loop before the gate passes, so a loop that is not getting anywhere does not run forever.
- A ledger is a file with one line per round, so you can read afterward what each round did.

The four parts work together. The gate grades, the loop retries, the stop rules end a stuck run, and the ledger tells you afterward what happened.
Day 12 set the rule that makes a retry work: give Claude the exact failure, not your summary of it. In a loop, the retry must show Claude the exact thing the gate graded. Today's script puts the gate's full output in the prompt, not a summary of it.
The task: Ticket Triage gets a billing category for tickets about charges and invoices. You write the tests, and the loop gets Claude to make them pass.
Check the starting state
cd ~/code/ticket-triage
git status
git log --oneline -1
npm test
Git prints nothing to commit, working tree clean, and the log shows Add a headless test check. The tests show ℹ fail 0. Both tested runs started at 22 tests. Your count can differ, so write it down.
Write the spec as failing tests
The tests are the spec. Create test/billing.test.js in your editor, or paste this into the terminal:
cat > test/billing.test.js <<'EOF'
import { test } from "node:test"
import assert from "node:assert/strict"
import { classify } from "../src/classify.js"
test("billing", () => {
assert.equal(classify("I was charged twice"), "billing")
})
test("invoice", () => {
assert.equal(classify("My invoice shows the wrong amount"), "billing")
})
test("billing loses to older rules", () => {
assert.equal(classify("I was charged for a crushed box"), "damaged")
})
EOF
The third test keeps the CLAUDE.md rule from Day 5: a new category needs a test with a ticket that also matches an older rule. Now stage the file and run the tests:
git add test/billing.test.js
npm test
Two tests fail, billing and invoice. The third already passes, because the damaged rule catches crushed. On the tested runs, the pass count went from 22 to 23.
You stage the tests so that the loop can guard them. git diff -- test/ compares the files in test/ with what you staged. If Claude edits or deletes a staged test, the diff shows it. An unstaged new file never shows in git diff.
Write the loop
Create scripts/loop.sh, in your editor or with this command:
cat > scripts/loop.sh <<'EOF'
#!/usr/bin/env bash
# A retry loop. The gate is npm test, and each round shows Claude the gate's full output.
# Stops when the gate passes, when the same tests fail twice, when Claude changes test/, or after 3 rounds.
task="$1"; last=""; mkdir -p .loop
for round in 1 2 3; do
npm test > .loop/gate.txt 2>&1 && { echo "round $round: gate passed" | tee -a .loop/ledger.txt; exit 0; }
failed=$(grep '^not ok' .loop/gate.txt)
echo "round $round: gate failed, $(echo "$failed" | wc -l | tr -d ' ') failing" | tee -a .loop/ledger.txt
[ "$failed" = "$last" ] && { echo "same tests failed twice: stop" | tee -a .loop/ledger.txt; exit 2; }
last="$failed"
claude -p "$task Do not change any file in test/. The gate is npm test. It failed with this output:
$(cat .loop/gate.txt)" --permission-mode acceptEdits > .loop/round-$round.txt
git diff --quiet -- test/ || { echo "round $round: Claude changed test/: stop" | tee -a .loop/ledger.txt; exit 3; }
done
npm test > .loop/gate.txt 2>&1 && { echo "after round 3: gate passed" | tee -a .loop/ledger.txt; exit 0; }
echo "gate still failing after 3 rounds: stop" | tee -a .loop/ledger.txt; exit 1
EOF
chmod +x scripts/loop.sh
echo ".loop/" >> .gitignore
chmod +x makes the script runnable. The last line keeps the loop's files out of git.
The script, line by line:
task="$1"takes the task from the first argument.lastholds the failing tests from the round before.mkdir -p .loopmakes the folder for the loop's files.for round in 1 2 3allows at most three rounds.- The
npm testline runs the gate and saves its full output in.loop/gate.txt.2>&1sends the error output to the same file. When the output goes to a file, Node's test runner writes a plain format with oneokornot okline per test. If the gate passes, the script writesgate passedand exits with code 0. failed=$(grep '^not ok' ...)collects thenot oklines, the failing tests.- The
echoline counts them.tee -aprints the line on your screen and adds it to.loop/ledger.txt, the ledger. - The
[ "$failed" = "$last" ]line is a stop rule. It compares the names in thenot oklines with the round before. If the same set of tests failed again, Claude's last try fixed none of them, so the script stops with code 2. Claude may still have changed files in that try. - The
claude -pcall is the retry. The prompt is your task, a rule not to touchtest/, and every line of.loop/gate.txt. Claude's reply goes to.loop/round-1.txt,.loop/round-2.txt, and so on. --permission-mode acceptEditsstarts the call in accept edits mode, where Claude edits files with no prompt. On Day 24, aclaude -prun with no mode set started in Manual mode and denied aWriteofnotes.txtwith no prompt. Without this flag, an edit tosrc/classify.jsneeds the same approval, and no one is there to give it.git diff --quiet -- test/is the second stop rule.--quietprints nothing and only sets the exit code: 0 whentest/matches what you staged. If Claude changed a staged test, the code is not 0,||runs the stop, and the script stops with code 3.- After round 3, the script runs the gate once more. A pass exits with 0, and a failure writes a stop line and exits with 1.

Every path out of the loop is a line of shell. The gate runs first in each round, and each stop has its own exit code.
Run the loop
Run it from the top of Ticket Triage, with the task in quotes:
./scripts/loop.sh "Add a billing category to src/classify.js for tickets about charges and invoices."
The tested runs took about 21 seconds and printed:
round 1: gate failed, 2 failing
round 2: gate passed
The gate runs at the top of each round. Round 1 graded your new tests before Claude did anything and found the 2 failures. Claude then worked once. Round 2 ran the gate again. It passed, so the script stopped.

Claude worked once, between two runs of the gate. The script, not Claude, decided that the run was over.
If the last line is a stop line instead of gate passed, read .loop/ledger.txt, .loop/gate.txt, and the last .loop/round-N.txt to find the cause. Fix it yourself, or ask Claude in a normal session, then run the loop again. Going deeper shows one loop that stops.
Manual mode: no prompts. Today runs no interactive session, and the loop sets the mode for its own claude -p calls, so both modes run the same calls.
Read what each round did
cat .loop/ledger.txt
cat .loop/round-1.txt
The ledger holds the same two lines. Claude's reply on the tested auto run began:
I added a billing category to `src/classify.js`, and `npm test` now passes all 25 tests.
The Manual run's reply began the same way. Read the whole reply. Both tested replies warned that charge also matches words such as recharge, which is a lead for you to judge, not a failure.
Check the result
git status --short
git diff -- src/
npm test
On the tested runs, git status --short printed:
M .gitignore
M src/classify.js
A test/billing.test.js
?? scripts/loop.sh
The A means that your test file is staged and unchanged. git diff -- src/ shows one new rule after the cancel rule, and it shows billing added to the comment above classify(). Both tested runs made the same change:
+ if (t.includes("charge") || t.includes("invoice")) return "billing"
The tests show ℹ fail 0 and three more tests than this morning: 25 on the tested runs.
The task named only src/, and Claude did not change CLAUDE.md. Its category list does not name billing, and it stays out of date until you run the one-line follow-up in Going deeper. The next day was tested without that follow-up.
Commit
git add -A
git commit -m "Add a billing category"
git status
git show --stat HEAD
git add -A stages every change. .loop/ is in .gitignore, so it stays out. Git prints nothing to commit, working tree clean, and git show --stat HEAD ends with:
.gitignore | 1 +
scripts/loop.sh | 16 ++++++++++++++++
src/classify.js | 3 ++-
test/billing.test.js | 15 +++++++++++++++
4 files changed, 34 insertions(+), 1 deletion(-)
You wrote a spec as tests, ran a loop that gave Claude the gate's full output, read the ledger, checked that the tests did not change, and committed.
You can stop here. Going deeper explains why this works and where it breaks.
Going deeper
This section is for readers who want to know why the loop is built this way, what it does when the spec cannot be met, how it compares with /goal, and when to move a loop into code. It assumes you finished Start here in Parts 1 to 25.
Why the prompt carries the whole gate output
Day 12 gave the rule for a loop that you run by hand: give Claude the exact failure, not your summary of it. The script follows the same rule. Each claude -p call is a new session with no memory of the round before, so the prompt and the files on disk are all that Claude has. A summary such as "2 tests fail" drops the test names, the expected values, and the actual values. The gate file has all of them.

Each round starts a new session. The prompt and the files on disk are all that Claude has, so the prompt carries the whole gate file.
The same rule holds in larger loops. Issue #347 in the reliable-agentic-lab repository records seven live runs and $103 with no finished paper. The cause was the retry loop, not the writer. The judge graded a section cut to its first 8,000 characters, and the editor got only the judge's failed rows, so it never heard about the Python length check that reported 1186 words, ceiling 1000. The fix showed the judge the whole section, passed every failed row to the editor, and ran the Python checks before the judge.

The larger loop failed for the same reason that a summary would fail today. The part that did the work never saw what the check reported.
Why the script runs the gate, not Claude
Claude's round-1 reply said that npm test passes all 25 tests. The script did not take its word. Round 2 ran the gate itself and decided from the exit code. A model can be wrong about its own work, and a loop that lets the doer grade itself can stop on a claim. The gate is a program, and the script reads its exit code.
The mode limits what Claude can do in a round. The permission modes docs say that accept edits mode approves file edits and a few filesystem commands, such as mkdir, mv, cp, and rm, inside your working directory. The headless docs add that other shell commands still need an --allowedTools entry or a permissions.allow rule. Day 4's allow rule for npm test lets Claude run the tests in its round. A command with no rule, such as git commit, has no one to approve it in a claude -p run, so it is denied. Because rm is allowed, Claude could delete a staged test. The git diff -- test/ guard catches that too. The docs also describe deny rules, which block an edit before it happens instead of after. This guide did not test a deny rule for test/.
The gate does not depend on Claude's permissions. In the stuck test below, the copy of the repository had never been trusted in an interactive session, and each claude -p call printed Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted. Day 4's allow rule for npm test did not apply there, but the script ran the gate in every round. Day 24 covers the trust check.
You could pass --permission-mode auto instead, and Day 24 showed that it lets a -p call write a file. Auto mode lets its classifier approve many more commands. Accept edits mode gives the round only what the task needs.
When the spec cannot be met
A stop rule matters most when the gate can never pass. The tested run added two tests that want different answers for the same ticket:
cat > test/stuck.test.js <<'EOF'
import { test } from "node:test"
import assert from "node:assert/strict"
import { classify } from "../src/classify.js"
test("invoice question is billing", () => {
assert.equal(classify("Where is my invoice?"), "billing")
})
test("invoice question is shipping", () => {
assert.equal(classify("Where is my invoice?"), "shipping")
})
EOF
git add test/stuck.test.js
rm -rf .loop
./scripts/loop.sh "Make every test pass."
rm -rf .loop starts a fresh ledger, because tee -a adds to the old one. The tested auto run stopped after 18 seconds with exit code 2:
round 1: gate failed, 1 failing
round 2: gate failed, 1 failing
same tests failed twice: stop
Claude changed no file. Its round-1 reply began I didn't change any files: these two tests can't both pass. and said that the fix belongs in test/, which the prompt told it not to edit. The loop spent one Claude call and stopped, instead of three. When a loop stops this way, read the round files before you spend more retries. The fault can be in the spec.

The same-failures rule ended the run after one Claude call. The round file then told you where the fault was.
A second tested run printed the same three lines, and echo $? right after it printed 2.

A loop that stops early is good news when the spec is wrong. You spend one Claude call, not three, and you fix the tests yourself.
Reset before you go on, because Day 26 starts from a clean tree with no failing tests:
git rm -f test/stuck.test.js
git status --short
npm test
On the tested copy, git status --short printed nothing, and the tests were back to 25 passing and 0 failing. Your count matches the one after your commit.
The other two exits were not tested here: a change to test/ (exit 3) and three rounds with no pass (exit 1).

Each way out of the loop has its own exit code, so a cron job or a CI step can tell a pass from each kind of stop.
/goal and a script-owned loop
Day 18 ran a /goal, which is also a retry loop. The difference is who owns each part:

Each part of a retry loop has an owner, and /goal and the script give each part to a different owner. The first column names the part of the loop. The /goal column says how a goal from Day 18 handles that part, and the scripts/loop.sh column says how today's script handles it.
- Who runs the check: with
/goal, Claude runs the check in the session, and withscripts/loop.sh, the script runs it. - Who decides it is done: with
/goal, an evaluator model that reads the conversation decides, and with the script, the exit code ofnpm testdecides. - What the next try sees: with
/goal, the next try sees the whole conversation and the evaluator's reason, and with the script, it sees the full gate output in a new session. - Stop rules: with
/goal, the evaluator judges whether the goal is met or impossible and judges a turn clause from the conversation, and with the script, the stop rules are code: a pass, the same failures twice, a change totest/, or 3 rounds.
The goal docs say that the evaluator does not run commands or read files, so it can judge only what Claude showed in the conversation. Use /goal when you are at the keyboard and the condition is easy to state. Use a script when you want the gate and the stop rules to be code that no model can talk past, or when the loop runs from cron or CI with no session open.

Both are retry loops. The difference is whether a model or a program decides that the work is done.
Budgets, hangs, and the Agent SDK
This section comes from the docs and was not tested here. The CLI reference lists two print-mode limits that a loop can pass to each claude -p call: --max-turns, which limits the agentic turns, and --max-budget-usd, which stops the run once its estimated spend reaches an amount. The docs say that the budget check uses a client-side estimate that can differ from your bill, and that spend can pass the cap. Day 24's JSON output has a total_cost_usd field, which a loop could add to each ledger line.
If a round hangs, the loop waits with it. Version 2.1.327 fixed one cause: claude -p hanging before its first request when the fetch of a model's settings stalled, such as at a proxy.
When a loop needs a budget across rounds, a judge, and a ledger with structure, move it into code. The Agent SDK runs the same agent as Claude Code as a library in Python or TypeScript, with its tools, permissions, hooks, and sessions. The docs' quickstart sets an API key, not a claude.ai sign-in. A judge, in this sense, is a second model call that can read but not write, so it cannot change the work it grades. Day 26 adds a stage before today's loop that writes the spec, and a stage after it that reviews the change.
Add billing to CLAUDE.md
One headless call fixes the category list:
claude -p "In CLAUDE.md, add billing to the category list and to the rule order, after cancel. Change no other file." --permission-mode acceptEdits
git diff --stat
On the tested copy, the reply began I updated CLAUDE.md in two places, and git diff --stat printed:
CLAUDE.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
The category list then read "cancel", "billing", or "other", and the rule order read damaged → refund → shipping → cancel → billing. Read git diff. If it is right, you can commit it as Add billing to CLAUDE.md. This guide did not test that commit. Day 26 was tested without it, so Day 26's starting check would show your new commit instead of Add a billing category.
To undo the change instead:
git restore CLAUDE.md
git status --short
On the tested copy, git status --short printed nothing.
Go further: Python Owns the Loop: Loop Engineering and the Harness Around It moves the same kind of loop into Python with the Agent SDK, with a judge that has no write tools and four stops that the model cannot talk its way past: the same failure twice, cost, turns, and rounds.
Do this today
Everyone
- Write the billing tests, stage them, run
scripts/loop.shwith the billing task, and commit. Check the win:git log --oneline -1showsAdd a billing category,git statusprintsnothing to commit, working tree clean,git show --stat HEADlists 4 files, andnpm testshowsℹ fail 0with three more tests than this morning. - Habit: stage the tests before you start a loop, and read the ledger and the round files when it ends. Check:
cat .loop/ledger.txtends withgate passed, andgit diff -- test/prints nothing.
If you read Going deeper
- Run the stuck test from "When the spec cannot be met". Check: the loop prints
same tests failed twice: stop, andecho $?right after it prints2. Reset: rungit rm -f test/stuck.test.js, then check thatgit status --shortprints nothing andnpm testshowsℹ fail 0with the same count as after your commit. - Add billing to CLAUDE.md with the call in Going deeper. Check:
git diff --statlists onlyCLAUDE.md, and the diff names billing after cancel. Commit it with the messageAdd billing to CLAUDE.md, a step this guide did not test, or reset withgit restore CLAUDE.mdand check thatgit status --shortprints nothing.
The exit code decides
A retry loop is only as honest as its gate. Claude said that the tests passed, and the script checked anyway. The gate ran in every round, the stop rules lived in shell, and the ledger kept a line for each round so that you could read the run afterward.
The retry also worked because each round saw exactly what the gate reported. Write the spec as tests, stage them, show Claude the full failure, and let the exit code decide.
This is Part 25 of "Claude Code, Day-to-Day (second edition)," a day-by-day guide to Claude Code in which each day takes 20 to 40 minutes. Part 25 is Day 25. All 28 days are out now.