Problem the loop everyone ships first
A turbine with the governor removed
Two failures show up in every agent loop that has no software around it. The first is the agent that declares a ticket done because it said so. The second is quieter and worse: the agent that will not stop, editing the same ticket forever, missing the same three headings it missed last round, reporting motion.
Both come from the same design. The stop condition lives inside the reply. A high powered model without a governor is a turbine spinning toward its own destruction, because continuation is what the model is for. The product you ship is the governor around that spin.
Here is that loop, written honestly. Every marked line is a failure.
ticket = open("ticket.md").read()
while True: # 1. no round count, no dollar cap
reply = model(
"Improve this ticket until it is DONE.\n" + ticket
)
if "DONE" in reply: # 2. stop is a token the model may emit
open("ticket.md", "w").write(reply) # 3. writer overwrites the ticket it graded
break
ticket = reply # 4. next turn's memory is a string in RAM
- 1. The loop never counts rounds or dollars. There is no exit except vocabulary.
- 2. Stop lives in the model.
"DONE"is a token the author is allowed to emit. - 3. The writer is the grader. The same string that claims completeness overwrites the ticket.
- 4. There is no evidence file. The next turn's memory is a string in RAM.
A clever prompt survives one ticket. At ten tickets a day it drifts. At a hundred, nobody remembers what good looked like, and the bottleneck is a person reading every diff to decide whether a confident paragraph counts as evidence.
Context rot
The chat is the only memory. Tool dumps bury the original ticket. The fact is still in the window and the model can no longer find it.
Runaway iteration
Unbounded authority: any tool, no allow list, no path check. The worker fires invented calls, burns tokens, and reports motion.
False completeness
The worker says done. DONE is in the reply. Nobody ran verify and nobody wrote an evidence file.
Stagnation
The same gap twice, dressed as progress. Missing criteria in round one, missing them in round two with a more confident paragraph around the hole.
If the same model, in the same thread, in its own words, is allowed to stop the run, you do not have a loop. You have a turbine with the governor removed.
Model what the two disciplines actually name
Five nodes, and the harness around them
Loop engineering is designing the cycle the agent runs, and deciding who is allowed to stop it. The unit of work is one controlled pass, not a single model reply. The model proposes the next action. Software owns five steps, and each step is a node.
Harness engineering is the software around that cycle: who may write, what counts as done, when to stop, and which context a role may see. Loop engineering designs the cycle. Harness engineering is what makes the cycle safe to run unattended.
The five nodes
Trigger. Something outside the model starts a turn: an issue, a poll, a webhook. The model does not get to feel like going again. A timer that fires when no state has changed is not a trigger, it is a token burn with no new evidence.
Action. One scoped job, done by a worker. An agent is a model given a role: a prompt, a declared tool list, a declared path check. The worker is the agent assigned the action, not every tool the platform owns.
Verify. A checker that is not the worker, plus a deterministic fact. Asking the author whether the homework is done is not verify.
Memory. Files a later process can read: a ticket, a JSON state file, a GitHub issue. Chat history is not a restore point.
Exit. The run has to end. The ticket ships, you try again because the gap can still change, or a person has to look. The model can type DONE. Software still picks which of the three actually happens.
Four properties of one iteration
Miss one and the next turn cannot be trusted.
| Property | What it means | Where it lives here |
|---|---|---|
| Explicit state | A later pass reads files, it does not replay the chat. | ticket markdown, .harness/*.json |
| Bounded authority | A role uses only the tools and paths its row in the table allows. | roleplan.py, PreToolUse |
| Observable evidence | A candidate file, a judge report, a test result. Something you can point at later. | *.enhancer-candidate.md |
| External transition | Software decides pass, retry, or escalate. The model does not get the vote. | check_fields, check_stop |
Three kinds of state, kept apart
On disk: the ticket, the trace, the plan. In process: the current budget, signature, and round. Never in the transcript: writes, stop, and the release token. Context windows reset. Ticket files do not.
Isolated context does two jobs. It keeps a child's window small so the agent is not overwhelmed, and it splits the problem into a smaller unit of work carrying only the context that job needs. That is mechanical sympathy for a machine whose recall sags in the middle of a long window, which is what Lost in the Middle measured. Put the patch, the research dump, and the plan on disk. Give the parent a short summary and a score.
Banking learned the other half a century ago. The person who writes the check is not the person who approves it. Maker and checker have to be different things, not different prompts in the same thread. A role name in a prompt is not a fence. The fence is the tool list, the path check, and a child session the parent does not share.
Example one program, start to finish
A ticket enhancer with three roles
Vague GitHub issue in. Rubric ready ticket out. GitHub is the inbox: a human opens an issue, the loop writes a local markdown ticket, scores it against a kind specific rubric, drafts a better body when the rubric is red, and waits for an exact LGTM comment before releasing the ticket downstream.
Nobody sits in a chat driving it. The orchestrator polls. The only human input that changes the state machine is that one exact comment. Three roles run the job, and only two of them are model subagents.
| File | Job |
|---|---|
| loop.py | CLI. Builds the runtime and starts one poll, then the process exits. |
| contract.py | Loads .loop.yml from the target repo. Budgets and write scope. |
| enhancer.py | Orchestrator. Discovery, GitHub, writes, exits. |
| roleplan.py | The role table. Tools and paths, not a prompt. |
| roles.py | SDK options plus the PreToolUse hook. |
| check_fields.py | Rubric. Turns {kind, present_fields} into ready. |
| check_stop.py | Exits the model cannot talk its way past. |
| adapter.py | Cost, text, and stop_reason as data. |
| turns.py | Judge and draft calls. Writes the hung dump. |
| tests/test_roles.py | Pins the deny envelope. No SDK, no key. |
Look at what is missing from the model. Discovery is Python. Writes are Python. Labels are Python. Stop is Python. The SDK runs two role turns under a tool list it did not choose.
Verify a judgment call and a fact are different things
Ready is a fact, not a vibe
The judge decides which required fields have real content. That is a model judgment call and there is no way around it. Whether the set adds up to ready is a fact, and facts live in Python.
REQUIRED = {
"bug": ["title", "steps", "expected", "actual", "environment"],
"feature": ["problem", "proposal", "value", "criteria"],
"ui": ["problem", "proposal", "value", "criteria", "wireframe"],
}
def check(kind: str, present_fields: list[str]) -> dict:
if kind not in REQUIRED:
raise ValueError(f"unknown ticket kind {kind!r}, expected one of {sorted(REQUIRED)}")
required = REQUIRED[kind]
present = set(present_fields)
missing_fields = [f for f in required if f not in present] # 1. computed here
return {
"kind": kind,
"present_fields": [f for f in required if f in present], # 2. intersection only
"missing_fields": missing_fields,
"ready": not missing_fields, # 3. a boolean
}
- 1. The function never reads a
readykey out of the model payload. - 2. A field the model invents is dropped rather than counted. It is not evidence.
- 3. A bug does not need problem, proposal and value. A UI ticket needs a wireframe on top of the feature fields.
Run the rubric
check_fields.check(kind, present_fields)Struck through chips are outside this kind's rubric. Toggle one on to watch it get dropped.
Trust a ready: true key in the judge's JSON and the model has quietly taken back the decision. A green rubric is still not permission to ship: without the exact human comment, green means waiting, and a red rubric never consumes that comment.
Exit four doors, all of them in software
Stopping is the feature
A failure signature is the missing field list from the last judge pass, for example ["criteria", "value"]. The round stores it. The next round builds a new one and compares. Equal lists mean the ticket did not move, which is stagnation wearing a confident paragraph. The signature is a fingerprint of the stall, not a cryptographic hash.
def check(
round_: int,
budget: int,
signature: list[str],
previous_signature: list[str] | None,
usd: float = 0.0,
max_usd: float | None = None,
turns: int = 0,
max_turns: int | None = None,
) -> dict:
if previous_signature is not None and signature == previous_signature:
return {"stop": True, "reason": "same signature two rounds running"} # 1.
if max_usd is not None and usd >= max_usd:
return {"stop": True, "reason": "cost budget spent"} # 2.
if max_turns is not None and turns >= max_turns:
return {"stop": True, "reason": "max turns"} # 3.
if round_ + 1 >= budget:
return {"stop": True, "reason": "budget spent"} # 4.
return {"stop": False, "reason": None}
- 1. Stable failure. Two rounds found exactly the same gaps.
- 2. Dollars are a hard cap. The figure comes from the SDK result, not from a role's word.
- 3. Turns are a hard cap too.
- 4. Four returns in Python. Completing a ticket is a different exit, owned by the orchestrator. The model cannot type a fifth return.
Trip a stop
first matching return winsOrder matters between the two checks. Ready is a fact about the ticket. Stop is a fact about the run. The orchestrator asks the rubric first, and only a still red ticket reaches the stop check. Reverse them and a green ticket that has spent its dollars picks up a needs-human label instead of waiting for a person to release it.
For any of that to work, dollars and timeouts have to be fields Python can read. The adapter turns the SDK result into usd and stop_reason, maps the SDK's own error subtypes onto the same strings the stop check already uses, and caps a query at 180 seconds. When a turn hangs, the raw events are written to .harness/last-doer-T<id>.md before the escalation, so the evidence survives the stop and the poll moves on to the next ticket.
Trigger the state machine, in order
What one poll does
One poll walks every open draft ticket. Opening GitHub issues is a separate setup step, and a poll never opens one.
Ingest
List open issues and write a local draft where one is missing. Keep every draft this loop owns. A leftover candidate file is not a second ticket.
Find the issue, never create one
First hit wins: state file, then ticket front matter, then a title search. A closed issue stops the ticket and asks you to reopen it. Closing an issue is not a reset.
Read the newest human comment for one exact token
A comment never starts an enhance round and a missing comment never stops one. Fuzzy thanks are not a release, and LGTM. with a period fails.
Respect a needs-human label
If a person has already been called in, wait for the person. Another poll is not a person.
Grade the real ticket
The judge reports kind and present fields. The rubric computes ready. The judge never claims the ticket is complete.
Decide from ready and the token only
Green plus the exact comment marks the ticket ready and hands it to the implementer loop. Green without it is waiting. A red rubric never consumes the comment.
Draft, then keep the draft only if it strictly closes gaps
The doer returns markdown. Python writes a candidate file and the judge grades that. The draft replaces the real ticket only when its missing set is a proper subset of the current one.
Ask for a stop
Same missing fields twice, cost spent, max turns, round budget spent. Any computed stop adds the needs-human label and the poll continues to later tickets.
def _improve(self, tkt, verdict: dict, comment: str | None, issue: int) -> list[str]:
"""Keep the draft only when it strictly closes gaps."""
before = set(verdict["missing_fields"])
candidate = self.draft(tkt, verdict["kind"], verdict["missing_fields"], comment)
try:
after_verdict = self.judge(candidate)
after = set(after_verdict["missing_fields"])
if after < before: # 1. proper subset, equal is not progress
shutil.copyfile(candidate, tkt.path) # 2. only Python copies
set_front_matter(
tkt.path, id=tkt.id, state="draft", loop="enhancer", github_issue=str(issue)
)
self.gh.set_body(issue, strip_front_matter(tkt.path.read_text(encoding="utf-8")))
self.gh.add_label(issue, "enhanced")
return sorted(after)
self.gh.comment(
issue,
f"The draft did not clear the rubric for a {verdict['kind']} ticket. "
f"Still missing {', '.join(verdict['missing_fields'])}.",
)
return sorted(before)
finally:
candidate.unlink(missing_ok=True) # 3. the candidate always dies
- 1. A draft that trades
valueforcriterialooks busy. That is how a loop spends its whole budget standing still. - 2. The doer and the judge hold no write tool, so the promotion from candidate to ticket is the orchestrator's move.
- 3. Accepted or rejected, the candidate file is removed in
finally.
Retry is not the model saying try again. Retry is the round counter going up, the signature getting stored, and a later poll being the next attempt.
| Outcome | Means |
|---|---|
| passed | Rubric green and a human typed the exact token. The ticket moves to the implementer loop. |
| waiting | Either green and waiting for a person, or still red after a round. Do not read every waiting as green. |
| escalated | Stop, hang, or budget. The issue gets a needs-human label. The poll continues. |
| blocked | Issue closed or missing. Reopen it. The poll does not invent a second issue. |
Harness the fence you did not test
The harness is production code
A deny envelope you do not assert, key by key, will fail open the first time somebody misspells it, and it will fail open silently. The tests stub the SDK, so they need no API key and no clone.
def test_the_hook_allows_a_write_inside_scope(repo, doer):
assert call(repo, doer, file_path=str(repo / "tickets" / "T001.md")) == {} # 1.
def test_the_hook_denies_a_write_outside_scope(repo, doer):
"""The full shape matters. A typo anywhere in it fails open."""
output = call(repo, doer, file_path=str(repo / "app" / "models.py"))["hookSpecificOutput"]
assert output["hookEventName"] == "PreToolUse" # 2.
assert output["permissionDecision"] == "deny"
assert "outside that scope" in output["permissionDecisionReason"]
def test_a_path_outside_the_repo_is_denied(repo, doer):
"""Fail closed. A path outside the repo matches no allow rule for any role."""
result = call(repo, doer, file_path="/etc/hosts") # 3.
output = result["hookSpecificOutput"]
assert output["permissionDecision"] == "deny"
assert "outside the target repo" in output["permissionDecisionReason"]
- 1. Recording the happy empty dictionary means a later change cannot silently start denying the doer's own scope.
- 2. Every key in the deny envelope is asserted, because any one of them is the typo that fails open.
- 3. A path that matches no allow rule does not get a free pass for failing to look like a ticket.
Run it before you spend a token
Three commands touch no model at all. The role table is the first one, and it is a pass or fail gate: if the judge prints yes in the writes column, the port is wrong and nothing else matters.
git clone https://github.com/RichardHightower/reliable-agentic-lab.git
cd reliable-agentic-lab/solutions/sol1_enhancer_agent_sdk
task setup # folder-local .venv plus the Agent SDK and pytest
task table # the role table. judge must print no in the writes column
task checks # both demo assertion scripts
task test # pytest against a stubbed SDK. no key, no clone
role writes scope
orchestrator no nothing
doer yes tickets/**
judge no nothing
Then one poll: task run --. A ticket that still needs work normally costs three model calls, judge, doer, judge again. Every ticket prints one final line. On a fresh fork nothing is passed on the first poll, because nobody has released anything yet.
The same loop can sit on a plugin host instead of the SDK, where the round budget lives in skill instructions. A skill can ask itself to stop, but an instruction is not an enforcing mechanism. The Python form turns the budget, the timeout and the signature into software the model cannot revise in its own reply. Swap the host, keep the loop: same rubric, same exits, same role table.
Grounding what the literature settles, and what it does not
The papers behind the design
None of these papers prescribe this program. Read together they draw a clear line: a model improves reliably against an external signal, and unreliably against its own opinion of its own work. Every fence in the enhancer sits on one side of that line.
ReAct: Synergizing Reasoning and Acting in Language Models
Yao et al., 2022
Names the inner cycle. Reason, act, observe, repeat. Every coding agent runs some version of it. The point for loop engineering is where it belongs: inside the action node, wrapped by control the model does not own.
Self-Refine: Iterative Refinement with Self-Feedback
Madaan et al., 2023
The optimistic case for iteration. A model critiques its own output and revises, and on many generation tasks the revision is better. Worth knowing, and worth noticing what it does not establish: that the same model should also decide when the iterating stops.
Large Language Models Cannot Self-Correct Reasoning Yet
Huang et al., 2023
The counterweight, and the reason for the judge. Without external feedback, intrinsic self correction on reasoning tasks can leave results unchanged or worse. Gains attributed to self correction often depend on an oracle that quietly told the model when it was wrong. In this program that oracle is a rubric in Python, not a second paragraph from the author.
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn et al., 2023
Reflection works when it is fed a real signal. The agent reflects on an outcome it did not get to define, such as a failing test or a task reward, and carries that reflection into the next attempt. The failure signature here is the same shape of idea, reduced to something a machine can compare: the sorted missing field list.
Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering
Ridnik, Kredo and Friedman, 2024
Flow design changes what the same model produces. On CodeContests validation, a planned flow with generated tests took GPT-4 from roughly 19% to roughly 44% pass@5. That is evidence that the outer flow matters, not a performance promise for any other program. The transferable part is smaller and harder: tests own done.
Lost in the Middle: How Language Models Use Long Contexts
Liu et al., 2023
Why durable state belongs outside the window. Long context models use information more reliably near the beginning or the end of a window than in the middle. Keep the ticket, the trace and the plan on disk. Give the parent a short summary and a score.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Jimenez et al., 2023
Done is decided by an external check, at benchmark scale. A patch counts when the repository's own tests pass, not when the model reports success. That is the same design decision as a rubric in Python, taken by an entire evaluation.
Terms defined once, used precisely
Glossary
- Loop engineering
- Designing the cycle an agent runs, and who is allowed to stop it.
- Harness engineering
- The software around that cycle: who may write, what counts as done, when to stop, and which context a role may see.
- Mechanical sympathy
- Adapting to how the machine actually behaves, then feeding it only what it can take. The harness does that for an agent.
- Production loop
- The cycle running when a human is not watching every turn.
- Node
- One of the five steps software owns: trigger, action, verify, memory, exit.
- Role graph
- A reusable arrangement of roles, authority and context boundaries. Orchestrator, doer, judge here. The same graph serves other jobs on other objects.
- Graph engineering
- Mapping intent into named steps that can be checked. A criterion becomes a test step and a code step.
- Worker
- The agent assigned the action. One scoped job. Here the doer. The judge is a subagent too, but it is the checker, not the worker.
- Subagent
- An agent the parent spawns, with its own context, separate from the parent and from every other subagent.
- Bounded authority
- The worker may use only the tools and paths its row in the role table allows. The opposite is any tool the platform owns, with no allow list and no path check.
- Write scope
- Paths a role may change. For the doer,
tickets/**unless the target repo says otherwise. - Maker / checker
- The doer writes a candidate, the judge scores fields, Python decides ready and stop. The same model must not do all three, and the checker must not inherit the maker's context.
- Failure signature
- A fingerprint of what still failed. Here the sorted missing field list, stored after a round and compared on the next one.
- Proper subset
- The draft's missing set must be strictly smaller than the current one, or Python discards the draft.
- Context rot
- Tool dumps bury the original ticket. The fact is still in the window and the model can no longer find it.
- Merge box
- A human owned decision to merge, spend money, or deploy. This program prepares a ticket and never takes that decision.