Loop engineering · harness engineering

Python owns
the loop

A production loop is not call the model until it says done. The model drafts and grades. Software holds every transition.

Worked example
Ticket enhancer, Claude Agent SDK port
Roles
orchestrator · doer · judge
Owner of stop
check_stop.py, never the model
Human gate
an exact LGTM comment
Code
sol1_enhancer_agent_sdk

Problem the loop everyone ships first

A turbine with the governor removed

Two failures show up in every agent loop that has no software around it. The first is the agent that declares a ticket done because it said so. The second is quieter and worse: the agent that will not stop, editing the same ticket forever, missing the same three headings it missed last round, reporting motion.

Both come from the same design. The stop condition lives inside the reply. A high powered model without a governor is a turbine spinning toward its own destruction, because continuation is what the model is for. The product you ship is the governor around that spin.

Here is that loop, written honestly. Every marked line is a failure.

Listing 1 The intern grades the homework. Do not ship this. negative example
ticket = open("ticket.md").read()

while True:                                 # 1. no round count, no dollar cap
    reply = model(
        "Improve this ticket until it is DONE.\n" + ticket
    )
    if "DONE" in reply:                     # 2. stop is a token the model may emit
        open("ticket.md", "w").write(reply) # 3. writer overwrites the ticket it graded
        break
    ticket = reply                          # 4. next turn's memory is a string in RAM
  1. 1. The loop never counts rounds or dollars. There is no exit except vocabulary.
  2. 2. Stop lives in the model. "DONE" is a token the author is allowed to emit.
  3. 3. The writer is the grader. The same string that claims completeness overwrites the ticket.
  4. 4. There is no evidence file. The next turn's memory is a string in RAM.

A clever prompt survives one ticket. At ten tickets a day it drifts. At a hundred, nobody remembers what good looked like, and the bottleneck is a person reading every diff to decide whether a confident paragraph counts as evidence.

failure

Context rot

The chat is the only memory. Tool dumps bury the original ticket. The fact is still in the window and the model can no longer find it.

failure

Runaway iteration

Unbounded authority: any tool, no allow list, no path check. The worker fires invented calls, burns tokens, and reports motion.

failure

False completeness

The worker says done. DONE is in the reply. Nobody ran verify and nobody wrote an evidence file.

failure

Stagnation

The same gap twice, dressed as progress. Missing criteria in round one, missing them in round two with a more confident paragraph around the hole.

If the same model, in the same thread, in its own words, is allowed to stop the run, you do not have a loop. You have a turbine with the governor removed.

Model what the two disciplines actually name

Five nodes, and the harness around them

Loop engineering is designing the cycle the agent runs, and deciding who is allowed to stop it. The unit of work is one controlled pass, not a single model reply. The model proposes the next action. Software owns five steps, and each step is a node.

Harness engineering is the software around that cycle: who may write, what counts as done, when to stop, and which context a role may see. Loop engineering designs the cycle. Harness engineering is what makes the cycle safe to run unattended.

SOFTWARE OWNS EVERY TRANSITION TRIGGER issue, poll, hook ACTION one scoped job VERIFY not the worker MEMORY files on disk EXIT pass, retry, or escalate retry = round + 1 INNER ReAct reason → act → observe all the model owns
One pass. The inner reason, act, observe cycle is real and every coding agent runs some version of it, but it sits entirely inside the action node. The product you ship is the outer control: what starts a turn, who checks the result, what is written down, and which of the three doors the run leaves by.

The five nodes

Trigger. Something outside the model starts a turn: an issue, a poll, a webhook. The model does not get to feel like going again. A timer that fires when no state has changed is not a trigger, it is a token burn with no new evidence.

Action. One scoped job, done by a worker. An agent is a model given a role: a prompt, a declared tool list, a declared path check. The worker is the agent assigned the action, not every tool the platform owns.

Verify. A checker that is not the worker, plus a deterministic fact. Asking the author whether the homework is done is not verify.

Memory. Files a later process can read: a ticket, a JSON state file, a GitHub issue. Chat history is not a restore point.

Exit. The run has to end. The ticket ships, you try again because the gap can still change, or a person has to look. The model can type DONE. Software still picks which of the three actually happens.

Four properties of one iteration

Miss one and the next turn cannot be trusted.

PropertyWhat it meansWhere it lives here
Explicit stateA later pass reads files, it does not replay the chat.ticket markdown, .harness/*.json
Bounded authorityA role uses only the tools and paths its row in the table allows.roleplan.py, PreToolUse
Observable evidenceA candidate file, a judge report, a test result. Something you can point at later.*.enhancer-candidate.md
External transitionSoftware decides pass, retry, or escalate. The model does not get the vote.check_fields, check_stop

Three kinds of state, kept apart

On disk: the ticket, the trace, the plan. In process: the current budget, signature, and round. Never in the transcript: writes, stop, and the release token. Context windows reset. Ticket files do not.

Isolated context does two jobs. It keeps a child's window small so the agent is not overwhelmed, and it splits the problem into a smaller unit of work carrying only the context that job needs. That is mechanical sympathy for a machine whose recall sags in the middle of a long window, which is what Lost in the Middle measured. Put the patch, the research dump, and the plan on disk. Give the parent a short summary and a score.

Banking learned the other half a century ago. The person who writes the check is not the person who approves it. Maker and checker have to be different things, not different prompts in the same thread. A role name in a prompt is not a fence. The fence is the tool list, the path check, and a child session the parent does not share.


Example one program, start to finish

A ticket enhancer with three roles

Vague GitHub issue in. Rubric ready ticket out. GitHub is the inbox: a human opens an issue, the loop writes a local markdown ticket, scores it against a kind specific rubric, drafts a better body when the rubric is red, and waits for an exact LGTM comment before releasing the ticket downstream.

Nobody sits in a chat driving it. The orchestrator polls. The only human input that changes the state machine is that one exact comment. Three roles run the job, and only two of them are model subagents.

own context window DOER maker · the worker rewrites the ticket body NO WRITE TOOL own context window JUDGE checker reports present fields NO WRITE TOOL ORCHESTRATOR Python, not an agent enhancer.py spawn markdown text spawn kind + fields NO SHARED CONTEXT writes the file · decides ready · decides stop
The role graph. The doer proposes, the judge observes fields, and Python decides. The judge never sees the doer's inner chain of thought, because it runs in a child session the doer's window never touched. Two prompts in one thread are still one intern.
FileJob
loop.pyCLI. Builds the runtime and starts one poll, then the process exits.
contract.pyLoads .loop.yml from the target repo. Budgets and write scope.
enhancer.pyOrchestrator. Discovery, GitHub, writes, exits.
roleplan.pyThe role table. Tools and paths, not a prompt.
roles.pySDK options plus the PreToolUse hook.
check_fields.pyRubric. Turns {kind, present_fields} into ready.
check_stop.pyExits the model cannot talk its way past.
adapter.pyCost, text, and stop_reason as data.
turns.pyJudge and draft calls. Writes the hung dump.
tests/test_roles.pyPins the deny envelope. No SDK, no key.

Look at what is missing from the model. Discovery is Python. Writes are Python. Labels are Python. Stop is Python. The SDK runs two role turns under a tool list it did not choose.


Action who may write, and where

The roles are a table, not a paragraph

Capability is not a sentence in a system prompt. It is a row in a table that any runtime can read. If the table and a runtime disagree, the runtime is wrong.

Listing 2 The cast is data roleplan.py
WRITE_TOOLS = ("Edit", "Write", "NotebookEdit")   # 1. tools that can change a file
READ_TOOLS = ("Read", "Glob", "Grep")

LOOPS = {
    "enhancer": ("orchestrator", "doer", "judge"),  # 2. exactly three names
}

PURPOSE = {
    "orchestrator": "Owns the budget and the order. Writes nothing.",
    "doer": "Edits the ticket body. Nothing else in the repo.",
    "judge": "Scores the attempt. Reads reports and the diff. Holds no write path.",
}

READERS = ("orchestrator", "judge", "researcher")  # 3. roles that hold no write tool

FALLBACK_SCOPE = {
    "doer": (("tickets/**",), ()),                 # 4. when .loop.yml is silent
}

@dataclass(frozen=True)
class RolePlan:
    name: str
    purpose: str
    tools: tuple[str, ...]
    allow: tuple[str, ...] = ()
    deny: tuple[str, ...] = ()

    @property
    def can_write(self) -> bool:
        return any(tool in WRITE_TOOLS for tool in self.tools)   # 5. computed, not declared
  1. 1. If a role's tool list contains none of these, the role cannot write. That is the definition, and nothing else gets a say in it.
  2. 2. The cast for this loop is three names. A runtime that starts a fourth role has drifted from the table.
  3. 3. A sentence saying "do not write files" can be ignored. A missing Write tool cannot be called.
  4. 4. The target repo's .loop.yml knows nothing about the doer, so Python falls back to tickets/** and nothing else.
  5. 5. can_write is membership in WRITE_TOOLS. If it is false and a runtime still hands the role Write, the runtime is wrong.

The table is the ceiling. This program is stricter than its own ceiling: the doer and the judge both hold no write tool at all, and Python writes the candidate file. A runtime may give a role fewer tools than the table. It may never give a role more.

Delete the table and the SDK still has to give someone tools. If that someone is the judge, the judge gets Write, and the same model that drafted the ticket can overwrite the ticket it is supposed to grade.

Listing 3 The judge has no write tool roles.py
NO_WRITE = ["Edit", "Write", "NotebookEdit", "Bash"]   # 1. one longer than WRITE_TOOLS

def options_for(contract, loop: str = DEFAULT_LOOP):
    # ...
        tools = list(source["tools"]) if source else list(role.tools)
        if enhancer:
            tools = [tool for tool in tools if tool not in NO_WRITE]   # 2. strip
            if role.name == "doer":
                tools.append("Agent")                                  # 3. read-only Explore
        agents[name] = AgentDefinition(
            description=description,
            prompt=prompt,
            tools=tools,
            disallowedTools=NO_WRITE if enhancer or not role.can_write else NO_SHELL,  # 4.
            maxTurns=DEFAULT_MAX_TURNS,
            background=False,
            model="sonnet",
        )
    # ...
    if enhancer:
        kwargs.update(
            allowed_tools=["Agent"],       # 5. the parent may only spawn a named subagent
            disallowed_tools=NO_WRITE,
        )
  1. 1. Bash is on the list because Bash is how a read only agent writes anyway.
  2. 2. Every enhancer subagent loses Edit, Write, NotebookEdit and Bash. The judge cannot write the ticket it grades and the doer cannot write it either.
  3. 3. The doer may spawn the built in read only explorer, which gets its own context too. Python still owns every candidate write.
  4. 4. Repeating the list on disallowedTools is how an inherited Write does not sneak back in from the parent session.
  5. 5. If the parent still held Write, it could skip the doer and edit the repo itself.

That is fence one: can this role write at all. Fence two answers a different question. If a write tool ever leaks back onto a role, which paths may it touch? A leaked write must not land in app/, and the first file an agent under pressure reaches for is a failing test.

WRITE CALL from a role FENCE 1 TOOL LIST NO_WRITE strips Edit Write Bash FENCE 2 PreToolUse is the path inside tickets/** ? leaked return {} no opinion, allowed hookSpecificOutput permissionDecision: deny in scope outside
Both fences run on a single tool call, and only the second one has a failure mode you can typo into. An empty dictionary means no opinion, which lets the call through, so a deny that is misspelled anywhere in its envelope fails open and looks exactly like success.
Listing 4 Deny must not fail open roles.py · scope_hook
def scope_hook(repo: Path, role: RolePlan):
    """A PreToolUse hook that denies a write outside this role's scope.

    Returning an empty dict means "no opinion", which lets the call through.
    Denying needs the full hookSpecificOutput shape, so a typo here fails open.
    """
    scope = WriteScope(allow=list(role.allow), deny=list(role.deny))

    async def check(input_data, tool_use_id, context):
        if input_data["tool_name"] not in ("Edit", "Write", "NotebookEdit"):
            return {}                       # 1. a tool that cannot write
        raw = next(
            (input_data["tool_input"][k] for k in PATH_KEYS if k in input_data["tool_input"]),
            None,
        )
        if raw is None:
            return {}                       # 2. no path to check
        relative = _relative(repo, raw)
        if relative is not None and scope.permits(relative):
            return {}                       # 3. inside tickets/** for the doer
        where = relative or f"{raw} (outside the target repo)"
        return {
            "hookSpecificOutput": {         # 4. deny is a nested object
                "hookEventName": "PreToolUse",
                "permissionDecision": "deny",     # 5. the string, not False
                "permissionDecisionReason": (
                    f"{role.name} may write {', '.join(role.allow) or 'nothing'}. "
                    f"{where} is outside that scope."
                ),
            }
        }

    return check
  1. 1. Three separate allow paths all return the same empty dictionary, which is also the shape a bug returns.
  2. 2. Nothing to check means nothing to deny.
  3. 3. A path the scope permits is allowed the same silent way.
  4. 4. Miss the outer key and the SDK reads the return as no opinion.
  5. 5. false, blocked and no are not deny. A path outside the target repo fails closed.

Verify a judgment call and a fact are different things

Ready is a fact, not a vibe

The judge decides which required fields have real content. That is a model judgment call and there is no way around it. Whether the set adds up to ready is a fact, and facts live in Python.

Listing 5 The rubric is a dict and ready is computed check_fields.py
REQUIRED = {
    "bug": ["title", "steps", "expected", "actual", "environment"],
    "feature": ["problem", "proposal", "value", "criteria"],
    "ui": ["problem", "proposal", "value", "criteria", "wireframe"],
}


def check(kind: str, present_fields: list[str]) -> dict:
    if kind not in REQUIRED:
        raise ValueError(f"unknown ticket kind {kind!r}, expected one of {sorted(REQUIRED)}")
    required = REQUIRED[kind]
    present = set(present_fields)
    missing_fields = [f for f in required if f not in present]   # 1. computed here
    return {
        "kind": kind,
        "present_fields": [f for f in required if f in present], # 2. intersection only
        "missing_fields": missing_fields,
        "ready": not missing_fields,                             # 3. a boolean
    }
  1. 1. The function never reads a ready key out of the model payload.
  2. 2. A field the model invents is dropped rather than counted. It is not evidence.
  3. 3. A bug does not need problem, proposal and value. A UI ticket needs a wireframe on top of the feature fields.

Run the rubric

check_fields.check(kind, present_fields)
Ticket kind
Fields the judge says have real content

Struck through chips are outside this kind's rubric. Toggle one on to watch it get dropped.

ready: false

Trust a ready: true key in the judge's JSON and the model has quietly taken back the decision. A green rubric is still not permission to ship: without the exact human comment, green means waiting, and a red rubric never consumes that comment.


Exit four doors, all of them in software

Stopping is the feature

A failure signature is the missing field list from the last judge pass, for example ["criteria", "value"]. The round stores it. The next round builds a new one and compares. Equal lists mean the ticket did not move, which is stagnation wearing a confident paragraph. The signature is a fingerprint of the stall, not a cryptographic hash.

Listing 6 Four computed stops, none from the model check_stop.py
def check(
    round_: int,
    budget: int,
    signature: list[str],
    previous_signature: list[str] | None,
    usd: float = 0.0,
    max_usd: float | None = None,
    turns: int = 0,
    max_turns: int | None = None,
) -> dict:
    if previous_signature is not None and signature == previous_signature:
        return {"stop": True, "reason": "same signature two rounds running"}   # 1.
    if max_usd is not None and usd >= max_usd:
        return {"stop": True, "reason": "cost budget spent"}                   # 2.
    if max_turns is not None and turns >= max_turns:
        return {"stop": True, "reason": "max turns"}                           # 3.
    if round_ + 1 >= budget:
        return {"stop": True, "reason": "budget spent"}                        # 4.
    return {"stop": False, "reason": None}
  1. 1. Stable failure. Two rounds found exactly the same gaps.
  2. 2. Dollars are a hard cap. The figure comes from the SDK result, not from a role's word.
  3. 3. Turns are a hard cap too.
  4. 4. Four returns in Python. Completing a ticket is a different exit, owned by the orchestrator. The model cannot type a fifth return.

Trip a stop

first matching return wins
stop: false

Order matters between the two checks. Ready is a fact about the ticket. Stop is a fact about the run. The orchestrator asks the rubric first, and only a still red ticket reaches the stop check. Reverse them and a green ticket that has spent its dollars picks up a needs-human label instead of waiting for a person to release it.

For any of that to work, dollars and timeouts have to be fields Python can read. The adapter turns the SDK result into usd and stop_reason, maps the SDK's own error subtypes onto the same strings the stop check already uses, and caps a query at 180 seconds. When a turn hangs, the raw events are written to .harness/last-doer-T<id>.md before the escalation, so the evidence survives the stop and the poll moves on to the next ticket.


Trigger the state machine, in order

What one poll does

One poll walks every open draft ticket. Opening GitHub issues is a separate setup step, and a poll never opens one.

01

Ingest

List open issues and write a local draft where one is missing. Keep every draft this loop owns. A leftover candidate file is not a second ticket.

memory
02

Find the issue, never create one

First hit wins: state file, then ticket front matter, then a title search. A closed issue stops the ticket and asks you to reopen it. Closing an issue is not a reset.

blocked
03

Read the newest human comment for one exact token

A comment never starts an enhance round and a missing comment never stops one. Fuzzy thanks are not a release, and LGTM. with a period fails.

human gate
04

Respect a needs-human label

If a person has already been called in, wait for the person. Another poll is not a person.

escalated
05

Grade the real ticket

The judge reports kind and present fields. The rubric computes ready. The judge never claims the ticket is complete.

verify
06

Decide from ready and the token only

Green plus the exact comment marks the ticket ready and hands it to the implementer loop. Green without it is waiting. A red rubric never consumes the comment.

passed waiting
07

Draft, then keep the draft only if it strictly closes gaps

The doer returns markdown. Python writes a candidate file and the judge grades that. The draft replaces the real ticket only when its missing set is a proper subset of the current one.

waiting
08

Ask for a stop

Same missing fields twice, cost spent, max turns, round budget spent. Any computed stop adds the needs-human label and the poll continues to later tickets.

escalated
Listing 7 Not worse is not good enough enhancer.py · _improve
    def _improve(self, tkt, verdict: dict, comment: str | None, issue: int) -> list[str]:
        """Keep the draft only when it strictly closes gaps."""
        before = set(verdict["missing_fields"])
        candidate = self.draft(tkt, verdict["kind"], verdict["missing_fields"], comment)
        try:
            after_verdict = self.judge(candidate)
            after = set(after_verdict["missing_fields"])
            if after < before:                            # 1. proper subset, equal is not progress
                shutil.copyfile(candidate, tkt.path)      # 2. only Python copies
                set_front_matter(
                    tkt.path, id=tkt.id, state="draft", loop="enhancer", github_issue=str(issue)
                )
                self.gh.set_body(issue, strip_front_matter(tkt.path.read_text(encoding="utf-8")))
                self.gh.add_label(issue, "enhanced")
                return sorted(after)
            self.gh.comment(
                issue,
                f"The draft did not clear the rubric for a {verdict['kind']} ticket. "
                f"Still missing {', '.join(verdict['missing_fields'])}.",
            )
            return sorted(before)
        finally:
            candidate.unlink(missing_ok=True)             # 3. the candidate always dies
  1. 1. A draft that trades value for criteria looks busy. That is how a loop spends its whole budget standing still.
  2. 2. The doer and the judge hold no write tool, so the promotion from candidate to ticket is the orchestrator's move.
  3. 3. Accepted or rejected, the candidate file is removed in finally.

Retry is not the model saying try again. Retry is the round counter going up, the signature getting stored, and a later poll being the next attempt.

OutcomeMeans
passedRubric green and a human typed the exact token. The ticket moves to the implementer loop.
waitingEither green and waiting for a person, or still red after a round. Do not read every waiting as green.
escalatedStop, hang, or budget. The issue gets a needs-human label. The poll continues.
blockedIssue closed or missing. Reopen it. The poll does not invent a second issue.

Harness the fence you did not test

The harness is production code

A deny envelope you do not assert, key by key, will fail open the first time somebody misspells it, and it will fail open silently. The tests stub the SDK, so they need no API key and no clone.

Listing 8 The tests pin the envelope tests/test_roles.py
def test_the_hook_allows_a_write_inside_scope(repo, doer):
    assert call(repo, doer, file_path=str(repo / "tickets" / "T001.md")) == {}   # 1.


def test_the_hook_denies_a_write_outside_scope(repo, doer):
    """The full shape matters. A typo anywhere in it fails open."""
    output = call(repo, doer, file_path=str(repo / "app" / "models.py"))["hookSpecificOutput"]
    assert output["hookEventName"] == "PreToolUse"                               # 2.
    assert output["permissionDecision"] == "deny"
    assert "outside that scope" in output["permissionDecisionReason"]


def test_a_path_outside_the_repo_is_denied(repo, doer):
    """Fail closed. A path outside the repo matches no allow rule for any role."""
    result = call(repo, doer, file_path="/etc/hosts")                            # 3.
    output = result["hookSpecificOutput"]
    assert output["permissionDecision"] == "deny"
    assert "outside the target repo" in output["permissionDecisionReason"]
  1. 1. Recording the happy empty dictionary means a later change cannot silently start denying the doer's own scope.
  2. 2. Every key in the deny envelope is asserted, because any one of them is the typo that fails open.
  3. 3. A path that matches no allow rule does not get a free pass for failing to look like a ticket.

Run it before you spend a token

Three commands touch no model at all. The role table is the first one, and it is a pass or fail gate: if the judge prints yes in the writes column, the port is wrong and nothing else matters.

Shell Scripts with no model go-task recipes
git clone https://github.com/RichardHightower/reliable-agentic-lab.git
cd reliable-agentic-lab/solutions/sol1_enhancer_agent_sdk

task setup     # folder-local .venv plus the Agent SDK and pytest
task table     # the role table. judge must print no in the writes column
task checks    # both demo assertion scripts
task test      # pytest against a stubbed SDK. no key, no clone
Output The gate you refuse to walk past task table
role              writes  scope
orchestrator      no      nothing
doer              yes     tickets/**
judge             no      nothing

Then one poll: task run --. A ticket that still needs work normally costs three model calls, judge, doer, judge again. Every ticket prints one final line. On a fresh fork nothing is passed on the first poll, because nobody has released anything yet.

The same loop can sit on a plugin host instead of the SDK, where the round budget lives in skill instructions. A skill can ask itself to stop, but an instruction is not an enforcing mechanism. The Python form turns the budget, the timeout and the signature into software the model cannot revise in its own reply. Swap the host, keep the loop: same rubric, same exits, same role table.


Grounding what the literature settles, and what it does not

The papers behind the design

None of these papers prescribe this program. Read together they draw a clear line: a model improves reliably against an external signal, and unreliably against its own opinion of its own work. Every fence in the enhancer sits on one side of that line.

2210.03629

ReAct: Synergizing Reasoning and Acting in Language Models

Yao et al., 2022

Names the inner cycle. Reason, act, observe, repeat. Every coding agent runs some version of it. The point for loop engineering is where it belongs: inside the action node, wrapped by control the model does not own.

2303.17651

Self-Refine: Iterative Refinement with Self-Feedback

Madaan et al., 2023

The optimistic case for iteration. A model critiques its own output and revises, and on many generation tasks the revision is better. Worth knowing, and worth noticing what it does not establish: that the same model should also decide when the iterating stops.

2310.01798

Large Language Models Cannot Self-Correct Reasoning Yet

Huang et al., 2023

The counterweight, and the reason for the judge. Without external feedback, intrinsic self correction on reasoning tasks can leave results unchanged or worse. Gains attributed to self correction often depend on an oracle that quietly told the model when it was wrong. In this program that oracle is a rubric in Python, not a second paragraph from the author.

2303.11366

Reflexion: Language Agents with Verbal Reinforcement Learning

Shinn et al., 2023

Reflection works when it is fed a real signal. The agent reflects on an outcome it did not get to define, such as a failing test or a task reward, and carries that reflection into the next attempt. The failure signature here is the same shape of idea, reduced to something a machine can compare: the sorted missing field list.

2401.08500

Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering

Ridnik, Kredo and Friedman, 2024

Flow design changes what the same model produces. On CodeContests validation, a planned flow with generated tests took GPT-4 from roughly 19% to roughly 44% pass@5. That is evidence that the outer flow matters, not a performance promise for any other program. The transferable part is smaller and harder: tests own done.

2307.03172

Lost in the Middle: How Language Models Use Long Contexts

Liu et al., 2023

Why durable state belongs outside the window. Long context models use information more reliably near the beginning or the end of a window than in the middle. Keep the ticket, the trace and the plan on disk. Give the parent a short summary and a score.

2310.06770

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Jimenez et al., 2023

Done is decided by an external check, at benchmark scale. A patch counts when the repository's own tests pass, not when the model reports success. That is the same design decision as a rubric in Python, taken by an entire evaluation.


Terms defined once, used precisely

Glossary

Loop engineering
Designing the cycle an agent runs, and who is allowed to stop it.
Harness engineering
The software around that cycle: who may write, what counts as done, when to stop, and which context a role may see.
Mechanical sympathy
Adapting to how the machine actually behaves, then feeding it only what it can take. The harness does that for an agent.
Production loop
The cycle running when a human is not watching every turn.
Node
One of the five steps software owns: trigger, action, verify, memory, exit.
Role graph
A reusable arrangement of roles, authority and context boundaries. Orchestrator, doer, judge here. The same graph serves other jobs on other objects.
Graph engineering
Mapping intent into named steps that can be checked. A criterion becomes a test step and a code step.
Worker
The agent assigned the action. One scoped job. Here the doer. The judge is a subagent too, but it is the checker, not the worker.
Subagent
An agent the parent spawns, with its own context, separate from the parent and from every other subagent.
Bounded authority
The worker may use only the tools and paths its row in the role table allows. The opposite is any tool the platform owns, with no allow list and no path check.
Write scope
Paths a role may change. For the doer, tickets/** unless the target repo says otherwise.
Maker / checker
The doer writes a candidate, the judge scores fields, Python decides ready and stop. The same model must not do all three, and the checker must not inherit the maker's context.
Failure signature
A fingerprint of what still failed. Here the sorted missing field list, stored after a round and compared on the next one.
Proper subset
The draft's missing set must be strictly smaller than the current one, or Python discards the draft.
Context rot
Tool dumps bury the original ticket. The fact is still in the window and the model can no longer find it.
Merge box
A human owned decision to merge, spend money, or deploy. This program prepares a ticket and never takes that decision.