Loop Engineering: From Writing Code to Shipping Correctly

Part one of a series on making vibe coding hold up at scale. This is the overview: why you need a Loop, what it changes, and when it's worth it. Packet fields, gate configuration and script details come later.

One-shot generation quality is decided by the model. Long-run delivery quality is decided by the engineering system around it.

That's the most concrete thing I've taken from a few months of AI coding. Models get better every year — but the gap between "it wrote this correctly" and "this project stays correct" has very little to do with the model.

1. The real problem with AI coding

To be clear: the model can write code. It writes it well.

The problem is elsewhere, and it only shows up once the task gets big.

The goal drifts. You ask for rate limiting; it also touches registration and two password-reset endpoints — because "those look like they should be limited too." Not wrong, exactly. Just not this round. By the time you notice, the diff spans three modules and you can't tell what belonged to this task.

Context gets lost. In a long task, the architectural semantics you settled on two days ago aren't in today's session. It re-derives them and lands somewhere equally reasonable and different.

It reasons in circles. This is the worst one. When the same model both implements and self-reviews, asking "is this right?" gets answered by the same reasoning chain that just produced the code. Self-review here is a tautology, not a check.

You can't see what it's doing. Three days into a cross-module task, you want to know "where are we, and why is this the next step" — and all you have is its output, which is always confident, complete and plausible.

Those four share something worth stating on its own:

None of them throw an error.

Nothing fails to compile. No test goes red. No tool anywhere says "hold on." The feedback loop you rely on for small tasks — compiler, tests, type checker — is completely silent on all of it.

They aren't a product of AI being weak. They're a product of AI being good: it writes smoothly enough that no single line looks off.

Human errors usually carry a tone. When a person writes "per So-and-so's confirmation," some part of them hesitates, and that hesitation bleeds into the text. AI errors have no tone.

2. What Loop Engineering is

First, what it isn't: not two AIs chatting forever, and not infinite retries.

It's a closed delivery loop:

     authoritative docs (requirements / architecture / plan)
                       │
                       ▼
        ┌──►  ① align: this round's work order
        │              │
        │              ▼
        │      ② implement in phases (executor)
        │              │
        │              ▼
        │      ③ machine checks: scope gate + tests
        │              │
        │              ▼
        │      ④ independent review (reads the real diff)
        │              │
        │       ┌──────┴──────┐
        │       │             │
        └───────┤ findings    │ approved
          another round       ▼
                      ⑤ freeze evidence (diff/tests/verdict)
                             │
                             ▼
                      ⑥ feed experience back → rules evolve

The whole thing turns on that back-arrow.

This isn't a pipeline of "write → review once → merge." It's a convergence process. That distinction isn't rhetorical — I have the data.

Why one review isn't enough

One architecture design, handed to an independent reviewer, ran five rounds:

23 substantive findings, 49 minutes, not one repeat.

That's the answer to "why must it be a Loop." Fix round one's six, and round two surfaces five brand-new ones — because clearing the first layer is what exposes the second.

And it gets deeper as it goes. Round one's findings are visible at the document level (red-line conflicts, inconsistent terms). Round five's require running the design in your head — for instance, "a CAS conflict when claiming a candidate is wrongly treated as the candidate list being exhausted." That's concurrency semantics. Round one never gets near it.

Review once and you get a design with six problems fixed and seventeen remaining — and you'll think it passed.

3. Why a prompt isn't enough

The natural reaction: can't I just say all this in the prompt? "Don't go out of scope." "Don't decide things yourself." "Tests must cover the real path."

No, for three reasons.

A prompt expresses a wish, not a constraint. "Please don't go out of scope" and "the machine actually blocks going out of scope" are different things. The first is a sentence. The second is a gate that returns a non-zero exit code.

Instruction weight decays over a long session. That "only touch these three files" from the top is, by turn 40, competing with a hundred other things in context. It hasn't been forgotten. It's just no longer the most important sentence in the room.

The executor can't verify its own compliance. You can ask it to self-check, but that's the same reasoning chain again.

So what Loop does is promote the critical constraints from text reminders to machine boundaries: a path allowlist becomes a diff gate, "must be tested" becomes an exit code, "can't approve your own work" becomes a second party that doesn't share your reasoning chain.

One hard-won rule: constraints with a code mechanism stop being violated once fixed; constraints that rely on documentation and memory get violated forever. I've been on the wrong side of this — I wrote a rule into a doc in the morning and broke it myself that afternoon, and only noticed two hours later.

4. Who does what

Four roles, with sharp boundaries:

The most important line here: Claude's explanations and progress reports are leads, not evidence.

"The tests pass" doesn't count; the exit code counts. "We settled this before" doesn't count; pulling up the actual decision record counts.

The trustworthy chain is only this:

authoritative docs + real diff + command exit codes + independent review + frozen evidence

5. Five core objects

In plain terms, Loop has exactly five things in it:

1. Authoritative docs — the blueprints Requirements, architecture, plan. They answer why and what it should look like. They don't record how each tool run went.

2. The work order (Task Packet) — this round's job ticket What is the single goal this round, which files may be touched, what counts as done, what's explicitly forbidden. It's the one pointer the executing session has to the main line — when it drifts, you check against this.

3. The scope gate (Guard) — the site fence It only answers "did you leave the lot," not "is what you built any good." It doesn't judge business semantics. It compares paths.

4. The review prompt — what to look hardest at this round Not a copy of the whole history. The one or two questions that matter most for this change.

5. The evidence pack — the as-built file The HEAD, the diff, the gate result, the test output, the review verdict. Three months later, when someone asks "why is it designed this way," you open this — not a chat log.

6. How one task flows

A simplified example.

The architect sets the rule: after an async request fails, retries must walk a fixed candidate sequence in order — no picking freely.

The work order: this round's single goal is implementing that sequence; the router and the state machine may be touched; done means normal, failure and fallback paths all have tests; billing code is explicitly off-limits.

Claude implements: writes the code, runs the tests. Normal path passes. Failure path passes. Fallback passes. It reports: done.

The gate compares: the real diff touches only the two allowed files. Pass.

Codex reviews independently: it gets the real diff, not Claude's account of it. It finds that under concurrency, two workers can claim the same candidate and double-submit. The tests missed it, because the tests are single-threaded.

Claude fixes only that: no incidental refactor, no "while I'm in here." Fix, run regression.

Review approves, evidence freezes: this moment's diff, test output and verdict get frozen. Round over.

Notice what happened: tests green, gate passed, executor reported done — and the code had a concurrency defect. What caught it wasn't a stricter testing policy. It was a second party that didn't participate in generation and read the real diff.

7. What Loop buys you

The most concrete one: defects surface before commit. Of those 23 MAJOR findings above, a good share would otherwise have become production incidents — or, worse, the three-months-later thing nobody dares touch because "that's how it was designed."

8. What Loop is not

This section matters more than the last one. Most people who wreck a process wreck it on one of these.

Not a project board. It doesn't track who's doing what or percent complete. Your existing plan and ledger still own that.

Not an approval flow for every change. Writing a work order and pulling in a reviewer to fix a typo is self-harm.

Not letting the AI run autonomously. The opposite: it narrows and fixes where humans decide — architectural semantics, risk acceptance, release boundaries — and hands the rest to machine boundaries.

Not measuring quality by document count. Two rounds in a row that only edit rules, prompts and ledgers with no code progress: that isn't rigor, that's spinning.

Not "more review is better." When review is down to wording and formatting, more review is waste.

Not a replacement for tests, architects, or whoever owns production. Independent review isn't a compiler — it can't run your tests. It also doesn't decide product risk. That's yours.

9. Cost and where it applies

Loop costs something, and not a little. So it comes in weights:

When in doubt, standard. But these signals mean lower the weight, not add more documents:

• the change fits in one low-risk file;
• two rounds in a row touched only rules, prompts or ledgers with no code progress;
• review is down to wording and formatting;
• process-building time starts approaching implementation time.

That last one is a hard line. The cost of the process has to be clearly smaller than the losses it prevents — otherwise it's a ritual.

10. How Loop evolves itself

This is the layer most people skip, and I think it's the critical one.

The natural urge: every time you hit a process problem, add a rule. Three months later you have a rulebook nobody reads — and a rule nobody reads is not a rule.

So promotion goes through this path:

problem observed in practice
        ↓
into a lightweight inbox (record the fact; don't legislate on the spot)
        ↓
periodic aggregation: how many times has this recurred?
        ↓
past a threshold → only then consider promoting to a general rule
        ↓
rules must be testable and measurable — and **allowed to be demoted and retired**

The core boundary: problems get discovered automatically; rules get promoted by a human.

There's a trap here I fell into myself, worth naming: the inbox has its own failure mode. Mine ran for a while and 96% of what it collected was "a guard correctly blocked something" — which isn't a problem, that's the guard working as designed. The real signal was buried in it.

Recording "the system is fine" manufactures noise, and noise teaches people to ignore the inbox — including the parts that matter. That one gets its own post later.

11. How to tell if your Loop is healthy

Not by document count. By these:

• the executor spends most of its time implementing and verifying, not maintaining process docs;
• you only handle real architecture, risk and release calls — you're not a message relay;
• review catches cross-file, concurrency, fallback and security problems — if it only finds typos, the seam isn't doing anything;
• each phase's progress can be stated in one sentence of business language;
• routine operations don't repeatedly prompt for confirmation;
• rule-building time is clearly smaller than delivery time;
• the same problems get rarer — recurring ones only become rules after aggregation, and rules can retire.

If you can't hit these for a while, shrink the phase, lower the weight, or fix the work order — don't add more constraint text.

Closing: from "generating code" to "staying correct"

Back to the opening line.

One-shot generation quality is decided by the model. Models have improved enormously this year and will keep going.

Long-run delivery quality is decided by the engineering system. That part doesn't improve for free as models improve. If anything it's the reverse: the stronger the model, the smoother, more self-consistent and more toneless its errors become — and the more you need a seam where the reasoning chain doesn't carry over.

The forged approval that got caught was written by Claude. Codex caught it. But what did the work wasn't one model being smarter — run two Claudes, or two Codexes, and as long as the seam holds, you get the same result.

Human teams worked this out decades ago. It's why code review exists and why you can't approve your own PR. Loop just applies the same rule to an author who writes faster, writes more confidently, and has no tone of voice.

The better it writes, the more that seam matters.

The value of Loop is turning accidental correctness into repeatable correctness.

---

Next: how to actually write a work order — what separates a useful one from a useless one, and why the "allowed change surface" field matters more than the "goal" field.