Software Is Not Made of Code

Software is not made of code. It is made of decisions — what to build, what to verify, and how much of the work to trust to machines. Code is only where those decisions come to rest.

For fifty years we believed otherwise. The idea of a “software factory” goes back to Bob Bemer’s 1968 paper, “The Economics of Program Production,” and for half a century the dream has rested on a single assumption: that development can be turned from an isolated individual craft into a repeatable, measurable production process. The dream has mostly failed. Stamping out ideas is not, in the end, like stamping out car parts.

Over the past two years, the situation has changed sharply. LLM-driven coding agents have given the old dream a new technical foundation — generation has become cheap, fast, and effectively unbounded. And precisely because the excitement is easy to get lost in, candor is required. Some of what looks new is genuinely new; some of it is a recurring trap wearing the clothes of a new opportunity. The difference is worth being exact about.

If a diff is an abstraction of the work and a test suite is a projection of its correctness, then verification is the underlying reality of software. Everything in this essay follows from that sentence.

The stack: loop, harness, factory

The whole architecture can be understood as three concepts layered on top of one another: the loop, the harness, and the factory.

The first kind of unit is a loop. A loop is the smallest unit of agentic work: an agent repeatedly running gather context → take action → check the result → go again, until some condition is met. Its quality standard is whether it terminates usefully, and whether that outcome can be checked cheaply. The central shift in loop engineering is that you no longer prompt the agent by hand, turn after turn; you design a small system that prompts the agent for you.

The second kind of unit is a harness. A harness is the boundary around the loop: the sandbox it runs in, the tools it can reach, the memory that persists between runs, and the gate that decides what “done” means. The loop is the behavior; the harness is the environment that behavior runs in. A raw model with no harness around it spins endlessly. The harness is everything around it that lets it run usefully and safely.

The third kind of unit is a factory. A factory is many harnessed loops running at once, fed by a work queue, funneled into production through review gates, with humans steering direction at the top. It is not a bigger agent; it is an org chart made of loops.

The final paradigm shift is from writing code to building and running the factory that writes the code. The unit of work moves up a level — to loops, to harnesses, to the flow between them — rather than a single code diff.

The closed loop and the stubborn node

An agentic software factory forms a closed loop: intent and production signals flow into a backlog queue, a harness picks items off the queue and builds the change, automated checks gate that change, deployment ships it, and monitoring turns the state of production back into the signal that drives the queue.

In this architecture nearly every box is extremely cheap. Generation, testing, scanning — all of them run at scale for negligible cost. Only one box proves stubbornly resistant to scaling: the review gate. That node, the one that stands for judgment, is where the entire argument lives.

The dark factory and comprehension debt

The term “dark factory” is borrowed from manufacturing. A physical plant runs with the lights off because there are only machines inside, and machines do not need light. In software, a dark factory means code that ships without any human reading it at any stage, validated only by other machines.

This is astonishingly easy to achieve, at least at first. It is easy because the review step that gets skipped is the one blocking everything else. Its absence makes a team’s perceived throughput rise suddenly and steeply. But surviving in a dark workflow over time is far harder than it looks, because such workflows carry a hidden cost: comprehension debt.

Comprehension debt is the widening gap between how much code exists and how much of it any human still understands. A dark factory does not pay that debt down; it takes it on as fast as it possibly can, with the tests green the whole way. Models perform well on certain tasks, but for any work that is not an immediate change to a small part of the codebase — especially inside a complex existing system — purely model-driven automated coding runs into an insurmountable barrier. An enterprise system a decade or more into its life has to be maintained in a professional environment, at a professional cadence. Three to six months into a project, the team is drowning in code no human has ever read.

What a dark factory is genuinely good at is consuming clean code quickly while the tests stay green. The reckoning is not a dramatic blow-up. It is quiet, and it arrives late.

Why verification is the linchpin

The fundamental limit of a software factory is not how much code we can produce, but how fast we can verify it.

Back pressure is the underlying rule: you can grant a loop exactly as much autonomy as the portion of its work you can verify cheaply and reliably — not an inch more. Verification, not generation, is the factory’s real constraint. There is a permanent tension between unbounded generative capacity and human attention, which is finite and does not scale. Volume alone is not the issue; what does the damage is a surplus of bad pull requests. When you have high output and no trustworthy gate, defects are inevitable.

Every production revolution answers a scarcity. The assembly line answered scarce labor. The software factory answers a scarcity that has never been cheaper to ignore: human attention. There is a second-order question here: why should improving the model not automatically close the gap between what it can generate and what can be verified? Because the cost function that measures architectural excellence is not counted in seconds or minutes but in months and years. A smooth gradient is functionally incomputable, so a system expected to deliver a clean, immediate judgment on complex design decisions cannot be trained on good examples. The signal for architectural quality is inherently sparse and delayed. The model will get better; the gap will not close by itself.

The lit factory: judgment, moved upstream

A lit factory is the same pipeline, with judgment left where it belongs. Agents still do most of the building, but humans read the output before it ships, and human attention is continuously present wherever a wrong call is expensive.

The key difference in the lit version is not review bolted onto the end; it is moving the node of human judgment upstream — into product, design and architecture — before the agent starts a loop. That up-front investment buys fewer implementation hours. It turns a long, dispiriting code review into a quick read of a two-hundred-line plan. The decision gets human scrutiny before its cost compounds.

That safety net is made of entirely ordinary architectural practices we have always known and mostly ignored: good types and method signatures, so errors are caught at the compiler; test seams, so behavior can be pinned down and change made observable; a clear code layout, so the next reader — human or model — knows where to look for what they care about; short, readable call stacks; well-defined component boundaries, so a single change does not have an enormous blast radius; and dependency injection, so one component can be swapped for another.

None of this is new. We have always said we cared about good architecture, but now that we are working with automated coding agents, that architecture finally has a second job: to serve as a cheap, hard-to-fake safety net against the mistakes agents make. That net has to exist outside the model, because the model will not supply it. The strongest coding agents are reinforcement-trained on their own harness and tools — fluent in the industry’s tooling and idioms, but not in concerns like long-term maintainability. Thoughtful architecture is the instrument that catches that debt, and our investment in it is how we buy autonomy back.

What earns a loop its automation

A loop earns full automation only when its checks are cheap, run at high frequency, and rest on something that is hard to fake. A green-or-red oracle, a type gate, property tests, and a review agent with a real rubric all qualify. You also need the oracle to answer immediately and not drift over time. Automation is reached when “done” can be proven not only by you but by a machine.

Short loops are easier to verify than long ones. A rule of thumb: an agent holds steady for three to ten steps, and starts losing the thread past twenty. The cause is context accumulation — the more an agent drags along, the more likely it is to wander off topic. When a loop is short, verifying it is cheap; sprawling loops hide their errors in the corners, which is another way of saying they never earned automation.

Keeping a human in the loop is the reverse case. When a wrong answer is expensive and only a person will catch it, the loop needs review: subtle production bugs the tests will not catch, a large blast radius, and decisions that shape the next year of work or more all qualify. In those cases human attention is the actual product — expensive and indispensable.

The real danger is not choosing all-dark or all-lit; it is forgetting to evaluate each loop on its own. All dark, and months later you are tearing everything out. All lit, and the review bottleneck swallows the whole process. The hard, skilled work is deciding where each switch goes.

Back pressure, drawn as a diagram

When you hand an agent a task, you will very likely end up building a graph around it — whether you call it a finite state machine or a set of conditionally linked service calls. It is a frame in which software is not merely following abstract rules but a structured workflow: every node an explicit step, every edge an explicit condition.

The genuinely notable insight here is that software always had that structure. Any code can be expressed as a control-flow graph. The truly new move was trying to throw that graph away and rely on a pure loop, letting the model pick its own path one tool call at a time until it declares itself done. That feels like liberation, right up until it meets a decade-old codebase. The discipline everyone is now rediscovering — owning your own control flow — is just walking that graph back around the loop.

The real value of the graph is that it is back pressure drawn as a diagram. You give up some of the agent’s freedom in exchange for enforced checks and a legible point of failure — when a run fails, you can point precisely at the node that killed it. This echoes a key observation: most so-called agents are not agentic at all; they are mostly deterministic code with an LLM step introduced at exactly the right point. The pattern recurs in LangGraph, LlamaIndex Workflows, and the various hybrid workflow-graph-over-agents architectures. At bottom, this is state machines and the actor model returning under contemporary technical conditions.

The human role: the outer loop

Humans never left the factory — they moved.

An agent can investigate a bug, write the diagnosis, implement the fix, run the tests and write the report. That is inner-loop execution, and an agent can do it as efficiently as anyone. But that was never the whole of the job. What humans own is the outer loop: deciding whether this is even the right way to handle the problem, verifying that the diagnosis and implementation are sound, approving the change, and carrying the consequences of a misjudgment.

The boundary between the two loops is evidence — the diff, the tests, the logs, and a short account that ties them together. Types, seams and rubrics let you maintain effective oversight without doing heavy work on every change.

There is a great deal you can do to make the model better and the harness more capable, but identifying the problems that prove expensive over the long run is usually not something that can be automated. What remains at the core of human work is exercising human judgment better than any flow of compute can.

Where the boundaries are collapsing, and what comes next

The central proposition of the software factory is not to replace people but to redefine where human judgment should be deployed. Dark and lit are not an either/or choice; they are a decision matrix that has to be evaluated loop by loop. The boundary between automation and review is not fixed — it moves wherever verification becomes cheap, frequent and hard to fake, and it holds wherever a wrong call is expensive.

What comes next is not a single mode, and not a single model. It is a factory in which autonomy is granted exactly as far as verification can reach — loop by loop, harness by harness — and in which the most valuable engineering work is no longer writing code at all. It is continuously judging how much autonomy each loop should get.

Software is not made of code. It is made of decisions, and the factory’s true product is knowing where each decision gets made. The lights are not on or off; they are a decision matrix, and someone has to flip every switch.