Harness Engineering for AI Agents: The Foundation for Loop Engineering

I want to start with a PR (pull-request) that changed how I think about shipping AI-written code and then show you why it turns out I wasn’t just having a bad week. The recent industry benchmarks on AI agent harness engineering suggest the issues I ran into are practically universal.

One of our engineers, working on a project, asked Claude for a webhook handler and had working code back before he could finish brainstorming with his colleague on the topic. Genuinely working, it compiled, the logic held, and the happy path was clean. Upon review, it turned out it had pulled in a logging library we had banned two years back after a CVE (vulnerability) was found. The code was written to update the database directly without a tenant scope, which is a severe security risk, as tenant isolation is the only thing preventing unauthorized cross-account data access. And it had also improvised its own error-handling routine that matched nothing else in the service.

The model wasn’t at fault; the code was correct. It just was not our code. So, a senior engineer spent the better part of an hour writing comments on the review. These comments were basically teaching us the rules we had written down a while ago on a wiki. Nobody looks at this wiki except when they’re new. This happens with every engineer, every repository, and every session.

Fresh out of the box, the model has no memory at all. Every API call starts from scratch, so whatever it learned on the day the webhook handler got generated is gone by the next day. The tooling is trying to fix this. Claude Code now ships memory files and even takes its own notes between sessions. Handy, right? But look closer at that “memory,” and it loses the plot fast. It’s just a pile of markdown files, one per repo and one per developer, kept up by whoever bothers, versioned by nobody, and checked by no one. So, your best prompter’s Claude keeps getting smarter, while everyone else’s shows up and draws a blank. The speed you paid for still leaks away at review time, not because memory is impossible, but because nobody made it into an organizational asset. And that quietly slows everything down.

What The Research Says About Agent Harness Engineering

Here’s the part that surprised me when I went looking. The interesting question in agentic AI right now isn’t “which model.” It’s how much of what you get depends on the harness, the software wrapped around the model that feeds it context, runs its tools, catches its mistakes, and checks its output before anyone downstream sees it.

Someone finally measured it properly. A benchmark called Harness-Bench ran the same 106 tasks through six different harnesses across eight model backends, a few thousand runs, holding the model constant and varying only the shell around it. The best harness scored 76.2%. The worst scored 52.4% on the same tasks and same models. A 24-point spread that had nothing to do with the model at all.

A 24-point difference is bigger than what you get when you upgrade to a whole new model generation. Another study that looked at results from across the field found the same thing from a different perspective. They found that making changes to the harness, such as changing the tool formats or restructuring the prompt and middleware layer, could lead to big improvements. They saw improvements of 10 X on coding benchmarks and big gains in accuracy, all without changing the model. This is much bigger than the 2-to-4-point improvement you usually get from upgrading the model. (Agent Harness Engineering: A Survey).

The data points to one conclusion: beyond a certain point, a better harness beats a bigger model. That messed up PR wasn’t bad luck; it was the plumbing, and that is why we built Hexaware’s Zerovity™.

What is an AI Agent Harness?

Let us clear up some common confusion first, since these terms often get mixed up. A model takes text in and gives text out; it has no memory between calls and no way to act on its own. A harness is everything around the model that turns a text generator into something that gets work done. It decides what the model sees, runs its tool calls, manages state, stores what matters, and evaluates the result. The equation is simple: agent = model + harness.

The most comprehensive breakdown I found divides an AI agent harness into seven layers: execution, tooling, context, lifecycle, observability, verification, and governance (the survey). The clear imbalance between well-supported layers and overlooked ones is particularly striking. Across 170-plus open-source projects, the execution, tooling, and verification layers are crowded. Observability and governance are thin; they mostly live in commercial products. The omission is no coincidence. It’s the whole reason enterprises can’t just glue together open-source parts and call it done. Production agentic AI infrastructure needs the layers nobody open sources. It also needs something for those layers to act on: an accurate, current picture of the application itself. That is where Hexaware’s Zerovity™ comes in.

Zerovity™ parses an application’s code, architecture, dependencies and business logic into a connected knowledge graph — a working model of how the system actually behaves, not how the documentation says it does. That understanding is the foundation for everything downstream: maintaining an application, modernizing it, and now handing an agent the context it needs to change it safely. A harness is only as good as what you can tell the model, and telling it precisely how your application works is the hard part. That is the part Zerovity™ solves.

Zerovity™ specializes in full-scale application understanding for maintenance and modernization. Now, we are applying that same intelligence to individual pull requests. Look at what enters the model and what gets checked on the way back out.

Research calls out four things carry the weight, and each one maps onto one of the layers.

• Context that sticks around. Before the request reaches the model, Zerovity™ assembles a repository-specific bundle: conventions, architecture decisions, data-handling rules, the dependency allowlist, a map of the modules in play, and hands it over as structured context, every single request. The model stops reconstructing your environment from whatever files happen to be open. Going back to the day the issue happened, the tenant scope rule and the banned library rule both existed. They just weren’t anywhere the model could see them when it mattered. This is the fix. And yes, you could hand-write some of this into a CLAUDE.md per repo — the difference is that Zerovity™ generates and governs it centrally, so the bundle is consistent across a hundred repositories and in sync with your actual architecture decisions, and not dependent on each team remembering to maintain their own markdown file.
• Memory that’s yours. This one’s about strategy, not just tech. LangChain made a point that stuck with me. Managing context is really managing memory, and whoever owns that memory owns something the model maker can’t copy. It’s a running record of how your company works. The model itself is rented, and you could swap it out tomorrow. But the conventions you’ve built up, the policies you’ve written down, the trail of what got approved and why, all that stays with you. That’s the real edge. Not a smarter Claude, but a system that knows your business so well, a generic model can’t catch up. As teams adopt vendor-specific features — skills, hooks, subagent configs, tool-native memory formats — pieces of “your memory” quietly get encoded in someone else’s format. Zerovity™ keeps context and policy in a vendor-neutral layer, so what you’ve accumulated survives a model swap and a tooling swap.
• Guardrails that are code, not vibes. (Governance + Verification) The diff doesn’t go straight to a human. It hits a set of AI coding guardrails — a policy gate, secret scanning, license and dependency checks, architecture fitness rules like “nothing touches the database outside the repository layer” and “every query is tenant-scoped.” A violation isn’t a polite suggestion the model can ignore. It’s a hard failure that bounces the diff back with the exact rule it broke, before a human spends a minute on it. You don’t ask a probabilistic system to remember the code security You check, deterministically.

Take dependencies, for example:  Ask an AI tool for code, and it’ll point you at packages that were never real — a 2025 study of 576,000 code snippets found that it happened 19.7% of the time. Attackers noticed this and registered the invented name, shipped malware under it, and waited for the next install. It’s called slopsquatting. A better model reduces this rate but never hits zero. The policy gate doesn’t care how smart the model is; if it is not on your allow-list, the write never happens.

Even the model vendors agree: Anthropic’s own documentation says instruction files are treated as context, not enforced configuration — if you need to actually block an action, you wire up a deterministic hook. Written-down rules are suggestions; enforcement is code. This matters more than it sounds: Harness-Bench, a benchmark that ran ~5,200 agent trajectories across 106 tasks, eight models, and six harnesses, found that the most common failure symptom among failed runs was output contract violations, which is the model producing something structurally wrong rather than reasoning wrong. The paper’s own conclusion is the point of this post: agent capability isn’t fully characterized by the base model alone — the execution layer that mediates observation, action, recovery, and output matters just as much. That’s a plumbing problem, and plumbing is fixable.

• One way of doing things, org-wide. (Lifecycle + Observability) Without a harness, output quality quietly tracks whoever wrote the prompt. Your prompt whisperer on team A gets gold; the new hire on team B gets bronze, and nobody can say why. Zerovity™ pulls the prompt, the context bundle, and the gate config into versioned artifacts the org owns, so two engineers on two teams get held to the same bar because the invocation is shared, not personal craft.

Centralized AI agent governance where one rule is authored once and enforced on every surface:

The reviewer’s job scope changes. It ceases being “catch the banned import for the hundredth time” and becomes “is this the right design?”, which is the only part that ever needed a human.

Our Benchmarks: What Changed with a Harness in Place

External benchmarks tell you the category works. Here’s what it did on our repos. Same Claude model version in both arms, same settings, same task set of tickets across the same repos. Only the harness changed.

The standout number: conventions passing on first emit jumped from 66.7% to 99.3% — The model now generates accurate, on-policy code on the first attempt & no review comment is needed to remove a banned import or add a missing scope.

Two unremarkable mechanisms account for the change, and both confirm what the research predicted. Context injection gives the model the rules outright instead of making it guesswork, so many more diffs come out clean on the first pass.

Where AI Agent Harness Engineering Falls Short

A harness isn’t magic, and anyone who says otherwise is selling. The honest failure modes, several of which the literature is blunt about:

The returns aren’t uniform. This is the one that keeps me honest. Harness-Bench found stronger models swing less across harnesses; the big gains cluster on mid-tier models. It doesn’t vanish; the governance and consistency value is real regardless.

Stale context is worse than no context. If an architecture decision changed in the code but the doc in your store still describes the old way, the harness will confidently march the model toward the “wrong but documented” pattern, and because it now arrives wearing the org’s authority, reviewers trust it more, not less. Wrong and credible is the nastiest failure mode there is. Treat the context store like code: an owner, a review cadence & an expiry.

Long jobs drift, and nobody’s fully solved it. The survey is candid that agents lose coherence past ~100 turns, repeating work, contradicting earlier decisions, losing the goal, even with aggressive context compaction. Reinjecting acceptance criteria mid-task (which we do) helps. It is not a cure. There’s a ceiling on how autonomous any of this can safely be, and we’re under it, not above it. And that ceiling matters more as agents are increasingly run for longer, with less supervision.

And it does not delete the reviewer. Every serious source in this space: the researchers, the guardrails vendors, lands in the same place: humans move up the abstraction stack, toward judgment, not out of the loop. Anyone pitching you a fully autonomous merge without review is treading a dangerous path.

Should You Build an AI Agent Harness?

Run the cheap experiment before you commit to anything I’ve said. Take one repo. Pull the last twenty AI-generated PRs. Count how many failed first review purely because of a rule the model could have been told up front, a banned dependency, a missing scope, or the wrong pattern. That count is your addressable rework, and it’s the ceiling on what a harness gives back. If it’s substantial, and in most enterprise repos I’ve looked at, it’s large, that’s your number.

The whole lesson summarized: the teams struggling to scale enterprise AI coding didn’t buy a weak model. They bought a strong one and dropped it into an org it knew nothing about. Correct and shippable are different words, and only your org knows the second one. Zerovity™ is just the part that teaches the model where it’s working, and then checks, before anyone else has to.

Everything above assumed a human reviews every change. That assumption is starting to break: the same coding agents are beginning to run unattended, for hours, with many agents running in parallel. Harness engineering makes one of them trustworthy. The next problem is supervising a hundred live agents without reading every transcript. The same gate primitive, now with a control plane behind it. That’s loop engineering. Coming next…

Want to run that benchmark against your own repos instead of taking my word for it? Request a demo