Reducing Blast Radius

A first step to build a trustworthy harness

Trust is the most essential component of any agentic harness. In a previous post I discussed how to start doing harness engineering on legacy projects. Most of that post is abstract; it talks about questions you should ask or things you should think about when getting started. While it's important to have answers to those questions, I also wanted to provide some specific approaches that make the first steps into harness engineering successful. A successful harness engineering workflow requires: A trustworthy harness, precise scoping, and good analytics. This post will focus on specific ways to reduce the scope of the agents changes. Reducing scope reduces risk, increases the ability of humans to verify what the agent did, makes it easier to get comfortable fixing the harness to correct mistakes, and lets developers see what the agent will do. These things will help developers build trust that their agent will not (or can not) go off the rails and do something dangerous.

Reducing the scope of what the agent does, has access to, or how the resulting code reaches customers can make it much easier to trust the agent's output. This approach also helps focus otherwise daunting questions: "How do we know this code is good?" can be rephrased to "How do we know the new user administration screen is good?", which is far easier to reason about. It also makes it much easier to measure the success and failure of the harness simply with human review so you can start to see where the harness is failing and succeeding.

Reduce task scope

Pick a small subset of tasks (a new feature, some technical debt, or maybe a specific view of your product) and use your harness to complete that small set of tasks. Remember that harness engineering is a process, so follow the process for these tasks. For example, you could create an epic in jira and add the tasks that you want your agent to complete to that epic. Then have your agent work each task on that epic fully autonomously. Make sure to only change the harness (don't iterate with the agent) when the output is incorrect.

Once you've completed the epic spend some time reviewing how the agent performed. Consider the feed forward and feedback adjustments you need to make. Did the agent introduce a bug that better e2e tests could have prevented? Did it spend way too much time figuring out how a specific feature works? Answering these kinds of questions and iterating on the parts of the harness that caused problems is harness engineering.

Other ways to reduce task scope

Finding a project that the team wants to complete but doesn't have the capacity to complete is a great place to start. Think about places where you / the team has wanted to make improvements (high reward) but haven't been able to. You may want to use stacked PRs to make it easier to review the full outcome without having to review many individual pull requests carefully (You can review the full result (final PR) for product bugs and then go back and gradually merge the lower PRs for code quality).

Here are some example ideas:

• Automatically triggered when a github issue has a specific tag

• Incrementally work through dependency upgrades

• Gradually replace deprecated functions

• Incrementally follow an existing pattern (Replace all class-based react components with function based ones. Use <link to a PR> as an example)

Reduce write scope

Reduce the "blast radius" that the agent can affect on its own. You can do this by restricting your agent to what directories the agent can write to. Either focus the agent on a specific part of your application (this probably would restrict it to several directories - at the very least the backend and frontend parts) or focus the agent on your directories with the best test coverage. Test coverage gives you an automated safety net, and feature-focus gives you a manual safety net. Below are two specific examples:

Restrict by test coverage

Imagine we have an application with a lot of legacy untested code, but all of the new functionality has good test coverage. This new code is somewhat isolated - the developers didn't want to mingle their nice new code with the legacy stuff - so they created a new directory inside the existing ui directory called app-react. The team has been good about writing tests for everything inside this directory, which will help catch bugs or hallucinations. The team is comfortable letting the AI "loose" on this work, trusting their existing guards to catch issues. Each AI provider has their own system for restricting access to directories, for Claude you can specify the permissions for directories in the .claude/settings.json file, like this:

{
  "permissions": {
    "allow": [
      "Edit(/ui/app-react/**)"
    ],
    "deny": [
      "Edit(/ui/legacy/**)",
      "Edit(/app/**)"
    ]
  }
}

The two lists do different jobs, and the difference matters more than it looks. allow means "don't stop to ask about this" - it is what lets the agent work through app-react without a human in the loop. deny is a hard block. Anything that matches neither list falls back to prompting, which during an autonomous run means the agent stops rather than proceeds. That fallback is doing a lot of the work here: you do not have to enumerate every directory in the codebase to keep the agent out of it.

The leading / anchors each pattern at the project root, because these rules live in the project's .claude/settings.json. A bare ui/app-react/** would anchor to whatever directory the agent was started from instead, which is the same thing right up until someone runs it from a subdirectory.

Restrict by feature

Restricting by section of the application still relies on the same directory approach from the test-based approach, but instead of focusing on areas with good test coverage you configure your agent to have access to whichever directories power the backend and front end of that part of the application. Below is an example configuration, restricting the agent to work on the Tasklists feature:

{
  "permissions": {
    "allow": [
      "Edit(/ui/app-react/tasklists/**)",
      "Edit(/app/models/tasklist.go)",
      "Edit(/app/api/tasklist.go)",
      "Edit(/app/db/tasklist.go)"
    ],
    "deny": [
      "Edit(/deploy/**)",
      "Edit(/.github/workflows/**)"
    ]
  }
}

This might get tedious to manage by hand, but you can also have your agent generate the permissions for you and then you can verify before using them.

Two details worth knowing before you rely on this. Write the rules as Edit(...) even when you mean "create a file" - Edit covers every built-in file editing tool, and a Write(...) path rule is accepted but never actually consulted. And these rules bind the agent's file tools and the shell commands Claude recognizes as file operations, but not a script that the agent writes and then runs to edit files on its own. If you need a boundary that holds no matter how the file gets written, you would need to use some kind of OS-level sandbox.

The deny list matters most if you run the agent with permission prompts skipped entirely, which is a tempting shortcut for autonomous work. In that mode the allow list stops meaning anything, because nothing was going to prompt anyway. Deny rules still apply. They are the only part of this config still holding the line.

Reduce exposure

Use feature flags or other mechanisms to prevent fully automated code from reaching customers. This moves the human verification out from the pull request and into the product (which has its own problems), and is a great way to prove that the validation part of your harness is working as expected. If your agent autonomously produced the entire feature you can review the new feature and see where the validations failed to prevent bugs, went off the rails from your feature docs, or missed critical design requirements.

This could be a very useful way to move forward if you trust the agent to operate within the whole application, but it definitely is the most risky as well. An agent could easily modify part of the application outside of the feature unintentionally so careful PR review or combining it with directory restrictions like above could be a good approach. The only downside to feature flags is they only apply to user facing work, limiting their usefulness in other tasks.

Conclusions

Restricting what an agent can do inside your codebase is a great way to start experimenting with true harness engineering practices without going "all in." You can choose to restrict by tasks, directories, features, or ideally mix and match these strategies to something that makes sense for your team. Restricting the agent from modifying code inside deployment related directories or files would be a good combination with any of the other approaches.

Working this way also gives you something to measure. Once the restrictions are in place and you are fixing the harness instead of the code, watch how often you have to step in: is it every run, once every few runs, or rarer than that? That rate is a way to tell how trustworthy the agent actually is, and watching it over an epic tells you both how the agent is failing and which part of the harness needs the work.

No matter what approach you take, using the harness engineering process to complete work should naturally make your team move faster - as you reduce the mistakes the agent makes you can trust its output more, and you can expand what the agent does for you.