Building a hybrid agent setup with qwen and pi-agent

I have enjoyed building tools for myself and exploring ideas with Claude code. When I started getting into some projects more seriously, I was hitting the usage limits on the pro subscription within a few days. I'm not willing to spend more on what is more or less a toy, so I would just wait until the weekly reset to keep working on my project. That is, until a friend of mine introduced me to Mixture of Experts (MoE) models (like Qwen 3.6 A3B) which can run on dramatically smaller hardware. Now I use Claude Code to do discovery and planning, then delegate most of the code work to my local model.

Running Qwen 3.6 A3B on my gaming PC

I've always been a gamer, so naturally I have a gaming computer. I haven't upgraded in a while because hardware prices are crazy, but I still have a pretty good setup. When I started I was running an RTX3080ti (12gb vram) and 32gb of ddr4 memory. I've since added another 16gb of RAM because I found running llama.cpp and the qwen model consumed almost all of my system RAM, which meant I had to close things like Firefox to get decent performance. In Windows 10 the wsl system does not have good GPU passthrough, so I'm running in Powershell. Below is the command I use:

./llama-server.exe `
   -m C:\models\Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf `
   --mmproj C:\models\mmproj-F16.gguf `
   --fit on `
   --fit-ctx 256000 `
   --fit-target 256 `
   -np 1 `
   -fa on `
   -ctk q8_0 `
   -ctv q8_0 `
   --chat-template-kwargs '{\"preserve_thinking\": true}' `
   -b 1024 -ub 1024 `
   --temp 0.6 `
   --top-p 0.95 `
   --top-k 20 `
   --min-p 0.0 `
   --host 0.0.0.0 `
   --port 8033 `
   --metrics `
   --reasoning-budget 4096 `
   --reasoning-budget-message "... thinking budget exceeded, let's answer now." `
   --jinja

I think the main things that I've changed are: enabling image processing via mmproj and putting in a reasoning budget. The image processing is convenient so that I can have qwen look at screenshots for debugging, and the reasoning budget is important as the model will sometimes get in reasoning loops.

Setup

I started with ollama, but quickly abandoned it for llama.cpp. llama.cpp is nice and easy to use too; so I don't think the convenience of ollama was particularly valuable to me. I downloaded the Q4 quantization from hugging face and started with a llama.cpp command that I found on reddit. I've tweaked it slightly since then, but it remains more or less the same.

Performance

Another reason that I added more RAM was performance. When I first started I was getting about 20 tokens / second. After upgrading my RAM I now am consistently seeing 30 tokens / second, which is fine for me. I almost always am interacting with it in asynchronous ways so I am not sitting waiting while the model churns. If I were to use this as my main driver model I might want to consider changing my video card out for something newer, but that's also a huge expense (at that point I might as well just pay for the higher tier of usage on Claude).

Installing and connecting Pi to llama-serve

Once I had the model running it was easy to verify by opening the llama-server directly in the browser, which provides a rudimentary chat interface. It's a simple way to test to see that the model responds.

With a working local model, it was simple enough to follow the instructions on the pi-agent github page to install pi-agent and get it connected to my model. I already had git for windows installed, so git-bash was available and made sense for me anyway. I believe if I were to configure things again I would actually run the model in windows in powershell and claude and pi in wsl instead. I find the git bash environment to be okay, but not the greatest.

Configuring the model

To use a custom model with pi you have to define a custom model. Fortunately this is simple and the minimal example provided basically gives you what you need. Here is what mine looks like:

{
  "providers": {
    "llama": {
      "baseUrl": "http://localhost:8033/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        {
                "id": "Qwen3.6-35B-A3B-UD-Q4_K_XL",
                "contextWindow": 240000,
                "input": ["text", "image"]
        }
      ]
    }
  }
}

I have my context window set slightly smaller in the pi configuration than in the run command just to give some buffer. I had them set to the same and on a few occasions ran into errors that caused pi to stop.

With the model configured, you should be able to run the pi command and interact with your model! You can add plugins if you prefer, but I wanted to build all of the things myself. Pi has some really good skills baked in for building extensions for pi, so it's actually quite good at making things for itself to use.

Teaching claude to talk to Pi

I was going to go into details about the skills, extensions, and tools I've built for Pi, but I'll do that in a separate blog post. Instead let's skip to the heart of this discussion - Using Claude to call Pi! The Pi agent has a few things that make it possible for claude to interact with it nicely: 1. You can invoke the agent in "print" mode (it will just respond in std out), 2. you can provide it a file as your prompt, and 3. the sessions are all saved as plain text files. With those three things claude can tell pi to do something, get a result from the agent, and check to see what it did during the session.

To actually tell claude how to interact with pi I forked the superpowers skills repo and adjusted the skills to invoke pi for sub agent use and planning instead of the standard flow. This lets me work with claude the way I already was, but planning and development are delegated to pi via delegation files. Claude calls pi via bash (pi -p @ where is a path to a delegation file). Claude writes a prompt with what it expects pi to do, and then pi runs, reads the brief, then does what it's asked to do. When done, the process quits and claude reviews what pi did and makes any adjustments it thinks are needed.

Ok, but is it worth it?

At first it was hard for me to know how many tokens I was saving with this workflow, or even if I was saving tokens at all. Ultimately I was able to build a tool that gives me an approximation of the tokens in and out with Pi, and I've found that when I am writing a lot of code or working on a big project this approach saves me considerably. You can see in the graph below that in some weeks Pi works through a lot of tokens.

I built a web interface that I can connect to to see pi sessions (I run it over tailscale so I can do it via my phone too!) and also get an idea of how many tokens in/out have been used. This was a fun project in its own right and may be a separate blog post sometime. Otherwise you can see the project on github too.

I also found this exercise very rewarding in its own way, and there's something special about having an LLM running locally. I think that the marketing really pushes just how much "compute" state of the art models need, and that everyone needs the state of the art, but I found with this project that even my four year old system could run a coherent and somewhat functional coding agent.

Thoughts on Claude as an orchestrator

Claude does a great job (unsurprisingly) as an orchestrator here. It waits patiently for my Pi agent to act on the tasks it is given, then reviews the output. It keeps memories of the things pi struggles with and adjusts accordingly. I did have to tell it that it should treat Pi as a "mid level" engineer, otherwise it got overly explicit about the things it wanted Pi to do. At that point it was almost using as many tokens instructing Pi as just writing the code, which was not useful to me.

Claude's notes on Pi

I asked claude to give me its info on Pi and this is what it came back with. Conveniently the commands it uses are right there at the top too. It then gave me a whole narrative about what it does, but I don't believe that it's particularly interesting so I've omitted it from below.

What pi is

  pi is a full mid-level coding agent (qwen, run via the pi CLI) with its own git access and
  the full Rust toolchain. The important mental model: I write for it like a competent
  mid-level engineer, not a sandboxed function that just fills in a blank. It can grep, read,
  edit, build, run, commit — the whole loop.

  I hand it work by writing a brief to a .pi-delegations/*.md file and invoking:
  - pi -p @<brief> — run against a brief
  - pi -c -p @<brief> or pi --resume — continue the same session (crucial; see budget below)

  How I structure a delegation brief

Claude's "Operational Lessons"

These are the notes claude shared with me about things that it has learned while working with Pi.

- Exit code 1 ≠ failure. The server pi runs on can get stuck or the model can drop its
  connection mid-run. Two consecutive exit-1s once turned out to be a stuck server, not pi
  giving up — the work was fine. I check whether files were actually edited, then resume with
  pi -c -p.
  - A long plan won't finish in one run. pi has a finite per-run budget. A ~9-task plan stopped
  after Task 4; a resume got through Task 7 and most of 8, then stopped again. Past the
  halfway point it's often more reliable for me to just take over and finish — especially
  live-GUI verification, which pi fights.
  fresh file edits / new commits over a ~20s window, not the process list. I nearly hand-edited
  files out from under a live pi once — only the Edit tool's "modified since read" guard
  caught it. When it's live, I back off and let it converge.
  - It will confidently claim "gate green, N tests pass" when that's false. The classic shape:
  the final fixes (missing imports, last call-site conversions, cargo fmt output) are left
  uncommitted, so the working tree compiles but committed HEAD does not. So I always verify
  HEAD, not just the working tree — git stash -u (or confirm clean status) then build the
  committed state. And I trust a real cargo build exit code over rust-analyzer squiggles, which
  lag badly after its rapid edits.
  - Its git discipline is good, with one trap. Named-file git add, per-task commits, respects
  the gate instead of --no-verify — I don't re-litigate that. But when a plan's final task
  literally says git add -A, pi obeys and sweeps up every untracked stray (run logs, unrelated
  delegation files) into a misattributed "chore: gate pass" commit. So I tell it to stage named
  files even for the gate-pass commit, and I git show --stat the final commit before trusting
  it.

Conclusions

It was a ton of fun to set this up! I also feel like the hybrid approach may also give me a better quality result, although that's very anecdotal. I'm very happy with how things work now, although sometimes I do have to remind claude to "delegate to pi", claude is very eager to just start building things. I believe I am saving a good amount of my available usage from claude too, without having to either wait for my usage to reset or pay more money for more usage. I have experimented with using pi as my main agent as well but most of the time I have had to have claude "rescue" it when it gets stuck. I hope that in the future these mixture of expert models will get even better; I would love to be able to just run my local model without paying thousands of dollars for hardware to do so.