Multi agent QA testing is when you use separate AI agents to spot problems, choose how to fix them, and then write the code changes, and you keep a written log for every step instead of one agent trying to do everything.
We don't split them up to go faster. The reason is that an agent which writes its own fixes will talk itself into believing they work, and one that checks its own code will just approve it.
We put this workflow through its paces on a big UI review in ChatFuse, and the clock was ticking. Here's what worked and what didn't.
Why does multi agent QA beat one agent doing everything?
Because looking, judging, and fixing each break down in their own way, and one AI trying to handle all 3 just buries those breakdowns. Tell a model to both find bugs and correct them, and it will only find the kind of bugs it already knows how to correct.
Keeping the jobs apart creates a paper trail between finding a problem and solving it, which means nothing gets lost along the way.
Telling that same model to check its own work means it verifies that the code matches its own instructions, but that isn't the same as verifying the actual behavior is correct. The main type of mistake we saw was "the interface appears right but the action it performs is wrong," and a solo model is simply built to miss that, because it's grading the very thing it just made.
What are the three roles in ChatFuse multi agent QA?
ChatFuse runs 3 roles: orchestrator, implementer and QA. An orchestrator figures out the root of a problem and manages the tracking doc. An implementer does all the application coding. QA hunts for bugs and checks that fixes work. The lines between these roles are the critical piece of the entire setup.
The human stays in charge for product calls, design choices, code commits, and anything with money or a third party dashboard. How this setup handles growth past QA is what we cover in the AI org chart.
Do the boundaries actually hold?
Not by themselves. A ChatFuse role boundary got crossed, and that cost us. Both agents did it. One tried to handle an implementation job. We had to tell it directly: you coordinate, you don't write code. The other put 3 findings right into the tracking doc, even after we said not to.
Those 3 findings ran right into the IDs the first one was giving out. Sorting that mess took real work. So you have to state the boundaries, again and again. They aren't a one time thing. An agent sliding back over the line is what usually happens. It isn't a breakdown.
Why does only one agent own the document?
If more than one AI agent can write to a ChatFuse tracking document at the same time, you get ID collisions. That's exactly what broke things when this rule wasn't followed. So we stick with one writer. The QA agent spots findings and tells the orchestrator. The orchestrator gives an ID and writes the entry.
It's not the fastest way to handle each finding. But it's why nothing disappears between the moment it's found and when it gets fixed.
What makes a finding useful to an implementer?
A clear writeup means the person building it doesn't have to figure out what you meant. You need 3 parts: how you got it to happen, what it actually does, and what it's supposed to do. Put each one in its own line.
A vague report just gets interpreted by someone else, and that's how the wrong solution gets made. That difference costs a lot more time and money later on.
Saying a button looks off starts a discussion. Saying the selected field shows 2 circles and the outer one gets cut off is a ticket that gets fixed.
When should you not use this?
When a job is a single, well defined feature, or when you don't have an agent that can run the show. A solo developer agent will get a straightforward feature built faster. The extra work of managing things only makes sense when your list of findings is too big for one context window, which is what you get from a full audit of a big system.
That second scenario is the big one. Without a dedicated coordinator, this whole process falls apart. Its main value is making sure every discovery actually makes it all the way to a fix. If you drop the coordinator, you just end up with 3 agents doing the same work and nothing gets written down.
Can you run more than one implementer?
Yes. ChatFuse brings in 2 implementers to rebuild a surface, but only if they first agree on who owns which specific files. We tried having a second one start while the first was still busy, and it only worked because each one had their own files. Without that clear ownership, they just crash into each other.
2 agents working in the same space without that agreement create problems they can't fix on their own. You end up spending more time cleaning up the mess than you ever saved by having them both work at once.
Does this need different models for each role?
It helps, but not because any single model is superior. They all fail in their own way. So one might miss something that another catches, while a group of the same agents will have the exact same blind spot. We run a capable model on the reasoning, a quick one for coding, and a third to check the work.
You can see this idea in action for cross model code review or read about why each job gets its own model in multi model agent team. You get over 100 models from Google, Meta, OpenAI, Anthropic and others on ChatFuse, so you can pick the right one for every task.
What is the actual failure mode of multi agent QA?
Silent dispatch. It just means you've told someone to do a job and you've logged it as done, but you never actually checked it went through. A message that's still sitting in a draft, an agent that got blocked by a rate limit, or a standing order that no one saw, all these look exactly like work is happening.
Nothing tells you they've stalled.
You can't just assume things got delivered. You have to look and see what really happened before you mark it sent.
How do you know an agent is working and not stuck?
A ChatFuse coordinator can't just look and see if an agent is stuck; from the outside, a broken one and a working one look exactly the same. We kept running into the same 3 problems. Sometimes a long message was just sitting there because no one told it to send.
Other times, an agent went quiet not because it was done, but because it got rate limited. And sometimes a standing brief was sent out but the agent never actually got it.
All 3 show up as a still, silent terminal. That's also how deep work appears from the outside.
The habits we developed aren't exciting. Always read the terminal's last line back to yourself before you log something as done. Take silence as a maybe, not a sure thing. If an agent stops talking, nudge it to keep going instead of giving its job to another one. Reassigning a task for an agent that's just rate limited means the work gets done twice, and you end up with 2 different versions of the same solution.
This isn't a model problem. A system that runs on terminals doesn't have a way to confirm delivery, so you have to make that habit yourself.
Is this the same as pair programming with AI?
No, it's not the same thing. Pair programming is about one person and one AI, working together on a single problem. Multi agent QA is built for a much larger job, one that's too big for a single session. Its real value is in the written trail it makes, not the dialogue inside any one task.
That record is what lets the team's discoveries move between different roles without getting lost.
Does the human still review the code?
Yes, and we built it that way on purpose. The agents never get to commit a change. They write their work to the working tree, and then a person checks it and does the commit if it's good. That keeps a single, real checkpoint before anything ever gets to a branch. That checkpoint is the one piece agents can't have.
Where can I read more about how ChatFuse is built?
We put these posts together as we build. If you want to know how the system's built, Inside the ChatFuse AI orchestration platform has the details. You can start free to see how it all works.
How we bundle up instructions for agents is its own topic. We wrote about that over here: AI agent skills.
Comments
Loading comments…