Field Notes

Field Notes: What I Learned Building an AI Agent Workforce

By Sean Doherty · June 26, 2026

The first time I watched two of my own agents talk to each other, I felt like a proud parent for about ninety seconds. One agent had drafted a customer reply. The second agent, whose job was to check it, approved it and passed it along. Clean handoff. Then I actually read the reply. It was polite, well structured, confident, and quoting a refund policy that did not exist. Both agents agreed on it. Neither of them was right.

That moment is the honest starting point for everything I want to write down here. I have spent the last couple of years building SoloBusinessAI, a platform meant to give one person, a solo operator or a very small team, a working crew of AI agents to run the ordinary functions of a business. Along the way I also built FloraLoop, a marketplace with an AI concierge, and Cura Mirai, which I will come back to because it grew directly out of the scars in this post. I have built and run multi-agent systems, not just drawn them on slides. These are field notes. Some of them cost me sleep.

Lesson one: an agent that sounds sure is not the same as an agent that is right

The refund policy that never existed taught me the most important thing on this list. Language models are fluent by design. Fluency reads as confidence, and confidence reads as competence, and none of those three things are the same thing. The output that scared me most was never the obviously broken one. Broken output you catch. The dangerous output is the one that is 90 percent correct, phrased beautifully, wrong in the one detail that matters.

I learned to stop trusting tone entirely. Now I treat every confident answer as a claim that needs a source, not a fact. The agents that earned my trust were the ones I forced to show their working, to cite where a number came from, to say plainly when they did not know. An agent that says "I am not sure, here is what I checked" is worth more to me than one that is smooth and occasionally catastrophic.

Lesson two: put a human on anything that actually matters

I resisted this one for a while, because the whole promise of an agent workforce is that you are not sitting there approving things all day. But I had the order of operations backwards. The point is not to approve everything. The point is to decide, in advance and on purpose, which actions are allowed to happen without a human and which are not.

Drafting an email? Let it run. Sending money, deleting records, making a promise to a customer, changing a price? A person looks first. I drew that line after the second time an agent did something reversible but embarrassing, and I have never regretted drawing it. Here is the rule I settled on:

• If an action is cheap to undo, let the agent do it and log it.
• If an action is expensive or impossible to undo, the agent proposes and a human commits.
• If you are not sure which bucket something is in, it goes in the second bucket.

This is not a lack of ambition. It is the difference between a tool you can actually put in front of a real business and a demo that works until the day it does not.

Lesson three: the hard part is not the agent, it is the space between agents

Any single agent doing a single job is, honestly, pretty achievable now. The difficulty explodes the moment two of them need to coordinate. One agent's output becomes another agent's input, and small misunderstandings compound. Agent A summarises a task slightly loosely. Agent B takes that summary as gospel and acts on it. Agent C cleans up after B and now the original intent is three translations away from what you meant.

I underestimated this badly. I thought coordination was a routing problem, a matter of wiring the outputs to the right inputs. It is closer to a management problem. You need shared context, clear ownership of each decision, and a way for an agent to say "this handoff does not make sense" instead of dutifully processing nonsense. The single agent was the easy 20 percent. The choreography was the other 80.

What surprised me here

Adding more agents did not linearly add capability. Past a certain point it added confusion. Two agents were manageable. Five agents with overlapping responsibilities produced a fog where no one, including me, could say who had actually decided what. Fewer agents with sharper boundaries beat more agents with fuzzy ones every single time.

Lesson four: keep the receipts, or you are flying blind

Early on, when something went wrong, I could not always reconstruct why. Which agent made the call? What did it see at the time? What did it pass downstream? I was debugging by vibe, rereading logs that were never designed to be read. That is a terrible place to be when the system is doing things on behalf of a real business.

So I started recording everything. Every agent action, the inputs it had, the decision it made, who or what approved it. A proper audit trail, not an afterthought. The moment I had that, two things happened. Debugging got faster, because I could walk backwards through an actual chain of events. And I could finally answer the question every serious operator eventually asks, which is "why did the system do that?" with a real answer instead of a shrug.

That work is the reason Cura Mirai exists. It started as my own internal plumbing for keeping agents auditable and under human governance, and it became a patent-pending governance kernel because I kept hitting the same truth: an agent that can take real actions needs oversight built in, not bolted on. An agent with the power to act and no record of its acts is not a workforce. It is a liability with good manners.

An agent that can act needs someone able to see what it did, question it, and overrule it. If you cannot do those three things, you do not have automation. You have a small unaccountable stranger with access to your accounts.

Lesson five: start with one agent doing one job

My original vision was a full crew from day one. A grand org chart of agents, each a specialist, humming together. It was a lovely picture and it was the wrong way to build. Every time I tried to stand up the whole thing at once, I could not tell which part was failing, because all the parts were failing a little and feeding each other.

What actually worked was almost boring. Pick one job that is real and annoying and well defined. Build one agent to do only that. Get it genuinely reliable, with a human checkpoint and an audit trail, before you let it touch anything else. Prove it in real use, not in a demo. Then, and only then, add the next agent. My honest advice to anyone starting: resist the org chart. Earn each agent.

What I would do differently

I would spend less time on clever coordination and more time making each individual agent honest and observable before connecting anything. I would write the audit trail on day one instead of month three. And I would define the human checkpoints before building the agents, not after an agent surprised me. Most of my worst weeks came from doing these in the wrong order.

The Kaizen point, which is really the whole thing

Between 1987 and 1997 I was Toyota's first Kaizen engineer, and that decade shaped how I think more than any framework since. Kaizen is not about the big leap. It is about the small proven improvement, made real, made stable, and only then built upon. You do not redesign the whole line. You fix one station, confirm it holds, and move to the next.

Building an agent workforce turned out to be the same discipline wearing new clothes. One small agent, proven in real conditions, with a human able to see and stop it. Then the next one. I still believe in the larger idea, that AI does not replace you, it hands you your own company. But you do not get handed a company. You build it one reliable station at a time, and you keep the receipts.

I am still in the middle of this. Agents still surprise me, and I still catch confident answers that are quietly wrong. The difference now is that I expect it, I have a checkpoint where it matters, and I can always go back and see what happened. That is not a triumphant ending. It is just where an honest builder actually stands.