Part 2 of 5 in the series “How We Built a Distributed AI Team.” A new post each week.
In the last post I described where this started: a single orchestrator, running in one shell session, wearing every hat. It read the mail, drafted the replies, tracked the compliance items, wrote the content, and kept the whole picture in its head. It worked — right up until it didn’t. One session meant one context window, one identity, one machine, and one thing happening at a time. Every new responsibility I added made the single session heavier, and every heavy session made the next mistake more expensive.
The fix wasn’t a bigger session. It was more sessions — separate ones. This post is about the pivot from one program doing everything to many independent agents doing their own jobs, and why that structural change mattered more than any single capability I added on top of it.
From sub-agents to first-class members
The first instinct is always to reach for sub-agents: keep the one orchestrator, but let it spin up helpers in-process to parallelize. I did that, and it buys you something — but the helpers still live inside the parent. They share its process, its credentials, its memory, its fate. If the parent dies, they die. If the parent’s context gets polluted, theirs is too. They’re threads, not colleagues.
What I actually wanted was colleagues. So the model changed: each agent became a first-class member of a shared team chat platform — the same kind of messaging platform a human team would use. Not a function the orchestrator calls, but a participant you can @-mention in a channel, the same way you’d mention a human teammate. You ask the finance agent a question in the finance channel; you ask the compliance agent something in the compliance channel. The chat platform is the coordination layer. There’s no separate console, no bespoke UI, no orchestration dashboard I have to keep open. If you can use a group chat, you can use the team.
Three things had to become true for that to work, and each one is a pillar of what “distributed” actually means here.
Its own identity
Every agent has its own bot identity and its own credentials. This sounds like plumbing, and it partly is — each one authenticates as itself, posts as itself, and shows up in the channel as a distinct member with a name. But the payoff is not cosmetic. It’s isolation of permission.
Because the finance agent authenticates as itself, it can hold exactly the access finance needs and nothing else. The compliance agent holds compliance access. The agent that reads mail can be scoped to reading mail. When every agent ran inside one orchestrator, there was effectively one keyring, and every capability I granted for one job was silently available to every other job. Splitting identities means a mistake in one lane is bounded to that lane — an agent literally cannot reach into a system it was never given keys to. Least privilege stops being a policy I have to remember and becomes a property of the architecture.
It also makes attribution honest. When something gets posted, drafted, or filed, it’s posted by a named agent. You can see who did what. In a system that acts on your behalf, that traceability is not a nice-to-have.
Its own memory
The second pillar: each agent keeps its own memory, separate from the others. The finance agent remembers finance context across runs. The compliance agent accumulates compliance context. They don’t share one giant brain that grows until it’s unmanageable — they each keep the state their own job needs.
This is the direct antidote to the single-session problem. One orchestrator meant one context window that had to hold everything, and everything is too much: the more you cram in, the worse the recall and the higher the cost of every single interaction. Give each agent its own memory and each one stays sharp on its own domain, because it’s not carrying the other eight domains around.
(There’s an obvious tension here — if every agent is siloed with its own memory, how does the team stay coherent? How do they avoid becoming nine amnesiac strangers who each know a slice and none of the whole? That’s a real problem, and it’s the entire subject of the next post. For now, take the isolation as the starting point; the shared brain comes later, deliberately, on top of it.)
Its own runtime
The third pillar is the one that’s easiest to overlook and hardest to give up once you have it: each agent runs as its own process, spun up on demand, torn down when it’s done.
An agent isn’t a long-lived daemon sitting there consuming resources and slowly corrupting its own state. It’s summoned for a task, does the task, and exits. A crash — a bad input, a timeout, a wedged run — takes down that invocation and nothing else. The rest of the team never notices. Compare that to the monolith, where a single unhandled error took down the one process that was doing everything, and you were back to square one.
Independent runtimes are also what make genuine parallelism possible, which for an AI team is the whole game.
Why distribution is the point, not a side effect
Here’s the part that changed how I think about building with agents.
A human team has a hard capacity ceiling: you have as many finance people as you hired. An agent team does not. Because each agent is just a process spun up on demand, I can run many instances of the same agent at once. If ten independent finance tasks land, I don’t queue them behind one finance agent — I spin up ten finance agents and they run simultaneously. The team scales sideways, per-role, on demand, for the duration of the work and no longer.
You simply cannot do that with a single orchestrator working through a to-do list one item at a time. Sequential thinking is the default when there’s one worker; parallel is the default when workers are free to summon. That inversion — from “what’s next in the queue” to “what can all run at once” — is the single biggest practical advantage this architecture has over the way I worked before, and it only exists because the agents are distributed and independent. Shared identity, shared memory, or a shared process would each, on its own, force everything back into a single lane.
And because no agent is welded to a single long-running process on a single machine, the team stops being tied to my laptop being awake. That independence is what eventually lets the whole thing move off the laptop entirely — but that’s the last post in this series, not this one.
The router: a channel message becomes an agent
If the agents are independent and only ever exist on demand, something has to notice a message in a channel and turn it into a running agent. That’s the router, and the striking thing about it is how little it does.
The router watches the chat platform. When a message mentions an agent, it works out who the message is for, gathers the relevant slice of the conversation so the agent has context, and spawns a fresh session for exactly that agent with exactly that task. The agent runs, posts its reply back into the channel as itself, and exits. The router goes back to watching.
That’s the whole loop: a channel message in, a spawned agent session out. The router doesn’t do the work, doesn’t hold the domain knowledge, doesn’t make the decisions. It’s a dispatcher — deliberately thin, so that all the intelligence lives in the agents and all the coordination lives in the chat platform. Keeping it thin is a choice, and it’s one I’d defend hard, because the more logic you pile into the coordinator, the more you’re just rebuilding the monolith with extra steps.
A thin router also has a quieter virtue: it’s a small, well-understood piece of code sitting on the one path every single agent invocation flows through. That makes it the natural place to enforce the rules that keep a team of autonomous, self-triggering agents from doing something stupid at scale. What those rules are — and the incident that taught me exactly why you need them — is a story for post four.
For now, the shape is enough: independent agents, each with its own identity, memory, and runtime, meeting on a shared chat platform, connected by a router thin enough to trust. Next, the hard part — how a team that distributed avoids forgetting everything it knows.