The agent platform, layer by layer
Gateways, registries, identity, PII scrubbing, durable runs: none of them matter with one agent, and each one becomes essential the week a particular number crosses a line.
Published: ·9 min read
Every layer in an agent platform is the answer to a number. With one agent, one model and one user, you need none of them: a script that calls an API is a complete platform. Then a count crosses a line (a second model provider, a fifth team sharing the same tools, the first user who mustn't see everything) and a layer you never needed becomes the only thing standing between you and an incident.
That's the lens I wish someone had handed me when I started looking at this space, because otherwise it reads as a blur of launches: gateways, registries, protocols, sandboxes, a new open-source project every week. Read it as a stack, and ask of each layer "which number makes this necessary?", and most of the confusion goes away.
Two of these layers are the ones I've spent the most time on, MCP gateways and PII protection, so I'll be more opinionated about those. The rest come from a few weeks of working through what a firm-wide platform needs, and from watching where the open-source projects are heading.
A second model provider brings the LLM gateway The first team calls one model with one key. So does the second, and the third, each with its own key, its own bill and its own idea of what a sensible timeout is. Nobody notices the problem until the provider has a bad afternoon and every team goes down at once, or until finance asks what the firm spends on models and the honest answer is three spreadsheets.
An LLM gateway puts one front door in front of every model. It holds the keys, so teams never see them. It gives each team a budget and a rate limit, keeps one log of every call, and when a provider falls over it sends the traffic somewhere else. The number that forces it is small: two providers, or two teams, whichever comes first. LiteLLM is the open-source project most teams reach for here, a proxy that speaks one API to many providers with keys, budgets and fallbacks built in.
Shared tools bring the MCP gateway MCP made it easy for an agent to use a tool. It also made it easy for every agent to connect to every tool directly, and that grows badly. Four agents and six tools is twenty-four connections, each with its own credentials, its own rate limits and its own idea of logging. Add a fifth agent and you add six more.
Put a gateway in the middle and it becomes four plus six. Authentication, rate limits, tool allow-lists and audit logs live in one place instead of twenty-four. This is the layer I'd build earliest after the LLM gateway, because the cost of skipping it compounds: every new tool multiplies the mess rather than adding to it. On the open-source side, agentgateway (Apache 2.0, under the Linux Foundation) is the one to watch, partly because it doesn't stop at MCP. It handles agent-to-agent (A2A) and LLM traffic through the same proxy, with JWT and OAuth, rate limits, observability and guardrails.
Too many agents brings the registry At five agents, everyone knows what exists. At twenty, nobody does. Teams build the same agent twice, the orchestrator's prompt grows a paragraph for every agent anyone has ever shipped, and adding one means changing the orchestrator's code.
A registry fixes that by turning agents into records: what the agent can do, where it lives, who owns it, which version is live. The orchestrator asks the registry for the agents that fit this request and this user, instead of carrying all of them around. And this is where the two words people mix up most finally separate. The registry is the control plane: it answers "what exists?", rarely. The gateway is the data plane: it checks every single call.
You don't need four products for agents, MCP servers, and a registry and gateway for each. One registry can catalogue agents and MCP servers together, and one gateway can carry both kinds of traffic. That's exactly how the open-source projects are being built: agentregistry catalogues agents, MCP servers, skills and prompts and is designed to pair with agentgateway, and the AGNTCY Directory (also Linux Foundation) goes a step further with discovery across registries in different organisations.
Real users bring identity Everything above works with a single service account. Then the platform gets real users, and some of them mustn't see what others can. The trap here is subtle, and it's the one I'd check first in any agent platform: somewhere between the user and the data, the call often switches from the user's identity to the orchestrator's service account, and the service account can see everything.
A gateway can check that the client agent is allowed to call the client database. It can't know which rows this person is allowed to read. Only the user's identity, carried through every hop on behalf of the person who asked, can do that. So identity has two halves: propagation (the token travels with the call) and entitlement (something decides what this user may touch). OpenFGA (Apache 2.0) is the open-source answer to the second half, relationship-based permissions in the style of Google's Zanzibar, so you can say things like "members of desk 2 can read clients covered by desk 2" and ask it on every call.
Customer data brings PII scrubbing The first prompt containing a real client's name is a turning point. From then on, every place that prompt goes is a place that data now lives: the model provider, the gateway logs, the trace store, the evaluation dataset someone builds from production traffic six months later. The fix is to scrub at the boundary, not to hope every downstream system is careful.
The trick that makes it work is reversibility. Names, emails and account numbers leave as placeholders, the model reasons perfectly well about <PERSON_1>, and the real values are put back only on the way to the person who's allowed to see them. The logs and traces keep the placeholder version forever. Microsoft's Presidio (MIT) is the usual open-source starting point for detection and anonymisation, and it's worth running at the gateway rather than inside each agent, for the same reason as everything else in this post: one place instead of twenty.
Long-running work brings durable execution A chat request lives for seconds. Real work doesn't. An agent that gathers filings, drafts a memo and then waits for someone to approve it might run for an afternoon, and a server that stays up for an afternoon without a restart is something nobody can promise.
Durable execution makes each step a checkpoint, so a crash resumes from the last finished step instead of the first one, and waiting six hours for a human costs nothing because the run is just a record in a database until the approval arrives. Temporal (MIT) is the established name here, and Hatchet (MIT) is a lighter option that self-hosts on Postgres with queues, retries, schedules and event triggers. The number that brings this in isn't users or agents. It's the first task that has to wait for a person.
Next to it sits the runtime layer, for agents that run code or touch files. Once an agent can execute something it wrote, you want it in a sandbox with its own scoped credentials. Kagent (Apache 2.0, a CNCF project) defines agents as Kubernetes resources, so they deploy, scale and get isolated like any other service, and the Kubernetes Agent Sandbox project adds stronger isolation through runtimes like gVisor and Kata Containers.
People bring the interface layer At the top of the stack is the part employees actually see, and it's moving fastest. AG-UI streams an agent's progress, state and tool calls into a frontend. A2UI lets an agent describe an interface (a table, a form, an approval) that the frontend renders from its own approved components. MCP Apps lets a tool ship a whole interactive app when a card isn't enough, and OpenAI's Intelligent UI shows the same idea arriving in consumer products. CopilotKit and assistant-ui (both MIT) are the open-source kits most teams build on. I wrote about what that workspace should feel like in Work that comes to you; the short version is that work arrives in an inbox instead of waiting for someone to ask.
Every layer needs evidence One layer doesn't sit in the stack, it runs through all of it: traces and evaluations. Without them you can't tell whether the gateway's failover worked, whether the registry routed to the right agent, or whether last week's prompt change broke something three agents away. OpenTelemetry for traces, Langfuse (MIT core) and Phoenix for agent observability, and Promptfoo (MIT) for evaluations are the usual open-source picks. The gap is in the second half. LangChain's survey of more than 1,300 practitioners in June found 89% of teams had agent observability but only 52% ran offline evaluations. Most teams can watch their agents. Far fewer can prove a change didn't make them worse, and that's the gate every rollout should pass through.
Open source is filling every layer, and the layers are merging The pattern across all of these projects is that they start in one layer and grow into the next. Gateways that began with MCP now carry A2A and LLM traffic too. Registries that began with agents now hold MCP servers, skills and prompts. Agent frameworks are absorbing workflow and evaluation features, and protocols are adding authorisation and extension points rather than being replaced. I expect the stack to settle into fewer, broader pieces: one registry, one gateway, one identity model, one durable runtime, one workspace, with traces through all of them.
Which makes the useful question less "which framework should we pick?" and more "which of these numbers have we already crossed?" Each layer costs something to run, so the right time to add one is the week its number shows up, and not much before.