Per-User Agent Architecture for SaaS Products

Per-user agent architecture decides whether a SaaS product running AI agents can ship, grow, and hold up under attack, all at the same time. That makes it a structural choice made at the outset rather than a control bolted on after launch to satisfy a security review.

Per-user agent architecture as a design primitive

The reasoning starts with what an agent actually does once a user gives it a prompt. The agent writes code and runs it, on the platform's own compute, and the platform has never seen that code before it executes. Conventional multi-tenant SaaS was built on the opposite premise: the vendor writes the code, the tenant supplies the data, and access control means keeping tenants from touching rows they don't own. Agents invert that relationship, because the thing being isolated is no longer just data. It's an arbitrary, freshly generated program running with the tenant's credentials attached.

That inversion is why so many agent products that work in a demo fail to survive contact with real users. Guides on multi-tenant AI agent architecture point to improper access control, not any shortfall in model capability, as the main reason production deployments break. The model does what it was asked. The infrastructure around it, built for a world where the platform controlled what ran, doesn't hold up.

A per-user agent architecture can carry a product to production only if the platform settles on an isolation model, persists agent state across sessions, wakes idle agents quickly, and keeps costs manageable once most of the user base is idle at any given moment. Each is treated here in turn, starting with isolation, because it sets limits the other three have to work within.

How the three isolation patterns produce different security ceilings

Fully siloed, fully shared, and hybrid isolation patterns appear across production agent platforms. They aren't adjustable settings on the same underlying design, picked to trade cost against convenience. Each one sets a different ceiling on what a security team can guarantee, and a platform built on the wrong one cannot patch its way to a higher ceiling later with application-layer rules.

Fully siloed infrastructure, with dedicated compute per tenant, gives the strongest containment available: a compromise in one tenant's agent has no path to another tenant's environment. This model runs per-tenant cost and operational load at their highest, so an update has to go through a separate pipeline for every tenant.

Fully shared infrastructure sits at the other end. It also means the security guarantee is only as good as that filtering logic, and one missed check in one code path is enough to cause a breach. Resource exhaustion runs alongside it as a second failure mode in shared compute: when workloads sit on the same hardware, they measurably slow each other down under contention, so one tenant's runaway agent becomes a latency problem for everyone sharing that pool.

Hybrid isolation has become the practical standard for platforms running agents in production: trusted, platform-authored background jobs run in ordinary containers, while untrusted, AI-generated code gets routed into microVMs. Amazon Bedrock AgentCore is built on exactly this logic, with a three-level hierarchy, tier, tenant, and user, enforcing isolation at every layer across knowledge base documents, memory, model access, and cost tracking. Platforms that offer tenant tiers can use isolation strength itself as a pricing lever, with premium tenants justifying the stronger, costlier guarantee.

The strongest objection is that hybrid isolation is good enough on its own, without the microVM half of the pair. It is, but only for the container side of the split, where the platform wrote the code and knows what it does. The moment an agent is generating and running code at the user's request, the shared kernel underneath that container becomes the thing an attacker is reaching for, and containers were never built to stop that reach.

Isolation for the vector database applies no matter which of the three patterns a platform chooses. Isolation there requires per-tenant namespaces with metadata filtering enforced on every query, not only when data is first written in, because semantic retrieval has no concept of application-layer permissions and will return whatever is closest in the embedding space. For tenants in finance or healthcare, the stronger recommendation is a separate index per tenant rather than namespace filtering inside a shared one.

What microVMs change when agents execute arbitrary code

Containers isolate at the process level, but every container on a host still shares that host's kernel, and the kernel is where the real risk sits once agents are running code nobody reviewed. That shared kernel is the attack path in agent workloads, and no patch can close it off, because the sharing is the design.

The CNCF's review of the runc breakouts from November 2025 names container escapes as a critical risk specifically in multi-tenant settings where users run workloads nobody vetted in advance, which describes agent-generated code exactly. Kubernetes' own documentation says that containers offer a weaker isolation boundary than hardware-based virtual machines.

MicroVMs change where the boundary sits. Instead of isolating at the process level on a shared kernel, each tenant's workload gets its own guest kernel, enforced by hardware virtualization through KVM, so a kernel exploit inside one VM has nowhere to go beyond that VM. The objection that used to carry weight here, that a VM per tenant costs too much in boot time and memory to run at scale, has been answered by the Firecracker numbers directly: a microVM boots in about 125ms and carries under 5 MiB of memory overhead. You also get a narrow attack surface by design, since only five virtual device types can be reached from inside the VM. AWS built its own production agent service on this same logic: in Bedrock AgentCore, every user session runs inside a dedicated microVM with its own CPU, memory, and filesystem.

AWS chose this same isolation standard for AgentCore, and it offers that to developers and startups at a flat rate built around the economics of running one agent per user.

These three isolation technologies cover different workload profiles, so they don't compete head to head. Firecracker microVMs give the strongest isolation available and are the right fit for arbitrary agent-generated code and for data under regulatory constraints. gVisor works by intercepting system calls and fits compute-heavy multi-tenant workloads where the overhead of a full VM is a real cost. Language-embedded isolates are limited to a single programming language but suit latency-critical, lightweight tasks where the code is trusted and the language is fixed in advance. SmolVM, open-sourced on April 5, 2026 under the Apache 2.0 license by Aniket Maurya and the Celesto AI team, is a new entrant addressing the same set of problems, which have grown as coding agents, app builders, and workflow automation tools get deployed at larger scale.

MicroVMs move the isolation boundary below the kernel itself, solving the shared-kernel vulnerability: each agent gets its own kernel, its own system call surface, and its own failure domain. That is why they've become the architectural standard for untrusted, agent-generated workloads where a container escape is a live possibility rather than a corner case.

State persistence turns an isolated agent into a usable product

If an isolated agent forgets everything between sessions, it is secure and useless in roughly equal measure. Isolation keeps the agent secure. It does nothing on its own to solve whether the agent can hold onto context, credentials, and work in progress long enough to be worth returning to.

Persistence has to be solved at three separate layers for that to happen. At the session layer, the tenant's identity, including tenant ID, tier, feature flags, and entitlements, has to travel into every tool call the agent makes, so the agent wakes up already knowing who it's working for instead of reconstructing that context from scratch.

Tenant context flows in via custom HTTP headers on every invocation. Microsoft took a similar approach with its Agent Framework, introduced in October 2025 and reaching general availability in April 2026, which was built from the start to pair scale-to-zero with persistence: agents cost nothing while sitting idle, and their files, disk state, and session identity all survive the trip across that scale-to-zero boundary.

A short checklist captures what you actually need to ask of the architecture here. Does the agent's identity and tenant context travel into every tool call it makes? Are the MCP tools it uses scoped to that tenant's own root directory, and nowhere else? Does every file access and every memory retrieval get logged against that tenant's ID, so the activity is traceable after the fact? Can one tenant's usage be rate-limited on its own, so a single runaway agent can't exhaust resources meant for everyone else?

That's the difference a user actually feels: the agent that remembers what it was doing, rather than the one that has to be reintroduced to its own task.

Snapshot-restore scheduling resolves the tension between persistence and idle cost

Persistence creates an obvious cost problem if it's solved the naive way, by simply leaving a VM running for every user at all times. At any real scale, most agents sit idle most of the time, so paying for continuous compute per user doesn't survive contact with a full user base. Snapshot-restore scheduling is the mechanism that breaks persistence away from that always-on cost, by separating the question of whether state survives from the question of whether compute is running.

The mechanism itself is straightforward. Hibernation follows the same logic one step further: hibernate is snapshot plus pause, so the agent stops drawing compute the moment it goes quiet and comes back with its full state intact on the next turn. When a session is actively running, persistent mode is the better fit, since it optimizes for how fast each turn responds. Hibernate mode is the better fit between bursts of activity, optimizing for what idle time costs. Both are legitimate operating modes, and which one applies depends on how actively a given user is working at that moment.

The same underlying mechanism pays off again for branching workloads, where an agent needs to try more than one path and keep whichever one works. A fork onto the same host takes roughly 400 to 750 milliseconds, and a fork onto a different host takes about 1.2 to 3.5 seconds, cheap enough that a platform can actually run tree-of-thought reasoning rather than treat it as a theoretical technique: each branch is a real, separately isolated machine diverging from shared warm state, and a dead end costs nothing more than a kill() call to discard.

AWS's own prescriptive guidance on multi-tenant agentic AI lists noisy-neighbor mitigation and cost attribution among the operational concerns a platform has to handle alongside tenant identity, tenant observability, and data isolation. Snapshot-restore addresses both at once: it bills compute only to active time, and it makes each tenant's actual cost visible instead of blended into a shared pool.

The hybrid isolation model, where untrusted, AI-generated code runs in dedicated microVMs while trusted platform work runs in containers, only holds up economically at scale if the VM overhead stays small. Maritime built its snapshot-restore scheduler around that constraint, so idle agents pay for storage instead of idle compute, and that is what makes a microVM-per-user model cost-competitive with the cheaper, fully shared alternative. Maritime's own scheduling model rests on the same observation, that most agents are idle most of the time, so users pay for storage rather than for compute sitting unused, which is what makes deploying one agent per user viable across an entire user base rather than just for a handful of the most active accounts.

Wake latency and whether a per-user agent feels like a product or a batch job

Wake latency functions as a contract term for a per-user agent, not just a performance nice-to-have. A conversational assistant that takes several seconds to resume every time a user sends a message stops feeling like something present and starts feeling like a job submitted to a queue, and users treat it accordingly.

The latency stack that determines how an agent feels breaks into a few distinct stages. Restoring a microVM from a snapshot takes 5 to 30 milliseconds on Firecracker. Waking the full agent, including rehydrating its session, runs at a p50 of about 179 milliseconds and a p99 of about 203 milliseconds on a well-optimized restore path. A true cold boot happens only the first time a template spins up, and it takes several seconds, which you can tolerate once per template's lifetime but not on every single user interaction.

A more subtle latency problem appears once agents start calling out to external sandboxes over MCP. Research from Beihang University, under the name SpecBox, identifies that running the LLM and then the sandbox in sequence creates tail latency that compounds across a multi-turn conversation, since each turn's delay stacks on the last. SpecBox's answer is to overlap LLM execution with sandbox preparation instead of running them in sequence, using intent-aware prewarming that speculates on execution intent from the LLM's streaming token output before generation even finishes. A second technique, reuse-aware data transmission, looks for semantic similarity across tool calls, so it can skip redundant execution. Tested across 200 multi-turn trajectories, the combination cut P99 end-to-end latency by a substantial margin against an on-demand baseline, and it also used substantially less peak memory than a deployment that keeps sandboxes permanently reserved.

Preparing a sandbox before the system is certain it's needed buys speed at the cost of doing work that sometimes turns out to be wasted, and any platform adopting the technique has to decide how much speculative execution it's willing to pay for in exchange for an agent that feels instant rather than queued.