What a Persistent Agent VM Holds in RAM Between Steps That Serverless Drops on the Floor
Persistent VMs retain loaded data and connections that serverless must rebuild on each invocation.
A persistent VM can retain loaded datasets, interpreter state, package import caches, open connections, and running subprocesses, while an agent placed in a fresh serverless environment must reconstruct them. For agents that return to unfinished work, RAM continuity is an execution requirement.
Why serverless functions and stateful agents are architecturally incompatible
Serverless execution is built around short, stateless request-response cycles. An agent follows a different pattern, installing dependencies, loading context, executing code, making tool calls, and returning to the environment to continue. Its next action often depends on what the previous action left running. An execution model that permits the environment to disappear between invocations cannot promise that an agent will find its working state intact.
When an invocation starts in a fresh environment, the agent must recover its context and rebuild whatever the next step needs. A chain of sequential tool calls can therefore become a chain of repeated initialization. Serverless can execute agent code. The limitation is that execution without guaranteed continuity forces the application to reconstruct state that a running process would otherwise retain.
What "in-memory state" actually means for an executing agent
In-memory state is the working state held by the agent's processes and their operating system. It includes interpreter variables, loaded data structures, socket descriptors, subprocess handles, and package import caches. These are not simply files waiting to be opened. They represent data and resources already prepared for execution, often through work that cannot be reproduced cheaply with a database call.
A persistent workspace is a different form of continuity. Source files, logs, and package caches written to durable disk can survive a clean stop and restart, while variables, live connections, and in-flight work do not. An ephemeral environment can lose both its local workspace and its RAM state. A persistent VM can retain the workspace and, when kept running or restored from a snapshot, retain the runtime state that a disk-only restart would discard.
Loaded datasets and in-process data structures
A data-analysis agent may load a multi-gigabyte dataset, clean it, build a feature pipeline, and generate a chart. The resulting pandas DataFrame or NumPy array exists in RAM as an object the interpreter can use directly. The source dataset is not the finished object. If the environment disappears, the next invocation must read the data again, parse it, and repeat the transformations before answering a follow-up question.
That reconstruction has a correctness cost as well as a latency cost. Random operations, timestamp-dependent joins, and external API enrichment can make a rebuilt dataset differ from the one used for the previous answer. A user's follow-up may then operate on changed data even though the conversation appears to concern the same analysis.
A persistent interpreter retains the processed object as the agent left it. The next question can use the existing transformed data rather than treating the original upload as the only surviving reference point.
Interpreter state: variables, call stacks, and execution context
The loaded dataset belongs to a larger interpreter session. In a Python REPL, Node.js runtime, or shell session, that session includes variables, defined functions, imported modules, intermediate results, and any active call stack. Those values do not automatically become files. Unless the agent explicitly writes them out, destroying the interpreter destroys the working context that gives subsequent code its meaning.
The familiar REPL analogy makes the dependency concrete. A later command can refer to a variable created by an earlier command because both execute in the same live session. Starting a new interpreter breaks that link, even if it still has the source code. The new process has the instructions, but not their results.
An LLM prompt cannot substitute for that missing execution state. The model can regenerate code, but the values produced by that code depend on the data and conditions present when it ran. A persistent VM keeps the interpreter available between tool calls, allowing the agent to use an earlier result without rerunning the computation that produced it.
Installed package caches and import state
Dependency state straddles disk and RAM, which makes it easy to mistake installation for readiness. Commands such as pip install, apt-get, and npm install place package files on disk. Importing a module then loads and initializes it inside the runtime. The installed files are only one layer. Python's sys.modules and the module registry in Node hold in-memory records that let the interpreter reuse modules already loaded.
A fresh interpreter begins without that warm import state, even when the package is included in its image. Startup must therefore read and initialize the modules it needs. For an agent that chooses dependencies dynamically, losing the local installation adds another cost before importing can begin. The agent must repeat the installation, consuming time and network bandwidth.
A persistent VM preserves these layers separately. Its workspace keeps the installed files, while its interpreter keeps running with the import cache. Subsequent tool calls can reuse an already imported module without repeating its initial loading. This is why durable package files alone do not provide the same execution continuity as a warm runtime.
Open sockets and live network connections
A database connection pool is working state with a direct operational value. It keeps established connections available so that successive queries can share the cost of connecting, negotiating TLS, and authenticating. The pool’s usefulness depends on live sockets and the connection state behind them, including sequence numbers and receive buffers held by the operating system, together with TLS state held in memory.
Terminating the execution environment closes those connections and can interrupt requests already in progress. This loss is more severe than a slow startup. An agent using a database, streaming API, or long-running RPC session may lose work when the environment shuts down, on top of waiting longer on its next invocation. Reconnecting also consumes resources on the remote service, and bursts of new connections can produce “too many simultaneous connects” errors.
A warm VM can keep the connection pool between steps. TLS session tickets provide another example of reusable client-side state, since the client keeps the ticket and associated session information in memory. A cold process without that state cannot use it to resume the prior session. Session IDs differ in requiring the server to cache session state.
Running subprocesses and background daemons
A headless browser extends the agent's working environment beyond its main interpreter. Chromium runs as a process tree with its own memory, file descriptors, and communication channels. The browser may already have open tabs, authenticated sessions, and a loaded page. Relaunching the executable does not recreate that state. The agent must recover the browser's working context before it can continue interacting with the page.
Environment termination removes those child processes as well as the parent. A termination signal and a grace period may allow a process to shut down, but shutdown is not serialization. It does not by itself produce a resumable browser session. Losing live browser state can also create a correctness problem when restoring authentication requires user interaction that the agent cannot repeat alone.
A local development server presents the same pattern. The agent needs not only the server's files, but also the initialized process, listening endpoint, and compiled assets already prepared for use. A persistent VM keeps that process available between calls. Language servers and background daemons also keep running as part of the working process tree, so they do not have to start again at every step.
The latency arithmetic when serverless reconstructs each artifact
Package installation, imports, dataset loading, browser startup, and database connection setup are distinct operations, but a single step may depend on all of them. Their costs accumulate along the execution path. Sequential tool calls then repeat the relevant preparation whenever they encounter another fresh environment, multiplying the reconstruction burden across the chain.
Workspace loss adds costs outside RAM reconstruction itself. A coding agent that must clone a large enterprise repository again has not yet reached the point of loading its interpreter or starting its development server. This is why the delay cannot be understood as a single runtime startup penalty.
A stateless agent that resends its full conversation history on every turn repeatedly pays to process old context. A model still needs input context when application state is kept in RAM, but rebuilding that state can add work that has already been done around the model call.
What a persistent VM actually does to preserve each artifact between steps
A persistent micro-VM retains the execution environment as a unit rather than treating each tool call as an independent process launch. While it remains running, its memory contains the processed dataset, interpreter namespace, and import caches. Its process tree contains the browser and development server. Its networking state supports the open connections.
Firecracker-based micro-VMs have hardware-enforced isolation and can pack VMs together at costs much like those of containers. Each VM has its own kernel, memory space, and virtual hardware. That boundary allows an agent session to retain its working environment without sharing its interpreter or process tree with another session.
Snapshot-restore helps with the cost of keeping an idle agent ready. The VM can be paused with its RAM, processes, and warm page cache captured, while its working directory and installed dependencies remain part of the preserved environment. Restoring that state saves the work of rebuilding it from a cold start. A snapshot of local socket state, however, should not be confused with a guarantee that a remote connection remains usable throughout suspension.
Copy-on-write memory further reduces duplication when agent VMs share a base image, with each VM's divergence accounting for the additional memory. Suspension shifts the idle cost from standing compute toward storage. Snapshots still take up disk space, and that cost adds up across a fleet.
Amazon Bedrock AgentCore Runtime illustrates session-level continuity in a shipping service. Its July 2025 preview and October 2025 general availability introduced a pattern in which a dedicated Firecracker micro-VM serves a session identified by runtimeSessionId. Subsequent invocations route to the same VM, preserving in-memory state, environment variables, running processes, and filesystem content across calls. Its October 2025 release set the maximum micro-VM lifetime at eight hours per instance.
The per-user agent pattern and why RAM continuity is its enabling condition
The per-user agent model assigns each user an environment containing their tools, working files, session state, and authenticated connections. A user can ask another question about the processed dataset or continue a browser task without treating every interaction as a new job. That experience depends on preserving execution state.
Without runtime continuity, the application has to rebuild what the user expects to find each time they return. Across many users, repeated loading, installation, authentication, and process startup become an aggregate compute and latency burden. Some state may also resist faithful reconstruction. A previous intermediate result or browser interaction is not necessarily recoverable merely because its originating instructions were recorded.
Snapshot-based persistence makes this model easier to put into practice because an idle environment can use storage without needing a compute instance that runs all the time. The architectural criterion is therefore precise. Products that depend on live execution state across interactions need a runtime that preserves it, or an explicit reconstruction design that accepts its costs and limits. A persistent agent VM makes the retained working environment the foundation of the product rather than a condition every tool call must recreate.
