Running Coding Agents Unattended

A field guide to agent platforms: what it takes, the options, and where teams get hurt

Keewan (X · LinkedIn) · 2026-10-15 · Version 1 · CC BY 4.0

About this guide

Who it's for. Mainly engineering leads and platform engineers deciding whether and how to
let coding agents (Claude Code, Codex, and similar) work beyond one developer's interactive
session: triggered from chat or tickets, running in parallel, testing against a real stack,
and opening pull requests. Anyone else curious about what that takes should be able to follow
it; section 1 and the checklist in section 7 stand on their own.

What it covers. The pieces an agent platform needs, the options for each, a recommended
default per piece, a staged path from one supervised runner to many agents, one hands-on case
study, a readiness checklist, and a landscape of products and public precedents.

What it doesn't. It isn't a product comparison with benchmarks, and it doesn't cover
training or evaluating models. Most of it is desk research; only the case study in section 6
was tested hands-on.

Evidence tags. Claims carry one of these:

Products in this space change monthly. Everything here is as of mid-October 2026.

1. The short version

1. The agent is not the hard part. Current models and harnesses already plan, implement,
   and test well inside a session. What most teams lack is the system around them: a trigger,
   isolation, credential handling, authority over what the agent may change, verification,
   and an audit trail.
2. Separate thinking from permission. Agents investigate, write, and propose; deterministic
   code decides what is allowed and owns every consequential state change. "You should not do
   X" in a prompt is a request. "You cannot do X" is a system property.
3. A worktree is not isolation, and a container is not enough. A worktree only stops file
   collisions. For untrusted workloads, use a VM boundary (a microVM per run), not a
   shared-kernel container alone.
4. Isolation without egress control and credential mediation is false security. The most
   likely incident is not a sandbox escape. It is an agent holding a real token, with open
   internet access, reading attacker-controlled text (an issue, a log, a web page) and sending
   the token somewhere.
5. The agent should hold no real secrets. A proxy outside the sandbox attaches credentials
   on the way out, only to the hosts that need them.
6. The work item is the source of truth. Not the chat thread, not the agent's context
   window. A durable record of ticket, base commit, sandbox, stage, attempts, evidence,
   approvals, and cost.
7. Bound everything. CI retries (two rounds is a common cap), steps per run, wall time,
   and spend. Unbounded retry loops are a known anti-pattern, not just a cost problem.
8. Ship evidence with every change. A PR from an agent should come with proof: tests run
   against a real stack, screenshots or a recording, a risk note, and the judgment calls the
   agent made where the spec was silent.
9. The bottleneck moves. Once agents produce changes faster, the constraint becomes
   review, CI capacity, and people knowing what shipped.
10. Start light. One runner, one workflow, one measurable pain. Add memory and one scheduled
    chore before attempting an autonomous supervisor.
11. Decide merge authority explicitly, and enforce it in the system (GitHub permissions,
    branch rules), not in the prompt. Section 4 lays out the options.

2. The shape of the system

2.1 Vocabulary

• Harness: what scopes one agent: prompts, tool definitions, retry control. It is
  ephemeral and gets rewritten when the model changes. Claude Code, Codex CLI, OpenCode, and
  LangChain's Deep Agents are harnesses.
• Platform: what scopes every agent: identity, sandboxing, tool registry, verification,
  observability. It survives a model or harness swap. Most of this guide is about the
  platform.
• Cognitive plane: the agents and skills that triage, spec, implement, review, and
  validate.
• Control plane: the run ledger, scheduler, sandbox lifecycle, identity, permissions,
  legal state transitions, CI gates, merge authority, escalation rules, and audit trail.
• Light vs dark factory (Addy Osmani's distinction, via Josh Rosen [opinion]): a dark
  factory is lights-out, where agents implement and ship and humans largely stop reading
  code. A light factory keeps humans in planning, implementation, review, and approval. Both
  are factories; the system around the work is the same, only the amount of automation
  differs.

2.2 Levels of autonomy

Two independent framings agree closely.

Agent Baseline's reference architecture [docs] keeps four boundaries constant (sandbox,
network, supply chain, identity/access) and varies who chooses the next step:

Kiro's maturity ladder [vendor] describes the same climb from the operator's side:

1. One human-driven session.
2. Several sessions, with the human as coordinator.
3. Shared memory, a dashboard, cron, monitors, and workflows; 10–20 sessions are manageable,
   but the human still chooses what happens next.
4. An asynchronous pipeline: triage, implementation, review, and merge are separate session
   types connected by queues.
5. An agent outside the sessions owns decomposition, dispatch, patrol, acceptance,
   sequencing, and cleanup; the human keeps the goal.

Most teams reading this are at level 1 / stage 1–2. The jump that needs real platform work is
to level 2 / stage 3–4.

2.3 Reference stack

 triggers: chat · tickets · red CI · webhooks · schedules · production signals
        │
 control plane: work-item ledger · scheduler · authorization · merge authority · audit
        │
 harness (Claude Code / Codex / custom) + your skills and repo context
        │
 sandbox: microVM per run ── egress proxy + credential broker ── tool / MCP gateway
        │
 verification: real stack · tests · previews · screenshots · traces · fresh-context review
        │
 review → merge → deploy → production signals ─────────────────► back to triggers

Everything below the harness line is platform. The rest of the guide walks it layer by layer.

3. The layers

Each layer follows the same pattern: what it does, what goes wrong without it, the options,
a recommended default, and gotchas.

3.1 Host

What it does. The operating system and machine the sandboxes run on.

Without it. Drift: hand-patched hosts that nobody can rebuild, with agent tooling installed
at floating versions.

Options.

• A standard Linux host or cloud image, plus a pinned sandbox template.
• An immutable host image such as AgenticLinux [docs]: a bootc/OCI image with read-only
  /usr, atomic upgrades, bootc rollback, published for x86_64 and aarch64, with Claude
  Code, Codex, Docker, and Docker Sandboxes preinstalled.
• Hosted machines (see 3.2 and Appendix A2), where the host is someone else's problem.

Default. A standard host plus a pinned, prebuilt sandbox template. Consider an immutable
image only once you run a fleet.

Gotchas.

• Membership in the docker group is root on the host. A read-only /usr does not make an
  agent with Docker daemon access safe; treat such a host as a trusted single-tenant worker.
• An immutable output image is not a reproducible build if its inputs float (latest tags,
  unpinned npm packages). Pin inputs, image digests, and agent versions.
• AgenticLinux reports Docker Sandboxes 0.40–0.43 failing to mount the workspace on composefs
  systems where /var/home is a separate mount (empty guest directory, lost writes)
  [docs].

3.2 Compute isolation (the sandbox)

What it does. Gives each run its own machine-shaped environment, so one agent cannot see or
damage another's work, the host, or your laptop.

Without it. Agents share a host, credentials, and network. A runaway or compromised agent
reaches everything any other agent can.

A useful taxonomy. George's agent sandbox taxonomy
[opinion] splits "sandboxing" into seven layers: compute isolation, resource limits,
filesystem boundary, network control, credential management, action governance, and audit
trail. No single product covers all seven well. Strong isolation with an open network and raw
credentials is false security. Agent Baseline adds an eighth concern the taxonomy lacks:
supply chain (3.5).

Two patterns (Browser Use's terms [docs]):

• Isolate the tool: the agent runs on your infrastructure; only dangerous code execution
  is sandboxed.
• Isolate the agent: the whole agent runs in the sandbox with zero secrets (Browser Use
  passes only a session token, control-plane URL, and session ID, then deletes them from the
  environment before agent code runs). Every credentialed operation, including LLM calls,
  goes through a control plane that holds the real tokens. Browser Use moved from the first
  pattern to the second for operational reasons too: a shared process meant a redeploy killed
  every running agent.

Options.

• Local microVMs: Docker Sandboxes (sbx) [tested] runs coding agents in a microVM with
  its own Docker daemon and git-worktree integration. Note that it does not use
  Firecracker; Docker wrote its own VMM on each OS's hypervisor (Hypervisor.framework on
  macOS, Windows Hypervisor Platform, KVM on Linux) [docs]. Same hardware-boundary
  concept, different and newer codebase.
• Process-level sandboxes: Anthropic's sandbox-runtime (srt) [docs] uses bubblewrap
  (Linux) or Seatbelt (macOS) with no container. It complements a microVM rather than
  replacing it; see 3.3 and 3.4.
• Self-hosted fleet runtimes: OpenSandbox [docs] (Kubernetes, with gVisor, Kata, or
  Firecracker as runtime classes); Agent Substrate with Google's AX [docs] (an early-stage
  actor model that suspends idle agents and resumes them onto warm workers from snapshots).
• Hosted VMs: exe.dev, Boat.dev, InstaBox, boxd (Appendix A2). Full Linux machines with
  Docker inside, snapshots, pause/resume, and in some cases fork.
• Edge containers: Cloudflare Sandbox, Vercel Sandbox. Fine for code-execution loops; no
  nested Docker, so they can't host a multi-container dev stack.

Default. One microVM per run. If the agent must run your real stack (databases, queues,
several services), pick a substrate that runs a Docker daemon inside the VM. Add a
process-level sandbox inside it as a second layer.

Gotchas.

• CPU architecture. Many official images still lack arm64 builds. A microVM on Apple
  Silicon has no x86 emulation, so an image that works on the host (through Colima's Rosetta
  emulation, for example) can fail inside the sandbox with exec format error. See the case
  study. An x86 cloud runner sidesteps this.
• Default permissions. Sandboxes built for agents often default to
  --dangerously-skip-permissions inside. That is fine only if the layers outside the agent
  (network, credentials, authority) are real.
• Cold start. Every fresh sandbox pays for toolchain install, image pulls, and setup.
  Snapshot a warm template (sbx template save, or a hosted VM's snapshot/fork) instead of
  rebuilding each run.
• Root on a persistent VM is not governance. A hosted VM where the agent is root contains
  the blast radius to that VM; it says nothing about what the agent can reach from it.
• Use a VM boundary for hostile workloads. OpenAI's agent-security lead [opinion]: use a
  VM-backed sandbox (Kata or Firecracker), and never let a shared-kernel container be your
  only control.

3.3 Network egress

What it does. Decides which hosts and operations an agent can reach.

Without it. Any secret the agent can read can leave. Any prompt injection can call home.

Options.

• The sandbox's own policy. Docker Sandboxes ships balanced (broad egress to registries
  and common hosts) and supports per-domain allow rules (sbx policy allow network <host>)
  and a policy log [tested].
• srt proxies [docs]: on Linux it removes the network namespace entirely and routes
  traffic through host-side HTTP and SOCKS5 proxies that enforce domain allow and deny lists.
  Before dialing an allowed name, the proxy resolves it and drops loopback, link-local,
  multicast, cloud-metadata, and host-interface addresses, so an allowed name that resolves
  to an internal address is still blocked.
• OpenSandbox's egress sidecar [docs]: FQDN and wildcard allow/deny, DNS filtering,
  nftables, live-patchable policy.
• gh-aw's firewall [docs], if you run agents in GitHub Actions; it also records token
  usage.
• Your own network boundary, which is mandatory for self-hosted Claude Managed Agents
  environments: the docs state the product cannot verify or enforce default-deny egress
  [docs].

Default. Default-deny, with an explicit allowlist: the git host, your model provider, the
package registries your build needs, and the specific APIs a task needs. Block the cloud
metadata endpoint.

Gotchas.

• Expect to discover the real allowlist by failing. In the case study, a full-stack build hit
  three separate blocks (a plain-HTTP Debian mirror, a toolchain host, a second toolchain
  domain) [tested]. Keep the policy log on while you build it.
• Domain filtering doesn't inspect traffic; domain fronting can get through [docs].
• Some programs ignore proxy environment variables. With the network namespace removed they
  fail rather than bypass, which is the right failure.
• Stripe's devboxes for agents reportedly have no internet and no production access at all
  [report]. A broad default like balanced is a testing convenience, not a production
  policy.

3.4 Credentials

What it does. Gives agents the access they need without giving them the secrets.

Without it. Tokens sit in the agent's environment, readable by any command it runs and any
text that convinces it to print them.

Options, from weakest to strongest.

1. Environment variables in the sandbox. Common, and the thing to move away from.
2. Short-lived, narrow tokens. A GitHub App installation token (one hour) instead of a
   personal token. Crabbox's coordinator mints a per-lease SSH key and keeps provider
   credentials central [docs]. Better, but the agent still holds a live token.
3. Proxy-side injection ("the agent holds no real secret"). The sandbox sees a fake or
   empty value; a proxy outside it substitutes the real one only on requests to the right
   host.
   • srt's credentials setting can deny an env var or file, or mask it: the process
     sees a sentinel and the proxy substitutes the real value on egress to listed hosts.
     Masking needs the experimental tlsTerminate option [docs].
   • OpenSandbox's Credential Vault does the same as a transparent MITM sidecar, redacting
     secrets from logs and audit output [docs].
   • DigitalOcean Managed Agents' Action Gateway claims to broker credentials at execution
     time so they never reach the model or sandbox [vendor].
   • Shopify's Aquifer has a named credentials proxy [docs].
4. Git without credentials in the runner. Claude Code's self-hosted environments can route
   git through Anthropic's proxy (--use-anthropic-git-proxy), using your organization's
   GitHub App installation token minted per session, so the runner image holds no git
   credentials [docs].

Default. Option 3 for anything that can write (git host, cloud, ticketing). At minimum,
option 2 plus a default-deny egress policy.

Gotchas.

• Read access is as dangerous as write access. Test what the sandbox can read: host
  environment variables, files outside the workspace mount, mounted sockets.
• Mounting the Docker socket gives host root. srt's own README flags
  allowUnixSockets: ["/var/run/docker.sock"] as exactly that [docs].
• Anything reachable is part of the boundary. If the agent can run commands in another
  container (a staging app container, say), that container's secrets and network are in
  scope too, and your sandbox's egress policy does not cover them.
• Hosted "vaults" vary. Check whether a vault injects secrets as environment variables
  (the agent can read them) or brokers them at the network edge (it can't).
• Multi-agent sessions share credentials. In Claude Managed Agents' multiagent mode, all
  agents in a session share the same sandbox, filesystem, and vault credentials; sub-agents
  get isolated context, not isolated credentials [docs].

3.5 Supply chain

What it does. Controls which images, packages, and sources an agent can pull and run.

Without it. An agent picks a community image because it happens to build on your
architecture, or installs a typo-squatted package, and nobody vets it.

Options. A registry allowlist enforced by the egress layer; hardened base images (Docker
Hardened Images advertises signed SBOMs, SLSA Build Level 3 provenance, and a free tier
[vendor]); image scanning (Docker Scout); a supply-chain firewall in front of package
registries (part of Agent Baseline's reference shape [docs]).

Default. Treat it as its own boundary from day one: allowlisted registries, pinned digests,
and a scan before any new image becomes a default.

Gotchas.

• Agent hooks and plugins are executable supply-chain code. ApexYard's own onboarding notes
  record a security issue caused by trusting arbitrary project hook files [docs].
• In the case study, the replacement database image was chosen only because it published an
  arm64 manifest. It worked, but its CVE and SBOM profile was never checked [tested].

3.6 Tools and context

What it does. Gives each run the right tools and the right knowledge, and nothing more.

Without it. "Whatever MCP servers are configured this session." Too many tool schemas
crowd the context, stale docs mislead, and every agent has every permission.

Options.

• A central tool layer. Stripe's Toolshed [report]: one internal MCP server with 400+
  tools; each agent gets a curated subset, so the MCP server is also the access-control
  point. Commercial equivalents: Docker MCP Catalog and Toolkit plus Docker AI Governance
  (org-wide tool allowlists, audit streaming) [vendor]; MintMCP's gateway [vendor];
  DigitalOcean's Action Gateway (16,000+ tools with model-backed tool search) [vendor].
• Progressive disclosure. Show only the tool families a step needs. Supermemory's harness
  hides sandbox and scheduler tools until the model explicitly unlocks them, and removes all
  tools at the final step [docs]. Uber caps context and avoids loading thousands of MCP
  schemas into every session [docs].
• Context. Keep docs and decisions in the repo, curated. Stripe's data-discovery tool
  found the biggest failure was finding the current, correct source, not generation
  [report]. Commercial context engines (Unblocked) aggregate code, docs, tickets, and chat
  with permissions [vendor].
• Memory. Separate durable knowledge (decisions, ownership, history) from live state
  (issue status, latest deploy), which should always be queried fresh [docs]. Scope memory
  by audience: shared, per-person, per-private-channel.

Default. A curated, task-scoped tool set per workflow, read-only by default, with writes
going through a small number of declared operations. Repo-resident docs as the main context.

3.7 Identity and authorization

What it does. Says who the agent acts for and what it may do on this task.

Without it. Agents run as a person, with that person's full access, and nobody can tell
afterwards which actions were the agent's.

Options.

• A bot identity per agent or workflow, such as a GitHub App, scoped to the repositories
  and permissions it needs. Bots should authenticate as themselves and never impersonate the
  human who invoked them (ClickClack's bot model is a clean reference [docs]).
• Per-task authorization. Agent Baseline's runtime authorization service grants scope per
  task; a request outside it is blocked until a new decision grants it [docs].
• Approved is not authorized. Keep "a human approved this" and "this is authorized to
  execute" as separate system states (one practitioner used execution_authorized=false
  blocking all writes even after approval) [opinion].
• Controls on who can dispatch. In Claude Code self-hosted environments, any member of the
  Anthropic organization can dispatch a session to any of its environments, and Slack routing
  can widen that further. Anthropic's guidance is to treat every runner host as reachable for
  code execution by everyone who can dispatch to it [docs].

Default. One bot identity per workflow, short-lived tokens, the narrowest repository
permissions that work, and branch rules that the bot cannot bypass.

3.8 The control plane

What it does. Owns the work: what exists, who has it, what state it is in, what is allowed
next, and what happened.

Without it. Work lives in chat threads and agent context. A restart loses state, two agents
pick up the same task, a crash between "pushed" and "recorded" causes a duplicate write.

What it needs, as converged across sources:

• A durable work item. Harvey's Spectre makes the run record, not the worker process, the
  durable object; follow-ups start fresh workers from archived state [docs]. At minimum,
  bind: ticket, spec version, base commit, sandbox/worktree, agent identity, stage, attempt,
  reviewed commit, validation evidence, human approval, cost, and failure history.
• Structured work, not free text. Headway's dispatcher creates typed tickets; labels assign
  them to worker pools; status columns map to lifecycle; progress goes in one edited
  structured comment. Free-text Slack coordination had caused duplicate work and colliding
  PRs [docs].
• Deterministic dispatch. Filtering, claiming, capacity checks, and state transitions are
  ordinary code. Leases, per-pool capacity (Headway caps three open PRs per pool), jitter,
  rate limits, and event-driven wake-ups instead of polling (hourly polling left review
  comments waiting 30 minutes on average) [docs].
• Typed workflow contracts. Monzo's workflow defines response shape, handling,
  throughput class, observability, team attribution, and cost accounting, keeping business
  logic out of the runner [docs].
• Single writer for shared state, with expected-revision fencing (compare-and-swap against
  current HEAD) and an idempotency key on every external write [opinion].
• Crash-safe retries. A durable-execution engine (Temporal is the common choice) catches a
  retry after a crash and returns the prior result instead of writing twice [opinion].
• Fresh context per stage, with resumption only through an explicit transition (for
  example, review feedback sending a lane back to implementation) [docs].

Options.

• Build a small kernel. Antfarm shows the minimum: YAML steps, SQLite state, cron polling,
  fresh context per step, machine-checkable acceptance criteria, bounded retries, escalation
  [docs]. Its reported verify_each deadlock is a reminder to test state transitions, not
  just the happy path.
• Reference designs. LIZA's deterministic supervisor owns transitions, role boundaries,
  commit-SHA fencing, leases, bounded iteration, and rollback; reviewers return typed
  verdicts and never own merge authority [docs].
• GitHub-native. gh-aw turns Markdown workflow descriptions into scoped GitHub Actions;
  gh-aw-cao adds a separate control repository so target repos receive only declared safe
  outputs, credentials are resolved inside each run, and live mutation needs an approved
  authority per target [docs]. Both are research prototypes from GitHub.
• Products. Omnara (open-source, self-hostable: durable sessions, RBAC, scoped secrets,
  approval pauses, append-only audit) [vendor]; Kiro Crew (open-source: Slack, Discord,
  Telegram, webhooks, cron, sub-agents, signed audit log) [vendor]; Mastra Factory (explicit
  intake → triage → plan → build → review → complete stages, server separate from the target
  repo) [vendor]; Fredrin (one ticket = one branch, worktree, worker, plan, acceptance
  checks, diff, PR) [vendor]; Tembo self-hosted (VM per agent in your cloud, approvals,
  audit) [vendor]; Claude Managed Agents (sessions, event stream, multiagent, outcomes)
  [docs].

Default. Write a one-page spec of the work-item record and its legal transitions before
picking a tool. Then pick the smallest thing that enforces it.

Gotchas.

• Every tool puts a different boundary in a different place. Several products advertise
  autonomous merge or lack a documented human gate; check before pointing one at a repository
  with write access.
• Claude Code Agent Teams (experimental) coordinates several CLI instances on one host with a
  shared task list and file mailboxes. It is coordination, not isolation: teammates share the
  lead's host, filesystem, and permission mode, and file contention is left to how you split
  the work [docs]. One useful detail: messages between teammates are treated as untrusted,
  so a denied action can't be relayed through another teammate.
• Hard limits exist. Claude Managed Agents: 25 concurrent threads per session, 20 agents per
  roster, one level of delegation [docs].

3.9 Triggers and the chat surface

What it does. Turns a request (a chat message, a ticket, a red CI run, a production alert,
a schedule) into a work item, and keeps the requester informed.

Without it. Agents only run when an engineer types a prompt.

Options by surface.

• Slack has the most first-party support: Claude Tag (Claude in Slack) [docs], and most
  public precedents (Stripe, Shopify, Block, Monzo) trigger from Slack [report].
• Microsoft Teams had no equivalent as of August 2026: Claude's Microsoft 365 connector
  can read Teams but cannot post; community Bot Framework bridges were hobby-grade; Microsoft
  Agent Framework has no Teams integration. Expect to build a Bot Framework bridge. Devin is
  one hosted product that triggers from Teams [vendor].
• Discord, Telegram, webhooks: Kiro Crew supports these directly [vendor].
• Tickets and repos: GitHub issues, Linear, and Jira through gh-aw, GitHub Apps, or the
  orchestrator's own integrations.
• Non-chat signals. Runlayer starts work from customer-call transcripts, customer Slack
  channels, red CI on main, and pins left on any UI element in production (a webhook carries
  the element, page, and note) [opinion].
• A dedicated chat app (ClickClack, LobeHub) sidesteps a hard-to-integrate surface but
  asks people to use a second app; the precedents all met people where they already were.

The chat bridge has a behavior layer, not just transport. Supermemory's open-sourced
"company brain" harness [docs] is the most concrete reference:

• Classify events before any model call: explicit (mention, DM), passive (could
  participate), context-only (keep, don't answer), ignored (bot traffic, unsupported types).
• Score with a small model, decide with code. The model scores usefulness, confidence,
  urgency, noise, and interruption cost; deterministic rules pick answer, investigate, react,
  or stay silent. An explicit mention always gets a response.
• Allow silent conclusions so background checks don't post "I checked and found nothing."
• A steering gate for messages that arrive mid-run: ignore, append (inject before the next
  step through a durable inbox), replace the request, or stop. Without it, "stop, wrong repo"
  waits until the run ends.
• A step budget with forced wrap-up: warn the model a few steps before the limit, remove
  tools at the last step so it must answer.
• Approval that suspends and resumes the same run, with serialized state, rather than
  handing off to another agent.

Intake should dedupe. Runlayer checks every request against Linear and open PRs; a
duplicate adds the requester to the existing issue instead of starting a second run
[opinion].

Default. Use chat for discussion and status, and a structured work item as the source of
truth. Design the bridge's behavior (steering, silence, approval) before writing it.

Gotchas.

• Webhook receivers fail. Anthropic's webhook infrastructure auto-disables a flaky endpoint,
  blackholing notifications until fixed [docs]. Use durable queuing and replay.
• Recover from a durable cursor, not the live socket: reconnects should resynchronize from
  stored state (ClickClack's "database truth, realtime pipe") [docs].

3.10 Verification and evidence

What it does. Establishes that a change works before a human spends time on it, and gives
the human what they need to judge it quickly.

Without it. Review becomes re-deriving the work, or a rubber stamp.

Patterns.

• A bounded loop. Stripe's "Blueprints" interleave deterministic nodes (lint, push) with
  agentic nodes (implementation): local lint (~5 seconds) → one CI pass with autofix → one
  more local retry → human review, capped at two CI rounds, because models show diminishing
  returns retrying the same problem [report].
• Real-stack testing. Run the change against real services (database, queues, the actual
  dev entrypoint), not only mocks. The case study shows this is feasible inside a microVM.
• Runtime evidence. Cloudflare Worker Previews give each branch its own URL, config,
  observability, and isolated state; agents deploy, run browser and API checks, inspect
  traces, patch, and redeploy [docs]. Ramp's Inspect adds telemetry, feature flags,
  screenshots, and visual verification [docs]. Make "commit + environment + state namespace
  • preview URL + traces + screenshots + CI result" one reviewable artifact.
• Evidence-driven review. Vercel's factory ships every change with a risk assessment and a
  reproducible validation proof [docs]. Runlayer attaches before/after screenshots or a
  recorded walkthrough so no reviewer checks out the branch [opinion].
• A decision ledger. Have the agent list every judgment call it made where the spec was
  silent, least-confident first, with what it guessed and what it would have asked. One
  practitioner reports ~30 decisions that mattered out of tens of thousands of unaudited
  lines [opinion].
• An independent reviewer that can't edit. Run review as a separate pass with fresh
  context; a model reviewing its own work rationalizes, and a reviewer that can fix code
  starts optimizing for a clean report [opinion]. Two agents agreeing can still be wrong for
  the same reason if they share assumptions or fixtures [opinion].
• Specialize by stage, not further. Vercel splits classification, reproduction, fix,
  analysis, implementation, review, docs, and backporting [docs]. Secondary summaries of a
  Princeton NLP paper report single agents matching or beating multi-agent systems on 64% of
  benchmarked tasks given equal tools and context; split only where tasks need different
  capabilities, parallelism, or separation of concerns [report].

Default. Bounded loop, real-stack tests where the change touches them, a fresh-context
read-only reviewer, and an evidence bundle on every PR.

3.11 Review and merge

See section 4. This is the decision everything else depends on.

3.12 Observability, audit, and data

What it does. Lets you reconstruct why an agent made a change, prove what it did, and
improve the system from what you learn.

Without it. You have a diff and nothing else. You can't tell which tasks needed help or
whether a prompt change made things better.

Patterns.

• Tamper-evident records outside the agent's control. Kiro Crew advertises a signed audit
  log [vendor]; Agent Baseline includes a tamper-evident decision record [docs]; OpenAI's
  agent-security lead stresses keeping evidence the agent cannot alter [opinion].
• Every dispatched action attributable to a control-plane run (gh-aw-cao) [docs].
• Structured events for model requests, tool calls, file operations, approvals, and token
  use (DigitalOcean Insights, Boat's events) [vendor].
• Treat it as data. Across many tasks, track which kinds of work need human intervention,
  where agents get stuck, and how prompts, models, and verification approaches compare on
  outcomes (Josh Rosen's "software factories are data factories") [opinion].
• Evaluations as tests. Keep a regression suite for agent behavior and rerun it when
  prompts, skills, models, tools, or context change (DigitalOcean AI Platform [vendor],
  LangSmith MDA's evals/ directory [docs]).
• Factory as code. Keep the factory's prompts, routing, verification rules, and approval
  points versioned in their own repository, so changes can be measured.

Default. An append-only run log keyed by work item, stored outside the sandbox, from day
one. Add evaluations once you change prompts regularly.

3.13 Cost

Patterns.

• Decompose spend. Uber: users × sessions per user × turns per session × requests per turn
  × tokens per request × price per token; then cost per outcome (merged PR, review, alert,
  cleanup) next to quality signals such as revert rate and MTTR [docs].
• Cap it in the control plane. Session budgets (Claude Managed Agents shares one budget
  across a session's threads) [docs]; per-lease and monthly caps (Crabbox) [docs]; cost
  caps in the credential proxy (Browser Use) [docs].
• Pay for activity, not waiting. Pause or hibernate sandboxes while the agent waits on a
  model, tool, or human (DigitalOcean, Boat, InstaBox, boxd) [vendor].
• Route cheap work to cheap models. Supervisory and triage steps don't need the strongest
  model (Uber, Kiro) [docs].
• Subscriptions vs API billing. Some orchestrators run on your existing agent
  subscriptions (Fredrin, Nimbalyst) [vendor]; managed platforms usually bill per token.

Default. Record cost per work item from the first run, and put a hard cap on every run.

3.14 After the merge

Most designs stop at the PR. Mature ones keep going:

• Deployment as its own step with its own evidence. OpenAI's reported factory has a
  per-change agent that determines success and failure signals, can build a dashboard for the
  change, and stays with the rollout [report].
• Production signals become work. A performance factory de-duplicates alerts and proposes
  fixes; an incident bot gathers context and proposes mitigations but doesn't execute them
  without an engineer [report]. Start with observe-and-propose.
• Post-merge QA. Runlayer runs a QA fuzzing agent against production after each merge
  [opinion].
• Release awareness. Runlayer shipped so fast that sales and field engineers didn't know
  what was live. Their fix: an append-only changelog (one Markdown file per PR that belongs in
  it), a weekly short video of the top changes, and direct messages to the people who asked
  for a feature or reported a bug [opinion].
• Capacity. OpenAI reportedly saw roughly 10x load growth on some version-control, CI/CD,
  and release systems over about six months [report]. Model pushes, builds, review jobs,
  retries, and deployments before scaling agent volume.

3.15 Adoption

The platform's value includes how many people can safely turn a need into a reviewed change.
Cursor's Grab case study [vendor]:

• Enablement beats mandate: help people solve a problem they already care about.
• Bug fixing is the universal entry point, because the starting evidence is concrete and the
  result is checkable.
• Distributed creation can keep local review: designers merged hundreds of UI fixes after
  design-manager review. The pattern is role-appropriate ownership with a nearby reviewer, not
  "everyone can merge anything."
• Measure completed work (time-to-fix, queue time, which roles ship), not just usage.

Monzo's lesson [docs]: agents move the bottleneck from finding an engineer with capacity to
finding a reviewer.

4. Merge authority: the spectrum

Who is allowed to merge an agent's change is the decision every other layer depends on. This
section presents the options without picking one; the right level depends on what a merge
triggers in your environment and how much of the platform you've built.

First, find out what a merge does. If merging to your main branch deploys to production
with no further gate, a merge is a production deploy, and every level below inherits that
risk. If a merge only lands code that a separate release process ships, the stakes are lower.

Whatever the level, two rules hold across every source:

• Enforce it in the system. Permissions on the bot identity, branch protection, and the
  merge worker's checks, not a sentence in the prompt.
• Keep "approved" and "authorized to execute" separate, so an approval can't be replayed
  against a different commit.

5. A staged path

Stage 0: interactive (where most teams are)

One developer, one session, maybe a few in parallel worktrees. Agents use whatever tools that
developer has configured.

Do now, even at this stage: keep agent skills and repo context in version control; stop
running agents with production credentials on laptops; learn which tasks agents do well.

Stage 1: one supervised remote runner

Shape. MindStudio's walkthrough is a reasonable minimum [vendor]: one remote machine, a
context-gathering interview turning a ticket into context files, explicit human handoffs for
infrastructure and identity, then a validated PR with human review.

Build.

• One sandbox per run (3.2), with a warm template.
• A bot identity with the narrowest permissions (3.7).
• Default-deny egress and proxy-side credentials, or at least short-lived tokens (3.3, 3.4).
• A bounded verification loop with an evidence bundle (3.10).
• An append-only run log outside the sandbox (3.12), and a cost cap per run (3.13).

Exit criteria. You can answer, for any agent PR: what triggered it, what it could reach,
what it ran, why it made its choices, and what it cost.

Stage 2: workflow-triggered

Build.

• A durable work-item record and its legal transitions (3.8).
• A trigger and a chat bridge with a behavior layer (3.9), plus intake dedupe.
• An independent read-only reviewer (3.10).
• An explicit merge-authority decision, enforced (section 4).
• One scheduled chore (CI-failure triage, dependency updates, doc fixes) before anything
  autonomous. Kiro's sequencing advice [vendor]: memory and one scheduled chore before an
  autonomous supervisor.

Exit criteria. Restarting any component loses no work; two requests for the same thing
produce one run; no agent can merge outside the chosen policy.

Stage 3: many agents, orchestrated

Build. Queue-backed session types (triage, implementation, review, merge) with capacity
limits; a supervisor that owns decomposition and acceptance but not permissions; a tool
gateway; evaluations; production feedback; a capacity model for CI and review.

Exit criteria. Cost and quality per delivered outcome are measured, and they improve when
you change the system.

How much work is it?

One planning-level estimate, from a team of roughly mid-size with an existing repo-aware skill
chain and a sandbox spike already done: about 9–13 weeks with 1–2 engineers to a supervised
MVP for one team, not yet safe for autonomous merges. It assumed a custom chat bridge had to
be built. Treat it as an order of magnitude, not a benchmark.

Beyond that, it's an ongoing team, not a project with an end date. Stripe reportedly runs
about a dozen engineers on developer-environment reliability alone [report]; Shopify's
Aquifer took a dedicated platform team months before River shipped [docs].

One advantage smaller teams have: Stripe's custom tooling was forced by scale (hundreds of
millions of lines, homegrown libraries no model has seen). A smaller codebase can lean on
off-the-shelf skills, MCP servers, and CI for most of what they had to build.

Build vs buy, by layer

There are three broad routes. Most teams mix them.

Whichever route, the repo-specific parts stay yours: the workflow contracts, the permission
policy, the verification adapters for your stack, and the merge decision.

6. Case study: a real dev stack inside Docker Sandboxes

[tested] throughout. One team, one macOS Apple Silicon machine, interactively supervised,
August 2026.

Setup

• The stack: a monorepo with a 17-service Docker Compose file behind a Tilt-based
  dev up entrypoint: PostgreSQL with PostGIS and row-level security, Redis, RabbitMQ, a
  Node API, several Vite frontends, and eight opt-in Python processing workers.
• The sandbox: Docker Sandboxes (sbx) v0.38.0, installed with Homebrew, balanced
  network policy.
• Isolation of the test itself: every run used a throwaway git worktree, never the main
  checkout; all sandboxes, worktrees, and branches were removed afterwards.

What happened

1. A real Docker daemon runs inside the sandbox. docker version, docker info, and
   docker compose all worked; the worktree was bind-mounted correctly. Compose parsed all 17
   services.

2. The database crashed on start: exec format error. The official postgis/postgis
   image has no arm64 build on any tag. The microVM is pure aarch64 with no x86 emulation
   (no binfmt_misc, nothing installable to add it). The host had only been getting away
   with it through Colima's Rosetta emulation. sbx has no platform or emulation flag.

3. A multi-arch replacement image worked natively. Two community PostGIS images publish
   real amd64+arm64 manifests with the same versions. With one swapped in, the container ran
   as aarch64, the existing init script created PostGIS and the other required extensions
   unmodified (two optional extensions skipped through the script's existing fallback), and
   the database accepted connections immediately.

4. Two sandboxes ran side by side with no collisions, even with identical container names
   and identical host port bindings in each: every sandbox is its own network namespace. This
   was tested at n=2 with lightweight services only.

5. The full 17-service stack did not come up in one pass.

   • One worker failed to build: its Dockerfile is pinned to amd64 because its geospatial
     toolchain isn't sourced for arm64. Swapping an image doesn't fix that.
   • Two others failed at apt-get update: the balanced egress proxy returned 403 for a
     plain-HTTP Debian mirror while allowing npm over HTTPS. A policy gap, fixable with one
     allow rule.

6. The real dev up entrypoint worked for the default footprint (core infrastructure,
   API, frontends; workers are opt-in). Bootstrap hit a third egress block (a toolchain
   installer domain), fixed with one allow rule. After that, the API's own health endpoint
   reported the database and Redis up, and the frontends served 200.

7. Viewing the sandbox's dev UIs from the host needed a relay. Publishing Tilt's UI port
   directly returned a connection reset, while publishing Redis's raw TCP port worked, so the
   publish mechanism was fine. Tilt's and Vite's dev servers reject the NAT'd source address
   (anti-DNS-rebinding guards, and IPv6-only loopback binding for Vite). A socat relay
   inside the sandbox, re-originating the connection from real loopback, fixed it:

   sbx exec <name> -- sh -c 'setsid socat TCP-LISTEN:20350,fork,reuseaddr TCP:127.0.0.1:10350 &'
   sbx ports <name> --publish 20350
   

Lessons

• A microVM per agent can run a real multi-service dev stack, including the daily-dev
  entrypoint, with no changes to that entrypoint. Teams that already have one deterministic
  dev up command get agent symmetry for free: agents run exactly what humans run.
• Single-tenant dev setups become a non-issue. Fixed container names and ports, a
  constraint for parallel humans on one machine, don't matter when each agent has its own VM.
• Check every image's architecture before you start. docker manifest inspect on each
  image in your Compose file takes minutes and finds the blockers.
• Build your egress allowlist from the policy log, and expect several rounds.
• Plan for dev-server security features when exposing sandboxed UIs to humans.
• What this did not test: credentials, egress beyond the default policy, what the sandbox
  can read, action governance, audit, concurrency beyond two, sustained load, unattended
  operation, database migrations against the replacement image, or the replacement image's
  CVE and SBOM profile. A database container coming up clean says nothing about those.

7. Readiness checklist

Score each yes / partial / no. The first 16 questions come from a platform-engineering
deck's "Platform Test" [opinion], which reads 14+ as a real agentic platform, 10–13 as agents
outrunning governance, and under 10 as automated chaos. Teams that have never run agents
unattended will score low; that measures what's unbuilt, not damage already done.

Sandboxing

1. Do agents act through versioned, auditable interfaces rather than raw shell access?
2. Can an agent get an isolated sandbox in one self-service step?
3. Are agents sandboxed from interfering with each other?
4. Are new agent permissions trialled in a scoped sandbox first?

Registry and context

5. Do agents query a live service catalog rather than stale docs?
6. Is there a live, queryable map of ownership and dependencies?
7. Is there a governed tool and SDK registry rather than improvised dependencies?
8. Do agents work from structured specs rather than open-ended prompts?

Verification

9. Does verification gate an agent's output before the next task builds on it?
10. Do automated verifier agents check output before a human sees it?
11. Is agent spend bounded, with circuit breakers for runaway loops?
12. Can you revoke an agent's access or roll back its change instantly?

Observability and identity

13. Is every agent-caused issue logged back to its run?
14. Do you spot-check agent reasoning logs when nothing is broken?
15. Does every agent have its own scoped identity and permissions?
16. Can you reconstruct why an agent made a change, not just the diff?

Additional checks this guide recommends

17. Is egress default-deny, with the cloud metadata endpoint blocked?
18. Does the agent hold no real secrets (proxy-side injection), or at least only short-lived,
    narrow tokens?
19. Have you tested what the sandbox can read, including host env, mounts, and sockets?
20. Is anything the agent can run commands in (other containers, staging) inside the same
    egress and credential boundary?
21. Are images and packages limited to allowlisted registries with pinned digests?
22. Is merge authority decided, written down, and enforced by permissions?
23. Has someone tried to break the sandbox with a model told to get a planted secret out?
24. Does a message sent during a run reach the agent (or stop it) before the run ends?
25. Do duplicate requests collapse into one work item?
26. Is cost recorded per work item?
27. Do the people who need to know what shipped find out?

For a more formal control list, see Agent Baseline's controls.yaml, which maps controls to
real incidents.

Appendix A: Landscape (as of 2026-10)

Short entries only. "Fit" says where each sits in section 3's layers.

A1. Sandboxes and execution substrates

A2. Hosted and self-hosted machines

A3. Managed agent platforms

A4. Orchestration and control planes

A5. Governance, tools, and context

A6. Chat and collaboration surfaces

A7. Previews and evidence

A8. Considered and not a fit for coding agents on a real stack

• Google Gemini Enterprise Agent Platform: built for connector-based business and RAG
  agents with a UI-driven agent gallery; no model of a monorepo, CI, or PR gate.
• Microsoft Agent Framework: a general multi-agent SDK (AutoGen + Semantic Kernel), no
  Teams integration; adopting it means leaving your existing coding harness.

Appendix B: Precedents

Reported numbers are the companies' own unless marked.

Appendix C: Further reading

Frameworks and principles

• Agent Baseline: The Secure Software Factory reference architecture
• Agent sandbox taxonomy
• Code is still law: prompts aren't permissions
• The agentic loop
• Joe, "It's not just the f*cking sandbox"
• Josh Rosen, "Software Factories for Skeptics"
• David Zhang on decision ledgers
• Jehanzeb Khan, "Scaling Engineering With Agents Looked Like Scaling a Distributed System"

Engineering write-ups

• Stripe Minions (ByteByteGo) ·
  MintMCP explainer ·
  Stripe's dev environment
• Shopify: cloud development with Spin
• Vercel: building a software factory for AI SDK
• OpenAI's software factory (The Pragmatic Engineer)
• Monzo Agent Chip ·
  Checkout HAL ·
  Headway ·
  Harvey Spectre ·
  Ramp Inspect ·
  Uber ·
  Block BuilderBot (The New Stack) ·
  MongoDB Sage ·
  Grab with Cursor ·
  Kiro: 1,000 PRs
• Browser Use's agent sandboxing
• Docker: why microVMs
• Runlayer's factory
• Supermemory company-brain harness

Primary docs

• Claude Managed Agents ·
  self-hosted sandboxes ·
  multiagent orchestration
• Claude Code self-hosted environments ·
  deploy to production ·
  Agent Teams
• Anthropic sandbox-runtime
• LangSmith Managed Deep Agents

This guide grew out of one team's internal research record on agent sandboxes and an agentic
SDLC, generalized for anyone building the same thing. Corrections and additions welcome: reach me on
X (@ahmedkeewan_) or LinkedIn.