The 5 deployment concepts

Tap any concept to jump to its 5-card cluster. Lab steps live on desktop.

• 1Three Deployment TargetsSame agent, three serving shapes
• 2Wrap Agent as HTTP ServiceWhy every runtime needs an API surface
• 3Containerize: The Docker PatternPack the runtime, ship it anywhere
• 4Docker vs Cloud Run vs LambdaThe trade-offs and when each wins
• 5Real-World Deployment PatternsPersistence, queues, multi-region

One agent, three deployment shapes

Your agent is a function: question goes in, answer comes out, with maybe a tool loop in the middle. That function does not care where it runs. What changes is the shape of the runtime around it: a long-lived process you control, a managed container that scales itself, or a serverless function that wakes up on demand.

Production teams routinely run the same agent on all three at once: Docker on the laptop for fast feedback, a managed container in production for steady customer traffic, and a serverless function for nightly batches or webhooks. The agent code is identical in every case — the deployment target is a configuration choice, not a rewrite.

One kitchen, three serving formats

BEFORE: Picture a restaurant with one kitchen, one set of recipes, one head chef. The food is always the same. What changes is how it reaches the customer: dine-in tables for sit-down service, a drive-through for quick orders, a food truck that rolls up to events.

PAIN: If you build a separate kitchen for each format, three menus drift apart, three line cooks blame each other when a dish goes wrong, and a recipe fix has to be made three times. Your "improvement" lands in one place and disappears in the others.

MAPPING: The agent core is the kitchen. The local container is dine-in — you control the lights and the music, and nothing scales without you. The managed container platform is the drive-through — spins up on demand, costs nothing when nobody is in line, has a small warm-up moment for the first customer. The serverless function is the food truck — pay only when you serve, with a slightly longer roll-up time and a hard cap on how long any one shift can last. Same recipes, different windows.

The kitchen never moves. Only the window changes.

The portability split

1. Write the core onceThe agent loop — tools, system prompt, message stack — lives in one function. It has no opinions about HTTP, processes, or cloud platforms.
2. Wrap it with a thin adapterThe adapter turns whatever the runtime hands you (an HTTP request, a function event, a CLI argument) into a call to the core function and serializes the result back out.
3. Pack runtime + adapter into a containerThe container becomes the unit of deployment. Same image runs on your laptop, on a managed container platform, or as the basis of a serverless function image.
4. Pick the target per workloadLocal for development. Managed container for steady customer traffic with long requests. Serverless for low-volume, spiky, event-driven, or batch work.
5. Same test command for all threeIf your contract is "POST a question, get JSON back," one curl command verifies every deployment target. The interface is what stays constant.

Order of operations: prove it locally, then push the same image to a managed runtime. Don't debug a cloud-only build from scratch.

Which target, when?

# What runtime should host this agent?

IF phase == "dev / debugging":
  USE local container
  # no cloud cost, no cold starts, fast iterate

ELIF traffic == "steady"
   AND request > 30s:
  USE managed container platform
  # scale-to-zero, long timeouts, native HTTPS

ELIF traffic == "spiky / event-driven"
   AND request < 15min:
  USE serverless function
  # pay-per-call, native event triggers

ELIF request > 15min
   OR needs GPUs
   OR persistent sockets:
  USE orchestrated containers
  # Kubernetes / ECS / managed worker pool

ELSE:
  DEFAULT to managed container platform
  # least surprising, fewest constraints

Pick the topology first; the platform follows from it. Do not pick a platform you "like" and bend the workload to fit.

Misconceptions

Wrap the agent as an HTTP service

Right now your agent is a script — you run it in a terminal, it answers, it exits. That is fine for development, but every deployment runtime in this module — local container, managed container, serverless — speaks one language: HTTP requests. Until the agent can answer an HTTP call, none of them can host it.

The fix is a web framework that exposes a /query endpoint and a /health endpoint. The query endpoint validates the incoming question, runs the agent, returns JSON. The health endpoint says "alive and configured." That is the whole shape: one function call, two endpoints. Everything else — auth, rate limiting, observability — layers on top of this skeleton.

Give the researcher a phone number

BEFORE: Imagine the agent as a brilliant researcher who only takes meetings in person. You walk to their office, sit down, ask the question, and wait. One person, one room, one question at a time.

PAIN: Nobody else can use this researcher. Your web app cannot call them. Your colleague in another building cannot call them. The researcher is locked in one room with one visitor — everything has to happen face-to-face.

MAPPING: Wrapping the agent in an HTTP service is like giving that researcher a phone number. Anyone in the world can dial in (a POST to /query), ask a question, and get the answer back — without being in the same room. The web framework is the phone system: it routes calls to the right researcher, hangs up cleanly when something breaks, and lets a separate line (/health) confirm the office is still open.

The two-endpoint skeleton

1. Validate the requestAn incoming JSON body is checked against a schema. Bad shape, bad type, missing field → the framework rejects with a 422 error and the agent never runs. Garbage requests get filtered before they cost any compute.
2. Call the agent coreOnce the request is clean, the handler calls the same agent function you would call from a script. No special HTTP knowledge needed inside the agent.
3. Wrap the answer in JSONThe handler serializes the answer (and timing, status, request ID) back as JSON. Errors are caught and turned into proper HTTP status codes — the service should never crash on a bad query.
4. Expose a health endpointA second endpoint, /health, returns 200 when the agent is alive and configured (API key set, dependencies imported). Load balancers and orchestrators ping this every 10–30 seconds to decide who gets traffic.
5. Listen on a configurable portThe framework binds to a port (often supplied by the runtime via an env var). All deployment targets in this module follow the same convention — the platform sets the port, you read it.

Two endpoints is the minimum. Add /metrics, /version, and request IDs once you ship.

Pseudocode

Same agent core, two HTTP endpoints. Validation in front, error handling behind:

# SCHEMA — what a valid request looks like
DEFINE QueryRequest:
  question: string (1..2000 chars, required)

# ENDPOINT 1 — the agent surface
ROUTE POST /query EXPECTS QueryRequest:
  VALIDATE body  # auto-422 on bad shape
  TRY:
    started = now()
    answer  = agent_core(body.question)
    RETURN 200 {
      answer:    answer,
      elapsed_s: now() - started,
      status:    "ok"
    }
  CATCH err:
    log(err)
    RETURN 500 {detail: "agent failed"}

# ENDPOINT 2 — liveness for orchestrators
ROUTE GET /health:
  IF NOT env.has("ANTHROPIC_API_KEY"):
    RETURN 503 {detail: "key missing"}
  RETURN 200 {status: "ok"}

# BIND — runtime supplies the port
SERVE on host=0.0.0.0 port=env.PORT or 8000

Validate at the edge. Catch every error. Bind to 0.0.0.0 — localhost is invisible from outside the container.

Misconceptions

Containerize: pack the runtime, ship it anywhere

A container image is a frozen snapshot of everything your agent needs to run: the language runtime, your dependencies, your code, the user it runs as, the port it listens on. Open the snapshot anywhere — your laptop, a colleague's machine, a managed cloud platform — and it boots into the same state.

This solves the deepest deployment headache in software: "works on my machine." Without a container, your laptop has Python 3.12, the server has 3.10, the cloud VM has different system libraries, and the same code behaves three different ways. With a container, the runtime is part of the package — the host OS contributes almost nothing.

Ship the kitchen, not the recipe

BEFORE: Imagine mailing a home-cooked meal to a friend. You would have to ship the recipe, the exact brand of every ingredient, the model number of your oven, and detailed instructions for adapting it to their kitchen layout. Anything different, the dish tastes wrong.

PAIN: That is software deployment without containers. "It runs on my laptop" but the production server has different OpenSSL, the staging machine has a different glibc, the cloud image is missing a system package. You spend a weekend hunting for the difference.

MAPPING: A container is shipping a fully-equipped kitchen with the meal already inside. The kitchen comes with its own oven, its own ingredients, and the recipe pre-loaded. Open it anywhere — the dish is identical because nothing about the kitchen depends on the destination. The "destination" only has to provide a power outlet (an OS kernel) and a counter to put it on.

What goes inside the image

1. Base layer: a minimal OS + runtimeStart from a small base (slim Python, slim Node) so the final image is hundreds of MB, not gigabytes. Smaller image → faster pulls, smaller attack surface, lower storage cost.
2. Dependencies layer (cached)Install all your libraries before copying source code. The dependency layer rarely changes, so the build cache reuses it — only your code rebuilds on edits, dropping rebuild time from minutes to seconds.
3. Source layer (changes most often)Copy your agent files last. Code changes invalidate only this thin layer; dependencies stay cached.
4. Non-root userCreate a dedicated user inside the image and switch to it before the entrypoint runs. If something exploits a bug, the blast radius is limited — root inside a container is still root.
5. Entrypoint + portDeclare which port the agent listens on and which command starts the server. Secrets like the API key are injected at runtime, never baked into the image.

Layer order matters: dependencies before code = orders-of-magnitude faster rebuilds.

Pseudocode

The image recipe in plain steps — never actual Dockerfile syntax:

# BUILD-TIME RECIPE (image construction)
START FROM minimal language runtime
SET working_dir = /app

# Cached layer: install libs first
COPY IN dependency_manifest
RUN install_dependencies()

# Volatile layer: source code last
COPY IN agent.py, server.py, mock_data.py

# Security: drop root
CREATE USER agent
SWITCH TO USER agent

# Contract: port + entrypoint
DECLARE PORT 8000
CMD start http_server(server.app, port=8000)

# RUN-TIME (one image, many hosts)
BUILD image   FROM recipe   TAG agent:1.0
RUN agent:1.0 PUBLISH port 8000
              INJECT ANTHROPIC_API_KEY FROM env

# Same image, three hosts:
#   laptop, managed container, serverless wrapper

Build once, run anywhere. The image is the artifact; the platform is just where you press play.

Misconceptions

Docker vs Cloud Run vs Lambda — the trade-offs

The three deployment shapes look interchangeable on paper — they all host the same image and answer HTTP requests. They are not interchangeable in practice. They differ on the things that hurt at 3am: cold start latency, maximum request duration, scaling behavior, and cost at idle vs cost under load.

Picking wrong is expensive. A Lambda hosting a 12-second steady-traffic API will out-bill an equivalent Cloud Run service every month. A Cloud Run service running a 30-minute research agent works fine; a Lambda hits its 15-minute cap and the user gets a hard error. The trade-offs do not reveal themselves until you ship — learn them before you commit.

Three vehicles for three commutes

BEFORE: You can drive to work in a car you own, hail a ride-share, or rent a scooter. All three get you to the office. They differ on cost per trip, how long you can use them, how fast they show up, and how big a load they can carry.

PAIN: Pick wrong and you feel it. A ride-share for a daily one-hour commute costs a fortune. An owned car for two trips a year is wasteful. A scooter on the highway is dangerous. Same destination, very different fit.

MAPPING: The local container is the owned car — always available, you pay even when it sits, you control everything. The managed container is the ride-share — appears in 1–2 seconds, no idle cost, scales naturally with demand. The serverless function is the rented scooter — fast for short trips, cheap when used briefly, hard cap on how far it can go in one session, slightly longer to roll up cold.

The four axes that matter

All three scale to zero cost when idle. The differences emerge under load — cold start, request duration, billing model.

Pick by workload, not by brand

# Inputs: traffic shape, request duration, latency tolerance

IF request_duration > 15 min:
  USE managed container OR orchestrated worker
  # serverless will time out mid-run

ELIF traffic == "steady" AND calls_per_day > 10000:
  USE managed container
  # per-ms billing on long agents adds up
  # warm instances handle steady load cheaper

ELIF traffic == "spiky" OR event-driven:
  USE serverless function
  # pays nothing between bursts; scales fast

ELIF latency_tolerance < 200ms p95
   AND agent_run_time < 1s:
  USE managed container with min_instances >= 1
  # keep one warm to avoid cold start

ELSE:
  USE managed container as the safe default

Most production agents end up running on two of the three: managed container for steady traffic, serverless for batch / events. Local stays for dev.

Misconceptions

Real-world deployment patterns

"Send a question, get an answer back" — the simple sync API — covers a real slice of production agents. But the moment your agent runs longer than ~5 minutes, needs to remember conversations across requests, or processes batches unattended, the simple recipe stops fitting. Production stretches the topology in three directions.

The four shapes you actually meet in the field are: sync API (the lab), streaming chat (copilots), async worker (long-running research), and scheduled batch (nightly jobs). Each one demands a different deployment topology and a different set of supporting services — queues, session stores, multi-region failover. Knowing which shape you're building changes which platform you reach for.

The restaurant grows up

BEFORE: Day one, the restaurant takes one order at a time. Customer walks in, orders, waits at the counter, takes the food, leaves. Simple, fast, fits on a napkin.

PAIN: Then reality arrives. A regular wants a dish that takes two hours to braise — nobody is going to wait at the counter. A bachelorette party of 40 walks in at once and overwhelms the kitchen. A corporate client wants nightly delivery to 200 offices on a schedule. The "one customer at the counter" model breaks under each of these.

MAPPING: Sync API is the counter. Streaming chat is a sit-down meal where courses arrive one at a time. Async worker is the two-hour braise — you take a buzzer and come back. Scheduled batch is the corporate catering run — 200 lunches prepared overnight, delivered at 11am. Same kitchen, four very different service models, each needing its own staffing and queue.

Three things production adds

1. Externalize session stateManaged containers and serverless are stateless by design — the next request can land on a brand-new instance with empty memory. If your agent needs to remember the last 20 turns, store that in a fast cache or database keyed by session ID. The compute is disposable; the state lives outside it.
2. Put a queue in front for async workFor long-running agents (research, deep analysis, code migration), the API immediately returns a job ID, drops the request into a queue, and a worker pool picks it up. The user polls or gets a webhook when done. This decouples request lifetime from agent runtime.
3. Throttle to respect model rate limitsThe cloud will happily scale to thousands of instances. The model API will not happily accept thousands of concurrent calls — you'll hit tokens-per-minute or requests-per-minute limits and start seeing 429 errors. A queue plus a configured concurrency cap on the worker pool absorbs bursts and keeps you under the rate limit.
4. Plan for multi-region (when latency or compliance demands)Mirror the deployment in two regions, route users to the closer one, replicate session state, and fail over on outage. Most agents do not need this on day one — but the moment the SLA crosses 99.9%, single-region is no longer enough.
5. Treat observability as non-negotiableAgents fail in ways CRUD APIs do not: wrong tool picked, model loop oscillated, planning step truncated. You need per-step traces, not just request logs. Wire it on day one — bolting it on later costs much more.

The lab builds the sync API. Production usually combines two or three of these patterns under the same agent core.

Match topology to workload

# Pick the topology FIRST. Platform follows.

IF request < 30s AND stateless:
  topology = "sync API"
  stack    = managed container + FastAPI
  # the lab you build on desktop

ELIF token-by-token UX (chat / copilot):
  topology = "streaming"
  stack    = managed container + SSE
           + cache for session state

ELIF minutes-to-hours of work
   OR rate limits will bite:
  topology = "async worker"
  stack    = queue + worker pool
           + job-state DB
           + webhook OR polling

ELIF nightly / scheduled bulk run:
  topology = "batch"
  stack    = scheduler + container jobs
           + Anthropic Batches API (50% off)

# Then layer in: secrets manager, OTel
# tracing, alerting, multi-region as needed

Sync path and async path can share the same agent core. The split is in how it's invoked, not what it does.

Misconceptions

One question per concept

Tap a card to reveal the answer.