The 4 concepts of cloud deployment

Tap any concept to jump to its 5-card cluster.

• 1Why Move From Local to Cloud?24/7 uptime, GPU speed, team access
• 2Provider ComparisonVast, GCP, AWS, Lambda — pick one
• 3Managed Open Source AlternativesGroq, Together, Fireworks — no VM
• 4Provider-Agnostic WrapperOne env var switches every backend

Same code, a machine that never sleeps

Local Ollama is perfect for building, but it dies when you close the lid, no teammate can reach it, and a 3 am job finds nothing listening. A cloud GPU VM fixes all three: it runs 24 / 7 and a T4's dedicated 16 GB VRAM delivers ~40 tokens/sec — five times a laptop CPU.

The magic is that nothing about your agent changes. Every call still points at base_url; you just swap localhost for the VM. It's a config change, not a migration.

The analyst who never sleeps

BEFORE: Picture a brilliant research analyst who only works at the one desk in your home office — your laptop. While you're there during the day, they're fast and helpful.

PAIN: But the moment you shut the office (close the lid), they vanish. A colleague overseas can't knock on your door, the 3 am batch job finds an empty desk, and when three people show up at once they all queue for the same single chair.

MAPPING: A cloud GPU VM hires that same analyst into an office that never closes. They work around the clock, sit behind a locked door only an SSH key opens, and their GPU desk is five times faster. You didn't retrain the analyst — you just gave them a better, always-open room.

Laptop to cloud, step by step

1. Provision a GPU VMRent a VM with a GPU (e.g. a T4) and install Ollama on it.
2. Run Ollama as a serviceRegister it so it starts on boot and restarts on crash — always listening.
3. Open a secure tunnelForward your local port to the VM over an encrypted SSH channel; no public port needed.
4. Point base_url at itRead the endpoint from an env var. Dev = localhost, prod = the VM. One value flips it.

laptop 8 tok/s → T4 40 tok/s → A100 200 tok/s — the code is identical.

Pseudocode

# the ONLY thing that changes: the host
base_url = env("OLLAMA_BASE_URL",
  default = "http://localhost:11434/v1")

# dev  -> localhost (unset var)
# prod -> set OLLAMA_BASE_URL to VM tunnel

client = open_ai_client(
  base_url = base_url,
  api_key  = "ollama")

reply = client.chat(model="mistral",
                    messages=[...])
# no if-branch, no adapter, no rewrite

Misconceptions

Not all GPU clouds are equal

Once you decide to host, you pick a provider — and they trade off on three axes: cost per hour, inference speed, and operational fit with your existing stack. A Vast.ai RTX 3090 spot instance is ~$0.20/hr and ~80 tok/s; a GCP T4 is ~$0.35/hr and ~40 tok/s; a CoreWeave A100 is ~$2.21/hr and ~200 tok/s.

There's no single "best" — the cheapest box for experiments is rarely the enterprise box for production. You match the provider to the job, not to a leaderboard.

Renting a car for the trip

BEFORE: Imagine every rental counter offered the exact same car at the same price. You'd just grab whichever line was shortest.

PAIN: Real lots aren't like that. A cheap subcompact is perfect for a solo errand but miserable for moving a family of six; a luxury SUV is overkill (and overpriced) for a quick grocery run. Pick wrong and you either can't fit the load or bleed money.

MAPPING: GPU providers are the rental lot. Vast.ai / RunPod are the cheap subcompact — great for dev and experiments. GCP / AWS are the mid-size sedan your company already has an account with. Lambda / CoreWeave are the heavy-duty haulers for real production throughput. Match the vehicle to the trip.

Read the comparison in 3 lenses

1. Cheapest & fastVast.ai / RunPod RTX 3090: ~$0.20–0.22/hr, ~80 tok/s. Best for dev, spiky experiments.
2. Native to your stackGCP T4 (~$0.35) or AWS g4dn (~$0.53): ~40 tok/s. Best if your team already lives there.
3. Production throughputLambda A10 (~$0.60, ~100 tok/s) or CoreWeave A100 (~$2.21, ~200 tok/s).

Which provider should I rent?

IF just experimenting / on a budget:
  → Vast.ai or RunPod (3090 spot)

IF team already uses GCP or AWS:
  → stay native: GCP T4 / AWS g4dn

IF steady production traffic:
  → Lambda A10 or CoreWeave A100

# also weigh: token volume vs a managed
# per-token API (see Concept 3)

Rough crossover: a T4 at 8 hr/day is ~$56/mo — stop the VM when idle or the meter keeps running.

Misconceptions

Someone else runs the GPU for you

You don't have to run a VM at all. Providers like Groq, Together AI, and Fireworks host open-source models and expose them through the same OpenAI-compatible API. You swap the URL and hand over a real API key instead of "ollama" — that's it.

Zero infra to manage, free tiers for development, and specialists like Groq deliver ~50 ms first-token latency on custom hardware. The price is per-token billing and shared rate limits instead of a flat hourly VM.

Owning a car vs. calling a taxi

BEFORE: Self-hosting a GPU VM is owning a car. Total control, but you buy it, park it, insure it, and it depreciates whether you drive or not.

PAIN: For occasional or bursty trips, ownership is wasteful — the car sits idle burning money, and you're on the hook for maintenance you never wanted.

MAPPING: A managed API is a taxi. You pay only per ride (per token), someone else owns and services the fleet, and a premium service (Groq) shows up almost instantly. You give up control of the vehicle and accept a meter — but for many trips that's exactly the right trade.

Swapping to a managed provider

1. Sign up & get a keyCreate an account, grab an API key, store it in an env var — never in code.
2. Swap the base_urlPoint the same OpenAI client at the provider's endpoint instead of localhost.
3. Use the real keyPass the provider key in place of the "ollama" placeholder.
4. Fix the model nameEach provider names models differently — check their docs for the exact string.

Groq ~50 ms TTFT, Together / Fireworks ~150–200 ms — all via the same SDK.

Managed API or self-hosted?

IF data may leave your infra
   AND usage is bursty / early dev:
  → managed API (Groq / Together / Fireworks)

IF you need the fastest latency, free:
  → Groq (LPU hardware, ~50ms)

IF data must stay in-house (HIPAA/GDPR)
   OR token volume is very high:
  → self-host Ollama on a VM

# crossover ~2-5M tokens/month

Below ~2–5M tokens/month, per-token APIs win; above it, the flat VM rate is cheaper.

Misconceptions

One dial controls every backend

With four possible backends (local Ollama, Groq, Together, Fireworks), you don't want base_url and key lookups scattered across your code. Centralise them in a single factory — a LocalModelClient that reads one env var, LOCAL_MODEL_PROVIDER, and returns a configured client plus that provider's default model name.

Now switching every agent in the app from Ollama to Groq is one env var, not a code sweep. The factory hides the URL, the key handling, and the provider-specific model string behind a single call.

A universal power adapter

BEFORE: Imagine every country's wall socket needed you to rewire your laptop charger by hand before you could plug in.

PAIN: You'd carry a box of loose plugs, re-solder at each border, and one wrong wire fries the device. Every new country means touching the hardware again — error-prone and slow.

MAPPING: The wrapper is a universal travel adapter. You flip one selector to the country (set LOCAL_MODEL_PROVIDER) and the same charger works everywhere. Your agents plug into one socket; the adapter quietly routes power from Ollama, Groq, or Together — no rewiring per provider.

Inside the factory

1. Read the providerTake the name from LOCAL_MODEL_PROVIDER, defaulting to ollama.
2. Look up its configEach provider has a stored base URL and a default model name.
3. Validate & fetch the keyReject unknown providers; pull the API key from its env var, erroring early if missing.
4. Return client + configHand back a ready OpenAI client and the config so callers use the right model.

Pseudocode

CONFIGS = {
  "ollama":    (localhost_url, "mistral"),
  "groq":      (groq_url,      "llama-3.3-70b"),
  "together":  (together_url,  "Mistral-7B"),
}

FUNCTION create(provider = null):
  name = provider OR env("LOCAL_MODEL_PROVIDER",
                          "ollama")
  IF name NOT IN CONFIGS: FAIL loudly

  url, model = CONFIGS[name]
  key = "ollama" IF name=="ollama"
        ELSE require_env(name + "_API_KEY")

  RETURN client(url, key), model

Misconceptions

Test the core insight

One question per concept. Tap to reveal the answer.