The 6 frontier concepts

Tap any concept to jump to its 5-card cluster.

• 1Model Landscape 2026Reasoning, vision, MoE, compact
• 2Multimodal via OllamaSend images to LLaVA
• 3Fine-Tuning with LoRATeach a model your domain
• 4Reasoning: DeepSeek-R1Visible chain-of-thought
• 5Frameworks: Need One?Manual vs LangChain / CrewAI
• 6Your 90-Day RoadmapThree paths forward

One installation, many specialists

The open source ecosystem is no longer one shelf of general 7B models. It now has specialists: reasoning models (DeepSeek-R1), vision models (LLaVA), compact on-device models (Phi-4 mini), and Mixture-of-Experts models that give big-model quality at small-model speed (Llama 4 Scout).

The gap to frontier APIs has shrunk hard. A 7B reasoning model now matches yesterday's $200/month cloud budget — running on your laptop. Your job shifted from "which model can I afford" to "which model fits this job," all inside one Ollama install.

A hammer vs a Swiss Army knife

BEFORE: Two years ago the shelf held Llama 2, Mistral 7B, and a few derivatives. If the 7B could not do the job, you upgraded to a costly 70B or paid a cloud API instead.

PAIN: The 7B made steady domain errors, could not reason through multi-step math, and could not look at an image. You were holding a hammer when the job needed a whole toolkit.

MAPPING: The 2026 landscape is a Swiss Army knife: a reasoning blade, a vision blade, a tiny on-device blade, and an MoE blade — all folded into one Ollama install. You match the right blade to the right cut instead of forcing one tool.

The four model families

1. WorkhorsesMistral 7B, Llama 3.1, Qwen 2.5 — tool use, coding, long context. Your everyday agents.
2. Reasoning modelsDeepSeek-R1 — a built-in chain-of-thought for hard multi-step problems.
3. Multimodal / MoELlama 4 Scout — natively sees images; routes each token to a few experts for speed.
4. CompactPhi-4 mini (3.8B) — reasoning-dense, tiny footprint, for edge and low-latency.

Match the model to the job

MoE routes each token to a few "experts" — 17B total, ~3.4B active per pass.

Misconceptions

Images are just another content type

A multimodal model takes text and images in a single call. In Ollama, vision is not a new endpoint or SDK — you send images as base64 data URIs inside the same message content array you already use for text.

Under the hood, the image is tokenized by a visual encoder and injected straight into the context window alongside your words. So "add vision" is a one-line change to the shape of your content, not a new integration.

Attaching a photo to a text

BEFORE: Your agent could only read what you typed. To ask "what's wrong with this screenshot?" you had to describe the screenshot in words first — slow and lossy.

PAIN: Describing a UI, a chart, or a receipt in prose loses detail and takes forever. You were transcribing images by hand just to feed the model.

MAPPING: Multimodal is like attaching a photo to a text message instead of typing a description. You drop the image into the same message, and the model "sees" it directly — same conversation, same API, just one extra attachment.

Sending an image

1. Encode the imageRead the file and base64-encode it into a data:image/...;base64, URI.
2. Build a content arrayInstead of a string, content becomes a list: one image_url item, one text item.
3. Call a vision modelUse llava, moondream, or llama4:scout on the same Ollama server.
4. Read the answerThe reply comes back exactly like a text call — the image tokens lived in the context.

Pseudocode

b64 = base64_encode(read_file("screenshot.png"))

message = {
  role: "user",
  content: [                       # an ARRAY, not a string
    { type: "image_url",
      image_url: {
        url: "data:image/png;base64," + b64 } },
    { type: "text",
      text: "What UI elements are here?" },
  ]
}
reply = model.chat("llava", [message])
# GOTCHA: MIME type in the URI must match the file

Misconceptions

Teach the weights, not the prompt

A prompt can't make a base model learn your domain vocabulary — the context window is not the same as learned weights. Fine-tuning on 500–1000 domain examples teaches terminology, format, and reasoning into the model itself.

LoRA makes this cheap: freeze the 7B base weights and train tiny adapter matrices (~0.1% of params) on top. QLoRA goes further, running LoRA on a 4-bit quantized base so a 7B fine-tune fits in 6–8 GB VRAM — a consumer RTX 3060, in 1–2 hours.

Onboarding a sharp new hire

BEFORE: Your support model is smart but generic. It confuses "510(k) clearance" with "CE marking," mangles your product numbers, and every reply needs editing before it reaches a customer.

PAIN: The base model learned the general internet, not your 800 submission docs and three years of validated Q&A. Re-explaining your domain in every prompt is like re-training a hire at the start of each task.

MAPPING: Fine-tuning is proper onboarding: you don't rebuild the person, you hand them your handbook and let them internalize it. LoRA adds thin "notes" (adapters) on frozen expertise — the hire now gets the calls right 80% of the time instead of 40%.

The Unsloth workflow

1. Load a 4-bit baseLoad the 7B model in 4-bit (QLoRA) so it fits in 6–8 GB VRAM.
2. Attach LoRA adaptersAdd small trainable matrices (rank r) to the attention projections; base stays frozen.
3. Train on your JSONLSupervised fine-tune on 500–1000 domain examples for a few epochs.
4. Export to GGUFSave as GGUF, write a Modelfile, then ollama create to run it like any model.

Pseudocode

model = load_base("mistral-7b", in_4bit=TRUE)

# add thin trainable adapters, base stays FROZEN
model = add_lora(model, rank=16,
                 target=[q_proj, v_proj])

data = load_jsonl("my_domain.jsonl")

train(model, data, epochs=3, lr=2e-4)

# export so Ollama can run it natively
save_gguf(model, quant="q4_k_m")
# then: ollama create my-model -f Modelfile

Misconceptions

A model that shows its work

A reasoning model like DeepSeek-R1 produces a visible chain-of-thought before its final answer, wrapped in <think>...</think> tags. It was trained (via GRPO reinforcement learning) to reward correct final answers, which pushed it to develop structured reasoning.

That thinking trace is gold for hard problems — and noise for end users. So the pattern is: let R1 think, then strip the <think> block before showing the answer. Keep it visible only in dev and eval, where it's your best debugging tool.

The candidate's scratch paper

BEFORE: A plain model blurts an answer instantly — like a student who writes only the final number on a hard math problem, with no working shown.

PAIN: On multi-step logic, that snap answer is often wrong, and you can't see where it went wrong. No scratch paper means no way to debug the reasoning.

MAPPING: R1 is the candidate who works the problem on scratch paper first, then boxes the final answer. You keep the scratch paper (the <think> block) for grading and debugging, but you hand the customer only the boxed answer.

Think, then strip

1. Call R1Send the hard problem to deepseek-r1:7b at temperature ~0.6.
2. Get think + answerThe reply holds a <think> trace (200–800 tokens) followed by the final answer.
3. Extract the traceRegex-match the <think>...</think> block if you want it for debugging.
4. Strip for usersRemove the block to leave a clean answer; show the trace only in dev/eval.

Pseudocode

reply = model.chat("deepseek-r1:7b",
                   problem, temp=0.6)

# pull out the reasoning trace
thinking = match(reply, "<think>...</think>")

# strip it to get the clean final answer
answer = remove(reply, "<think>...</think>")

IF production:
  show(answer)                # hide the trace
ELSE:
  show(thinking, answer)      # debug in dev/eval

Misconceptions

Frameworks compress; they don't replace understanding

LangChain, CrewAI, AutoGen, and LlamaIndex are faster for shipping common patterns — but they hide the ReAct loop, the handoff, the RAG pipeline you built by hand. When something breaks (wrong tools, infinite loops, hallucinated schemas), you need to know what's underneath.

Because you built every primitive manually, that knowledge is now your edge. Each framework abstraction maps to something you already understand — so you can read its source, adopt it surgically, and debug it when it fails.

Learning to cut before the CNC

BEFORE: A joiner learning woodworking is handed a CNC machine on day one — fast, precise, and completely opaque.

PAIN: Start with the CNC and you never learn to read the grain or feel why a joint holds. When the machine jams (and it will), you're helpless — you can't fix what you never understood.

MAPPING: Frameworks are the CNC machines of agent building. Because you first cut every joint by hand — the ReAct loop, the handoff, the RAG pipeline — you can now run the CNC and fix it, because you know exactly which cut each button makes.

What each framework is for

1. Manual (this course)Full control, any architecture. Always the starting point — you own it all.
2. LangChainMedium control; best for linear pipelines and reaching 50+ integrations fast.
3. CrewAI / AutoGenRole-based or conversation-first multi-agent patterns when they map to your problem.
4. LlamaIndexIndex and retrieval abstractions for complex RAG and knowledge bases.

Manual or framework?

ALWAYS start manual
  → you understand every primitive

IF RAG grows brittle & complex:
  → adopt LlamaIndex

IF a clear crew/role model fits:
  → adopt CrewAI

IF you need 50+ integrations fast:
  → adopt LangChain

NEVER start a new project on a
  framework — debugging is 10x harder
  when you can't see underneath

Misconceptions

Three paths, one next step

You finished the track — now pick a direction. There are three 90-day paths: deepen open source (fine-tune, RAG, deploy to a GPU), add Claude + cloud (swap the client, take the main course, build hybrids), or specialize (MCP, eval, monitoring, production hardening).

Each path is 12 weeks of concrete, buildable steps that stack on what you already made. The point isn't to do all three — it's to pick the one that matches your goal and start this week, using your Module 12 agent as the scaffold.

Trailheads at the summit

BEFORE: You've climbed to a ridge and finished the marked trail. Ahead, the map just says "you are here" — no obvious next move, easy to wander in circles.

PAIN: Without a chosen route, "keep learning" becomes aimless tab-hoarding — a dozen half-read tutorials and no shipped project three months later.

MAPPING: The roadmap is three signed trailheads leaving the summit: Deepen, Add Claude, or Specialize. Each has week-by-week markers so you always know the next step. You pick one path and walk it — not stand at the sign forever.

Path A, week by week

1. Weeks 1–4: Fine-tunePrepare 500–1000 Q&A examples, run QLoRA with Unsloth, measure the accuracy delta.
2. Weeks 5–8: Domain RAGIngest your own docs, add hybrid search and the reranking from M10.
3. Weeks 9–12: DeployProvision a T4 GPU VM (M21B), serve real users, measure p50/p95 latency.
4. Scaffold itReuse the reference project tree: agent, tools, memory, rag/, eval/, guardrails/, api.

Which path is yours?

Path B: your OpenAI-compatible code needs only a base_url + key change to point at Claude.

Misconceptions

Test the core insight

One question per concept. Tap to reveal the answer.