Module 4 of 10 · CLI Comparison Track
40%
AI CLI Tools Compared  ·  M04

Code Generation Head-to-Head

Same prompt. Three tools. How does each one translate "write me a TypeScript fetch function with retries" into actual production-grade code? We run two real tasks, show all outputs side-by-side, and score them on the dimensions that actually matter.

Claude Code
Gemini CLI
GitHub Copilot CLI
Task 1 of 2

The TypeScript Fetch Function

Why this task?

BEFORE: You open a blank file. You know you need a typed HTTP utility with retry logic and exponential backoff. You could copy-paste from Stack Overflow, but the result will be un-typed, brittle, and won't match your project's error conventions.

PAIN: Writing this properly — correct generics, typed error hierarchy, configurable backoff with jitter, test-friendly structure — takes 45–90 minutes even for an experienced TypeScript developer. And you'll write it slightly differently every time, making the codebase inconsistent.

MAPPING: This is exactly the task where AI CLI tools should shine: a well-defined, self-contained problem with clear quality criteria. We'll see how each tool interprets "TypeScript quality" and whether it produces something you'd actually ship.

Exact prompt sent to all three tools
"Create a TypeScript function that fetches user data from an API, handles errors, retries 3 times with exponential backoff, and returns typed results."
Generation speed comparison — lines of useful output per second
Tool Output — Claude Code

Claude Code — Clean, Idiomatic TypeScript

Claude Code produces a tightly scoped solution. Notice the proper generic constraints, a typed error class hierarchy, and inline JSDoc comments that explain each parameter. No dead imports, no unnecessary abstractions.

Claude Code Output 52 lines · TypeScript
TypeScript
// WHAT: A typed HTTP client utility with retry + exponential backoff
// WHY:  Centralises retry logic so callers never reimplement it
// GOTCHA: jitter prevents "thundering herd" when many requests retry simultaneously

export class FetchError extends Error {
  constructor(
    public readonly status: number,
    public readonly url: string,
    message: string,
  ) {
    super(message);
    this.name = 'FetchError';
  }
}

export interface FetchOptions extends RequestInit {
  /** Maximum number of retry attempts (default: 3) */
  retries?: number;
  /** Base delay in ms for exponential backoff (default: 300) */
  baseDelayMs?: number;
}

// WHAT: Generic typed fetch wrapper — T is the shape of the JSON response
// WHY:  Callers get full type-safety without casting
// GOTCHA: Only 5xx and network errors trigger retries; 4xx are not retried
export async function fetchWithRetry<T>(
  url: string,
  options: FetchOptions = {},
): Promise<T> {
  const { retries = 3, baseDelayMs = 300, ...fetchOptions } = options;

  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      const response = await fetch(url, fetchOptions);

      if (!response.ok) {
        // 4xx: do not retry — the request itself is malformed
        if (response.status < 500) {
          throw new FetchError(response.status, url,
            `HTTP ${response.status}: ${response.statusText}`);
        }
        // 5xx: retriable server error
        throw new FetchError(response.status, url,
          `Server error ${response.status} — attempt ${attempt + 1}/${retries + 1}`);
      }

      return response.json() as Promise<T>;

    } catch (err) {
      const isLastAttempt = attempt === retries;
      if (isLastAttempt) throw err;

      // WHAT: Exponential backoff with ±25% jitter
      const delay = baseDelayMs * 2 ** attempt * (0.75 + Math.random() * 0.5);
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }

  // TypeScript requires an explicit throw here; the loop always throws or returns
  throw new FetchError(0, url, 'fetchWithRetry: unreachable');
}
Strengths: proper generic <T>, typed FetchError extends Error, 4xx vs 5xx distinction, jitter, JSDoc. Test-ready: FetchError is exported and instanceof-checkable in test assertions.
Tool Output — Gemini CLI

Gemini CLI — Functional, Slightly More Verbose

Gemini CLI produces a working solution with a broader feature set. It adds an AbortController timeout, explicit retry-on-specific-status logic, and node-fetch polyfill handling. The trade-off is slightly more boilerplate.

Gemini CLI Output 68 lines · TypeScript
TypeScript
import fetch, { Response } from 'node-fetch'; // polyfill for Node.js <18

interface RetryConfig {
  maxRetries: number;
  baseDelay: number;
  maxDelay: number;
  retryOnStatus: number[];
  timeoutMs: number;
}

const DEFAULT_CONFIG: RetryConfig = {
  maxRetries: 3,
  baseDelay: 300,
  maxDelay: 10000,
  retryOnStatus: [429, 500, 502, 503, 504],
  timeoutMs: 30000,
};

export interface ApiResponse<T> {
  data: T | null;
  error: string | null;
  attempts: number;
}

function sleep(ms: number): Promise<void> {
  return new Promise(resolve => setTimeout(resolve, ms));
}

function getBackoffDelay(attempt: number, config: RetryConfig): number {
  const exponential = config.baseDelay * Math.pow(2, attempt);
  const jittered = exponential * (0.5 + Math.random() * 0.5);
  return Math.min(jittered, config.maxDelay);
}

export async function fetchUserData<T>(
  url: string,
  options: RequestInit = {},
  config: Partial<RetryConfig> = {},
): Promise<ApiResponse<T>> {
  const cfg: RetryConfig = { ...DEFAULT_CONFIG, ...config };
  let lastError: string = 'Unknown error';

  for (let attempt = 0; attempt <= cfg.maxRetries; attempt++) {
    const controller = new AbortController();
    const timeoutId = setTimeout(() => controller.abort(), cfg.timeoutMs);

    try {
      const response: Response = await fetch(url, {
        ...options,
        signal: controller.signal,
      });
      clearTimeout(timeoutId);

      if (!cfg.retryOnStatus.includes(response.status) && !response.ok) {
        return {
          data: null,
          error: `HTTP ${response.status}: ${response.statusText}`,
          attempts: attempt + 1,
        };
      }

      if (response.ok) {
        const data = (await response.json()) as T;
        return { data, error: null, attempts: attempt + 1 };
      }

      lastError = `HTTP ${response.status} — retrying (${attempt + 1}/${cfg.maxRetries + 1})`;
    } catch (err) {
      clearTimeout(timeoutId);
      lastError = err instanceof Error ? err.message : 'Network error';
      if (err instanceof Error && err.name === 'AbortError') {
        lastError = `Request timed out after ${cfg.timeoutMs}ms`;
      }
    }

    if (attempt < cfg.maxRetries) {
      await sleep(getBackoffDelay(attempt, cfg));
    }
  }

  return { data: null, error: lastError, attempts: cfg.maxRetries + 1 };
}
Strengths: AbortController timeout, explicit retryOnStatus list, maxDelay cap, ApiResponse wrapper. Trade-offs: ApiResponse<T> wraps errors instead of throwing — callers must check .error, not catch.
Tool Output — GitHub Copilot CLI

Copilot CLI — Shell Scaffold, Not Code Generation

This is where the architectural difference becomes concrete. gh copilot suggest is not a code generator — it is a shell command translator. When you ask it to generate TypeScript code, it suggests a shell command to scaffold the file — it does not write the TypeScript itself.

GitHub Copilot CLI Output gh copilot suggest
bash — what Copilot CLI actually outputs
$ gh copilot suggest "Create a TypeScript function that fetches user data, retries 3 times with exponential backoff"

? What kind of shell command?
  1. generic shell command
  2. git command
  3. gh command
> 1

Suggestion:
  npx ts-node -e "
    async function fetchWithRetry(url, retries=3) {
      // stub — implement logic here
    }
  "

? Select an option
  1. Copy command to clipboard
  2. Explain command
  3. Revise command
  4. Rate response
  5. Exit

# Note: Copilot CLI does NOT generate a full .ts file.
# It suggests a shell command that runs a ts-node stub.
# For full code generation, use claude or gemini interactive mode.
PowerShell — Copilot CLI suggest
PS> gh copilot suggest "Create a TypeScript function that fetches user data, retries 3 times with exponential backoff"

# What kind of shell command? > generic shell command

Suggestion:
  npx ts-node -e "async function fetchWithRetry(url, retries=3) { /* stub */ }"

# Select an option: Copy command to clipboard

# The correct way to use Copilot CLI for TypeScript is to ask for
# the scaffolding command, then write the actual code yourself (or
# use Claude/Gemini for the code generation step):
gh copilot suggest "create a new TypeScript project with tsconfig and package.json"
# → suggests: npx tsc --init && npm init -y && npm i typescript @types/node
This is not a weakness — it is intentional design. Copilot CLI's job is shell command translation. For code generation from the terminal, you need Claude Code or Gemini CLI's interactive mode.
Architectural clarification

GitHub Copilot in VS Code absolutely generates full TypeScript functions — it is excellent at it. But gh copilot suggest in the terminal is a different product with a different purpose: shell command assistance. This module compares CLI tools only.

Quality Analysis — Task 1

Scoring the TypeScript Function

Quality radar — Correctness / TS Quality / Error Handling / Test-readiness / Conciseness
Dimension Claude Code Gemini CLI Copilot CLI
Correctness ✓ Correct, ships as-is ✓ Correct, ships as-is N/A — no code produced
TypeScript quality ✓ Proper generics, typed errors, no any Good generics, ApiResponse wrapper adds complexity N/A
Error handling ✓ FetchError class, 4xx vs 5xx distinction ✓ Timeout via AbortController, retryOnStatus list N/A
Test-readiness ✓ FetchError exported, throws on failure (easy to mock) Returns ApiResponse — callers check .error, harder to test with catch N/A
Lines of code 52 (lean) 68 (more features, more code) N/A
Inline comments WHAT/WHY/GOTCHA per block Minimal N/A
What Just Happened?

Both Claude Code and Gemini CLI produce correct, shippable TypeScript. The key differences:

  • Claude Code's output is more idiomatic — it throws typed errors, which is the TypeScript convention for unexpected failures.
  • Gemini CLI's output is more configurable — it adds timeout, retryOnStatus list, and a wrapper type. Good choice when you need more control.
  • Copilot CLI is disqualified from this comparison — it is a shell tool, not a code generator.
Task 2 of 2

Python Flask REST API with Full CRUD

The second task tests a more substantial generation request: a multi-endpoint REST API with create, read, update, and delete operations for a blog post model. This requires understanding conventions — route naming, HTTP status codes, error response shape, and separation of concerns.

Exact prompt sent to all three tools
"Generate a Python Flask REST API with CRUD operations for a blog post."
Claude Code — Flask API 78 lines · Python
Python (Flask)
"""
WHAT: Blog post CRUD REST API using Flask + in-memory store
WHY:  Demonstrates clean route structure and proper HTTP semantics
GOTCHA: In-memory dict resets on restart — replace with SQLAlchemy for production
"""
from __future__ import annotations
from flask import Flask, jsonify, request, abort
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
import uuid

app = Flask(__name__)

# WHAT: In-memory store keyed by post UUID string
posts: dict[str, dict] = {}


@dataclass
class BlogPost:
    id: str
    title: str
    content: str
    author: str
    created_at: str
    updated_at: str

    @staticmethod
    def create(title: str, content: str, author: str) -> BlogPost:
        now = datetime.now(timezone.utc).isoformat()
        return BlogPost(
            id=str(uuid.uuid4()),
            title=title, content=content, author=author,
            created_at=now, updated_at=now,
        )


def _get_or_404(post_id: str) -> dict:
    post = posts.get(post_id)
    if post is None:
        abort(404, description=f"Post '{post_id}' not found")
    return post


# ── CREATE ──────────────────────────────────────────────────────────
@app.post("/posts")
def create_post():
    body = request.get_json(silent=True) or {}
    required = {"title", "content", "author"}
    missing = required - body.keys()
    if missing:
        return jsonify(error=f"Missing fields: {', '.join(missing)}"), 400

    post = BlogPost.create(body["title"], body["content"], body["author"])
    posts[post.id] = asdict(post)
    return jsonify(posts[post.id]), 201


# ── READ ALL / READ ONE ──────────────────────────────────────────────
@app.get("/posts")
def list_posts():
    return jsonify(list(posts.values())), 200


@app.get("/posts/<string:post_id>")
def get_post(post_id: str):
    return jsonify(_get_or_404(post_id)), 200


# ── UPDATE ───────────────────────────────────────────────────────────
@app.patch("/posts/<string:post_id>")
def update_post(post_id: str):
    post = _get_or_404(post_id)
    body = request.get_json(silent=True) or {}
    for field in ("title", "content", "author"):
        if field in body:
            post[field] = body[field]
    post["updated_at"] = datetime.now(timezone.utc).isoformat()
    return jsonify(post), 200


# ── DELETE ───────────────────────────────────────────────────────────
@app.delete("/posts/<string:post_id>")
def delete_post(post_id: str):
    _get_or_404(post_id)
    del posts[post_id]
    return "", 204


@app.errorhandler(404)
def not_found(err):
    return jsonify(error=str(err.description)), 404


@app.errorhandler(400)
def bad_request(err):
    return jsonify(error=str(err.description)), 400


if __name__ == "__main__":
    app.run(debug=True)
Uses modern Flask 2.x decorators (@app.post, @app.patch, @app.delete), typed dataclass, PATCH over PUT (correct REST semantics), structured 404/400 error responses.
Gemini CLI — Flask API 94 lines · Python
Python (Flask)
from flask import Flask, jsonify, request
from flask_sqlalchemy import SQLAlchemy
from marshmallow import Schema, fields, ValidationError
from datetime import datetime
import os

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = os.getenv('DATABASE_URL', 'sqlite:///blog.db')
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)


class BlogPost(db.Model):
    __tablename__ = 'posts'
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(200), nullable=False)
    content = db.Column(db.Text, nullable=False)
    author = db.Column(db.String(100), nullable=False)
    created_at = db.Column(db.DateTime, default=datetime.utcnow)
    updated_at = db.Column(db.DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)

    def to_dict(self):
        return {
            'id': self.id, 'title': self.title, 'content': self.content,
            'author': self.author,
            'created_at': self.created_at.isoformat(),
            'updated_at': self.updated_at.isoformat(),
        }


class PostSchema(Schema):
    title = fields.Str(required=True, validate=lambda s: len(s) > 0)
    content = fields.Str(required=True)
    author = fields.Str(required=True)

post_schema = PostSchema()
partial_schema = PostSchema(partial=True)


@app.route('/posts', methods=['POST'])
def create_post():
    try:
        data = post_schema.load(request.get_json() or {})
    except ValidationError as e:
        return jsonify({'errors': e.messages}), 400
    post = BlogPost(**data)
    db.session.add(post)
    db.session.commit()
    return jsonify(post.to_dict()), 201


@app.route('/posts', methods=['GET'])
def list_posts():
    page = request.args.get('page', 1, type=int)
    per_page = request.args.get('per_page', 20, type=int)
    paginated = BlogPost.query.order_by(BlogPost.created_at.desc()).paginate(
        page=page, per_page=per_page, error_out=False)
    return jsonify({
        'posts': [p.to_dict() for p in paginated.items],
        'total': paginated.total,
        'pages': paginated.pages,
        'current_page': page,
    }), 200


@app.route('/posts/<int:post_id>', methods=['GET'])
def get_post(post_id):
    post = db.get_or_404(BlogPost, post_id)
    return jsonify(post.to_dict()), 200


@app.route('/posts/<int:post_id>', methods=['PUT'])
def update_post(post_id):
    post = db.get_or_404(BlogPost, post_id)
    try:
        data = partial_schema.load(request.get_json() or {})
    except ValidationError as e:
        return jsonify({'errors': e.messages}), 400
    for key, value in data.items():
        setattr(post, key, value)
    db.session.commit()
    return jsonify(post.to_dict()), 200


@app.route('/posts/<int:post_id>', methods=['DELETE'])
def delete_post(post_id):
    post = db.get_or_404(BlogPost, post_id)
    db.session.delete(post)
    db.session.commit()
    return '', 204


if __name__ == '__main__':
    with app.app_context():
        db.create_all()
    app.run(debug=True)
Adds SQLAlchemy ORM + marshmallow validation + pagination. More production-ready but requires additional dependencies. Uses PUT (full replacement) instead of PATCH (partial update) — a minor REST convention difference.
Copilot CLI — Flask Scaffold Shell suggestion
bash
$ gh copilot suggest "Generate a Python Flask REST API with CRUD for a blog post"

Suggestion:
  pip install flask && \
  python -c "
from flask import Flask; app = Flask(__name__)
# Add routes here
app.run()
"

# Copilot CLI suggests the pip install + a minimal skeleton.
# It does NOT write the full CRUD routes.
# Correct usage pattern: use Copilot CLI for the setup commands,
# then use an interactive agent for code generation:
gh copilot suggest "create a python virtual environment and install flask sqlalchemy"
# → python -m venv venv && source venv/bin/activate && pip install flask flask-sqlalchemy
PowerShell
PS> gh copilot suggest "Generate a Python Flask REST API with CRUD for a blog post"

# Suggestion:
#   pip install flask; python -c "from flask import Flask; app = Flask(__name__); app.run()"

# For actual code generation, use:
#   claude  →  start Claude Code interactive session
#   gemini  →  start Gemini CLI interactive session
# Both will write the full app.py file in your project directory.
Copilot CLI suggests the environment setup commands correctly. For code generation, hand off to claude or gemini.
Diff highlight — what each tool adds beyond the baseline
Dimension Claude Code Gemini CLI Copilot CLI
Completeness ✓ Full CRUD, zero deps beyond Flask ✓ Full CRUD + ORM + validation Setup only
REST correctness PATCH for partial update, correct status codes PUT (full replace), needs PATCH for strict REST N/A
Production readiness In-memory store — needs DB for production SQLAlchemy + marshmallow + pagination N/A
Zero-dependency run pip install flask only Needs flask-sqlalchemy marshmallow N/A
Annotation quality WHAT/WHY/GOTCHA block comments Minimal inline comments N/A
Section 7

Key Insight

Pattern Observed

Claude Code produces cleaner, more idiomatic output on average. It defaults to the minimum necessary — no extra deps, explicit error typing, correct HTTP conventions (PATCH not PUT), inline pedagogical comments. It behaves like a senior developer who writes the lean version and explains it.

Gemini CLI is competitive and benefits from larger context. When your codebase is already loaded (@src/ or @package.json), Gemini detects that you're using SQLAlchemy and naturally extends the generated code to match. On a fresh prompt it makes more assumptions. Its output is often more feature-rich — pagination, validation — which is better for production but adds setup cost.

Copilot CLI is architecturally excluded from code generation comparison. This is not a flaw — gh copilot suggest is a shell command translator. It excels at environment setup, git commands, and workflow automation. Use it as the first step (set up the project), then hand to Claude or Gemini for the actual code.

Practical workflow: chain the tools

The most productive pattern many teams use: 1) Use gh copilot suggest to scaffold the project and install deps. 2) Use claude or gemini to generate the actual code. 3) Use gh copilot suggest again for git commands, Docker commands, and CI configuration. Each tool handles what it was designed for.

What Just Happened?

You now have a concrete, code-level understanding of how these tools differ on generation quality:

  • Claude Code: cleaner types, correct conventions, pedagogical comments, leaner output.
  • Gemini CLI: more features, larger assumptions, better when context is already loaded.
  • Copilot CLI: excellent for shell scaffolding; not a code generator from the terminal.

Next: M05 covers multi-file editing and large codebase handling — where context window size becomes the dominant variable.

Knowledge Check

Five questions on code generation quality and tool architecture.

Question 1 of 5
When Claude Code generates a TypeScript fetch function, it throws a typed FetchError for server errors, but does NOT retry on 4xx status codes. Why?
Correct. A 4xx status (400 Bad Request, 401 Unauthorized, 404 Not Found) means the request itself is wrong. Retrying the identical request will return the same error. 5xx errors (500 Internal Server Error, 502 Bad Gateway, 503 Unavailable) indicate a transient server problem that may resolve on retry — which is why only those are retried. Note: 429 (Too Many Requests) is technically a 4xx that benefits from retry — this is a valid edge case to add manually.
Question 2 of 5
Gemini CLI's Flask output uses SQLAlchemy and marshmallow without being asked. What does this tell you about its generation behavior?
Correct. Gemini CLI tends to generate "production-oriented" output by default — it assumes you eventually want a real database, input validation, and pagination. This is often the right call for a backend API, but it does mean you need to install additional packages (flask-sqlalchemy, marshmallow) before running the code. Claude Code takes the opposite default: minimal dependencies, correct code, and a comment telling you to add a real DB for production.
Question 3 of 5
A developer runs gh copilot suggest "write a TypeScript retry function" and receives a stub shell command instead of full code. What should they do next?
Correct. gh copilot suggest is designed to translate natural language into shell commands. It will always produce a shell command output, even when the input is a programming request. For code generation from the terminal, Claude Code and Gemini CLI are the right tools. There is no --code-mode flag.
Question 4 of 5
What is exponential backoff with jitter, and why does jitter matter?
Correct. Without jitter, all clients whose requests failed at the same time will retry at the same moment (e.g., all at t+300ms). This creates a "thundering herd" that overwhelms the server again. By adding ±25-50% random variation to each retry delay, clients spread out their retries in time, giving the server a chance to recover. This is a well-known distributed systems pattern — AWS documented it in their "Exponential Backoff and Jitter" blog post in 2015.
Question 5 of 5
Claude Code's Flask API uses PATCH for updates while Gemini CLI's uses PUT. Which is more correct according to REST semantics, and why?
Correct. RFC 5789 defines PATCH as a method for applying partial modifications to a resource. PUT replaces the entire resource — if you PUT a post with only a new title, you must also send all other fields (content, author) or they will be deleted/reset. PATCH lets you send only the fields you want to change. For a blog post API where clients commonly update just the title or content, PATCH is the semantically correct choice. Gemini's PUT output works, but a client that sends only {"title": "new title"} via PUT would accidentally clear the content field.