Code Generation Head-to-Head
Same prompt. Three tools. How does each one translate "write me a TypeScript fetch function with retries" into actual production-grade code? We run two real tasks, show all outputs side-by-side, and score them on the dimensions that actually matter.
The TypeScript Fetch Function
BEFORE: You open a blank file. You know you need a typed HTTP utility with retry logic and exponential backoff. You could copy-paste from Stack Overflow, but the result will be un-typed, brittle, and won't match your project's error conventions.
PAIN: Writing this properly — correct generics, typed error hierarchy, configurable backoff with jitter, test-friendly structure — takes 45–90 minutes even for an experienced TypeScript developer. And you'll write it slightly differently every time, making the codebase inconsistent.
MAPPING: This is exactly the task where AI CLI tools should shine: a well-defined, self-contained problem with clear quality criteria. We'll see how each tool interprets "TypeScript quality" and whether it produces something you'd actually ship.
Claude Code — Clean, Idiomatic TypeScript
Claude Code produces a tightly scoped solution. Notice the proper generic constraints, a typed error class hierarchy, and inline JSDoc comments that explain each parameter. No dead imports, no unnecessary abstractions.
// WHAT: A typed HTTP client utility with retry + exponential backoff
// WHY: Centralises retry logic so callers never reimplement it
// GOTCHA: jitter prevents "thundering herd" when many requests retry simultaneously
export class FetchError extends Error {
constructor(
public readonly status: number,
public readonly url: string,
message: string,
) {
super(message);
this.name = 'FetchError';
}
}
export interface FetchOptions extends RequestInit {
/** Maximum number of retry attempts (default: 3) */
retries?: number;
/** Base delay in ms for exponential backoff (default: 300) */
baseDelayMs?: number;
}
// WHAT: Generic typed fetch wrapper — T is the shape of the JSON response
// WHY: Callers get full type-safety without casting
// GOTCHA: Only 5xx and network errors trigger retries; 4xx are not retried
export async function fetchWithRetry<T>(
url: string,
options: FetchOptions = {},
): Promise<T> {
const { retries = 3, baseDelayMs = 300, ...fetchOptions } = options;
for (let attempt = 0; attempt <= retries; attempt++) {
try {
const response = await fetch(url, fetchOptions);
if (!response.ok) {
// 4xx: do not retry — the request itself is malformed
if (response.status < 500) {
throw new FetchError(response.status, url,
`HTTP ${response.status}: ${response.statusText}`);
}
// 5xx: retriable server error
throw new FetchError(response.status, url,
`Server error ${response.status} — attempt ${attempt + 1}/${retries + 1}`);
}
return response.json() as Promise<T>;
} catch (err) {
const isLastAttempt = attempt === retries;
if (isLastAttempt) throw err;
// WHAT: Exponential backoff with ±25% jitter
const delay = baseDelayMs * 2 ** attempt * (0.75 + Math.random() * 0.5);
await new Promise(resolve => setTimeout(resolve, delay));
}
}
// TypeScript requires an explicit throw here; the loop always throws or returns
throw new FetchError(0, url, 'fetchWithRetry: unreachable');
}
Gemini CLI — Functional, Slightly More Verbose
Gemini CLI produces a working solution with a broader feature set. It adds an AbortController timeout, explicit retry-on-specific-status logic, and node-fetch polyfill handling. The trade-off is slightly more boilerplate.
import fetch, { Response } from 'node-fetch'; // polyfill for Node.js <18
interface RetryConfig {
maxRetries: number;
baseDelay: number;
maxDelay: number;
retryOnStatus: number[];
timeoutMs: number;
}
const DEFAULT_CONFIG: RetryConfig = {
maxRetries: 3,
baseDelay: 300,
maxDelay: 10000,
retryOnStatus: [429, 500, 502, 503, 504],
timeoutMs: 30000,
};
export interface ApiResponse<T> {
data: T | null;
error: string | null;
attempts: number;
}
function sleep(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
function getBackoffDelay(attempt: number, config: RetryConfig): number {
const exponential = config.baseDelay * Math.pow(2, attempt);
const jittered = exponential * (0.5 + Math.random() * 0.5);
return Math.min(jittered, config.maxDelay);
}
export async function fetchUserData<T>(
url: string,
options: RequestInit = {},
config: Partial<RetryConfig> = {},
): Promise<ApiResponse<T>> {
const cfg: RetryConfig = { ...DEFAULT_CONFIG, ...config };
let lastError: string = 'Unknown error';
for (let attempt = 0; attempt <= cfg.maxRetries; attempt++) {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), cfg.timeoutMs);
try {
const response: Response = await fetch(url, {
...options,
signal: controller.signal,
});
clearTimeout(timeoutId);
if (!cfg.retryOnStatus.includes(response.status) && !response.ok) {
return {
data: null,
error: `HTTP ${response.status}: ${response.statusText}`,
attempts: attempt + 1,
};
}
if (response.ok) {
const data = (await response.json()) as T;
return { data, error: null, attempts: attempt + 1 };
}
lastError = `HTTP ${response.status} — retrying (${attempt + 1}/${cfg.maxRetries + 1})`;
} catch (err) {
clearTimeout(timeoutId);
lastError = err instanceof Error ? err.message : 'Network error';
if (err instanceof Error && err.name === 'AbortError') {
lastError = `Request timed out after ${cfg.timeoutMs}ms`;
}
}
if (attempt < cfg.maxRetries) {
await sleep(getBackoffDelay(attempt, cfg));
}
}
return { data: null, error: lastError, attempts: cfg.maxRetries + 1 };
}
Copilot CLI — Shell Scaffold, Not Code Generation
This is where the architectural difference becomes concrete. gh copilot suggest is not a code generator — it is a shell command translator. When you ask it to generate TypeScript code, it suggests a shell command to scaffold the file — it does not write the TypeScript itself.
$ gh copilot suggest "Create a TypeScript function that fetches user data, retries 3 times with exponential backoff"
? What kind of shell command?
1. generic shell command
2. git command
3. gh command
> 1
Suggestion:
npx ts-node -e "
async function fetchWithRetry(url, retries=3) {
// stub — implement logic here
}
"
? Select an option
1. Copy command to clipboard
2. Explain command
3. Revise command
4. Rate response
5. Exit
# Note: Copilot CLI does NOT generate a full .ts file.
# It suggests a shell command that runs a ts-node stub.
# For full code generation, use claude or gemini interactive mode.
PS> gh copilot suggest "Create a TypeScript function that fetches user data, retries 3 times with exponential backoff"
# What kind of shell command? > generic shell command
Suggestion:
npx ts-node -e "async function fetchWithRetry(url, retries=3) { /* stub */ }"
# Select an option: Copy command to clipboard
# The correct way to use Copilot CLI for TypeScript is to ask for
# the scaffolding command, then write the actual code yourself (or
# use Claude/Gemini for the code generation step):
gh copilot suggest "create a new TypeScript project with tsconfig and package.json"
# → suggests: npx tsc --init && npm init -y && npm i typescript @types/node
GitHub Copilot in VS Code absolutely generates full TypeScript functions — it is excellent at it. But gh copilot suggest in the terminal is a different product with a different purpose: shell command assistance. This module compares CLI tools only.
Scoring the TypeScript Function
| Dimension | Claude Code | Gemini CLI | Copilot CLI |
|---|---|---|---|
| Correctness | ✓ Correct, ships as-is | ✓ Correct, ships as-is | N/A — no code produced |
| TypeScript quality | ✓ Proper generics, typed errors, no any |
Good generics, ApiResponse wrapper adds complexity | N/A |
| Error handling | ✓ FetchError class, 4xx vs 5xx distinction | ✓ Timeout via AbortController, retryOnStatus list | N/A |
| Test-readiness | ✓ FetchError exported, throws on failure (easy to mock) | Returns ApiResponse — callers check .error, harder to test with catch | N/A |
| Lines of code | 52 (lean) | 68 (more features, more code) | N/A |
| Inline comments | WHAT/WHY/GOTCHA per block | Minimal | N/A |
Both Claude Code and Gemini CLI produce correct, shippable TypeScript. The key differences:
- Claude Code's output is more idiomatic — it throws typed errors, which is the TypeScript convention for unexpected failures.
- Gemini CLI's output is more configurable — it adds timeout, retryOnStatus list, and a wrapper type. Good choice when you need more control.
- Copilot CLI is disqualified from this comparison — it is a shell tool, not a code generator.
Python Flask REST API with Full CRUD
The second task tests a more substantial generation request: a multi-endpoint REST API with create, read, update, and delete operations for a blog post model. This requires understanding conventions — route naming, HTTP status codes, error response shape, and separation of concerns.
"""
WHAT: Blog post CRUD REST API using Flask + in-memory store
WHY: Demonstrates clean route structure and proper HTTP semantics
GOTCHA: In-memory dict resets on restart — replace with SQLAlchemy for production
"""
from __future__ import annotations
from flask import Flask, jsonify, request, abort
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
import uuid
app = Flask(__name__)
# WHAT: In-memory store keyed by post UUID string
posts: dict[str, dict] = {}
@dataclass
class BlogPost:
id: str
title: str
content: str
author: str
created_at: str
updated_at: str
@staticmethod
def create(title: str, content: str, author: str) -> BlogPost:
now = datetime.now(timezone.utc).isoformat()
return BlogPost(
id=str(uuid.uuid4()),
title=title, content=content, author=author,
created_at=now, updated_at=now,
)
def _get_or_404(post_id: str) -> dict:
post = posts.get(post_id)
if post is None:
abort(404, description=f"Post '{post_id}' not found")
return post
# ── CREATE ──────────────────────────────────────────────────────────
@app.post("/posts")
def create_post():
body = request.get_json(silent=True) or {}
required = {"title", "content", "author"}
missing = required - body.keys()
if missing:
return jsonify(error=f"Missing fields: {', '.join(missing)}"), 400
post = BlogPost.create(body["title"], body["content"], body["author"])
posts[post.id] = asdict(post)
return jsonify(posts[post.id]), 201
# ── READ ALL / READ ONE ──────────────────────────────────────────────
@app.get("/posts")
def list_posts():
return jsonify(list(posts.values())), 200
@app.get("/posts/<string:post_id>")
def get_post(post_id: str):
return jsonify(_get_or_404(post_id)), 200
# ── UPDATE ───────────────────────────────────────────────────────────
@app.patch("/posts/<string:post_id>")
def update_post(post_id: str):
post = _get_or_404(post_id)
body = request.get_json(silent=True) or {}
for field in ("title", "content", "author"):
if field in body:
post[field] = body[field]
post["updated_at"] = datetime.now(timezone.utc).isoformat()
return jsonify(post), 200
# ── DELETE ───────────────────────────────────────────────────────────
@app.delete("/posts/<string:post_id>")
def delete_post(post_id: str):
_get_or_404(post_id)
del posts[post_id]
return "", 204
@app.errorhandler(404)
def not_found(err):
return jsonify(error=str(err.description)), 404
@app.errorhandler(400)
def bad_request(err):
return jsonify(error=str(err.description)), 400
if __name__ == "__main__":
app.run(debug=True)
from flask import Flask, jsonify, request
from flask_sqlalchemy import SQLAlchemy
from marshmallow import Schema, fields, ValidationError
from datetime import datetime
import os
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = os.getenv('DATABASE_URL', 'sqlite:///blog.db')
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)
class BlogPost(db.Model):
__tablename__ = 'posts'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(200), nullable=False)
content = db.Column(db.Text, nullable=False)
author = db.Column(db.String(100), nullable=False)
created_at = db.Column(db.DateTime, default=datetime.utcnow)
updated_at = db.Column(db.DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
def to_dict(self):
return {
'id': self.id, 'title': self.title, 'content': self.content,
'author': self.author,
'created_at': self.created_at.isoformat(),
'updated_at': self.updated_at.isoformat(),
}
class PostSchema(Schema):
title = fields.Str(required=True, validate=lambda s: len(s) > 0)
content = fields.Str(required=True)
author = fields.Str(required=True)
post_schema = PostSchema()
partial_schema = PostSchema(partial=True)
@app.route('/posts', methods=['POST'])
def create_post():
try:
data = post_schema.load(request.get_json() or {})
except ValidationError as e:
return jsonify({'errors': e.messages}), 400
post = BlogPost(**data)
db.session.add(post)
db.session.commit()
return jsonify(post.to_dict()), 201
@app.route('/posts', methods=['GET'])
def list_posts():
page = request.args.get('page', 1, type=int)
per_page = request.args.get('per_page', 20, type=int)
paginated = BlogPost.query.order_by(BlogPost.created_at.desc()).paginate(
page=page, per_page=per_page, error_out=False)
return jsonify({
'posts': [p.to_dict() for p in paginated.items],
'total': paginated.total,
'pages': paginated.pages,
'current_page': page,
}), 200
@app.route('/posts/<int:post_id>', methods=['GET'])
def get_post(post_id):
post = db.get_or_404(BlogPost, post_id)
return jsonify(post.to_dict()), 200
@app.route('/posts/<int:post_id>', methods=['PUT'])
def update_post(post_id):
post = db.get_or_404(BlogPost, post_id)
try:
data = partial_schema.load(request.get_json() or {})
except ValidationError as e:
return jsonify({'errors': e.messages}), 400
for key, value in data.items():
setattr(post, key, value)
db.session.commit()
return jsonify(post.to_dict()), 200
@app.route('/posts/<int:post_id>', methods=['DELETE'])
def delete_post(post_id):
post = db.get_or_404(BlogPost, post_id)
db.session.delete(post)
db.session.commit()
return '', 204
if __name__ == '__main__':
with app.app_context():
db.create_all()
app.run(debug=True)
$ gh copilot suggest "Generate a Python Flask REST API with CRUD for a blog post"
Suggestion:
pip install flask && \
python -c "
from flask import Flask; app = Flask(__name__)
# Add routes here
app.run()
"
# Copilot CLI suggests the pip install + a minimal skeleton.
# It does NOT write the full CRUD routes.
# Correct usage pattern: use Copilot CLI for the setup commands,
# then use an interactive agent for code generation:
gh copilot suggest "create a python virtual environment and install flask sqlalchemy"
# → python -m venv venv && source venv/bin/activate && pip install flask flask-sqlalchemy
PS> gh copilot suggest "Generate a Python Flask REST API with CRUD for a blog post"
# Suggestion:
# pip install flask; python -c "from flask import Flask; app = Flask(__name__); app.run()"
# For actual code generation, use:
# claude → start Claude Code interactive session
# gemini → start Gemini CLI interactive session
# Both will write the full app.py file in your project directory.
| Dimension | Claude Code | Gemini CLI | Copilot CLI |
|---|---|---|---|
| Completeness | ✓ Full CRUD, zero deps beyond Flask | ✓ Full CRUD + ORM + validation | Setup only |
| REST correctness | PATCH for partial update, correct status codes | PUT (full replace), needs PATCH for strict REST | N/A |
| Production readiness | In-memory store — needs DB for production | SQLAlchemy + marshmallow + pagination | N/A |
| Zero-dependency run | ✓ pip install flask only |
Needs flask-sqlalchemy marshmallow | N/A |
| Annotation quality | WHAT/WHY/GOTCHA block comments | Minimal inline comments | N/A |
Key Insight
Claude Code produces cleaner, more idiomatic output on average. It defaults to the minimum necessary — no extra deps, explicit error typing, correct HTTP conventions (PATCH not PUT), inline pedagogical comments. It behaves like a senior developer who writes the lean version and explains it.
Gemini CLI is competitive and benefits from larger context. When your codebase is already loaded (@src/ or @package.json), Gemini detects that you're using SQLAlchemy and naturally extends the generated code to match. On a fresh prompt it makes more assumptions. Its output is often more feature-rich — pagination, validation — which is better for production but adds setup cost.
Copilot CLI is architecturally excluded from code generation comparison. This is not a flaw — gh copilot suggest is a shell command translator. It excels at environment setup, git commands, and workflow automation. Use it as the first step (set up the project), then hand to Claude or Gemini for the actual code.
The most productive pattern many teams use: 1) Use gh copilot suggest to scaffold the project and install deps. 2) Use claude or gemini to generate the actual code. 3) Use gh copilot suggest again for git commands, Docker commands, and CI configuration. Each tool handles what it was designed for.
You now have a concrete, code-level understanding of how these tools differ on generation quality:
- Claude Code: cleaner types, correct conventions, pedagogical comments, leaner output.
- Gemini CLI: more features, larger assumptions, better when context is already loaded.
- Copilot CLI: excellent for shell scaffolding; not a code generator from the terminal.
Next: M05 covers multi-file editing and large codebase handling — where context window size becomes the dominant variable.
Knowledge Check
Five questions on code generation quality and tool architecture.
FetchError for server errors, but does NOT retry on 4xx status codes. Why?gh copilot suggest "write a TypeScript retry function" and receives a stub shell command instead of full code. What should they do next?gh copilot suggest is designed to translate natural language into shell commands. It will always produce a shell command output, even when the input is a programming request. For code generation from the terminal, Claude Code and Gemini CLI are the right tools. There is no --code-mode flag.