Deployment & Maintenance

gemini "@Dockerfile @requirements.txt Improve this Dockerfile with:
1. Multi-stage build: builder stage (install deps) + runtime stage (copy only what's needed)
2. Non-root user named 'appuser' with UID 1001
3. Health check that hits GET /health every 30s
4. Pinned base image with digest (not just :3.12-slim)
5. Explain each addition with a comment in the Dockerfile"

gemini "@Dockerfile @main.py @database.py Generate a docker-compose.yml for TaskFlow with:
1. A service named 'api' using the Dockerfile in the current directory
2. A named volume 'taskflow-data' mounted to where database.py stores the SQLite file
3. Environment variables loaded from a .env file
4. PORT mapped to 8000 on the host
5. restart: unless-stopped
6. A depends_on health check so the service only starts when healthy
7. Comments explaining each section"

# docker-compose.yml for TaskFlow
# Usage: docker compose up -d
# Requires: .env file with SECRET_KEY and DATABASE_URL

version: "3.9"

services:
  api:
    # ── Build from the Dockerfile in the current directory ──────
    build:
      context: .
      dockerfile: Dockerfile

    # ── Map host port 8000 to container port 8000 ───────────────
    ports:
      - "8000:8000"

    # ── Load environment variables from .env file ────────────────
    # Create .env from .env.example before running compose up
    env_file:
      - .env

    # ── Persist the SQLite database across container restarts ────
    # database.py: sqlite:///./taskflow.db → /app/taskflow.db
    volumes:
      - taskflow-data:/app/taskflow.db

    # ── Restart automatically unless manually stopped ────────────
    restart: unless-stopped

    # ── Health check (matches Dockerfile HEALTHCHECK) ────────────
    healthcheck:
      test: ["CMD", "python", "-c",
             "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 10s

# ── Named volume — data survives container replacement ───────────
volumes:
  taskflow-data:
    driver: local

gemini "@requirements.txt @Dockerfile Generate a .github/workflows/ci.yml for TaskFlow that:
1. Triggers on push to main and on pull requests
2. Runs on ubuntu-latest
3. Sets up Python 3.12 and caches pip dependencies
4. Installs dependencies from requirements.txt
5. Runs pytest with --cov and fails the workflow if coverage is below 80%
6. Builds the Docker image (but does not push — no registry credentials needed)
7. Uses a SECRET_KEY secret from GitHub Secrets for the test run
8. Annotates each step clearly"

name: TaskFlow CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test-and-build:
    runs-on: ubuntu-latest

    steps:
      # ── CHUNK 1: Checkout and Python setup ──────────────────────
      # WHAT: Gets the code and sets up Python with pip caching.
      # WHY:  Caching pip packages saves ~45s per run for TaskFlow's
      #       dependency set.
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python 3.12
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"

      # ── CHUNK 2: Install dependencies ────────────────────────────
      # WHAT: Installs all packages from requirements.txt plus pytest-cov.
      # WHY:  pytest-cov is not in requirements.txt (it's a dev dependency)
      #       so we install it separately.
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest-cov

      # ── CHUNK 3: Run tests with coverage gate ────────────────────
      # WHAT: Runs pytest, generates a coverage report, and fails if
      #       overall coverage is below 80%.
      # WHY:  --cov-fail-under=80 makes the CI step exit non-zero
      #       (red) if coverage drops. This prevents coverage regressions
      #       from silently merging.
      # GOTCHA: We set SECRET_KEY from GitHub Secrets so auth.py does
      #         not fall back to the insecure dev default.
      - name: Run tests with coverage
        env:
          SECRET_KEY: ${{ secrets.SECRET_KEY }}
        working-directory: ./sample-project/taskflow
        run: |
          pytest tests/ \
            --cov=. \
            --cov-report=term-missing \
            --cov-omit="tests/*" \
            --cov-fail-under=80 \
            -v

      # ── CHUNK 4: Build Docker image ───────────────────────────────
      # WHAT: Builds (but does not push) the Docker image.
      # WHY:  Verifies the Dockerfile is valid and the build succeeds
      #       without requiring any registry credentials.
      - name: Build Docker image
        working-directory: ./sample-project/taskflow
        run: |
          docker build -t taskflow:${{ github.sha }} .
          echo "Image built: taskflow:${{ github.sha }}"

gemini "@Dockerfile @requirements.txt Generate a cloudbuild.yaml for TaskFlow that:
1. Runs pytest (fail if tests fail — do NOT deploy broken code)
2. Builds the Docker image
3. Pushes to Artifact Registry at us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api
4. Deploys to Cloud Run service named 'taskflow-api' in us-central1 ONLY if tests pass
5. Uses $COMMIT_SHA as the image tag
6. Sets SECRET_KEY from Secret Manager (not hardcoded)
7. Annotates each step"

# cloudbuild.yaml for TaskFlow
# Trigger: push to main branch via Cloud Build trigger
# Required IAM: Cloud Run Admin, Artifact Registry Writer, Secret Manager Accessor

steps:
  # ── STEP 1: Run tests ─────────────────────────────────────────
  # WHAT: Runs pytest in the Python 3.12 container.
  # WHY:  If tests fail, Cloud Build stops here and does NOT build
  #       or deploy — broken code never reaches production.
  - name: 'python:3.12-slim'
    id: 'run-tests'
    dir: 'sample-project/taskflow'
    entrypoint: bash
    args:
      - '-c'
      - |
        pip install -r requirements.txt pytest pytest-cov -q
        pytest tests/ --cov=. --cov-fail-under=80 -q
    secretEnv: ['SECRET_KEY']

  # ── STEP 2: Build Docker image ────────────────────────────────
  # WHAT: Builds the image and tags it with the commit SHA.
  # WHY:  $COMMIT_SHA is automatically set by Cloud Build — each
  #       image is uniquely traceable to its source commit.
  - name: 'gcr.io/cloud-builders/docker'
    id: 'build-image'
    dir: 'sample-project/taskflow'
    args:
      - 'build'
      - '-t'
      - 'us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api:$COMMIT_SHA'
      - '-t'
      - 'us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api:latest'
      - '.'

  # ── STEP 3: Push to Artifact Registry ────────────────────────
  # WHAT: Pushes both the SHA-tagged and latest-tagged images.
  # WHY:  The Cloud Run deploy step references the SHA-tagged image
  #       for precision; the latest tag is for manual pulls.
  - name: 'gcr.io/cloud-builders/docker'
    id: 'push-image'
    args:
      - 'push'
      - '--all-tags'
      - 'us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api'

  # ── STEP 4: Deploy to Cloud Run ──────────────────────────────
  # WHAT: Updates the Cloud Run service to use the new image.
  # WHY:  --no-traffic means the new revision is deployed but
  #       does not receive traffic until you manually promote it.
  #       Remove --no-traffic for fully automated blue/green.
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    id: 'deploy-cloud-run'
    entrypoint: gcloud
    args:
      - 'run'
      - 'deploy'
      - 'taskflow-api'
      - '--image=us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api:$COMMIT_SHA'
      - '--region=us-central1'
      - '--platform=managed'
      - '--allow-unauthenticated'
      - '--set-secrets=SECRET_KEY=taskflow-secret-key:latest'

# ── Secret Manager integration ───────────────────────────────────
# Pulls SECRET_KEY from Secret Manager at build time.
# Requires the Cloud Build service account to have the
# Secret Manager Secret Accessor role.
availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/taskflow-secret-key/versions/latest
      env: 'SECRET_KEY'

images:
  - 'us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api:$COMMIT_SHA'
  - 'us-central1-docker.pkg.dev/$PROJECT_ID/taskflow/api:latest'

options:
  logging: CLOUD_LOGGING_ONLY

gemini "@main.py @database.py @Dockerfile @auth.py Write an operations runbook for TaskFlow covering:
1. Service startup — local and Docker
2. Health check verification
3. Database reset procedure (for development)
4. Graceful shutdown
5. Common errors and their fixes:
   - 'detail: Could not validate credentials' in response body
   - Database lock errors
   - 422 Unprocessable Entity from the tasks endpoint
6. Format as numbered steps with expected outputs"

# TaskFlow Operations Runbook

## 1. Service Startup

### Local (development)
1. Navigate to the project directory: `cd sample-project/taskflow`
2. Ensure .env exists: `cp .env.example .env`
3. Install dependencies: `pip install -r requirements.txt`
4. Start the server: `uvicorn main:app --reload`
5. Expected output:
   ```
   INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
   INFO:     Started reloader process [PID]
   ```
6. Verify: `curl http://localhost:8000/health`
   Expected: `{"status": "ok", "service": "taskflow"}`

### Docker
1. Build image: `docker build -t taskflow:local .`
2. Run container: `docker run -p 8000:8000 --env-file .env taskflow:local`
3. Verify health: `docker inspect --format='{{.State.Health.Status}}' `
   Expected: `healthy`

---

## 2. Health Check Verification

The `/health` endpoint returns HTTP 200 with `{"status": "ok", "service": "taskflow"}`.

- If it returns 200: Service is running normally.
- If it returns 503 or times out: Check logs with `docker logs  --tail 50`.
- If the endpoint does not exist: The container is running but the app failed to start.
  Check for import errors: `docker logs  2>&1 | grep "Error"`

---

## 5. Common Errors and Fixes

### "Could not validate credentials" (HTTP 401)
**What it means:** `auth.py get_current_user()` raised `credentials_exception`.
This happens when:
- No Authorization header is present
- The token has expired (default: 30 minutes, set by ACCESS_TOKEN_EXPIRE_MINUTES)
- The token was signed with a different SECRET_KEY than the current one
- The user account was deleted after the token was issued

**Fix:** Re-authenticate via POST /users/login. If the issue is SECRET_KEY mismatch
after a redeploy, all existing tokens are invalidated — users must log in again.

### Database lock error
**What it means:** SQLite is single-writer. If multiple requests try to write
simultaneously, one will get `sqlite3.OperationalError: database is locked`.
**Fix (development):** Restart the server. SQLite auto-releases locks.
**Fix (production):** This indicates high concurrent write load. Either:
  1. Add a connection pool timeout in database.py: `connect_args={"timeout": 30}`
  2. Migrate to PostgreSQL for production workloads above 50 concurrent users.

### 422 Unprocessable Entity on POST /tasks/
**What it means:** Pydantic validation failed. The request body is missing a
required field or has the wrong type.
**Diagnosis:** The response body includes a `detail` array specifying which field
failed. Example: `{"detail": [{"loc": ["body", "title"], "msg": "field required"}]}`
**Fix:** Check the request body against the TaskCreate schema in schemas.py.

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File ".../sqlalchemy/pool/base.py", line 686, in checkout
    rec = self._do_get()
  File ".../sqlalchemy/pool/impl.py", line 145, in _do_get
    self._dec_overflow()
  File ".../sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(exc_value, with_traceback=exc_tb)
  File ".../sqlalchemy/pool/impl.py", line 142, in _do_get
    raise exc.TimeoutError(...)
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10
is reached, connection timed out, timeout 30

During handling of the above exception, another exception occurred:
  ...
  File ".../taskflow/database.py", line 22, in get_db
    db = SessionLocal()
fastapi.exceptions.ResponseValidationError: 1 validation error ...

gemini "TaskFlow returned this error in production:

[paste the error log above]

Look at @database.py and @models.py and identify:
1. The root cause of this specific error
2. Whether the current database.py configuration would cause this
3. The immediate fix and the long-term fix"

Root cause analysis:

The error is QueuePool limit exceeded. SQLAlchemy's connection pool has
hit its maximum size (default: 5 connections + 10 overflow = 15 total).
New requests cannot acquire a database connection within 30 seconds.

Why this is happening in TaskFlow:

Looking at database.py, the SessionLocal is created with default pool
settings:
  SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

And the engine uses SQLite's default pool (StaticPool for SQLite in-memory,
or the default NullPool — but your code uses a file-based SQLite with the
default QueuePool). The real issue is likely in get_db():

  def get_db():
      db = SessionLocal()
      try:
          yield db
      finally:
          db.close()

If an exception occurs in a route handler BEFORE the finally block executes
(e.g., in a Pydantic validation step), the session may not be closed, leaking
connections back into the pool.

IMMEDIATE FIX:
Add pool_pre_ping=True to the engine to automatically discard stale connections:

  engine = create_engine(
      DATABASE_URL,
      connect_args={"check_same_thread": False},
      pool_pre_ping=True,  # ← add this
  )

Also add a connection timeout to give queued requests more time:
  engine = create_engine(
      DATABASE_URL,
      connect_args={"check_same_thread": False},
      pool_timeout=30,
      pool_recycle=300,  # recycle connections every 5 minutes
  )

LONG-TERM FIX:
SQLite is not designed for more than ~10 concurrent write connections.
At this traffic level, migrate to PostgreSQL. Change DATABASE_URL to:
  postgresql+psycopg2://user:password@host/taskflow

and remove the check_same_thread connect_arg (SQLite-specific).