docs: document migration file modification failure pattern

Add CRITICAL section documenting the failure pattern where migration
files are modified after deployment, causing Drizzle to skip them
(it tracks by name, not content). This pattern has caused three
production outages in December 2025.

Includes:
- Explanation of why this happens
- The correct pattern (complete SQL before committing)
- How to fix with new migrations
- Emergency fix procedure

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Thomas Hallock 2025-12-30 07:56:09 -06:00
parent 474c4da05a
commit a8f55c9f4f
1 changed files with 99 additions and 0 deletions

View File

@ -145,6 +145,105 @@ sqlite3 data/sqlite.db "ALTER TABLE session_plans ADD COLUMN is_paused INTEGER;"
# SELECT 1; -- This does nothing on production!
```
## CRITICAL: Never Modify Migration Files After Deployment
**NEVER modify a migration file after it has been deployed to production.**
Drizzle tracks migrations by name/tag, not by content. Once a migration is recorded in `__drizzle_migrations`, it will NEVER be re-run, even if the file content changes.
### The Failure Pattern (December 2025)
1. Migration 0047 was created and deployed (possibly as a stub or incomplete)
2. Production recorded it in `__drizzle_migrations` as "applied"
3. Developer modified 0047.sql locally with the actual CREATE TABLE statement
4. New deployment saw 0047 was already "applied" → skipped it
5. Production crashed: `SqliteError: no such column: "entry_prompt_expiry_minutes"`
6. Required emergency migration to fix
**This exact pattern has caused THREE production outages:**
- Migration 0043 (December 2025): `is_paused` column missing
- Migration 0047 (December 2025): `entry_prompts` table missing
- Migration 0048 (December 2025): `entry_prompt_expiry_minutes` column missing
### Why This Happens
```
Timeline:
1. Create migration file (empty or stub) → deployed → recorded as "applied"
2. Modify migration file with real SQL → deployed → SKIPPED (already "applied")
3. Production crashes → missing tables/columns
```
Drizzle's migrator checks: "Is migration 0047 in `__drizzle_migrations`?" → Yes → Skip it.
It does NOT check: "Has the content of 0047.sql changed?"
### The Correct Pattern
**Before committing a migration, ensure it contains the FINAL SQL:**
```bash
# 1. Generate migration
npx drizzle-kit generate --custom
# 2. IMMEDIATELY edit the generated SQL with the actual statements
# DO NOT commit an empty/stub migration!
# 3. Run locally to verify
npm run db:migrate
# 4. Only THEN commit
git add drizzle/
git commit -m "feat: add entry_prompts table"
```
### If You Need to Fix a Deployed Migration
**DO NOT modify the existing migration file.** Instead:
```bash
# Create a NEW migration with the fix
npx drizzle-kit generate --custom
# Name it something like: 0050_fix_missing_entry_prompts.sql
# Add the missing SQL (with IF NOT EXISTS for safety)
CREATE TABLE IF NOT EXISTS `entry_prompts` (...);
# OR for SQLite (which doesn't support IF NOT EXISTS for columns):
# Check if column exists first, or just let it fail silently
```
### Red Flags
If you find yourself:
- Editing a migration file that's already been committed
- Thinking "I'll just update this migration with the correct SQL"
- Seeing "migration already applied" but schema is wrong
**STOP.** Create a NEW migration instead.
### Emergency Fix for Production
If production is down due to missing schema, create a new migration immediately:
```bash
# 1. Generate emergency migration
npx drizzle-kit generate --custom
# Creates: drizzle/0050_emergency_fix.sql
# 2. Add the missing SQL with safety checks
# For tables:
CREATE TABLE IF NOT EXISTS `entry_prompts` (...);
# For columns (SQLite workaround - will error if exists, but migration still records):
ALTER TABLE `classrooms` ADD COLUMN `entry_prompt_expiry_minutes` integer;
# 3. Commit and deploy
git add drizzle/
git commit -m "fix: emergency migration for missing schema"
git push
```
The new migration will run on production startup and fix the schema.
## CRITICAL: @svg-maps ES Module Imports Work Correctly
**The @svg-maps packages (world, usa) USE ES module syntax and this WORKS correctly in production.**