Alembic in CI/CD Pipelines: Automate Deployments
Integrating Alembic migrations into your CI/CD pipeline ensures every code change is tested for schema compatibility before deployment. Your pipeline should (1) test migrations on staging, (2) verify the migrated schema matches models, (3) generate migration reports for review, and (4) safely apply migrations to production. This automation prevents schema errors from reaching production and gives teams confidence in deployments.
Basic CI/CD Workflow
A typical migration workflow:
Code commit → Run tests → Test migrations → Schema check → Deploy to staging → Migrate production
- Code commit — Developer pushes code with model changes
- Run tests — Unit and integration tests run
- Test migrations — Alembic migrations applied to a test database
- Schema check — Verify migrated schema matches models
- Deploy to staging — Code and migrations deployed to staging environment
- Migrate production — Run
alembic upgrade headon production (manual approval step)
GitHub Actions Example
A complete GitHub Actions workflow that tests and validates migrations:
# .github/workflows/migrations.yml
name: Test and Validate Migrations
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
jobs:
test-migrations:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
POSTGRES_DB: testdb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install -e .
pip install pytest alembic sqlalchemy
# Test 1: Run migration on test database
- name: Test migrations
env:
DATABASE_URL: postgresql://testuser:testpass@localhost:5432/testdb
run: |
alembic upgrade head
echo "✓ All migrations applied successfully"
# Test 2: Verify schema matches models
- name: Verify schema integrity
run: |
python -m pytest tests/test_migrations.py -v
# Test 3: Run application tests
- name: Run application tests
env:
DATABASE_URL: postgresql://testuser:testpass@localhost:5432/testdb
run: |
pytest tests/ -v --cov=app
# Test 4: Generate migration report
- name: Generate migration report
run: |
alembic history -v > migration-report.txt
cat migration-report.txt
# Test 5: Check for migration conflicts
- name: Detect migration conflicts
run: |
python scripts/check_migration_conflicts.py
# Optional: Upload report as artifact
- name: Upload migration report
if: always()
uses: actions/upload-artifact@v3
with:
name: migration-report
path: migration-report.txt
Migration Conflict Detection Script
Detect if multiple migrations share the same parent (branching problem):
# scripts/check_migration_conflicts.py
#!/usr/bin/env python
"""
Detect migration conflicts: multiple revisions with the same parent.
"""
import os
import re
from pathlib import Path
def check_migration_conflicts():
"""Find migrations that could conflict."""
migrations_dir = Path('migrations/versions')
parent_map = {} # Map of parent -> [children]
conflicts = []
for migration_file in sorted(migrations_dir.glob('*.py')):
if migration_file.name == '__pycache__':
continue
with open(migration_file) as f:
content = f.read()
# Extract down_revision
match = re.search(r"down_revision\s*=\s*['\"]([^'\"]+)['\"]", content)
if match:
parent = match.group(1)
if parent not in parent_map:
parent_map[parent] = []
parent_map[parent].append(migration_file.name)
# Check for conflicts
for parent, children in parent_map.items():
if len(children) > 1:
conflicts.append((parent, children))
print(f"⚠️ CONFLICT: Multiple migrations depend on {parent}:")
for child in children:
print(f" - {child}")
if conflicts:
print(f"\n❌ Found {len(conflicts)} migration conflict(s)")
print(" Resolve by creating a merge migration")
exit(1)
else:
print("✓ No migration conflicts detected")
exit(0)
if __name__ == '__main__':
check_migration_conflicts()
Pre-Deployment Verification Script
Before deploying to production, verify migrations are safe:
# scripts/verify_migrations.py
#!/usr/bin/env python
"""
Verify that migrations are safe to deploy.
Checks:
1. No empty migrations
2. All migrations have reversible downgrade()
3. No duplicate migration names
4. Migration history is linear (no conflicts)
"""
import os
import re
from pathlib import Path
from alembic.config import Config
from alembic.script import ScriptDirectory
def verify_migrations():
"""Run all migration safety checks."""
config = Config('alembic.ini')
script = ScriptDirectory.from_config(config)
issues = []
# Check 1: No empty migrations
versions_dir = Path('migrations/versions')
for migration_file in versions_dir.glob('*.py'):
if migration_file.name.startswith('__'):
continue
with open(migration_file) as f:
content = f.read()
# Check if upgrade and downgrade are empty
if 'def upgrade():\n pass' in content:
issues.append(f"Empty upgrade in {migration_file.name}")
if 'def downgrade():\n pass' in content:
issues.append(f"Empty downgrade in {migration_file.name}")
# Check 2: No duplicate names
migration_names = [f.stem for f in versions_dir.glob('*.py') if not f.name.startswith('__')]
if len(migration_names) != len(set(migration_names)):
issues.append("Duplicate migration file names detected")
# Check 3: Verify linear history (no conflicts)
heads = script.get_heads()
if len(heads) > 1:
issues.append(f"Multiple migration heads (branches): {heads}")
issues.append(" Create a merge migration to resolve")
# Report
if issues:
print("❌ Migration verification FAILED:")
for issue in issues:
print(f" {issue}")
exit(1)
else:
print("✓ All migration checks passed")
print(f" - No empty migrations")
print(f" - All migrations are reversible")
print(f" - No duplicate names")
print(f" - Linear migration history")
exit(0)
if __name__ == '__main__':
verify_migrations()
Staging Deployment Script
Before production, test migrations on a staging database:
#!/bin/bash
# scripts/deploy-staging.sh
set -e # Exit on any error
echo "=== Deploying to staging ==="
# 1. Build and push Docker image
echo "Building Docker image..."
docker build -t myapp:staging .
docker push myregistry.azurecr.io/myapp:staging
# 2. Deploy to staging Kubernetes cluster
echo "Deploying to staging cluster..."
kubectl apply -f k8s/staging/ --image=myregistry.azurecr.io/myapp:staging
# 3. Wait for pods to be ready
echo "Waiting for pods to be ready..."
kubectl rollout status deployment/myapp -n staging --timeout=5m
# 4. Run migrations on staging database
echo "Applying migrations to staging..."
kubectl exec -n staging deployment/myapp -- alembic upgrade head
# 5. Verify schema integrity
echo "Verifying schema..."
kubectl exec -n staging deployment/myapp -- python -m pytest tests/test_migrations.py
# 6. Run smoke tests
echo "Running smoke tests..."
kubectl exec -n staging deployment/myapp -- pytest tests/smoke_tests.py
echo "✓ Staging deployment successful"
Production Deployment: Manual Approval
For production, require human approval before migrations run. This prevents accidental data loss:
# .github/workflows/deploy-production.yml
name: Deploy to Production
on:
workflow_dispatch: # Manual trigger
inputs:
migration_revision:
description: 'Migration revision to deploy (default: head)'
required: false
default: 'head'
jobs:
deploy-production:
runs-on: ubuntu-latest
environment:
name: production
# GitHub requires manual approval from code owners
steps:
- uses: actions/checkout@v3
- name: Set up gcloud CLI
uses: google-cloud/setup-gcloud@v1
- name: Generate migration plan
run: |
export DATABASE_URL=${{ secrets.PROD_DATABASE_URL }}
alembic current > current-revision.txt
alembic history -v > migration-plan.txt
echo "## Migrations to be applied:"
cat migration-plan.txt
- name: Apply migrations to production
run: |
export DATABASE_URL=${{ secrets.PROD_DATABASE_URL }}
# Show what will happen
alembic upgrade ${{ github.event.inputs.migration_revision }} --sql
# Ask for confirmation (this step requires manual approval in GitHub)
echo "🚨 WARNING: This will modify your production database"
echo "Press Enter to continue or Ctrl+C to cancel"
read -p "Continue? (yes/no): " -n 3 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
alembic upgrade ${{ github.event.inputs.migration_revision }}
echo "✓ Migrations applied successfully"
else
echo "❌ Deployment cancelled"
exit 1
fi
- name: Post-deployment verification
run: |
export DATABASE_URL=${{ secrets.PROD_DATABASE_URL }}
alembic current
python -m pytest tests/smoke_tests.py
- name: Notify Slack
if: success()
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "Production migrations deployed successfully",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "✓ Production migrations applied\nRevision: ${{ github.event.inputs.migration_revision }}"
}
}
]
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
Rollback Automation
If production deployments fail, have a rollback procedure:
#!/bin/bash
# scripts/rollback-production.sh
REVISION=$1 # Specific revision to rollback to
if [ -z "$REVISION" ]; then
echo "Usage: ./rollback-production.sh <revision>"
echo "Example: ./rollback-production.sh abc123def456"
exit 1
fi
echo "⚠️ WARNING: Rolling back to revision $REVISION"
read -p "Are you sure? Type 'yes' to continue: " -r
if [[ ! $REPLY =~ ^[Yy][Ee][Ss]$ ]]; then
echo "Rollback cancelled"
exit 0
fi
export DATABASE_URL=$PROD_DATABASE_URL
# Show what will be rolled back
alembic downgrade $REVISION --sql
read -p "Execute rollback? (yes/no): " -n 3 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
alembic downgrade $REVISION
echo "✓ Rollback completed"
else
echo "Rollback cancelled"
exit 1
fi
Docker Integration
For containerized apps, include migrations in the Docker image:
# Dockerfile
FROM python:3.11
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
# Run migrations on container startup
CMD ["sh", "-c", "alembic upgrade head && gunicorn app:app"]
Alternatively, run migrations as a separate job before deploying the app:
# kubernetes-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: db-migration
spec:
template:
spec:
containers:
- name: migration
image: myapp:latest
command: ["alembic", "upgrade", "head"]
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
restartPolicy: Never
Key Takeaways
- Integrate migration testing into CI/CD pipelines to catch schema errors before deployment
- Use GitHub Actions (or similar) to test migrations on staging databases
- Write scripts to detect migration conflicts and verify migration safety
- Generate migration reports and require human review for production deployments
- Use environment approval gates for production migrations (manual approval)
- Implement rollback procedures for quick recovery if migrations fail
- For containerized apps, run migrations as a separate job before deploying app code
- Monitor post-deployment to catch schema-related bugs early
Frequently Asked Questions
Can I deploy migrations and code changes at the same time?
Yes, but be careful. If the migration happens before the code deploys, the old code may break on the new schema. Better approach: (1) deploy backward-compatible migrations first (add new columns, don't remove old ones), (2) deploy code, (3) in a follow-up, remove old columns. This two-phase approach allows rollback without data loss.
What if a migration fails during production deployment?
Alembic wraps each migration in a transaction. If any statement fails, the entire migration is rolled back. Your database reverts to the previous state, and you can fix the migration and retry. For zero-downtime deployments, use shadow tables or feature flags.
Should I apply migrations before or after deploying new code?
Migrations should apply before code. New code expects the new schema. If you deploy code first and the migration fails, the app breaks. Order: (1) run migrations, (2) deploy code.
Can I test migrations on production data without modifying it?
Use a production data clone. Snapshot your production database, restore it to a staging environment, run migrations, and verify. This is slower but catches edge cases that test data misses.
How do I handle zero-downtime migrations?
Use shadow tables, feature flags, or blue-green deployments. For example, (1) add a new column, (2) have the app write to both old and new columns, (3) migrate data in batches, (4) switch code to read from new column only. This avoids downtime.