Skip to main content

Development vs Production Secrets: Environment

One of the most common security failures is using production secrets in development, or vice versa. A developer accidentally tests against the production database, corrupting real user data. A test suite uses a hardcoded fake key that gets committed to Git. A staging environment shares the production API key, so a compromise of staging leaks production access. This article covers the architecture and practices needed to maintain strict separation between development, staging, and production secrets, preventing environment-bleeding bugs and reducing blast radius when credentials are compromised.

The Environment Separation Pattern​

Define distinct configurations for each environment: development (your laptop), staging (mirror of production), and production (live traffic). Each environment has its own secrets that are never shared:

Development Secrets:
  - DATABASE_HOST: localhost (or dev-db.example.com)
  - DATABASE_PASSWORD: changeme123
  - API_KEY: sk_test_fake_12345
  - DEBUG: true

Staging Secrets:
  - DATABASE_HOST: staging-db.internal
  - DATABASE_PASSWORD: staging_password_xyz (different from prod)
  - API_KEY: sk_test_staging_67890
  - DEBUG: false

Production Secrets:
  - DATABASE_HOST: prod-db-primary.internal
  - DATABASE_PASSWORD: <highly protected, rotated monthly>
  - API_KEY: sk_live_xyz789abc (different key, different account)
  - DEBUG: false

Never reuse secrets across environments. If a staging API key is compromised, it should not grant access to the production API account.

Managing Secrets Across Environments​

Create separate secret sources for each environment:

import os
from pathlib import Path

class EnvironmentManager:
    """Manage secrets based on the ENVIRONMENT variable."""
    
    VALID_ENVS = {"development", "staging", "production"}
    
    @staticmethod
    def get_environment() -> str:
        """Get the current environment from env var."""
        env = os.getenv("ENVIRONMENT", "development").lower()
        if env not in EnvironmentManager.VALID_ENVS:
            raise ValueError(f"Invalid ENVIRONMENT: {env}. Must be one of {EnvironmentManager.VALID_ENVS}")
        return env
    
    @staticmethod
    def load_secrets() -> dict:
        """Load secrets for the current environment."""
        env = EnvironmentManager.get_environment()
        
        # Load base secrets first
        secrets = {
            "DATABASE_HOST": "localhost",
            "DATABASE_PORT": 5432,
            "DEBUG": False,
        }
        
        # Load environment-specific overrides
        if env == "development":
            secrets.update({
                "DATABASE_PASSWORD": "dev_password",
                "API_KEY": "sk_test_dev_123",
                "DEBUG": True,
                "LOG_LEVEL": "DEBUG",
            })
        
        elif env == "staging":
            secrets.update({
                "DATABASE_HOST": "staging-db.internal",
                "DATABASE_PASSWORD": os.getenv("DATABASE_PASSWORD"),
                "API_KEY": os.getenv("API_KEY"),
                "DEBUG": False,
                "LOG_LEVEL": "INFO",
            })
        
        elif env == "production":
            # Production secrets MUST come from secret manager, never from code
            import hvac
            client = hvac.Client(url=os.getenv("VAULT_ADDR"))
            client.auth.approle.login(
                role_id=os.getenv("VAULT_ROLE_ID"),
                secret_id=os.getenv("VAULT_SECRET_ID")
            )
            
            vault_secret = client.secrets.kv.v2.read_data_latest(path="prod/database")
            secrets.update({
                "DATABASE_HOST": vault_secret["data"]["data"]["host"],
                "DATABASE_PASSWORD": vault_secret["data"]["data"]["password"],
                "API_KEY": os.getenv("API_KEY"),  # From env var injected at deploy
                "DEBUG": False,
                "LOG_LEVEL": "WARNING",
            })
        
        # Validate required secrets
        required = ["DATABASE_PASSWORD", "API_KEY"]
        missing = [k for k in required if not secrets.get(k)]
        if missing and env != "development":
            raise RuntimeError(f"Missing required secrets: {missing}")
        
        return secrets

# At application startup
secrets = EnvironmentManager.load_secrets()

Preventing Production Data Leaks in Development​

A common accident: a developer copies production data locally for debugging, accidentally commits it, then shares it with the team. Prevent this with strict data policies:

import os
import warnings

def check_development_data_safety():
    """Warn if using production data in development environment."""
    env = os.getenv("ENVIRONMENT", "development")
    
    if env == "development":
        # Development should never connect to production database
        db_host = os.getenv("DATABASE_HOST", "localhost")
        
        if "prod" in db_host.lower() or "production" in db_host.lower():
            raise RuntimeError(
                "Development environment is configured to use production database! "
                "This is a critical security error. Please reset DATABASE_HOST to a local or staging database."
            )
        
        if os.getenv("API_KEY", "").startswith("sk_live_"):
            raise RuntimeError(
                "Development environment is using a live (production) API key! "
                "Switch to a test key (sk_test_*)."
            )

# Call at application startup
check_development_data_safety()

For test fixtures, use minimal mock data that is never based on production:

import pytest
from faker import Faker

@pytest.fixture
def mock_user():
    """Generate a mock user for testing (never based on production data)."""
    fake = Faker()
    return {
        "id": 12345,  # Fake ID, never a real user ID
        "email": fake.email(),
        "name": fake.name(),
    }

def test_user_registration(mock_user):
    """Test registration with mock data, never production."""
    result = register_user(mock_user["email"], mock_user["name"])
    assert result["success"]

Managing Secrets in CI/CD Pipelines​

CI/CD platforms (GitHub Actions, GitLab CI, Jenkins) provide secret vaults separate from the codebase. Store all secrets there, never in .yml files:

# GitHub Actions: .github/workflows/deploy.yml

name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      # Secrets are injected as environment variables from GitHub's vault
      - name: Deploy to production
        env:
          DATABASE_PASSWORD: ${{ secrets.PROD_DATABASE_PASSWORD }}
          API_KEY: ${{ secrets.PROD_API_KEY }}
          VAULT_ROLE_ID: ${{ secrets.VAULT_ROLE_ID }}
          VAULT_SECRET_ID: ${{ secrets.VAULT_SECRET_ID }}
        run: |
          python -m pip install -r requirements.txt
          export ENVIRONMENT=production
          python deploy.py

Never echo secrets in build logs:

# WRONG: logs the secret
echo "Deploying with API key: $API_KEY"

# CORRECT: no secret in output
echo "Deploying with API key: sk_***"

Testing with Different Secret Sources​

Your tests should pass with all secret sources (environment variables, secret managers, .env files):

import os
import pytest
from unittest.mock import MagicMock, patch

class TestSecretManagement:
    
    def test_secrets_from_env_vars(self):
        """Test loading secrets from environment variables."""
        os.environ["ENVIRONMENT"] = "staging"
        os.environ["DATABASE_PASSWORD"] = "test_password"
        os.environ["API_KEY"] = "sk_test_123"
        
        secrets = EnvironmentManager.load_secrets()
        
        assert secrets["DATABASE_PASSWORD"] == "test_password"
        assert secrets["API_KEY"] == "sk_test_123"
    
    @patch("hvac.Client")
    def test_secrets_from_vault(self, mock_vault):
        """Test loading secrets from Vault in production."""
        os.environ["ENVIRONMENT"] = "production"
        os.environ["VAULT_ADDR"] = "http://vault:8200"
        os.environ["VAULT_ROLE_ID"] = "test_role"
        os.environ["VAULT_SECRET_ID"] = "test_secret"
        
        mock_vault.return_value.secrets.kv.v2.read_data_latest.return_value = {
            "data": {"data": {"host": "prod-db", "password": "prod_secret"}}
        }
        
        secrets = EnvironmentManager.load_secrets()
        
        assert secrets["DATABASE_PASSWORD"] == "prod_secret"
        assert secrets["DATABASE_HOST"] == "prod-db"
    
    def test_production_data_safety(self):
        """Ensure development mode rejects production data."""
        os.environ["ENVIRONMENT"] = "development"
        os.environ["DATABASE_HOST"] = "prod-db.internal"  # Oops!
        
        with pytest.raises(RuntimeError, match="production database"):
            check_development_data_safety()

Secret Audit Trail for Compliance​

For regulated industries (finance, healthcare), maintain an audit log of who accessed which secrets:

import logging
import json
from datetime import datetime

class SecretAccessLogger:
    """Log all secret accesses for compliance auditing."""
    
    def __init__(self, log_file: str):
        self.logger = logging.getLogger("secret_access")
        handler = logging.FileHandler(log_file)
        self.logger.addHandler(handler)
    
    def log_access(self, secret_name: str, actor: str, action: str):
        """Log a secret access event."""
        event = {
            "timestamp": datetime.utcnow().isoformat(),
            "secret": secret_name,
            "actor": actor,
            "action": action,
        }
        self.logger.info(json.dumps(event))

# Usage
audit_logger = SecretAccessLogger("/var/log/secret_access.log")

# When fetching a secret
audit_logger.log_access("database_password", "app.deploy", "read")

Environment-Specific Configuration Table​

AspectDevelopmentStagingProduction
DataTest/fake dataCopy of production (masked)Real user data
Secrets Storage.env file, environment varsSecret Manager (encrypted)Vault, AWS Secrets Manager
API KeysTest/sandbox keysTest/staging keysLive keys (separate account)
Debug LoggingVerbose (DEBUG)Normal (INFO)Minimal (WARNING)
External API CallsMocked / sandboxReal but staging serviceReal production service
Database ReplicationNoneDaily from productionReal-time transactions
Access ControlAny developerLimited (QA, product)Strict IAM (ops, oncall)

Key Takeaways​

  • Maintain separate, non-overlapping secrets for development, staging, and production; never share API keys or database passwords across environments.
  • Load environment-specific secrets based on the ENVIRONMENT variable; development can use .env files, production must use secret managers.
  • Prevent production data leaks by rejecting production database hosts and API keys in development mode.
  • Store all CI/CD secrets in the platform's vault (GitHub Actions secrets:, GitLab CI variables:), never in .yml files.
  • Test your application with different secret sources to ensure it works with environment variables, files, and secret managers.

Frequently Asked Questions​

How do I safely copy production data for debugging in development?​

Create an anonymized or masked copy using tools like Anonymizer or pgMasker (PostgreSQL). Redact PII (personally identifiable information), remove payment details, and hash identifiers. Store this copy in a staging database, not your laptop. Access it only through a VPN or bastion host.

What if I accidentally used production secrets during development?​

Immediately revoke the secret (database password, API key) in the production service. Check audit logs to see if the secret was actually used (queries, API calls). If compromised, rotate all other secrets in that environment. Update your .env file to use development secrets. Add pre-commit hooks (see Article 10) to prevent this.

Can I use the same API key across development and staging?​

No. Create separate API keys for each environment in your service provider's dashboard. Staging uses sk_test_staging_*, production uses sk_live_*. This limits damage if staging is compromised.

How do I rotate production secrets during a deployment?​

For databases: use AWS RDS Proxy or Vault dynamic credentials (automatically rotates). For API keys: run a blue-green deployment where new instances use the new key, old instances use the old key, then gradually shift traffic. Keep old keys active for 24 hours to catch any missed clients.

Should my test fixtures use production-like data?​

Never real production data. Use the Faker library to generate realistic but synthetic data. Keep fixture sizes small < 100 records so tests are fast. For integration tests requiring real-like data structures, use a staging database copy (masked).

Further Reading​