Skip to main content

Vault and Credential Rotation: HashiCorp Vault

HashiCorp Vault is an enterprise-grade secret management platform that goes beyond static secret storage: it generates dynamic credentials on demand, automatically rotates them, revokes old ones, and logs every access. Instead of storing a database password forever, Vault creates a temporary username and password with a TTL (time-to-live), lets your application use it, and automatically revokes it when the TTL expires. This dramatically reduces the blast radius of a compromised credential: an attacker can only use a stolen credential until it expires, typically 1–24 hours.

Vault supports multiple backends (databases, PKI, SSH, cloud credentials) and is used by companies like Netflix, HashiCorp itself, and Databricks. This article covers Vault architecture, integrating Python applications with the Vault client library, requesting dynamic credentials, and handling credential renewal.

Understanding Vault Architecture​

Vault has three core concepts:

  1. Secrets Engine: A backend that generates or stores secrets. Examples: database (generates temporary DB passwords), pki (issues certificates), kv (static key-value store).

  2. Auth Method: A way to authenticate and get a token. Examples: approle (for applications), ldap (for users), kubernetes (for pods).

  3. Policy: Controls what a token can access. Example: a token can read database credentials but not write audit logs.

A typical flow:

1. Application authenticates with Vault (AppRole auth method)
2. Vault issues a token
3. Application requests database credentials using the token
4. Vault generates a temporary username/password with TTL
5. Application uses the credentials
6. Vault automatically revokes the credentials when TTL expires

Installing and Configuring hvac​

Install the HashiCorp Vault Python client:

pip install hvac

For local development, start a dev Vault server:

vault server -dev

Vault starts on 127.0.0.1:8200 with an unencrypted in-memory backend (development only). The init output shows the root token.

In your Python application, authenticate with AppRole:

import hvac
import os

# Initialize Vault client
vault_addr = os.getenv("VAULT_ADDR", "http://127.0.0.1:8200")
role_id = os.getenv("VAULT_ROLE_ID")
secret_id = os.getenv("VAULT_SECRET_ID")

client = hvac.Client(url=vault_addr)

# Authenticate using AppRole
client.auth.approle.login(role_id=role_id, secret_id=secret_id)

print(f"Authenticated to Vault. Token: {client.token[:20]}...")

The AppRole auth method is designed for applications: it uses a role_id (non-secret, can be in code) and secret_id (secret, must come from the environment or Vault) to authenticate.

Requesting Dynamic Database Credentials​

Configure Vault to manage database credentials (this is typically done by Vault administrators):

# Enable the database secrets engine
vault secrets enable database

# Configure a database connection
vault write database/config/postgres \
  plugin_name=postgresql-database-plugin \
  allowed_roles="readonly" \
  connection_url="postgresql://{{username}}:{{password}}@localhost:5432/postgres" \
  username="vault_admin" \
  password="admin_password"

# Create a role that generates temporary credentials
vault write database/roles/readonly \
  db_name=postgres \
  creation_statements="CREATE USER \"{{name}}\" WITH PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  ttl=1h \
  max_ttl=24h

Now, in your Python application:

import hvac
import os

client = hvac.Client(url=os.getenv("VAULT_ADDR", "http://127.0.0.1:8200"))

# Authenticate
client.auth.approle.login(
    role_id=os.getenv("VAULT_ROLE_ID"),
    secret_id=os.getenv("VAULT_SECRET_ID")
)

# Request dynamic database credentials
db_creds = client.secrets.database.read_dynamic_credentials(
    name="readonly",
    mount_point="database"
)

username = db_creds["data"]["data"]["username"]
password = db_creds["data"]["data"]["password"]
ttl = db_creds["data"]["lease_duration"]

print(f"Got credentials for {username} (valid for {ttl} seconds)")

# Use the credentials
import psycopg2
conn = psycopg2.connect(
    host="localhost",
    user=username,
    password=password,
    database="postgres"
)

# Fetch data...
cursor = conn.cursor()
cursor.execute("SELECT * FROM users LIMIT 1")
print(cursor.fetchone())

conn.close()

# Vault automatically revokes the credentials after TTL expires

Each request for credentials gets a unique username and password with a short TTL. When the TTL expires, Vault revokes the database user, preventing further access.

Handling Token Renewal and Lease Management​

Vault tokens and credentials expire. Your application must handle renewal:

import hvac
import os
from threading import Thread
import time

class VaultCredentialManager:
    def __init__(self, vault_addr: str, role_id: str, secret_id: str):
        self.client = hvac.Client(url=vault_addr)
        self.client.auth.approle.login(role_id=role_id, secret_id=secret_id)
        self._credentials = None
        self._token_ttl = None
        self._renew_thread = None
        self.start_renewal()
    
    def start_renewal(self):
        """Start a background thread to renew token periodically."""
        self._renew_thread = Thread(target=self._renew_loop, daemon=True)
        self._renew_thread.start()
    
    def _renew_loop(self):
        """Periodically renew the Vault token."""
        while True:
            try:
                # Renew token (Vault default is to renew for the same TTL)
                self.client.auth.token.renew_self()
                self._token_ttl = self.client.auth.token.lookup_self()["data"]["ttl"]
                print(f"Token renewed. TTL: {self._token_ttl} seconds")
                
                # Renew in half the TTL
                time.sleep(self._token_ttl / 2)
            except Exception as e:
                print(f"Token renewal failed: {e}")
                time.sleep(10)  # Retry after 10 seconds
    
    def get_db_credentials(self, role: str) -> dict:
        """Get dynamic database credentials."""
        response = self.client.secrets.database.read_dynamic_credentials(
            name=role,
            mount_point="database"
        )
        self._credentials = response["data"]["data"]
        return self._credentials

# Usage
manager = VaultCredentialManager(
    vault_addr="http://127.0.0.1:8200",
    role_id=os.getenv("VAULT_ROLE_ID"),
    secret_id=os.getenv("VAULT_SECRET_ID")
)

creds = manager.get_db_credentials("readonly")
print(f"Using credential for {creds['username']}")

The background thread periodically renews the token, keeping the application authenticated without manual intervention.

Storing Static Secrets in Vault​

For secrets that do not change (like API keys), use Vault's key-value store:

# Set a secret (via CLI or API)
vault kv put secret/my-app/api-keys \
  external_api_key="sk_live_123456" \
  internal_api_key="sk_internal_789"

# Read in Python
api_keys = client.secrets.kv.v2.read_data_latest(path="my-app/api-keys")
external_key = api_keys["data"]["data"]["external_api_key"]

Or mount a v1 kv store (simpler):

vault kv put secret/db-password value="mypass"

creds = client.secrets.kv.v1.read_secret_version(path="db-password")
password = creds["data"]["data"]["value"]

Vault vs Secrets Manager Comparison​

FeatureVaultAWS Secrets Manager
Dynamic CredentialsYes (database, SSH, PKI)No (static only)
Automatic RotationYes (built-in)Yes (via Lambda)
EncryptionYes (TLS, encryption at rest)Yes (KMS)
On-PremisesYes (self-hosted)No (AWS only)
CostFree (open-source) or $5k+/year (enterprise)$0.40/secret/month
Learning CurveSteepShallow
Best ForLarge enterprises, complianceAWS-only teams, simplicity

Vault is more powerful but requires more operational overhead. Secrets Manager is simpler but AWS-only.

Key Takeaways​

  • Vault generates dynamic credentials on demand with automatic expiration, reducing the blast radius of compromised secrets to minutes or hours instead of indefinitely.
  • Use AppRole auth for applications; authenticate with role_id and secret_id, then request dynamic credentials for databases, SSH, or PKI.
  • Implement token renewal in a background thread to keep your application perpetually authenticated without restarting.
  • Store static secrets (API keys) in Vault's key-value store; store temporary credentials in specialized backends (database, PKI).
  • Vault is ideal for large, regulated organizations that need fine-grained control and audit trails; Secrets Manager is better for AWS-only teams prioritizing simplicity.

Frequently Asked Questions​

How do I deploy Vault in production?​

Use Vault Enterprise (SaaS-hosted by HashiCorp) or self-host using Vault Raft storage or Consul as a backend. Enable TLS, set up high availability with multiple nodes, and use encryption keys from a KMS (AWS KMS, Google Cloud KMS). For details, see the Vault Operations guide.

What happens if my application crashes and loses the temporary credential before TTL expires?​

Vault revokes the credential after the TTL expires (typically 1 hour), so the leaked credential is automatically invalidated. This is far better than a static password that persists forever. Always use short TTLs (1–24 hours) for dynamic credentials.

Can I use Vault to generate SSH keys instead of passwords?​

Yes. Vault's SSH secrets engine generates ephemeral SSH key pairs with built-in principal constraints. Your application requests a key pair, uses it to connect to a server, and Vault revokes the key after the TTL. This is more secure than managing SSH keys manually.

Does Vault work with Kubernetes?​

Yes. Enable the Kubernetes auth method and use Vault in a sidecar or init container. Kubernetes ServiceAccounts authenticate directly to Vault, eliminating the need to manage AppRole credentials. See the Vault Kubernetes guide.

How do I integrate Vault with Pydantic settings?​

Similar to AWS Secrets Manager: use a validator that calls client.secrets.database.read_dynamic_credentials() and returns the credential. Combine with local .env fallback for development.

Further Reading​