API Keys and Authentication Tokens Storage
API keys and authentication tokens are credentials that grant access to third-party services (Stripe, OpenAI, AWS, GitHub). Unlike database passwords, which are rarely rotated and persist for years, API keys should be treated as short-lived tokens, rotated frequently, and revoked when no longer needed. A stolen API key for Stripe or OpenAI can cost you thousands of dollars in unauthorized charges within minutes. This article covers secure key storage patterns, token rotation strategies, refresh token flows, and how to prevent keys from leaking in logs, error messages, or memory.
Storing API Keys Securely
Never hardcode API keys or commit them to version control. Store them in environment variables or secret managers:
import os
from typing import Optional
class APIClient:
def __init__(self):
# Read from environment (or secret manager)
self.api_key = os.getenv("STRIPE_API_KEY")
if not self.api_key:
raise ValueError("STRIPE_API_KEY environment variable not set")
self.api_url = "https://api.stripe.com/v1"
def charge_card(self, amount: int, token: str) -> dict:
"""
Charge a card using Stripe API.
Never log the api_key or token.
"""
import requests
response = requests.post(
f"{self.api_url}/charges",
auth=(self.api_key, ""), # HTTP Basic Auth with key as username
data={"amount": amount, "source": token, "currency": "usd"}
)
return response.json()
# Usage
client = APIClient()
result = client.charge_card(1000, "tok_visa")
For multiple API keys (different providers, environments), use a configuration class:
from pydantic import Field, field_validator
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
stripe_api_key: str = Field(description="Stripe API secret key")
openai_api_key: str = Field(description="OpenAI API key")
github_token: str = Field(description="GitHub personal access token")
@field_validator("stripe_api_key", mode="before")
@classmethod
def validate_stripe_key(cls, v):
if v and not v.startswith("sk_"):
raise ValueError("Invalid Stripe API key format")
return v
settings = Settings()
# Keys are now typed and validated
stripe_client = stripe.Client(api_key=settings.stripe_api_key)
openai_client = openai.OpenAI(api_key=settings.openai_api_key)
Rotating API Keys Without Downtime
API keys should be rotated every 30–90 days. Most services (Stripe, AWS, GitHub) allow multiple active keys simultaneously, enabling zero-downtime rotation:
- Generate a new API key in the service dashboard.
- Add it to your application's configuration.
- Update all client instances to use the new key.
- Revoke the old key.
For gradual migration (canary rotation):
from typing import List
import random
class APIClientWithRotation:
def __init__(self, active_keys: List[str], old_keys: List[str] = None):
self.active_keys = active_keys
self.old_keys = old_keys or []
self.current_key_index = 0
def _get_next_key(self):
"""Round-robin through active keys for load distribution."""
self.current_key_index = (self.current_key_index + 1) % len(self.active_keys)
return self.active_keys[self.current_key_index]
def call_api(self, endpoint: str, data: dict) -> dict:
"""
Call API with automatic fallback to old keys on 401.
"""
import requests
keys_to_try = self.active_keys + self.old_keys
for key in keys_to_try:
try:
response = requests.post(
f"https://api.example.com{endpoint}",
headers={"Authorization": f"Bearer {key}"},
json=data,
timeout=10
)
if response.status_code == 200:
return response.json()
elif response.status_code == 401:
# Key is invalid, try next
continue
else:
response.raise_for_status()
except requests.RequestException:
# Network error, try next key
continue
raise RuntimeError("All API keys failed. Check key validity and IP allowlists.")
# Usage
old_key = os.getenv("STRIPE_API_KEY_OLD")
new_key = os.getenv("STRIPE_API_KEY_NEW")
client = APIClientWithRotation(
active_keys=[new_key],
old_keys=[old_key] if old_key else []
)
result = client.call_api("/charges", {"amount": 1000})
Once all traffic is using the new key, remove old keys from the system.
Handling Refresh Tokens and Bearer Tokens
Many OAuth 2.0 services (Google, GitHub) use refresh tokens (long-lived) to obtain access tokens (short-lived). Your application should:
- Store the refresh token securely (secret manager).
- Periodically request new access tokens.
- Cache access tokens in memory with a TTL.
import requests
import time
from typing import Optional
class OAuthTokenManager:
def __init__(self, client_id: str, client_secret: str, refresh_token: str):
self.client_id = client_id
self.client_secret = client_secret
self.refresh_token = refresh_token
self._access_token: Optional[str] = None
self._token_expiry: float = 0
def get_access_token(self) -> str:
"""
Get a valid access token, refreshing if needed.
"""
now = time.time()
# Return cached token if still valid (with 5-minute buffer)
if self._access_token and now < self._token_expiry - 300:
return self._access_token
# Request new access token
response = requests.post(
"https://oauth.googleapis.com/token",
data={
"client_id": self.client_id,
"client_secret": self.client_secret,
"refresh_token": self.refresh_token,
"grant_type": "refresh_token"
}
)
if response.status_code != 200:
raise RuntimeError(f"Failed to refresh access token: {response.text}")
data = response.json()
self._access_token = data["access_token"]
self._token_expiry = now + data.get("expires_in", 3600)
return self._access_token
def call_api(self, url: str, method: str = "GET", **kwargs) -> dict:
"""
Call an API with automatic token refresh.
"""
token = self.get_access_token()
headers = kwargs.pop("headers", {})
headers["Authorization"] = f"Bearer {token}"
response = requests.request(
method=method,
url=url,
headers=headers,
**kwargs
)
if response.status_code == 401:
# Token expired despite cache; force refresh
self._token_expiry = 0
token = self.get_access_token()
headers["Authorization"] = f"Bearer {token}"
response = requests.request(method=method, url=url, headers=headers, **kwargs)
response.raise_for_status()
return response.json()
# Usage
token_manager = OAuthTokenManager(
client_id=os.getenv("GOOGLE_CLIENT_ID"),
client_secret=os.getenv("GOOGLE_CLIENT_SECRET"),
refresh_token=os.getenv("GOOGLE_REFRESH_TOKEN")
)
user_data = token_manager.call_api("https://www.googleapis.com/oauth2/v2/userinfo")
Preventing Key Leaks in Logs and Error Messages
API keys often leak in error messages, stack traces, or verbose logging. Implement automatic redaction:
import logging
import re
from typing import Any
class RedactingFilter(logging.Filter):
"""Filter that redacts API keys, tokens, and passwords from logs."""
PATTERNS = [
(r"api[_-]?key[=:\s]+([^\s,;'\"]+)", "api_key=***"),
(r"token[=:\s]+([^\s,;'\"]+)", "token=***"),
(r"password[=:\s]+([^\s,;'\"]+)", "password=***"),
(r"authorization[:\s]+Bearer\s+([^\s,;'\"]+)", "Authorization: Bearer ***"),
(r"sk_[a-zA-Z0-9]{20,}", "sk_***"), # Stripe-like keys
]
def filter(self, record: logging.LogRecord) -> bool:
"""Redact sensitive data from log records."""
record.msg = self._redact(str(record.msg))
if record.args:
if isinstance(record.args, dict):
record.args = {k: self._redact(str(v)) for k, v in record.args.items()}
elif isinstance(record.args, tuple):
record.args = tuple(self._redact(str(arg)) for arg in record.args)
return True
@staticmethod
def _redact(text: str) -> str:
"""Redact sensitive patterns from text."""
for pattern, replacement in RedactingFilter.PATTERNS:
text = re.sub(pattern, replacement, text, flags=re.IGNORECASE)
return text
# Apply to all loggers
redacting_filter = RedactingFilter()
for handler in logging.root.handlers:
handler.addFilter(redacting_filter)
Never log API responses directly (they often contain keys). Log only summary information:
import logging
logger = logging.getLogger(__name__)
def call_external_api(key: str, data: dict) -> dict:
import requests
response = requests.post(
"https://api.example.com/endpoint",
headers={"Authorization": f"Bearer {key}"},
json=data
)
# WRONG: logs the entire response which may contain secrets
# logger.info(f"API response: {response.json()}")
# CORRECT: log only safe metadata
logger.info(
f"API call succeeded",
extra={"status_code": response.status_code, "request_id": response.headers.get("x-request-id")}
)
return response.json()
API Key Policies and Scope Limitation
Modern API key systems support scopes and permissions. Always request the minimum scope needed:
# Stripe: Request only necessary permissions
stripe_api_key = "sk_live_..." # Full access—avoid if possible
# GitHub: Use personal access token with limited scopes
github_token_with_public_repo = "ghp_..." # Can read/write public repos only
# Google OAuth: Request only 'userinfo' scope, not all Gmail access
google_scopes = ["https://www.googleapis.com/auth/userinfo.profile"]
# AWS: Use IAM roles with least-privilege policies
# Example: S3 read-only access, not full admin
Comparison: Key Storage Methods
| Method | Security | Rotation | Complexity | Best For |
|---|---|---|---|---|
| Hardcoded | Worst | Manual | Low | Never |
| Environment variables | Good | Requires redeploy | Low | Development, small projects |
| .env file | Fair | Requires manual update | Low | Local development |
| Secret Manager (static) | Excellent | Manual or automatic | Medium | Production |
| OAuth 2.0 refresh tokens | Excellent | Automatic | High | Third-party services |
Key Takeaways
- Store API keys in environment variables or secret managers, never hardcode them; rotate keys every 30–90 days.
- Implement a token manager that caches access tokens and automatically refreshes them before expiration.
- Use refresh tokens for long-lived OAuth flows; keep them encrypted in a secret manager.
- Implement redacting log filters to prevent keys from appearing in error messages or logs.
- Request minimum scopes and permissions when creating API keys.
Frequently Asked Questions
How do I detect if an API key has been compromised?
Most services provide audit logs or suspicious activity alerts. GitHub shows all personal access token uses with timestamps. Stripe sends emails on unusual charges. AWS CloudTrail logs API calls. Monitor these regularly. If a key is compromised, revoke it immediately and rotate to a new one.
Can I rotate API keys during running requests?
Yes, if you maintain multiple active keys (old and new) simultaneously. Route new requests to the new key and let old requests complete with the old key. Once all old requests finish, revoke the old key. This takes minutes to hours depending on average request duration.
Should I commit .env files with dummy API keys?
Yes, commit .env.example with placeholder keys (like STRIPE_API_KEY=sk_test_example), but never commit the real .env file. This lets developers know which keys they need to set up and what format they should be in.
How do I safely pass API keys to background jobs or cron tasks?
Use the same environment variable or secret manager approach as your main application. For Celery: pass the secret name (not the key) to the task, then fetch it inside the task. Never pass the key as a task argument—task logs may expose it.
What if I accidentally logged an API key?
Immediately revoke the key in the service dashboard and rotate to a new one. Search your log aggregation system (CloudWatch, Splunk, ELK) for any queries made with that key to detect unauthorized usage. For public leaks (GitHub, Stack Overflow), contact the service provider's security team—many offer key revocation before the leak spreads.