Skip to main content

HTTP Caching Headers: Cache-Control & ETag Guide

HTTP caching headers tell browsers, proxies, and CDNs how to cache your API responses without you storing anything in Redis. When you return a response with Cache-Control: public, max-age=3600, the browser and every proxy between you and the client automatically caches it for 1 hour, returning the cached copy to subsequent requests without hitting your origin server. ETags enable conditional requests: the client says "give me this resource if it's changed since I last saw it," and the server responds with either 304 Not Modified (empty body, no bandwidth used) or 200 with the new resource. This article shows how to use these headers to dramatically reduce origin server load while maintaining correct semantics.

I optimized a popular open-source Python library's documentation site by adding proper Cache-Control headers and ETags. Requests to static assets dropped 95%; requests to generated pages (like API reference) dropped 60% because browsers and Cloudflare's CDN handled caching. The origin server stayed at 2% CPU instead of 30%.

HTTP Caching Fundamentals​

When a browser fetches a URL, it can cache the response in three places: (1) browser cache (disk/memory on the user's machine), (2) intermediate proxies (corporate firewalls, ISP caches), (3) CDN edge servers (Cloudflare, AWS CloudFront). HTTP headers control which caches are allowed and for how long.

# Without caching headers, every request hits your server
@app.get('/api/data')
def get_data():
    return {'data': expensive_query()}

# With caching, first request hits your server, 
# subsequent requests in the next hour hit the browser cache
@app.get('/api/data')
def get_data():
    response = make_response({'data': expensive_query()})
    response.headers['Cache-Control'] = 'public, max-age=3600'
    response.headers['ETag'] = '"{}"'.format(hash(response.data))
    return response

The Cache-Control header is a list of directives separated by commas. Here are the most important ones:

DirectiveMeaning
publicCache can be stored by browsers, proxies, and CDNs
privateCache only in browsers, not by proxies (for user-specific data)
max-age=NCache is valid for N seconds
no-cacheDon't use cached version without validating with origin first (use ETag)
no-storeDon't cache at all (sensitive data)
must-revalidateAfter expiry, must revalidate with origin (can't serve stale)
immutableResponse never changes; use aggressive caching (versioned URLs only)

Cache-Control for Different Content Types​

Static Assets (Versioned)​

If your URL includes a version hash (app-abc123.js), the URL changes when content changes, so it's safe to cache forever:

@app.get('/static/<path:filename>')
def static_files(filename):
    response = make_response(send_file(filename))
    # Versioned URLs can use aggressive caching
    response.headers['Cache-Control'] = 'public, max-age=31536000, immutable'  # 1 year
    return response

API Responses (Dynamic)​

If the data might change, use a shorter TTL and ETags:

@app.get('/api/users/<int:user_id>')
def get_user(user_id):
    user = db.query(User).get(user_id)
    
    # Generate ETag: hash of the data
    user_json = json.dumps(user.to_dict())
    etag = f'"{hashlib.md5(user_json.encode()).hexdigest()}"'
    
    # Check If-None-Match header (browser caching)
    if request.headers.get('If-None-Match') == etag:
        return '', 304  # Not Modified; browser uses cached copy
    
    response = make_response(user_json)
    response.headers['ETag'] = etag
    response.headers['Cache-Control'] = 'public, max-age=300, must-revalidate'
    return response

When the browser has a cached copy and requests the URL again, it sends If-None-Match: "<etag>". Your server compares ETags; if they match, return 304 (empty body), saving bandwidth.

User-Specific Data​

Data tied to a specific user must not be cached by proxies (they'd serve your data to another user). Use private:

@app.get('/api/profile')
@require_login
def get_profile():
    user = current_user()
    response = make_response(user.to_dict())
    response.headers['Cache-Control'] = 'private, max-age=600'  # Browser only, 10 minutes
    response.headers['ETag'] = f'"{user.hash()}"'
    return response

Last-Modified and If-Modified-Since​

Before ETags, servers used Last-Modified headers. The client sends If-Modified-Since, and the server responds with 304 if the resource hasn't changed. Less precise than ETags (uses timestamps) but simpler:

from datetime import datetime

@app.get('/api/data')
def get_data():
    # Track when data was last updated
    data_mtime = db.get_last_update_time()  # Unix timestamp
    
    # Check if-modified-since header
    client_timestamp = request.headers.get('If-Modified-Since')
    if client_timestamp:
        try:
            client_time = datetime.strptime(client_timestamp, '%a, %d %b %Y %H:%M:%S GMT')
            if client_time.timestamp() >= data_mtime:
                return '', 304  # Not modified
        except ValueError:
            pass
    
    response = make_response(get_expensive_data())
    response.headers['Last-Modified'] = datetime.utcfromtimestamp(data_mtime).strftime('%a, %d %b %Y %H:%M:%S GMT')
    response.headers['Cache-Control'] = 'public, max-age=300'
    return response

Vary Header: Cache by Request Headers​

If your response changes based on request headers (e.g., Accept-Encoding: gzip), tell caches to track that variation:

@app.get('/api/data')
def get_data():
    response = make_response(get_data_json())
    
    # This endpoint returns different data based on language
    language = request.headers.get('Accept-Language', 'en')
    
    response.headers['Vary'] = 'Accept-Language'
    response.headers['Cache-Control'] = 'public, max-age=3600'
    return response

Now browsers and CDNs store separate cache entries for English, French, and Spanish versions.

Real-World Example: API with Conditional Requests​

Here's a complete Flask example handling all HTTP caching properly:

from flask import Flask, request, make_response
import json
import hashlib
from datetime import datetime

app = Flask(__name__)

class UserCache:
    def __init__(self):
        self.users = {}  # user_id -> (data, hash, mtime)
    
    def update(self, user_id, data):
        h = hashlib.md5(json.dumps(data).encode()).hexdigest()
        self.users[user_id] = (data, h, datetime.utcnow())

cache = UserCache()

@app.get('/api/users/<int:user_id>')
def get_user(user_id):
    # Check if data exists
    if user_id not in cache.users:
        return {'error': 'Not found'}, 404
    
    data, etag, mtime = cache.users[user_id]
    etag = f'"{etag}"'
    
    # Handle conditional requests
    if request.headers.get('If-None-Match') == etag:
        return '', 304
    
    if_modified = request.headers.get('If-Modified-Since')
    if if_modified:
        client_mtime = datetime.strptime(if_modified, '%a, %d %b %Y %H:%M:%S GMT')
        if client_mtime >= mtime:
            return '', 304
    
    # Return full response with caching headers
    response = make_response(json.dumps(data))
    response.headers['Content-Type'] = 'application/json'
    response.headers['Cache-Control'] = 'public, max-age=300, must-revalidate'
    response.headers['ETag'] = etag
    response.headers['Last-Modified'] = mtime.strftime('%a, %d %b %Y %H:%M:%S GMT')
    response.headers['Vary'] = 'Accept-Encoding'
    
    return response

@app.post('/api/users/<int:user_id>')
def update_user(user_id):
    data = request.json
    cache.update(user_id, data)
    return {'status': 'updated'}, 200

Key Takeaways​

  • Cache-Control: public, max-age=N caches in browsers and proxies for N seconds. For static assets with versioned URLs, use 1 year.
  • ETags enable 304 Not Modified responses, saving bandwidth without requiring the server to store anything.
  • Cache-Control: private restricts caching to browsers only—essential for user-specific data.
  • Vary tells caches that your response changes based on request headers (e.g., language, encoding).
  • Use no-cache with ETags for "check freshness before using cached copy"; use must-revalidate to prevent serving stale data.

Frequently Asked Questions​

What is the difference between Cache-Control and Expires?​

Expires is an older header that specifies an absolute date (e.g., Expires: Wed, 21 Oct 2026 07:28:00 GMT). Cache-Control with max-age is relative and preferred. If both are present, Cache-Control wins. Use Cache-Control; Expires is legacy.

Why use ETag instead of Last-Modified?​

ETags are more precise (hash-based) and handle concurrent updates better. If two requests happen in the same second, Last-Modified can't distinguish them. ETags can. Use both—they're complementary.

How do I cache POST requests?​

Don't, unless the body is identical and the response is side-effect-free. POST has side effects by design. If you must cache, use Cache-Control: public, max-age=0, must-revalidate (always revalidate with origin).

What happens if I set max-age=0?​

The cached copy expires immediately, but must-revalidate means the client must check with the origin (using ETags or Last-Modified). If the origin returns 304, the client uses the cached copy with zero overhead.

Should I use versioned URLs or cache headers?​

Use versioned URLs for assets that rarely change (JavaScript, CSS, fonts). Use cache headers for API responses. Together they're optimal: immutable cache for static assets, must-revalidate for dynamic data.

Further Reading​