Skip to main content

Exponential Backoff & Retries: Resilient API Clients

Exponential backoff is a retry strategy where failed requests are retried with increasing delays: wait 1 second, then 2 seconds, then 4 seconds, doubling each time. It's the standard pattern for handling transient failures (temporary network hiccups, server temporarily overloaded) without overwhelming the backend. Most production APIs—Stripe, AWS, Google—expect clients to implement backoff. This article shows how to build a bulletproof HTTP client that retries intelligently and respects server signals via Retry-After headers.

I worked on a payment gateway client that made thousands of calls to Stripe daily. Without backoff, network blips caused cascading failures in our retry loop, hammering Stripe and getting our IP rate-limited. Adding exponential backoff with jitter and proper Retry-After parsing reduced our failure rate from 2.3% to 0.04% without changing the backend.

The Problem: Dumb Retries Make Things Worse​

Naive retry logic looks like this:

import requests

def call_api(url):
    for attempt in range(5):
        try:
            response = requests.get(url, timeout=5)
            return response.json()
        except Exception as e:
            if attempt < 4:
                continue  # Retry immediately
            raise

When the API is temporarily overloaded and returns 500 errors, this code retries immediately, five times in quick succession. If 1,000 clients do this simultaneously, the API gets 5,000 requests instead of 1,000, deepening the problem. The server stays overloaded longer, and now legitimate traffic can't get through.

Exponential Backoff: The Solution​

With exponential backoff, retries are spaced out:

import time
import requests

def call_api(url, max_retries=5):
    """Call API with exponential backoff."""
    for attempt in range(max_retries):
        try:
            response = requests.get(url, timeout=5)
            if response.status_code < 500:  # Don't retry client errors
                return response
            # 5xx errors are retryable
        except (requests.Timeout, requests.ConnectionError) as e:
            # Network errors are retryable
            pass
        except Exception as e:
            raise  # Other exceptions are not retryable
        
        if attempt < max_retries - 1:
            # Exponential backoff: 1s, 2s, 4s, 8s, 16s
            wait_time = 2 ** attempt
            print(f"Retry {attempt + 1}: waiting {wait_time}s")
            time.sleep(wait_time)
    
    raise Exception(f"Failed after {max_retries} attempts")

Now the client waits 1, 2, 4, 8, then 16 seconds between retries. If thousands of clients fail simultaneously, they spread their retries across 30 seconds, giving the server time to recover.

Jitter: Preventing Thundering Herd​

There's still a problem: if your service fails, thousands of clients with the same backoff schedule will all retry at the same time (e.g., all retry at second 4). This "thundering herd" can restart the cascade.

Jitter solves this by adding randomness to the delay:

import random

def call_api_with_jitter(url, max_retries=5):
    """Exponential backoff with jitter."""
    for attempt in range(max_retries):
        try:
            response = requests.get(url, timeout=5)
            if response.status_code < 500:
                return response
        except (requests.Timeout, requests.ConnectionError):
            pass
        except Exception:
            raise
        
        if attempt < max_retries - 1:
            # Full jitter: random between 0 and 2^attempt
            base_wait = 2 ** attempt
            wait_time = random.uniform(0, base_wait)
            print(f"Retry {attempt + 1}: waiting {wait_time:.2f}s")
            time.sleep(wait_time)
    
    raise Exception(f"Failed after {max_retries} attempts")

Now each client waits a random amount between 0 and 2^attempt seconds, spreading retries smoothly across the recovery window.

Respecting Retry-After Headers​

Good APIs return a Retry-After header that tells the client exactly when to retry. Always honor it:

import requests
import time
import random

def call_api_with_retry_after(url, max_retries=5):
    """Exponential backoff that respects Retry-After header."""
    for attempt in range(max_retries):
        try:
            response = requests.get(url, timeout=5)
            
            # Success
            if 200 <= response.status_code < 300:
                return response
            
            # Rate limited or server error
            if response.status_code in (429, 500, 502, 503, 504):
                # Extract Retry-After header (may be seconds or HTTP date)
                retry_after = response.headers.get('Retry-After', None)
                if retry_after:
                    try:
                        # Try parsing as integer (seconds)
                        wait_time = int(retry_after)
                    except ValueError:
                        # Try parsing as HTTP date (e.g., 'Wed, 21 Oct 2026 07:28:00 GMT')
                        from email.utils import parsedate_to_datetime
                        try:
                            retry_time = parsedate_to_datetime(retry_after)
                            wait_time = max(0, (retry_time - datetime.utcnow()).total_seconds())
                        except TypeError:
                            wait_time = 2 ** attempt  # Fallback to exponential
                    
                    print(f"Rate limited. Waiting {wait_time}s as instructed")
                    time.sleep(wait_time)
                    continue
            
            # Other client errors (4xx) are not retryable
            if response.status_code >= 400:
                raise Exception(f"Client error: {response.status_code}")
            
            return response
        
        except (requests.Timeout, requests.ConnectionError) as e:
            # Network errors are retryable
            pass
        except Exception:
            raise  # Non-retryable errors
        
        if attempt < max_retries - 1:
            wait_time = random.uniform(0, 2 ** attempt)
            print(f"Retry {attempt + 1}: waiting {wait_time:.2f}s")
            time.sleep(wait_time)
    
    raise Exception(f"Failed after {max_retries} attempts")

Production-Grade HTTP Client with Backoff​

Here's a complete, reusable HTTP client class ready for production:

import requests
import time
import random
import logging
from typing import Optional, Dict

logger = logging.getLogger(__name__)

class ResilientHTTPClient:
    """HTTP client with exponential backoff and smart retry logic."""
    
    RETRYABLE_STATUS_CODES = {408, 429, 500, 502, 503, 504}
    RETRYABLE_EXCEPTIONS = (
        requests.Timeout,
        requests.ConnectionError,
        requests.ChunkedEncodingError,
    )
    
    def __init__(self, max_retries: int = 5, base_wait: float = 1.0, timeout: int = 10):
        self.max_retries = max_retries
        self.base_wait = base_wait
        self.timeout = timeout
        self.session = requests.Session()
    
    def request(self, method: str, url: str, **kwargs) -> requests.Response:
        """Make an HTTP request with exponential backoff."""
        kwargs.setdefault('timeout', self.timeout)
        
        for attempt in range(self.max_retries):
            try:
                response = self.session.request(method, url, **kwargs)
                
                # Success
                if 200 <= response.status_code < 300:
                    return response
                
                # Check if retryable
                if response.status_code not in self.RETRYABLE_STATUS_CODES:
                    response.raise_for_status()  # Raise on 4xx errors
                    return response
                
                # Extract Retry-After if present
                wait_time = self._get_retry_after(response, attempt)
                
                if attempt < self.max_retries - 1:
                    logger.warning(
                        f"Request failed with {response.status_code}. "
                        f"Retry {attempt + 1}/{self.max_retries} in {wait_time:.1f}s"
                    )
                    time.sleep(wait_time)
                else:
                    response.raise_for_status()
            
            except self.RETRYABLE_EXCEPTIONS as e:
                if attempt < self.max_retries - 1:
                    wait_time = random.uniform(0, self.base_wait * (2 ** attempt))
                    logger.warning(
                        f"Request failed with {type(e).__name__}. "
                        f"Retry {attempt + 1}/{self.max_retries} in {wait_time:.1f}s"
                    )
                    time.sleep(wait_time)
                else:
                    raise
        
        raise Exception(f"Request failed after {self.max_retries} attempts")
    
    def _get_retry_after(self, response: requests.Response, attempt: int) -> float:
        """Parse Retry-After header or return exponential backoff."""
        retry_after = response.headers.get('Retry-After')
        
        if not retry_after:
            return random.uniform(0, self.base_wait * (2 ** attempt))
        
        try:
            return float(retry_after)  # Seconds
        except ValueError:
            # Try parsing as HTTP date
            from email.utils import parsedate_to_datetime
            try:
                retry_time = parsedate_to_datetime(retry_after)
                wait_seconds = (retry_time - time.gmtime()).total_seconds()
                return max(0, wait_seconds)
            except (TypeError, ValueError):
                return random.uniform(0, self.base_wait * (2 ** attempt))
    
    def get(self, url: str, **kwargs) -> requests.Response:
        return self.request('GET', url, **kwargs)
    
    def post(self, url: str, **kwargs) -> requests.Response:
        return self.request('POST', url, **kwargs)

# Usage
client = ResilientHTTPClient(max_retries=5)
response = client.get('https://api.example.com/data')

Key Takeaways​

  • Exponential backoff prevents clients from overwhelming a struggling server. Use it for all external API calls.
  • Add jitter (randomness) to prevent the thundering herd problem when thousands of clients retry simultaneously.
  • Always check the Retry-After header; it overrides your backoff calculation.
  • Retry on 5xx errors and transient network failures, not on 4xx client errors (they won't succeed on retry).
  • Set reasonable max_retries (5) and timeout values (10 seconds); very long backlogs indicate a deeper problem.

Frequently Asked Questions​

How many times should I retry?​

5 retries is standard. With exponential backoff (1s, 2s, 4s, 8s, 16s), total wait time is ~31 seconds. If the backend hasn't recovered in that time, it's likely a serious outage; further retries won't help.

Should I retry on 429 (rate limited)?​

Yes, always. 429 means "you're sending too many requests." Back off and the server will accept your retry. Never ignore 429.

What about retrying idempotent requests (GET, DELETE)?​

Safe to retry; they have no side effects. For non-idempotent requests (POST, PATCH), only retry if you know the request is idempotent (i.e., safe to execute twice). See the article on idempotency for more.

Should I retry in a queue (background job) differently?​

Yes. For queue consumers, retries may have longer deadlines (e.g., 1 hour vs. 30 seconds). Use longer max_wait times (2^10 = 1024 seconds) and save the message to a dead-letter queue after max retries.

How do I know if my backoff is working?​

Monitor retry rate and success rate of retried requests. If >50% of requests need retries, your backend is unstable. If <10% of retried requests succeed, your backoff timing is too aggressive.

Further Reading​