Resource cleanup exception handling tutorial
Resource cleanup is the primary purpose of context managers. A context manager guarantees that cleanup code runs, even if the with block raises an exception or returns early. This deterministic cleanup prevents resource leaks, file handle exhaustion, database connection saturation, and memory pressure — the silent killers of long-running applications.
Exception handling in context managers is subtle: the __exit__ method receives exception information and must decide whether to suppress or propagate the exception. Understanding exception semantics is essential for writing context managers that fail safely and provide useful debugging information. This article explores cleanup patterns, exception propagation, exception chaining, and defensive strategies.
Guaranteeing Cleanup: The Core Guarantee
The fundamental contract of a context manager is that __exit__ is called unconditionally when exiting the with block. This guarantee holds for normal execution, early returns, exceptions, and even KeyboardInterrupt. This makes context managers fundamentally more reliable than manual try-finally patterns.
# Example 1: Cleanup runs in all exit paths
import sys
class ResourceTracker:
def __init__(self, name):
self.name = name
def __enter__(self):
print(f"{self.name}: acquired")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print(f"{self.name}: cleaned up")
return False
print("--- Normal exit ---")
with ResourceTracker("Resource1"):
print("Using resource")
print("\n--- Early return ---")
def early_return():
with ResourceTracker("Resource2"):
print("About to return")
return "result"
# Cleanup still runs before returning
result = early_return()
print(f"Returned: {result}")
print("\n--- Exception exit ---")
try:
with ResourceTracker("Resource3"):
print("About to raise")
raise ValueError("Error in block")
except ValueError:
print("Exception caught")
# Output:
# --- Normal exit ---
# Resource1: acquired
# Using resource
# Resource1: cleaned up
#
# --- Early return ---
# Resource2: acquired
# About to return
# Resource2: cleaned up
# Returned: result
#
# --- Exception exit ---
# Resource3: acquired
# About to raise
# Resource3: cleaned up
# Exception caught
Exception Suppression: Suppress or Propagate?
The __exit__ method receives three arguments describing any exception that occurred: exc_type, exc_val, and exc_tb. If no exception occurred, all three are None. The return value of __exit__ determines whether the exception is suppressed (return True) or propagated (return False or None).
Most context managers should not suppress exceptions — exceptions indicate errors that the caller should know about. However, in rare cases, a context manager is specifically designed to absorb certain exceptions. For example, a context manager for temporary directories might suppress FileNotFoundError when cleaning up an already-deleted directory.
# Example 2: Selective exception suppression
class LoggingContextWithSuppression:
def __init__(self, suppress_errors=False):
self.suppress_errors = suppress_errors
def __enter__(self):
print("Entering")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None:
print(f"Exception detected: {exc_type.__name__}: {exc_val}")
if self.suppress_errors and exc_type in (ValueError, KeyError):
print("Suppressing exception")
return True # Suppress
else:
print("Propagating exception")
return False # Propagate
else:
print("Normal exit")
return False
print("--- Exception with suppression enabled ---")
with LoggingContextWithSuppression(suppress_errors=True):
raise ValueError("This will be suppressed")
print("Continued after exception")
print("\n--- Exception with suppression disabled ---")
try:
with LoggingContextWithSuppression(suppress_errors=False):
raise ValueError("This will propagate")
except ValueError:
print("Exception caught by caller")
# Output:
# --- Exception with suppression enabled ---
# Entering
# Exception detected: ValueError: This will be suppressed
# Suppressing exception
# Continued after exception
#
# --- Exception with suppression disabled ---
# Entering
# Exception detected: ValueError: This will propagate
# Propagating exception
# Exception caught by caller
Defensive Cleanup: Handling Errors During Cleanup
__exit__ itself can raise exceptions, but this is dangerous: it replaces the original exception and may hide the root cause. Defensive cleanup should catch and handle any errors that might occur during resource cleanup, logging them without propagating.
# Example 3: Defensive cleanup with error handling
import io
class RobustResource:
def __init__(self):
self.resource = None
self.log = io.StringIO()
def __enter__(self):
self.resource = "open resource"
print(f"Resource acquired: {self.resource}")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
try:
if self.resource is not None:
print(f"Closing resource: {self.resource}")
# Simulate cleanup that might fail
if self.resource == "broken":
raise IOError("Cleanup failed")
self.resource = None
except Exception as e:
# Log the cleanup error, but don't propagate it
# unless it's more important than the original error
print(f"Warning: cleanup failed with {type(e).__name__}: {e}")
if exc_type is None:
# No original exception, so propagate cleanup error
raise
else:
# Original exception exists; log cleanup error but propagate original
return False
return False
print("--- Normal cleanup ---")
with RobustResource() as res:
print("Using resource")
print("\n--- Cleanup error with no original exception ---")
try:
resource = RobustResource()
resource.resource = "broken"
with resource:
print("Using resource")
except IOError as e:
print(f"Caught cleanup error: {e}")
# Output:
# --- Normal cleanup ---
# Resource acquired: open resource
# Using resource
# Closing resource: open resource
#
# --- Cleanup error with no original exception ---
# Resource acquired: broken resource
# Using resource
# Closing resource: broken resource
# Warning: cleanup failed with IOError: Cleanup failed
# Caught cleanup error: Cleanup failed
Exception Chaining and Context
When an exception occurs in the with block and __exit__ decides to suppress it or raise a different exception, Python's implicit exception chaining preserves the original exception as __context__ or __cause__. This helps debugging: you see both the original and the new exception.
# Example 4: Exception chaining for better debugging
class ChainedContextManager:
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is ValueError:
# Raise a new exception while preserving the original
raise RuntimeError("Cleanup encountered an error") from exc_val
return False
print("--- Exception chaining ---")
try:
with ChainedContextManager():
raise ValueError("Original error")
except RuntimeError as e:
print(f"Caught: {e}")
print(f"Caused by: {e.__cause__}")
# Output:
# --- Exception chaining ---
# Caught: Cleanup encountered an error
# Caused by: Original error
Cleanup Patterns: Resource Types
Different resource types require different cleanup patterns. Files should be closed; database connections should be disconnected; locks should be released; temporary files should be deleted; and timers should be stopped.
| Resource Type | Acquisition | Cleanup |
|---|---|---|
| File | open() | close() |
| Database | connect() | disconnect() |
| Lock | acquire() | release() |
| Temp Directory | create() | remove() with error handling |
| Timer | start() | cancel() |
| Transaction | begin() | commit() or rollback() |
# Example 5: Real-world cleanup pattern for a file wrapper
class SafeFileWriter:
def __init__(self, filename):
self.filename = filename
self.file = None
self.bytes_written = 0
def __enter__(self):
print(f"Opening {self.filename} for writing")
self.file = open(self.filename, 'w')
return self
def write(self, data):
if self.file is None:
raise ValueError("File not open")
self.file.write(data)
self.bytes_written += len(data)
def __exit__(self, exc_type, exc_val, exc_tb):
try:
if self.file is not None:
print(f"Flushing {self.bytes_written} bytes")
self.file.flush()
self.file.close()
self.file = None
if exc_type is not None:
print(f"Exception: {exc_type.__name__}")
except Exception as cleanup_error:
print(f"Cleanup failed: {cleanup_error}")
if exc_type is None:
raise cleanup_error
return False
import tempfile
import os
# Create a temporary file for testing
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as tmp:
tmp_name = tmp.name
try:
with SafeFileWriter(tmp_name) as f:
f.write("Hello, ")
f.write("World!")
finally:
os.unlink(tmp_name)
# Output:
# Opening /path/to/tmp.txt for writing
# Flushing 13 bytes
Key Takeaways
- Context managers guarantee cleanup by calling
__exit__unconditionally, even on exceptions, early returns, orKeyboardInterrupt. __exit__receives exception information and must decide whether to suppress (returnTrue) or propagate (returnFalse) the exception.- Most context managers should not suppress exceptions; let them propagate so callers can respond appropriately.
- Defensive cleanup handles errors during
__exit__itself, logging them without replacing the original exception. - Exception chaining via
raise ... from excpreserves the original exception, improving debugging.
Frequently Asked Questions
If exit raises an exception, does the original exception get hidden?
Yes. If __exit__ raises a new exception, it replaces the original exception. To preserve both, use raise new_exception from original_exception for explicit chaining, which stores the original in __cause__.
Should I use finally inside exit?
Yes, if __exit__ has multiple steps. Use try-finally inside __exit__ to ensure the final cleanup step (like file close) always runs, even if earlier steps fail.
How do I log cleanup errors without suppressing them?
Catch the exception in __exit__, log it, and then decide: if there's no original exception from the with block, propagate the cleanup error by re-raising it. If there is an original exception, log the cleanup error and return False to propagate the original.
Can I inspect the original exception to decide cleanup behavior?
Yes. The exc_type, exc_val, and exc_tb arguments give you full access to the original exception. You can inspect its type, message, or traceback to make cleanup decisions — for example, rollback on transaction errors but commit on success.
What is the difference between context and cause?
__context__ is set implicitly when an exception occurs during exception handling (implicit chaining). __cause__ is set explicitly via raise ... from ... (explicit chaining). Both preserve the exception chain for debugging.