Skip to main content

Thread Synchronization: Locks and Thread Safety

A race condition occurs when multiple threads access shared data without coordination, resulting in unpredictable behavior or data corruption. Synchronization primitives like locks (mutexes) prevent race conditions by ensuring only one thread can modify shared data at a time. Python provides threading.Lock for mutual exclusion, RLock for reentrant code, and Condition for more complex coordination patterns.

I spent a full week debugging a production memory leak caused by a race condition in a multi-threaded caching layer—two threads incremented a counter simultaneously, overwriting each other's updates. This article teaches you the synchronization patterns that prevent such disasters.

Understanding Race Conditions​

A race condition occurs when the correctness of a program depends on the relative timing of events in different threads. Consider incrementing a shared counter:

import threading

counter = 0

def increment_counter(iterations):
    """Increment the global counter without synchronization."""
    global counter
    for _ in range(iterations):
        # This operation is NOT atomic
        counter += 1

# Without synchronization, final counter value is unpredictable
threads = [
    threading.Thread(target=increment_counter, args=(1_000_000,))
    for _ in range(4)
]

for t in threads:
    t.start()
for t in threads:
    t.join()

print(f"Counter: {counter} (expected 4,000,000)")
# Output: Counter: 1,234,567 (varies each run—race condition!)

The problem is that counter += 1 is not atomic. Under the hood, it's three operations: read the current value, add 1, write the new value back. If two threads interleave their reads and writes, they can overwrite each other's increments.

Thread 1 reads counter (value 5), adds 1 to get 6. Before thread 1 writes, thread 2 reads counter (still 5), adds 1 to get 6. Both threads write 6. The final value is 6, but it should be 7—an update was lost.

The Lock Pattern​

A lock (mutex) is a synchronization primitive that allows only one thread to hold it at a time. To protect shared data, you acquire the lock before accessing the data and release it afterward. Python's threading.Lock provides this:

import threading

counter = 0
counter_lock = threading.Lock()

def increment_counter_safely(iterations):
    """Increment the global counter with synchronization."""
    global counter
    for _ in range(iterations):
        with counter_lock:  # Acquire the lock (blocks if held by another thread)
            counter += 1     # Now only this thread can execute here
                             # Release happens automatically when exiting the with block

threads = [
    threading.Thread(target=increment_counter_safely, args=(1_000_000,))
    for _ in range(4)
]

for t in threads:
    t.start()
for t in threads:
    t.join()

print(f"Counter: {counter}")
# Output: Counter: 4,000,000 (correct every time)

The with statement (context manager) handles acquisition and release automatically. When a thread enters the with counter_lock: block, it tries to acquire the lock. If the lock is held by another thread, the current thread blocks until the lock is available. Once acquired, only that thread can execute the indented block. When the block exits, the lock is released, and waiting threads wake up.

Critical Sections and Lock Scope​

A critical section is a block of code that must be executed by only one thread at a time. Keep critical sections as small as possible—longer critical sections reduce concurrency:

import threading
import time

class BankAccount:
    def __init__(self, balance):
        self.balance = balance
        self.lock = threading.Lock()
    
    def withdraw_carefully(self, amount):
        """Good: small critical section."""
        with self.lock:
            if self.balance >= amount:
                self.balance -= amount
                return True
        return False
    
    def withdraw_poorly(self, amount):
        """Bad: large critical section, doing I/O while holding the lock."""
        with self.lock:
            # WRONG: Never do I/O or expensive operations inside a lock
            time.sleep(1)  # Blocking I/O blocks all other threads from accessing balance
            if self.balance >= amount:
                self.balance -= amount
                return True
        return False

# Good design: multiple threads can check balance concurrently
# Bad design: even reading the balance blocks while time.sleep() holds the lock

The poor pattern is a classic deadlock/contention hazard. Always minimize the time a thread holds a lock.

Reentrant Locks (RLock)​

Sometimes a thread needs to acquire the same lock multiple times. A regular Lock will deadlock if the same thread tries to acquire it twice:

import threading

lock = threading.Lock()

def deadlock_example():
    """This code will deadlock."""
    with lock:
        print("First acquisition successful")
        # Now try to acquire the same lock again (from the same thread)
        with lock:  # DEADLOCK: This thread is waiting for a lock it already holds
            print("Second acquisition successful")

thread = threading.Thread(target=deadlock_example)
thread.start()
# thread hangs forever

For recursive patterns where a function calls itself (or calls other functions that also need the lock), use RLock (reentrant lock):

import threading

rlock = threading.RLock()

def recursive_function(depth):
    """Recursively acquire the same lock."""
    with rlock:
        print(f"Depth {depth}")
        if depth > 0:
            recursive_function(depth - 1)

thread = threading.Thread(target=recursive_function, args=(3,))
thread.start()
thread.join()
# Output:
# Depth 3
# Depth 2
# Depth 1
# Depth 0

RLock tracks which thread holds it and allows the same thread to acquire it multiple times. Other threads block until the owning thread releases all its acquisitions.

Condition Variables for Coordination​

A Condition variable allows threads to wait until a specific condition is true, then wake up and proceed. This is useful for producer-consumer patterns:

import threading
import time
from collections import deque

queue = deque()
queue_lock = threading.Condition()

def producer():
    """Produce items and notify consumers."""
    for item in range(5):
        time.sleep(0.5)
        with queue_lock:
            queue.append(item)
            print(f"Produced: {item}")
            queue_lock.notify()  # Wake up one waiting consumer

def consumer():
    """Consume items when available."""
    while True:
        with queue_lock:
            # Wait until the queue is not empty
            while not queue:
                queue_lock.wait()  # Release lock and sleep; re-acquire when notified
            item = queue.popleft()
            print(f"Consumed: {item}")

prod = threading.Thread(target=producer, daemon=True)
cons = threading.Thread(target=consumer, daemon=True)

prod.start()
cons.start()
time.sleep(5)

The wait() method releases the lock and sleeps until another thread calls notify(). The notify() method wakes one waiting thread (or notify_all() for all). This is essential for producer-consumer queues and event-driven systems.

Common Synchronization Mistakes​

Holding a lock while acquiring another lock can cause deadlock if two threads try to acquire locks in opposite orders:

# Mistake: Lock ordering
lock_a = threading.Lock()
lock_b = threading.Lock()

def thread1():
    with lock_a:
        with lock_b:  # Acquires locks in order A, B
            pass

def thread2():
    with lock_b:
        with lock_a:  # Acquires locks in opposite order B, A
            pass      # Deadlock: thread1 waits for B while holding A;
                      # thread2 waits for A while holding B

Solution: Always acquire locks in the same order.

Comparison Table: Synchronization Primitives​

PrimitiveUse CaseReentrantOverhead
LockProtecting shared dataNoLow
RLockRecursive functions, complex stateYesLow
ConditionProducer-consumer, signalingN/AMedium
SemaphoreLimiting concurrent accessNoLow
EventOne-way signaling (simple condition)N/AVery low

Key Takeaways​

  • A race condition occurs when threads access shared data without synchronization, causing unpredictable behavior.
  • Use with lock: to protect critical sections; keep them as small as possible.
  • RLock allows the same thread to acquire a lock multiple times; use it for recursive or nested lock patterns.
  • Condition variables coordinate threads with wait/notify semantics.
  • Always acquire locks in the same order across all threads to prevent deadlock.

Frequently Asked Questions​

What happens if a thread tries to acquire a lock that's held by another thread?​

The thread blocks (suspends execution) until the lock is released. This is the whole point: the thread waits its turn, then proceeds with exclusive access.

Is it safe to access a shared variable without a lock if all accesses are reads?​

In CPython, many read operations appear atomic due to the GIL, but it's not guaranteed across Python implementations. For portability and clarity, always use locks for shared data, even if reads seem atomic.

How do I detect a deadlock in my code?​

Use a debugger to inspect thread stacks. If two threads are permanently blocked, look for circular lock acquisition patterns. Python 3.13+ has improved debugging tools; in earlier versions, set a timeout on lock operations as a safety mechanism.

Can a lock be acquired with a timeout?​

Yes, use lock.acquire(timeout=2.0) which returns True if the lock was acquired and False if the timeout expired. However, with lock: does not support timeout; you must use explicit acquire()/release().

Why not just use a global threading.Lock() for everything?​

A single global lock would serialize all threads—only one runs at a time, defeating concurrency. Use fine-grained locks (one per data structure) to maximize parallel execution while protecting individual resources.

Further Reading​