Skip to main content

Troubleshooting Common Metaprogramming Pitfalls in Python

Metaprogramming is powerful but easy to misuse; debugging descriptor infinite loops, metaclass conflicts, and decorator order issues can be frustrating without the right tools and patterns. Common pitfalls include accessing self.x inside __getattribute__ (infinite recursion), mixing metaclasses with conflicting __new__ methods (TypeError), applying decorators in the wrong order, and accidentally shadowing class methods with instance attributes. This troubleshooting guide covers the most common mistakes, how to detect them, and practical solutions. Mastering these debugging techniques transforms you from writing metaprogramming code that "sort of works" to confident, production-grade implementations.

Pitfall 1: Infinite Recursion in __getattribute__​

The classic mistake: accessing self.x inside __getattribute__ triggers __getattribute__ again:

class Buggy:
    def __getattribute__(self, name):
        print(f'Getting {name}')
        return self.__dict__.get(name)  # BUG: self.__dict__ calls __getattribute__ again!


# This crashes with RecursionError
try:
    obj = Buggy()
    obj.foo = 'bar'
    print(obj.foo)
except RecursionError as e:
    print(f'RecursionError: too many recursive calls')

Solution: Use object.__getattribute__() to bypass your custom logic:

class Fixed:
    def __getattribute__(self, name):
        print(f'Getting {name}')
        # Use object's __getattribute__ to safely access __dict__
        return object.__getattribute__(self, '__dict__').get(name)


obj = Fixed()
obj.foo = 'bar'
print(obj.foo)  # Output: Getting __dict__\nGetting foo\nbar

General rule: Inside __getattribute__ or __setattr__, always use object.__getattribute__(self, 'attr_name') to access your own state.

Pitfall 2: Descriptor Not Invoked Because Instance Attribute Shadows It​

Data descriptors (with __set__) should take precedence over instance attributes, but non-data descriptors (with only __get__) do not:

class NonDataDescriptor:
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return 'From descriptor'


class Container:
    prop = NonDataDescriptor()


obj = Container()

# First access: descriptor is used
print(obj.prop)  # Output: From descriptor

# Assign an instance attribute (shadows the descriptor)
obj.__dict__['prop'] = 'From instance'

# Now instance attribute takes precedence (non-data descriptor rule)
print(obj.prop)  # Output: From instance

Solution: Make your descriptor a data descriptor by implementing both __get__ and __set__:

class DataDescriptor:
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return 'From descriptor'
    
    def __set__(self, obj, value):
        # Even if __set__ does nothing, its presence makes this a data descriptor
        pass


class Container:
    prop = DataDescriptor()


obj = Container()
obj.__dict__['prop'] = 'From instance'

# Data descriptor still takes precedence
print(obj.prop)  # Output: From descriptor

Diagnostic tool: Check if a descriptor is "data" or "non-data":

def is_data_descriptor(obj):
    return hasattr(obj, '__set__') or hasattr(obj, '__delete__')

print(is_data_descriptor(NonDataDescriptor()))  # Output: False
print(is_data_descriptor(DataDescriptor()))  # Output: True

Pitfall 3: Metaclass Conflict in Multiple Inheritance​

When multiple parent classes have incompatible metaclasses, Python raises TypeError:

class MetaA(type):
    pass

class MetaB(type):
    pass

class A(metaclass=MetaA):
    pass

class B(metaclass=MetaB):
    pass

# TypeError: metaclass conflict
try:
    class C(A, B):
        pass
except TypeError as e:
    print(f'Error: {e}')

Solution: Create a metaclass that inherits from both conflicting metaclasses:

class MetaC(MetaA, MetaB):
    pass

class C(A, B, metaclass=MetaC):
    pass

print(f'C created successfully with metaclass {type(C).__name__}')

Prevention: When designing frameworks with metaclasses, provide a base class with the metaclass already set:

class BaseModel(metaclass=FrameworkMeta):
    pass

# Users only inherit from BaseModel; no conflicts
class User(BaseModel):
    pass

Pitfall 4: __init_subclass__ Not Calling super()​

Forgetting super().__init_subclass__(**kwargs) breaks the chain in multiple inheritance:

class Plugin:
    def __init_subclass__(cls, **kwargs):
        # BUG: not calling super()
        print(f'Subclass {cls.__name__} registered')


class Mixin:
    def __init_subclass__(cls, **kwargs):
        print(f'Mixin subclass {cls.__name__} created')
        super().__init_subclass__(**kwargs)


class MyPlugin(Plugin, Mixin):
    pass
# Output: Subclass MyPlugin registered
# Missing: "Mixin subclass MyPlugin created" because Plugin didn't call super()

Solution: Always call super().__init_subclass__(**kwargs):

class Plugin:
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        print(f'Subclass {cls.__name__} registered')


class MyPlugin(Plugin, Mixin):
    pass
# Output: Mixin subclass MyPlugin created
#         Subclass MyPlugin registered

Pitfall 5: Decorator Order Matters​

Applying decorators in the wrong order produces unexpected results:

def add_repr(cls):
    cls.__repr__ = lambda self: f'Added by add_repr'
    return cls

def add_str(cls):
    cls.__str__ = lambda self: f'Added by add_str'
    return cls

# Order 1: add_repr then add_str
@add_str
@add_repr
class Order1:
    pass

# Order 2: add_str then add_repr
@add_repr
@add_str
class Order2:
    pass

obj1 = Order1()
obj2 = Order2()

print(repr(obj1))  # Output: Added by add_repr
print(str(obj1))   # Output: Added by add_str

print(repr(obj2))  # Output: Added by add_repr
print(str(obj2))   # Output: Added by add_str

In this case, order doesn't matter because the decorators affect different methods. But order matters when decorators modify the same behavior:

def decorator_a(cls):
    original_init = cls.__init__
    def new_init(self, *args, **kwargs):
        print('A')
        original_init(self, *args, **kwargs)
    cls.__init__ = new_init
    return cls

def decorator_b(cls):
    original_init = cls.__init__
    def new_init(self, *args, **kwargs):
        print('B')
        original_init(self, *args, **kwargs)
    cls.__init__ = new_init
    return cls

@decorator_a
@decorator_b
class Ordered:
    def __init__(self):
        print('Init')

obj = Ordered()
# Output: A\nB\nInit
# decorator_a is outermost (called first), decorator_b is inner

Best practice: Document decorator order in docstrings, or use a single "main" decorator that applies sub-decorators in a fixed sequence.

Debugging Tools and Introspection​

Use Python's built-in introspection to understand metaprogramming code:

import inspect

class Example:
    attr = "class attribute"
    
    def method(self):
        pass

# Inspect all attributes and their types
print("\nAll attributes:")
for name, value in inspect.getmembers(Example):
    if not name.startswith('_'):
        print(f'  {name}: {type(value).__name__}')

# Check MRO (Method Resolution Order)
print(f"\nMRO: {[c.__name__ for c in Example.__mro__]}")

# Get the descriptor type
print(f"\nDescriptor test:")
print(f'  isinstance(Example.method, property): {isinstance(Example.method, property)}')

# Inspect source code (for debugging)
print(f"\nSource of Example.method:")
print(inspect.getsource(Example.method))

Key introspection functions:

FunctionPurpose
inspect.getmembers(cls)List all attributes (class and inherited)
inspect.getmro(cls)Print Method Resolution Order
type(obj) / isinstance(obj, type)Determine if obj is a class
hasattr(obj, '__dict__')Check if obj has an instance dictionary
inspect.getsource(func)Print the source code of a function
dir(obj)List all accessible attributes

Pitfall 6: Circular Dependencies in Metaclass Field Collection​

When a metaclass tries to access class attributes during creation, it may encounter fields that reference the class:

class ValidatingMeta(type):
    def __new__(mcs, name, bases, namespace):
        # Try to access class before it exists
        for key, value in namespace.items():
            if hasattr(value, 'get_type_hint'):
                # This might fail if value references the class being created
                hint = value.get_type_hint()
        
        return super().__new__(mcs, name, bases, namespace)

Solution: Defer access to the class itself; perform initialization in __init__ of the metaclass or on first use:

class ValidatingMeta(type):
    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        
        # Now the class exists; safe to access it
        for key in dir(cls):
            value = getattr(cls, key)
            if hasattr(value, 'get_type_hint'):
                hint = value.get_type_hint()
        
        return cls

Key Takeaways​

  • Use object.__getattribute__() inside custom __getattribute__ to avoid infinite recursion.
  • Make descriptors data descriptors (with __set__) if they must take precedence over instance attributes.
  • In multiple inheritance with metaclasses, create a unifying metaclass inheriting from all conflicting bases.
  • Always call super().__init_subclass__(**kwargs) to maintain the inheritance chain.
  • Use introspection tools (inspect, dir(), type()) to understand and debug metaprogramming code.

Frequently Asked Questions​

How do I detect infinite recursion early?​

Set a recursion limit check: add a counter in __getattribute__ or __getattr__ and raise RecursionError if it exceeds a threshold. Better: use a sentinel value (e.g., object.__getattribute__(self, '__dict__')) to access core state safely.

Can I debug a metaclass by printing its state during creation?​

Yes. Add print() statements in __new__ to inspect the namespace dictionary. You can also print the class after creation: print(f'{name}: {dir(cls)}' to see generated methods.

What does "method resolution order" (MRO) tell me?​

MRO determines which parent class method is called in inheritance chains. Print it with ClassName.__mro__ or use inspect.getmro(). If a method isn't found where you expect, check the MRO to see if it's shadowed earlier in the chain.

How do I check if an attribute is a descriptor?​

Check for __get__, __set__, or __delete__: hasattr(attr, '__get__'). To check if it's a data descriptor: hasattr(attr, '__set__') or hasattr(attr, '__delete__').

Is it safe to use __dict__.items() in a descriptor's __set__?​

Only if you know the instance has a __dict__ (not using __slots__). To be safe, use object.__getattribute__(obj, '__dict__') explicitly or check for slots first.

Further Reading​