Extending Swig Builtin Classes

The -builtin option of SWIG has the advantage of being faster, and of being exempt of a bug with multiple inheritance. The setback is I can't set any attribute on the generated cla

Solution 1:

I found a solution quite by accident. I was experimenting with metaclasses, thinking I could manage to override the setattr and getattr functions of the builtin type in the subclass.

Doing this I discovered the builtins already have a metaclass (SwigPyObjectType), so my metaclass had to inherit it.

And that's it. This alone solved the problem. I would be glad if someone could explain why :

SwigPyObjectType = type(SWIGBuiltinClass)

classMeta(SwigPyObjectType):
    passclassThing(SWIGBuiltinClass):
    __metaclass__ = Meta

Thing.myattr = 'anything'# Works fine this time

Solution 2:

The problem comes from how swig implemented the classes in "-builtin" to be just like builtin classes (hence the name).

builtin classes are not extensible - try to add or modify a member of "str" and python won't let you modify the attribute dictionary.

I do have a solution I've been using for several years.

I'm not sure I can recommend it because:

1. It's arguably evil - the moral equivalent of casting away const-ness in C/C++
2. It's unsupported and could break in future python releases
3. I haven't tried it with python3
4. I would be a bit uncomfortable using "black-magic" like this in production code - it could break and is certainly obscure - but at least one giant corporation IS using this in production code

But.. I love how well it works to solve some obscure features we wanted for debugging.

The original idea is not mine, I got it from: https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader

The basic idea is to access the const dictionary in the swig-created type object as a non-const dictionary and add/override any desired methods.

FYI, the technique of runtime modification of classes is called monkeypatching, see https://en.wikipedia.org/wiki/Monkey_patch

First - here's "monkeypatch.py":

''' monkeypatch.py:
I got this from https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader,
his comment: "found this from Armin R. on Twitter, what a beautiful gem ;)"
I made a few changes for coding style preferences
- Rudy Albachten   April 30 2015
'''import ctypes
from types import DictProxyType, MethodType

# figure out the size of _Py_ssize_t
_Py_ssize_t = ctypes.c_int64 ifhasattr(ctypes.pythonapi, 'Py_InitModule4_64') else ctypes.c_int

# python without tracingclass_PyObject(ctypes.Structure):
    pass
_PyObject._fields_ = [
    ('ob_refcnt', _Py_ssize_t),
    ('ob_type', ctypes.POINTER(_PyObject))
]

# fixup for python with tracingifobject.__basicsize__ != ctypes.sizeof(_PyObject):
    class_PyObject(ctypes.Structure):
        pass
    _PyObject._fields_ = [
        ('_ob_next', ctypes.POINTER(_PyObject)),
        ('_ob_prev', ctypes.POINTER(_PyObject)),
        ('ob_refcnt', _Py_ssize_t),
        ('ob_type', ctypes.POINTER(_PyObject))
    ]

class_DictProxy(_PyObject):
    _fields_ = [('dict', ctypes.POINTER(_PyObject))]

defreveal_dict(proxy):
    ifnotisinstance(proxy, DictProxyType):
        raise TypeError('dictproxy expected')
    dp = _DictProxy.from_address(id(proxy))
    ns = {}
    ctypes.pythonapi.PyDict_SetItem(ctypes.py_object(ns), ctypes.py_object(None), dp.dict)
    return ns[None]

defget_class_dict(cls): 
    d = getattr(cls, '__dict__', None)
    if d isNone:
        raise TypeError('given class does not have a dictionary')
    ifisinstance(d, DictProxyType):
        return reveal_dict(d)
    return d

deftest():
    import random
    d = get_class_dict(str)
    d['foo'] = lambda x: ''.join(random.choice((c.upper, c.lower))() for c in x)
    print"and this is monkey patching str".foo()

if __name__ == '__main__':
    test()

Here's a contrived example using monkeypatch:

I have a class "myclass" in module "mystuff" wrapped with swig -python -builtin

I want to add an extra runtime method "namelen" that returns the length of the name returned by myclass.getName()

import mystuff
import monkeypatch

# add a "namelen" method to all "myclass" objectsdefnamelen(self):
    returnlen(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen

x = mystuff.myclass("xxxxxxxx")
print"namelen:", x.namelen()

Note that this can also be used to extend or override methods on builtin python classes, as is demonstrated in the test in monkeypatch.py: it adds a method "foo" to the builtin str class that returns a copy of the original string with random upper/lower case letters

I would probably replace:

# add a "namelen" method to all "myclass" objectsdefnamelen(self):
    returnlen(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen

with

# add a "namelen" method to all "myclass" objects
monkeypatch.get_class_dict(mystuff.myclass)['namelen'] = lambda self: returnlen(self.getName())

to avoid extra global variables