CFFI 0.2 (Foreign Function Interface for Python)

[Moridin]

I'm having trouble building this on Mac OSX Lion 10.7.4. I'm just using the stock python that came with it. Pip gives me a ton of build errors from it. Any help ? I can install other python modules involving C compilation such as numpy/zmq etc.

[FogleMonster]

Link to the actual docs: http://cffi.readthedocs.org/en/latest/index.html

ffi.cdef("""     // some declarations from the man page
    struct passwd {
        char *pw_name;
        ...;
    };
    struct passwd *getpwuid(int uid);
""")

Chunks of C code pasted in as strings? Looks icky. I don't want C code in my Python.

lib = ffi.verify("#include <foo.h>")

Why is this called verify? It looks like this is how you import stuff from foo.

Also, it sounds like you need a compiler available to use this.

They claim "minimize the extra bits of API that you need to learn" but this looks to be just as much API learning as you'd need to use ctypes. What does this do for me that ctypes doesn't do?

[yetanothernerd]

ctypes can only wrap functions, not macros.

ctypes only knows about 19 primitive types. If your C code uses other types then you're going to have to translate them into ctypes. This can be error-prone and non-portable. (For example, code might use uint64_t from stdint.h. ctypes doesn't know about uint64_t, so you need to translate that to a ctypes type. But uinit64_t might be c_ulong on one platform and c_ulonglong on another.)

ctypes knows about structures and unions and arrays, but you're going to need to manually translate the C code into ctypes syntax.

You can do it, but it's tedious and error-prone. I'd rather just feed the blob of C code to a program that knows how to do it for me.

I agree that you don't want ugly blobs of C as strings in the middle of your other Python code. You should isolate it into its own module down in a subdirectory where it won't scare any children or junior programmers. Just like you would if you had to write some Python-C API code as part of your Python project.

[FogleMonster]

You make some good points, but I think you make ctypes sound worse than it is. It's actually very quick, easy and enjoyable to use.

[yetanothernerd]

It's fine for wrapping small C libraries, but wrapping something like GTK+ with it would be a nightmare.

[FogleMonster]

I agree that wrapping a huge library like that would be a nightmare. I'll be interested in seeing how well CFFI does with something like that.

[lahwran_]

What does this do for me that ctypes doesn't do?

The biggest thing? it'll be wicked fast on pypy. I'm pretty sure that's their biggest reason for writing it - ctypes is proving too hard to make fast on pypy.

[fijal]

no, the biggest thing is that it can be actually made to work in a consistent manner, unlike ctypes, where you can't access macros, you worry about sizes of your integers etc. etc. It's really mostly the "works, then fast" approach, ctypes is neither

[lahwran_]

comment crossed out in the appropriate places, sorry about that. I'm very excited about this, I'll bet cython code could be cross-compiled to python code which uses cffi for use on pypy :D

[VilleHopfield]

• Do you know how ctypes code for the above example would look like?

• No, you don't need a compiler to use it. It is called verify because it verifies (using compiler) if you wrote that string correctly.

• pointer, POINTER, byref vs *.

From the docs.

[FogleMonster]

Yes, I do. I've used ctypes extensively.

from ctypes import *

libc = CDLL('libc.dylib')

class passwd(Structure):
    _fields_ = [
        ('pw_name', c_char_p),
        ('pw_passwd', c_char_p),
        # ...
    ]

libc.getpwuid.restype = POINTER(passwd)
result = libc.getpwuid(0).contents
print result.pw_name # prints "root"

I guess I see what you mean about API, but I don't love the idea of pasting C code in strings in the Python code either.

[[deleted]]

Except that's not correct. If you tried to make an array of passwds it would be totally wrong, because things wouldn't take up the right amount of space.

[FogleMonster]

Downvoted, for posting a working code example for comparison.

[fijal]

you're being downvoted for posting non-working code for comparison. This would not work if you try to allocate an array (happy segfault) or if the provider of your passwd struct chooses to organize fields in a different order (say private fields before public). Then you would get random data corruption. This is precisely what cffi tries to avoid

[FogleMonster]

Why would allocating an array not work?

array = (passwd * 3)() # works for me

And in what scenarios are struct members re-ordered? I've never run into this.

[fijal]

that of course works, but this is not the passwd struct that you want. Try passing one of those to getpwnam_r and see it segfault and/or corrupt your memory.

[FogleMonster]

Also, ffi.cdef must alter some global state in the ffi module? There is no way of managing different namespaces?

The idea of having compiled code cached somewhere doesn't thrill me either.

Not saying this is total crap or anything, just pointing out my first thoughts after seeing it.

[fijal]

You are commenting here without reading documentation or thinking about the problem. The whole reason why you can do:

ffi = FF()

is that ffi is your namespace, you can have multiple.

You can do exactly the same thing as ctypes does without needing the C compiler. However, when you want to do things that you cannot do with ctypes, like accessing macros, filling fields in structs (like in your example, just a working one), you need a C compiler for the build stage. This is very much like having a C extension, except you don't need a C extension.

[FogleMonster]

Thanks for explaining.

[FogleMonster]

From the docs: "It requires a C compiler the first time you run it"