├── cffi_example ├── __init__.py ├── utils.py ├── fnmatch.h ├── build_fnmatch.py ├── fnmatch.py ├── build_person.py └── person.py ├── MANIFEST.in ├── requirements-testing.txt ├── tox.ini ├── Makefile ├── .gitignore ├── test ├── test_fnmatch.py └── test_person.py ├── setup.py ├── LICENSE.txt └── README.rst /cffi_example/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.rst 2 | include cffi_example/*.h 3 | -------------------------------------------------------------------------------- /requirements-testing.txt: -------------------------------------------------------------------------------- 1 | pytest>=2.7 2 | cffi>=1.0 3 | tox>=1.7 4 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist=py26,py27,py33,pypy 3 | 4 | [testenv] 5 | deps=-rrequirements-testing.txt 6 | commands=py.test test/ 7 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: cffi_example/_person.so cffi_example/_fnmatch.so 2 | 3 | cffi_example/_person.so: cffi_example/build_person.py 4 | python $< 5 | 6 | cffi_example/_fnmatch.so: cffi_example/build_fnmatch.py 7 | python $< 8 | 9 | clean: 10 | rm cffi_example/_*.c cffi_example/_*.o cffi_example/_*.so 11 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.class 2 | *.o 3 | *.so 4 | *.pyc 5 | *.sqlite3 6 | *.sw[a-z] 7 | *~ 8 | .DS_Store 9 | bin-debug/* 10 | bin-release/* 11 | bin/* 12 | tags 13 | *.beam 14 | *.dump 15 | env/ 16 | .env/ 17 | *egg-info* 18 | misc/ 19 | dist/ 20 | Icon? 21 | node_modules/ 22 | .tox/ 23 | .eggs/ 24 | build/ 25 | __pycache__ 26 | 27 | cffi_example/_*.c 28 | cffi_example/_*.so 29 | -------------------------------------------------------------------------------- /test/test_fnmatch.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from cffi_example import fnmatch 4 | 5 | @pytest.mark.parametrize("pattern,name,flags,expected", [ 6 | ("foo", "bar", 0, False), 7 | ("f*", "foo", 0, True), 8 | ("f*bar", "f/bar", 0, True), 9 | ("f*bar", "f/bar", fnmatch.FNM_PATHNAME, False), 10 | ]) 11 | def test_fnmatch(pattern, name, flags, expected): 12 | assert fnmatch.fnmatch(pattern, name, flags) == expected 13 | -------------------------------------------------------------------------------- /test/test_person.py: -------------------------------------------------------------------------------- 1 | from cffi_example.person import Person 2 | 3 | class TestPerson(object): 4 | def setup(self): 5 | self.p = Person(u"Alex", u"Smith", 72) 6 | 7 | def test_get_age(self): 8 | assert self.p.get_age() == 72 9 | 10 | def test_get_full_name(self): 11 | assert self.p.get_full_name() == u"Alex Smith" 12 | 13 | def test_long_name(self): 14 | p = Person(u"x" * 100, u"y" * 100, 72) 15 | assert p.get_full_name() == u"x" * 100 16 | -------------------------------------------------------------------------------- /cffi_example/utils.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | PY2 = sys.version_info[0] == 2 4 | PY3 = sys.version_info[0] == 3 5 | 6 | if PY3: 7 | text_type = str 8 | binary_type = bytes 9 | else: 10 | text_type = unicode 11 | binary_type = str 12 | 13 | 14 | def to_bytes(s): 15 | if isinstance(s, binary_type): 16 | return s 17 | return text_type(s).encode("utf-8") 18 | 19 | 20 | def to_unicode(s): 21 | if isinstance(s, text_type): 22 | return s 23 | return binary_type(s).decode("utf-8") 24 | -------------------------------------------------------------------------------- /cffi_example/fnmatch.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Definitions for fnmatch, copy+pasted from with some small 3 | * cleanups by hand. 4 | */ 5 | 6 | /* Bits set in the FLAGS argument to `fnmatch'. */ 7 | #define FNM_PATHNAME ... /* No wildcard can ever match `/'. */ 8 | #define FNM_NOESCAPE ... /* Backslashes don't quote special chars. */ 9 | #define FNM_PERIOD ... /* Leading `.' is matched only explicitly. */ 10 | 11 | /* Value returned by `fnmatch' if STRING does not match PATTERN. */ 12 | #define FNM_NOMATCH 1 13 | 14 | /* Match NAME against the filename pattern PATTERN, 15 | returning zero if it matches, FNM_NOMATCH if not. */ 16 | extern int fnmatch (const char *__pattern, const char *__name, int __flags); 17 | 18 | -------------------------------------------------------------------------------- /cffi_example/build_fnmatch.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from cffi import FFI 4 | 5 | ffi = FFI() 6 | 7 | ffi.set_source("cffi_example._fnmatch", 8 | # Since we are calling fnmatch directly no custom source is necessary. We 9 | # need to #include , though, because behind the scenes cffi 10 | # generates a .c file which contains a Python-friendly wrapper around 11 | # ``fnmatch``: 12 | # static PyObject * 13 | # _cffi_f_fnmatch(PyObject *self, PyObject *args) { 14 | # ... setup ... 15 | # result = fnmatch(...); 16 | # return PyInt_FromLong(result); 17 | # } 18 | "#include ", 19 | # The important thing is to inclue libc in the list of libraries we're 20 | # linking against: 21 | libraries=["c"], 22 | ) 23 | 24 | with open(os.path.join(os.path.dirname(__file__), "fnmatch.h")) as f: 25 | ffi.cdef(f.read()) 26 | 27 | if __name__ == "__main__": 28 | ffi.compile() 29 | -------------------------------------------------------------------------------- /cffi_example/fnmatch.py: -------------------------------------------------------------------------------- 1 | from ._fnmatch import lib 2 | from .utils import to_bytes 3 | 4 | FNM_PATHNAME = lib.FNM_PATHNAME 5 | FNM_NOESCAPE = lib.FNM_NOESCAPE 6 | FNM_PERIOD = lib.FNM_PERIOD 7 | 8 | def fnmatch(pattern, name, flags=0): 9 | """ Matches ``name`` against ``pattern``. Flags are a bitmask of: 10 | 11 | * ``FNM_PATHNAME``: No wildcard can match '/'. 12 | * ``FNM_NOESCAPE``: Backslashes don't quote special chars. 13 | * ``FNM_PERIOD``: Leading '.' is matched only explicitly. 14 | 15 | For example:: 16 | 17 | >>> fnmatch("init.*", "init.d") 18 | True 19 | >>> fnmatch("foo", "bar") 20 | False 21 | """ 22 | # fnmatch expects its arguments to be ``char[]``, so make sure that the 23 | # ``params`` and ``name`` are bytes/str (cffi will convert 24 | # bytes/str --> char[], but will refuse to convert unicode --> char[]). 25 | res = lib.fnmatch(to_bytes(pattern), to_bytes(name), flags) 26 | return res == 0 27 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | import sys 5 | 6 | from setuptools import setup, find_packages 7 | 8 | os.chdir(os.path.dirname(sys.argv[0]) or ".") 9 | 10 | setup( 11 | name="cffi-example", 12 | version="0.1", 13 | description="An example project using Python's CFFI", 14 | long_description=open("README.rst", "rt").read(), 15 | url="https://github.com/wolever/python-cffi-example", 16 | author="David Wolever", 17 | author_email="david@wolever.net", 18 | classifiers=[ 19 | "Development Status :: 4 - Beta", 20 | "Programming Language :: Python :: 2", 21 | "Programming Language :: Python :: 3", 22 | "Programming Language :: Python :: Implementation :: PyPy", 23 | "License :: OSI Approved :: BSD License", 24 | ], 25 | packages=find_packages(), 26 | install_requires=["cffi>=1.0.0"], 27 | setup_requires=["cffi>=1.0.0"], 28 | cffi_modules=[ 29 | "./cffi_example/build_person.py:ffi", 30 | "./cffi_example/build_fnmatch.py:ffi", 31 | ], 32 | ) 33 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | tl;dr: all code code is licensed under simplified BSD, unless stated otherwise. 2 | 3 | Unless stated otherwise in the source files, all code is copyright 2015 David 4 | Wolever . All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, 10 | this list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND ANY EXPRESS OR 17 | IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 18 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO 19 | EVENT SHALL OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, 20 | INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 21 | BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 22 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 23 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 24 | OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 25 | ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | 27 | The views and conclusions contained in the software and documentation are those 28 | of the authors and should not be interpreted as representing official policies, 29 | either expressed or implied, of David Wolever. 30 | -------------------------------------------------------------------------------- /cffi_example/build_person.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from cffi import FFI 4 | 5 | ffi = FFI() 6 | 7 | person_header = """ 8 | typedef struct { 9 | wchar_t *p_first_name; 10 | wchar_t *p_last_name; 11 | int p_age; 12 | } person_t; 13 | 14 | /* Creates a new person. Names are strdup(...)'d into the person. */ 15 | person_t *person_create(const wchar_t *first_name, const wchar_t *last_name, int age); 16 | 17 | /* Writes the person's full name into *buf, up to n characters. 18 | * Returns the number of characters written. */ 19 | int person_get_full_name(person_t *p, wchar_t *buf, size_t n); 20 | 21 | /* Destroys a person, including free()ing their names. */ 22 | void person_destroy(person_t *p); 23 | """ 24 | 25 | ffi.set_source("cffi_example._person", person_header + """ 26 | #include 27 | #include 28 | #include 29 | 30 | person_t *person_create(const wchar_t *first_name, const wchar_t *last_name, int age) { 31 | person_t *p = malloc(sizeof(person_t)); 32 | if (!p) 33 | return NULL; 34 | p->p_first_name = wcsdup(first_name); 35 | p->p_last_name = wcsdup(last_name); 36 | p->p_age = age; 37 | return p; 38 | } 39 | 40 | int person_get_full_name(person_t *p, wchar_t *buf, size_t n) { 41 | return swprintf(buf, n, L"%S %S", p->p_first_name, p->p_last_name); 42 | } 43 | 44 | void person_destroy(person_t *p) { 45 | if (p->p_first_name) 46 | free(p->p_first_name); 47 | if (p->p_last_name) 48 | free(p->p_last_name); 49 | free(p); 50 | } 51 | """) 52 | 53 | ffi.cdef(person_header) 54 | 55 | if __name__ == "__main__": 56 | ffi.compile() 57 | -------------------------------------------------------------------------------- /cffi_example/person.py: -------------------------------------------------------------------------------- 1 | from ._person import ffi, lib 2 | from .utils import text_type 3 | 4 | class Person(object): 5 | def __init__(self, first_name, last_name, age): 6 | # Because person_create expects wchar_t we need to guarantee that the 7 | # first_name and last_name are unicode; cffi will convert 8 | # unicode <--> wchar_t[], but will throw an error if asked to implictly 9 | # convert a str/bytes --> wchar_t[]. 10 | if not isinstance(first_name, text_type): 11 | raise TypeError("first_name must be unicode (got %r)" %(first_name, )) 12 | if not isinstance(last_name, text_type): 13 | raise TypeError("last_name must be unicode (got %r)" %(last_name, )) 14 | 15 | # Because ``person_create`` calls ``strdup`` on the inputs we can 16 | # safely pass Python byte strings directly into the constructor. If 17 | # the constructor expected us to manage the lifecycle of the strings we 18 | # would need to allocate new char buffers for them, then make sure that 19 | # those character buffers are not GC'd by storing a reference to them:: 20 | # first_name = ffi.new("wchar_t[]", first_name) 21 | # last_name = ffi.new("wchar_t[]", last_name) 22 | # self._gc_keepalive = [first_name, last_name] 23 | p = lib.person_create(first_name, last_name, age) 24 | if p == ffi.NULL: 25 | raise MemoryError("Could not allocate person") 26 | 27 | # ffi.gc returns a copy of the cdata object which will have the 28 | # destructor (in this case ``person_destroy``) called when the Python 29 | # object is GC'd: 30 | # https://cffi.readthedocs.org/en/latest/using.html#ffi-interface 31 | self._p = ffi.gc(p, lib.person_destroy) 32 | 33 | def get_age(self): 34 | # Because the ``person_t`` struct is defined in the ``ffi.cdef`` call 35 | # we can directly access members. 36 | return self._p.p_age 37 | 38 | def get_full_name(self): 39 | # ffi.new() creates a managed buffer which will be automatically 40 | # free()'d when the Python object is GC'd: 41 | # https://cffi.readthedocs.org/en/latest/using.html#ffi-interface 42 | buf = ffi.new("wchar_t[]", 101) 43 | # Note: ``len(buf)`` (101, in this case) is used instead of 44 | # ``ffi.sizeof(buf)`` (101 * 4 = 404) because person_get_full_name 45 | # expects the maximum number of characters, not number of bytes. 46 | lib.person_get_full_name(self._p, buf, len(buf)) 47 | 48 | # ``ffi.string`` converts a null-terminated C string to a Python byte 49 | # array. Note that ``buf`` will be automatically free'd when the Python 50 | # object is GC'd so we don't need to do anything special there. 51 | return ffi.string(buf) 52 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | cffi-example: an example project showing how to use Python's CFFI 2 | ================================================================= 3 | 4 | A complete project which demonstrates how to use CFFI 1.0 during development of 5 | a modern Python package: 6 | 7 | * Development 8 | * Testing across Python interpreters (Py2, Py3, PyPy) 9 | * Packaging and distribution 10 | 11 | 12 | Status 13 | ------ 14 | 15 | This repository is the result of a weeks worth of frustrated googling. I don't 16 | think anything is *too* misleading, but I would deeply appreciate feedback from 17 | anyone who actually knows what's going on. 18 | 19 | 20 | Examples 21 | -------- 22 | 23 | * ``cffi_example/person.py`` and ``cffi_example/build_person.py`` is an example 24 | of a Python class which wraps a C class, including proper memory management 25 | and passing around string buffers:: 26 | 27 | >>> p = Person(u"Alex", u"Smith", 72) # --> calls person_create(...) 28 | >>> p.get_full_name() # --> calls person_get_full_name(p, buf, len(buf)) 29 | u"Alex Smith" 30 | 31 | * ``cffi_example/fnmatch.py`` and ``cffi_example/build_fnmatch.py`` is an 32 | example of wrapping a shared library (in this case ``libc``'s `fnmatch`__):: 33 | 34 | >>> fnmatch("f*", "foo") # --> calls libc's fnmatch 35 | True 36 | 37 | __ http://man7.org/linux/man-pages/man3/fnmatch.3.html 38 | 39 | 40 | Development 41 | ----------- 42 | 43 | 1. Clone the repository:: 44 | 45 | $ git clone git@github.com:wolever/python-cffi-example.git 46 | 47 | 2. Make sure that ``cffi``, ``py.test``, and ``tox`` are installed:: 48 | 49 | $ pip install -r requirements-testing.txt 50 | 51 | 3. Run ``python setup.py develop`` to build development versions of the 52 | modules:: 53 | 54 | $ python setup.py develop 55 | ... 56 | Finished processing dependencies for cffi-example==0.1 57 | 58 | 4. Test locally with ``py.test``:: 59 | 60 | $ py.test test/ 61 | =========================== test session starts =========================== 62 | platform darwin -- Python 2.7.2 -- py-1.4.28 -- pytest-2.7.1 63 | rootdir: /Users/wolever/code/python-cffi-example, inifile: 64 | collected 7 items 65 | 66 | test/test_fnmatch.py .... 67 | test/test_person.py ... 68 | 69 | ======================== 7 passed in 0.03 seconds ========================= 70 | 71 | 5. Test against multiple Python environments using ``tox``:: 72 | 73 | $ tox 74 | ... 75 | _________________________________ summary _________________________________ 76 | py26: commands succeeded 77 | py27: commands succeeded 78 | py33: commands succeeded 79 | pypy: commands succeeded 80 | congratulations :) 81 | 82 | 6. I prefer to use ``make`` to clean and rebuild libraries during development 83 | (since it's faster than ``setup.py develop``):: 84 | 85 | $ make clean 86 | ... 87 | $ make 88 | python cffi_example/build_person.py 89 | python cffi_example/build_fnmatch.py 90 | 91 | 92 | Packaging 93 | --------- 94 | 95 | This example uses `CFFI's recommended combination`__ of ``setuptools`` and 96 | ``cffi_modules``:: 97 | 98 | $ cat setup.py 99 | from setuptools import setup 100 | 101 | setup( 102 | ... 103 | install_requires=["cffi>=1.0.0"], 104 | setup_requires=["cffi>=1.0.0"], 105 | cffi_modules=[ 106 | "./cffi_example/build_person.py:ffi", 107 | "./cffi_example/build_fnmatch.py:ffi", 108 | ], 109 | ) 110 | 111 | __ https://cffi.readthedocs.org/en/latest/cdef.html#distutils-setuptools 112 | 113 | This will cause the modules to be built with ``setup.py develop`` or ``setup.py 114 | build``, and installed with ``setup.py install``. 115 | 116 | **Note**: Many examples you'll see online use either the ``distutils`` 117 | ``ext_modules=[cffi_example.ffi.distribute_extension()]`` method, or the 118 | more complex ``keywords_with_side_effects`` method. To the best of my 119 | knowledge these methods are only necessary with CFFI < 1.0 or ``distutils``. 120 | `dstufft`__ has written a fantastic post — 121 | https://caremad.io/2014/11/distributing-a-cffi-project/ — which details the 122 | drawbacks of the ``ext_modules`` method and explains the 123 | ``keywords_with_side_effects`` method, but I believe it was written before CFFI 124 | 1.0 so it does not include the now preferred ``cffi_modules`` method. 125 | 126 | __ https://twitter.com/dstufft/ 127 | 128 | 129 | Distribution 130 | ------------ 131 | 132 | Distribution is just like any other Python package, with the obvious caveat 133 | that wheels will be platform-specific:: 134 | 135 | $ python setup.py sdist bdist_wheel 136 | ... 137 | $ ls dist/ 138 | cffi-example-0.1.tar.gz 139 | cffi_example-0.1-cp27-none-macosx_10_8_intel.whl 140 | 141 | And the package can be uploaded to PyPI using ``upload``:: 142 | 143 | $ python setup.py sdist upload 144 | 145 | 146 | Note that users of the source package will need to have ``cffi`` (and a C 147 | compiler, and development headers of any libraries you're linking against) 148 | installed to build and install your package. 149 | 150 | Note also that the ``MANIFEST.in`` file will need to be updated to include any 151 | new source or headers you may add during development. The ``tox`` tests will 152 | catch this error, but it may not be obvious how to correct it. 153 | 154 | 155 | Caveats 156 | ------- 157 | 158 | * Doesn't yet cover using ``dlopen(...)`` to dynamically load ``.so`` files 159 | because I haven't figured out any best practices for building custom shared 160 | libraries along with a Python package's lifecycle, and the 161 | `CFFI documentation on loading dynamic libraries`__ covers the details of 162 | making the ``lib.dlopen(...)`` call. 163 | 164 | * Using ``make`` to build modules during development is less than ideal. Please 165 | post here if there's a better way to do this: 166 | http://stackoverflow.com/q/30823397/71522 167 | 168 | 169 | __ https://cffi.readthedocs.org/en/latest/overview.html#out-of-line-abi-level 170 | --------------------------------------------------------------------------------