├── tests ├── python │ ├── __init__.py │ └── test_pybind11_numpy_example.py └── cpp │ ├── tests.cpp │ ├── CMakeLists.txt │ └── pybind11_numpy_example_t.cpp ├── lib ├── pybind11_numpy_example.cpp └── CMakeLists.txt ├── scripts ├── time.png ├── memory.png ├── mem_list.txt ├── mem_array.txt ├── time_list.txt ├── mem_array_nocopy.txt ├── time.py ├── memory.py ├── time_array.txt ├── time_array_nocopy.txt ├── plot.py └── benchmarks.sh ├── .gitmodules ├── src ├── pybind11_numpy_example │ ├── python_code.py │ └── __init__.py ├── CMakeLists.txt └── pybind11_numpy_example_python.cpp ├── .readthedocs.yml ├── include └── pybind11_numpy_example │ └── pybind11_numpy_example.hpp ├── doc ├── CMakeLists.txt ├── conf.py └── index.rst ├── .pre-commit-config.yaml ├── LICENSE.md ├── .github └── workflows │ ├── ci.yml │ └── pypi.yml ├── pyproject.toml ├── CMakeLists.txt ├── .gitignore └── README.md /tests/python/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/cpp/tests.cpp: -------------------------------------------------------------------------------- 1 | #define CATCH_CONFIG_MAIN 2 | #include "catch2/catch.hpp" 3 | -------------------------------------------------------------------------------- /lib/pybind11_numpy_example.cpp: -------------------------------------------------------------------------------- 1 | #include "pybind11_numpy_example/pybind11_numpy_example.hpp" 2 | -------------------------------------------------------------------------------- /scripts/time.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/HEAD/scripts/time.png -------------------------------------------------------------------------------- /scripts/memory.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/HEAD/scripts/memory.png -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "ext/Catch2"] 2 | path = ext/Catch2 3 | url = https://github.com/catchorg/Catch2.git 4 | -------------------------------------------------------------------------------- /src/pybind11_numpy_example/python_code.py: -------------------------------------------------------------------------------- 1 | def pure_python_list(size: int): 2 | return list(range(size)) 3 | -------------------------------------------------------------------------------- /lib/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_library(pybind11_numpy_example pybind11_numpy_example.cpp) 2 | target_include_directories( 3 | pybind11_numpy_example 4 | PUBLIC $ 5 | $) 6 | -------------------------------------------------------------------------------- /tests/cpp/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(tests pybind11_numpy_example_t.cpp) 2 | target_link_libraries(tests PUBLIC pybind11_numpy_example 3 | Catch2::Catch2WithMain) 4 | 5 | # allow user to run tests with `make test` or `ctest` 6 | catch_discover_tests(tests) 7 | -------------------------------------------------------------------------------- /src/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | find_package(pybind11 CONFIG REQUIRED) 2 | pybind11_add_module(_pybind11_numpy_example pybind11_numpy_example_python.cpp) 3 | target_link_libraries(_pybind11_numpy_example PUBLIC pybind11_numpy_example) 4 | install(TARGETS _pybind11_numpy_example DESTINATION pybind11_numpy_example) 5 | -------------------------------------------------------------------------------- /src/pybind11_numpy_example/__init__.py: -------------------------------------------------------------------------------- 1 | """An example of how to use pybind11 and numpy""" 2 | 3 | # here we import the contents of our compiled C++ module 4 | from ._pybind11_numpy_example import * 5 | 6 | # we can also import from python modules as usual: 7 | from .python_code import pure_python_list 8 | -------------------------------------------------------------------------------- /scripts/mem_list.txt: -------------------------------------------------------------------------------- 1 | #n memory (kb) 2 | 1000 192 3 | 10000 384 4 | 100000 4224 5 | 1000000 41088 6 | 10000000 410304 7 | 50000000 2050752 8 | 100000000 4101504 9 | 200000000 8203200 10 | 300000000 12304704 11 | 400000000 16405824 12 | 600000000 24609216 13 | 800000000 32811840 14 | 1000000000 41014464 15 | 1200000000 49218432 16 | -------------------------------------------------------------------------------- /.readthedocs.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | 3 | build: 4 | os: "ubuntu-22.04" 5 | tools: 6 | python: "3.12" 7 | 8 | sphinx: 9 | builder: html 10 | configuration: doc/conf.py 11 | fail_on_warning: false 12 | 13 | formats: all 14 | 15 | python: 16 | install: 17 | - method: pip 18 | path: . 19 | extra_requirements: 20 | - docs 21 | -------------------------------------------------------------------------------- /scripts/mem_array.txt: -------------------------------------------------------------------------------- 1 | #n memory (kb) 2 | 1000 20648 3 | 10000 20664 4 | 100000 20840 5 | 1000000 24520 6 | 10000000 59672 7 | 50000000 215920 8 | 100000000 411412 9 | 200000000 801848 10 | 300000000 1192548 11 | 400000000 1583112 12 | 600000000 2363992 13 | 800000000 3145808 14 | 1000000000 3926680 15 | 1200000000 4708296 16 | 2000000000 7832800 17 | 3000000000 11739192 18 | 4000000000 15645488 19 | 6000000000 23458288 20 | 8000000000 31270592 21 | 10000000000 39083108 22 | 12000000000 46895616 23 | -------------------------------------------------------------------------------- /scripts/time_list.txt: -------------------------------------------------------------------------------- 1 | #n time (seconds) 2 | 1000 1.1281892404451172e-05 3 | 10000 0.00012381860035491158 4 | 100000 0.0021609368014822504 5 | 1000000 0.028091967190531166 6 | 10000000 0.3382009760243818 7 | 50000000 1.6539601690601557 8 | 100000000 3.3066364750266075 9 | 200000000 6.4062930109212175 10 | 300000000 9.456814253004268 11 | 400000000 12.655741214985028 12 | 600000000 18.71708781796042 13 | 800000000 24.94151852594223 14 | 1000000000 31.0570098310709 15 | 1200000000 37.80632794194389 16 | -------------------------------------------------------------------------------- /tests/cpp/pybind11_numpy_example_t.cpp: -------------------------------------------------------------------------------- 1 | #include "pybind11_numpy_example/pybind11_numpy_example.hpp" 2 | #include 3 | 4 | using namespace pybind11numpyexample; 5 | 6 | TEST_CASE("make_vector", "[make_vector]") { 7 | REQUIRE(make_vector(0) == std::vector{}); 8 | REQUIRE(make_vector(5) == std::vector{0, 1, 2, 3, 4}); 9 | REQUIRE(make_vector(3) == std::vector{0, 1, 2}); 10 | REQUIRE(make_vector(4) == std::vector{0, 1, 2, 3}); 11 | } 12 | -------------------------------------------------------------------------------- /include/pybind11_numpy_example/pybind11_numpy_example.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | namespace pybind11numpyexample { 7 | 8 | /** @brief Helper function that returns a vector of given size and type 9 | * 10 | * @tparam T The type of element 11 | * @param size The size of the vector to return 12 | * @returns a vector of given size and type 13 | */ 14 | template std::vector make_vector(std::size_t size) { 15 | std::vector v(size, 0); 16 | std::iota(v.begin(), v.end(), 0); 17 | return v; 18 | } 19 | 20 | } // namespace pybind11numpyexample 21 | -------------------------------------------------------------------------------- /scripts/mem_array_nocopy.txt: -------------------------------------------------------------------------------- 1 | #n memory (kb) 2 | 1000 20656 3 | 10000 20668 4 | 100000 20844 5 | 1000000 22596 6 | 10000000 39936 7 | 50000000 118080 8 | 100000000 215964 9 | 200000000 411264 10 | 300000000 606336 11 | 400000000 801716 12 | 600000000 1192512 13 | 800000000 1583148 14 | 1000000000 1973368 15 | 1200000000 2364404 16 | 2000000000 3926908 17 | 3000000000 5880000 18 | 4000000000 7832964 19 | 6000000000 11739264 20 | 8000000000 15645624 21 | 10000000000 19551908 22 | 12000000000 23457968 23 | 14000000000 27364404 24 | 16000000000 31270648 25 | 18000000000 35176512 26 | 20000000000 39083136 27 | 24000000000 46895548 28 | -------------------------------------------------------------------------------- /scripts/time.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # simple script to print approx time taken 4 | 5 | import pybind11_numpy_example 6 | import timeit 7 | import sys 8 | 9 | n = int(sys.argv[1]) 10 | data_type = int(sys.argv[2]) 11 | iters = 1 + 10000000 // n 12 | 13 | 14 | def doit(): 15 | if data_type == 0: 16 | return pybind11_numpy_example.vector_as_list(n) 17 | elif data_type == 1: 18 | return pybind11_numpy_example.vector_as_array(n) 19 | elif data_type == 2: 20 | return pybind11_numpy_example.vector_as_array_nocopy(n) 21 | 22 | 23 | print(timeit.timeit(doit, number=iters) / iters) 24 | -------------------------------------------------------------------------------- /scripts/memory.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # simple script to print approx memory usage 4 | 5 | import pybind11_numpy_example 6 | import resource 7 | import sys 8 | 9 | n = int(sys.argv[1]) 10 | data_type = int(sys.argv[2]) 11 | max_mem_before = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss 12 | 13 | if data_type == 0: 14 | a = pybind11_numpy_example.vector_as_list(n) 15 | elif data_type == 1: 16 | a = pybind11_numpy_example.vector_as_array(n) 17 | elif data_type == 2: 18 | a = pybind11_numpy_example.vector_as_array_nocopy(n) 19 | 20 | max_mem_after = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss 21 | 22 | print(max_mem_after - max_mem_before) 23 | -------------------------------------------------------------------------------- /scripts/time_array.txt: -------------------------------------------------------------------------------- 1 | #n time (seconds) 2 | 1000 8.885241385522114e-06 3 | 10000 8.064530965231068e-05 4 | 100000 0.0007929159005605938 5 | 1000000 0.009228698179041121 6 | 10000000 0.05545463500311598 7 | 50000000 0.16033962194342166 8 | 100000000 0.23489250801503658 9 | 200000000 0.387445722008124 10 | 300000000 0.5446825119433925 11 | 400000000 0.6931000800104812 12 | 600000000 1.0074213709449396 13 | 800000000 1.3038605999900028 14 | 1000000000 1.6112821800634265 15 | 1200000000 1.8958396930247545 16 | 2000000000 3.080139480996877 17 | 3000000000 4.569697203929536 18 | 4000000000 6.015317940968089 19 | 6000000000 8.836932464968413 20 | 8000000000 11.728594742016867 21 | 10000000000 14.558276921976358 22 | 12000000000 17.50521888397634 23 | -------------------------------------------------------------------------------- /doc/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | find_package(Doxygen REQUIRED) 2 | set(DOXYGEN_EXCLUDE_PATTERNS "${CMAKE_SOURCE_DIR}/ext/*") 3 | set(DOXYGEN_SHORT_NAMES YES) 4 | set(DOXYGEN_GENERATE_XML YES) 5 | doxygen_add_docs( 6 | doxygen ${CMAKE_SOURCE_DIR} 7 | WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} 8 | COMMENT Building doxygen documentation...) 9 | add_custom_target( 10 | sphinx-doc 11 | COMMAND 12 | sphinx-build -b html 13 | -Dbreathe_projects.pybind11-numpy-example="${CMAKE_CURRENT_BINARY_DIR}/xml" 14 | -c ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR} 15 | ${CMAKE_CURRENT_BINARY_DIR}/sphinx 16 | WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} 17 | COMMENT "Generating documentation with Sphinx...") 18 | add_dependencies(sphinx-doc doxygen) 19 | -------------------------------------------------------------------------------- /scripts/time_array_nocopy.txt: -------------------------------------------------------------------------------- 1 | #n time (seconds) 2 | 1000 8.883211479922996e-06 3 | 10000 7.990702798778003e-05 4 | 100000 0.0007843043065012092 5 | 1000000 0.007740272727625614 6 | 10000000 0.050859106006100774 7 | 50000000 0.1441949950531125 8 | 100000000 0.239397105993703 9 | 200000000 0.3419582910137251 10 | 300000000 0.48297904699575156 11 | 400000000 0.6020817440003157 12 | 600000000 0.8661073229741305 13 | 800000000 1.1198496220167726 14 | 1000000000 1.3745229849591851 15 | 1200000000 1.6183152759913355 16 | 2000000000 2.622669712989591 17 | 3000000000 3.8534785190131515 18 | 4000000000 5.067567399004474 19 | 6000000000 7.537025678087957 20 | 8000000000 9.918707602075301 21 | 10000000000 12.325743645080365 22 | 12000000000 14.685017141047865 23 | 14000000000 17.10181719996035 24 | 16000000000 19.59815236995928 25 | 18000000000 21.906724045053124 26 | 20000000000 24.42910428403411 27 | 24000000000 29.188139538047835 28 | -------------------------------------------------------------------------------- /tests/python/test_pybind11_numpy_example.py: -------------------------------------------------------------------------------- 1 | import pybind11_numpy_example as pne 2 | import numpy as np 3 | import pytest 4 | 5 | 6 | n_values = [0, 1, 2, 17, 159] 7 | 8 | 9 | @pytest.mark.parametrize("list_func", [pne.pure_python_list, pne.vector_as_list]) 10 | @pytest.mark.parametrize("n", n_values) 11 | def test_pybind11_numpy_example_list(list_func, n): 12 | l = list_func(n) 13 | assert isinstance(l, list) 14 | assert len(l) == n 15 | for i in range(n): 16 | assert l[i] == i 17 | 18 | 19 | @pytest.mark.parametrize( 20 | "ndarray_func", [pne.vector_as_array, pne.vector_as_array_nocopy] 21 | ) 22 | @pytest.mark.parametrize("n", n_values) 23 | def test_pybind11_numpy_example_ndarray(ndarray_func, n): 24 | a = ndarray_func(n) 25 | assert isinstance(a, np.ndarray) 26 | assert len(a) == n 27 | assert a.dtype == np.int16 28 | for i in range(n): 29 | assert a[i] == i 30 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v5.0.0 4 | hooks: 5 | - id: check-yaml 6 | - id: end-of-file-fixer 7 | - id: trailing-whitespace 8 | - id: mixed-line-ending 9 | 10 | - repo: https://github.com/psf/black 11 | rev: 25.1.0 12 | hooks: 13 | - id: black 14 | 15 | - repo: https://github.com/cheshirekow/cmake-format-precommit 16 | rev: v0.6.13 17 | hooks: 18 | - id: cmake-format 19 | additional_dependencies: [pyyaml] 20 | 21 | - repo: https://github.com/kynan/nbstripout 22 | rev: 0.8.1 23 | hooks: 24 | - id: nbstripout 25 | 26 | - repo: https://github.com/pre-commit/mirrors-clang-format 27 | rev: v20.1.8 28 | hooks: 29 | - id: clang-format 30 | 31 | - repo: https://github.com/pre-commit/mirrors-prettier 32 | rev: v4.0.0-alpha.8 33 | hooks: 34 | - id: prettier 35 | ci: 36 | autoupdate_schedule: quarterly 37 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright 2020, The copyright holders according to COPYING.md 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 8 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: push 4 | 5 | jobs: 6 | cpp: 7 | name: "${{ matrix.os }} :: C++" 8 | runs-on: ${{matrix.os}} 9 | strategy: 10 | matrix: 11 | os: [ubuntu-latest, macos-latest, windows-latest] 12 | steps: 13 | - uses: actions/checkout@v4 14 | with: 15 | submodules: "recursive" 16 | 17 | - name: Build and run c++ tests 18 | run: | 19 | mkdir build 20 | cd build 21 | cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_DOCS=OFF -DBUILD_PYTHON=OFF -DBUILD_TESTING=ON .. 22 | cmake --build . 23 | ctest 24 | 25 | python: 26 | name: "${{ matrix.os }} :: Python ${{ matrix.python-version }}" 27 | runs-on: ${{matrix.os}} 28 | strategy: 29 | fail-fast: false 30 | matrix: 31 | os: [ubuntu-latest, macos-latest, windows-latest] 32 | python-version: ["3.11", "3.12", "3.13"] 33 | steps: 34 | - uses: actions/checkout@v4 35 | 36 | - uses: actions/setup-python@v5 37 | with: 38 | python-version: ${{ matrix.python-version }} 39 | allow-prereleases: true 40 | 41 | - name: Install Python bindings using pip 42 | run: python -m pip install .[test] -v 43 | 44 | - name: Run Python tests 45 | run: python -m pytest -v 46 | -------------------------------------------------------------------------------- /.github/workflows/pypi.yml: -------------------------------------------------------------------------------- 1 | name: Build Wheels + PyPI deploy 2 | 3 | on: push 4 | 5 | jobs: 6 | build-wheels: 7 | name: Build wheels on ${{ matrix.os }} 8 | runs-on: ${{ matrix.os }} 9 | strategy: 10 | matrix: 11 | os: [ubuntu-latest, windows-latest, macos-13, macos-latest] 12 | steps: 13 | - uses: actions/checkout@v4 14 | 15 | - name: Build wheels 16 | uses: pypa/cibuildwheel@v3.1 17 | 18 | - uses: actions/upload-artifact@v4 19 | with: 20 | name: cibw-wheels-${{ matrix.os }} 21 | path: ./wheelhouse/*.whl 22 | 23 | build-sdist: 24 | name: Build source distribution 25 | runs-on: ubuntu-latest 26 | steps: 27 | - uses: actions/checkout@v4 28 | 29 | - name: Build sdist 30 | run: pipx run build --sdist 31 | 32 | - uses: actions/upload-artifact@v4 33 | with: 34 | name: cibw-sdist 35 | path: dist/*.tar.gz 36 | 37 | upload_pypi: 38 | needs: [build-wheels, build-sdist] 39 | runs-on: ubuntu-latest 40 | environment: release 41 | permissions: 42 | id-token: write 43 | if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/') 44 | steps: 45 | - uses: actions/download-artifact@v4 46 | with: 47 | pattern: cibw-* 48 | merge-multiple: true 49 | path: dist 50 | 51 | - uses: pypa/gh-action-pypi-publish@release/v1 52 | with: 53 | verbose: true 54 | -------------------------------------------------------------------------------- /scripts/plot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | 7 | mem_list = np.loadtxt("mem_list.txt") 8 | mem_array = np.loadtxt("mem_array.txt") 9 | mem_array_nocopy = np.loadtxt("mem_array_nocopy.txt") 10 | 11 | plt.figure() 12 | plt.title("RAM used vs number of elements") 13 | plt.xlabel("Number of elements (million)") 14 | plt.ylabel("RAM used (GB)") 15 | plt.plot(mem_list[:, 0] / 1e6, mem_list[:, 1] / 1e6, label="list", marker="o") 16 | plt.plot(mem_array[:, 0] / 1e6, mem_array[:, 1] / 1e6, label="array (copy)", marker="x") 17 | plt.plot( 18 | mem_array_nocopy[:, 0] / 1e6, 19 | mem_array_nocopy[:, 1] / 1e6, 20 | label="array (move)", 21 | marker=".", 22 | ) 23 | plt.legend() 24 | plt.savefig("memory.png", bbox_inches="tight") 25 | 26 | time_list = np.loadtxt("time_list.txt") 27 | time_array = np.loadtxt("time_array.txt") 28 | time_array_nocopy = np.loadtxt("time_array_nocopy.txt") 29 | 30 | plt.figure() 31 | plt.title("Time used vs number of elements") 32 | plt.xlabel("Number of elements (million)") 33 | plt.ylabel("Time used (seconds)") 34 | plt.plot(time_list[:, 0] / 1e6, time_list[:, 1], label="list", marker="o") 35 | plt.plot(time_array[:, 0] / 1e6, time_array[:, 1], label="array (copy)", marker="x") 36 | plt.plot( 37 | time_array_nocopy[:, 0] / 1e6, 38 | time_array_nocopy[:, 1], 39 | label="array (move)", 40 | marker=".", 41 | ) 42 | plt.legend() 43 | plt.savefig("time.png", bbox_inches="tight") 44 | -------------------------------------------------------------------------------- /scripts/benchmarks.sh: -------------------------------------------------------------------------------- 1 | # simple bash script to benchmark memory & runtime 2 | 3 | echo "#n memory (kb)" > mem_list.txt 4 | cp mem_list.txt mem_array.txt 5 | cp mem_list.txt mem_array_nocopy.txt 6 | 7 | echo "#n time (seconds)" > time_list.txt 8 | cp time_list.txt time_array.txt 9 | cp time_list.txt time_array_nocopy.txt 10 | 11 | for n in 1000 10000 100000 1000000 10000000 50000000 100000000 200000000 300000000 400000000 600000000 800000000 1000000000 1200000000 12 | do 13 | echo $n 14 | m_list=$(./memory.py $n 0) 15 | m_array=$(./memory.py $n 1) 16 | m_array_nocopy=$(./memory.py $n 2) 17 | echo "${n} ${m_list}" >> mem_list.txt 18 | echo "${n} ${m_array}" >> mem_array.txt 19 | echo "${n} ${m_array_nocopy}" >> mem_array_nocopy.txt 20 | 21 | t_list=$(./time.py $n 0) 22 | t_array=$(./time.py $n 1) 23 | t_array_nocopy=$(./time.py $n 2) 24 | echo "${n} ${t_list}" >> time_list.txt 25 | echo "${n} ${t_array}" >> time_array.txt 26 | echo "${n} ${t_array_nocopy}" >> time_array_nocopy.txt 27 | done 28 | for n in 2000000000 3000000000 4000000000 6000000000 8000000000 10000000000 12000000000 29 | do 30 | echo $n 31 | m_array=$(./memory.py $n 1) 32 | echo "${n} ${m_array}" >> mem_array.txt 33 | m_array_nocopy=$(./memory.py $n 2) 34 | echo "${n} ${m_array_nocopy}" >> mem_array_nocopy.txt 35 | 36 | t_array=$(./time.py $n 1) 37 | echo "${n} ${t_array}" >> time_array.txt 38 | t_array_nocopy=$(./time.py $n 2) 39 | echo "${n} ${t_array_nocopy}" >> time_array_nocopy.txt 40 | done 41 | 42 | for n in 14000000000 16000000000 18000000000 20000000000 24000000000 43 | do 44 | echo $n 45 | m_array_nocopy=$(./memory.py $n 2) 46 | echo "${n} ${m_array_nocopy}" >> mem_array_nocopy.txt 47 | 48 | t_array_nocopy=$(./time.py $n 2) 49 | echo "${n} ${t_array_nocopy}" >> time_array_nocopy.txt 50 | done 51 | 52 | ./plot.py 53 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["scikit-build-core", "pybind11"] 3 | build-backend = "scikit_build_core.build" 4 | 5 | [project] 6 | name = "pybind11-numpy-example" 7 | version = "1.0.1" 8 | description = "An example of using numpy with pybind11" 9 | readme = "README.md" 10 | license = {text = "MIT"} 11 | authors=[{name="Liam Keegan", email="liam@keegan.ch"}] 12 | maintainers=[{name="Liam Keegan", email="liam@keegan.ch"}] 13 | requires-python = ">=3.8" 14 | dependencies = ["numpy"] 15 | keywords = ["pybind11", "cibuildwheel", "c++", "pypi", "numpy", "simple", "example", "wheel", "pypi", "conda-forge"] 16 | classifiers=[ 17 | "Programming Language :: C++", 18 | "Programming Language :: Python :: 3 :: Only", 19 | "Programming Language :: Python :: 3.8", 20 | "Programming Language :: Python :: 3.9", 21 | "Programming Language :: Python :: 3.10", 22 | "Programming Language :: Python :: 3.11", 23 | "Programming Language :: Python :: 3.12", 24 | "Programming Language :: Python :: 3.13", 25 | "Programming Language :: Python :: 3.14", 26 | "Programming Language :: Python :: Implementation :: CPython", 27 | "Operating System :: MacOS :: MacOS X", 28 | "Operating System :: Microsoft :: Windows", 29 | "Operating System :: POSIX :: Linux", 30 | "License :: OSI Approved :: MIT License", 31 | ] 32 | 33 | [project.urls] 34 | Github = "https://github.com/ssciwr/pybind11-numpy-example" 35 | Documentation = "https://pybind11-numpy-example.readthedocs.io" 36 | 37 | [project.optional-dependencies] 38 | test = ["pytest"] 39 | docs = ["cmake", "breathe", "sphinx_rtd_theme"] 40 | 41 | [tool.scikit-build.cmake.define] 42 | BUILD_CPP = "OFF" 43 | BUILD_PYTHON = "ON" 44 | BUILD_TESTING = "OFF" 45 | BUILD_DOCS = "OFF" 46 | 47 | [tool.cibuildwheel] 48 | test-extras = "test" 49 | test-command = "python -m pytest {project}/tests/python -v" 50 | test-skip = "*-musllinux* *-manylinux_i686" 51 | -------------------------------------------------------------------------------- /CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 3.16...4.0) 2 | 3 | # Set a name and a version number for your project: 4 | project( 5 | pybind11-numpy-example 6 | VERSION 1.0.2 7 | LANGUAGES CXX) 8 | 9 | # Initialize some default paths 10 | include(GNUInstallDirs) 11 | 12 | # Define the minimum C++ standard that is required 13 | set(CMAKE_CXX_STANDARD 17) 14 | set(CMAKE_CXX_STANDARD_REQUIRED ON) 15 | 16 | set(CMAKE_POSITION_INDEPENDENT_CODE ON) 17 | 18 | # Configuration options 19 | option(BUILD_CPP "Enable building of C++ interface" ON) 20 | option(BUILD_PYTHON "Enable building of Python interface" ON) 21 | option(BUILD_DOCS "Enable building of documentation" ON) 22 | 23 | # Build the core c++ library 24 | add_subdirectory(lib) 25 | 26 | # Build the c++ tests 27 | include(CTest) 28 | if(BUILD_TESTING) 29 | add_subdirectory(ext/Catch2) 30 | include(./ext/Catch2/extras/Catch.cmake) 31 | add_subdirectory(tests/cpp) 32 | endif() 33 | 34 | # Build the documentation 35 | if(BUILD_DOCS) 36 | add_subdirectory(doc) 37 | endif() 38 | 39 | # Build the python interface 40 | if(BUILD_PYTHON) 41 | add_subdirectory(src) 42 | endif() 43 | 44 | # Install c++ interface 45 | if(BUILD_CPP) 46 | # Add an alias target for use if this project is included as a subproject in 47 | # another project 48 | add_library(pybind11_numpy_example::pybind11_numpy_example ALIAS 49 | pybind11_numpy_example) 50 | 51 | # Install targets and configuration 52 | install( 53 | TARGETS pybind11_numpy_example 54 | EXPORT pybind11_numpy_example_config 55 | RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} 56 | LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} 57 | ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) 58 | 59 | install( 60 | EXPORT pybind11_numpy_example_config 61 | NAMESPACE pybind11_numpy_example:: 62 | DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/pybind11_numpy_example) 63 | 64 | install(DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/include/ 65 | DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) 66 | endif() 67 | 68 | # This prints a summary of found dependencies 69 | include(FeatureSummary) 70 | feature_summary(WHAT ALL) 71 | -------------------------------------------------------------------------------- /src/pybind11_numpy_example_python.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include "pybind11_numpy_example/pybind11_numpy_example.hpp" 6 | 7 | namespace py = pybind11; 8 | 9 | // helper function to avoid making a copy when returning a py::array_t 10 | // author: https://github.com/YannickJadoul 11 | // source: https://github.com/pybind/pybind11/issues/1042#issuecomment-642215028 12 | template 13 | inline py::array_t as_pyarray(Sequence &&seq) { 14 | auto size = seq.size(); 15 | auto data = seq.data(); 16 | std::unique_ptr seq_ptr = 17 | std::make_unique(std::move(seq)); 18 | auto capsule = py::capsule(seq_ptr.get(), [](void *p) { 19 | std::unique_ptr(reinterpret_cast(p)); 20 | }); 21 | seq_ptr.release(); 22 | return py::array(size, data, capsule); 23 | } 24 | 25 | namespace pybind11numpyexample { 26 | 27 | /** @brief Return a vector as a Python List 28 | * 29 | * @param size The size of the vector to return 30 | * @returns the vector as a Python List 31 | */ 32 | static std::vector vector_as_list(std::size_t size) { 33 | return make_vector(size); 34 | } 35 | 36 | /** @brief Return a vector as a NumPy array 37 | * 38 | * Makes a copy of an existing vector of data 39 | * 40 | * @param size The size of the vector to return 41 | * @returns the vector as a NumPy array 42 | */ 43 | static py::array_t vector_as_array(std::size_t size) { 44 | auto temp_vector = make_vector(size); 45 | return py::array(size, temp_vector.data()); 46 | } 47 | 48 | /** @brief Return a vector as a NumPy array 49 | * 50 | * Moves the contents of an existing vector of data 51 | * 52 | * @param size The size of the vector to return 53 | * @returns the vector as a NumPy array 54 | */ 55 | static py::array_t vector_as_array_nocopy(std::size_t size) { 56 | auto temp_vector = make_vector(size); 57 | return as_pyarray(std::move(temp_vector)); 58 | } 59 | 60 | PYBIND11_MODULE(_pybind11_numpy_example, m) { 61 | m.doc() = "Python Bindings for pybind11-numpy-example"; 62 | m.def("vector_as_list", &vector_as_list, 63 | "Returns a vector of 16-bit ints as a Python List"); 64 | m.def("vector_as_array", &vector_as_array, 65 | "Returns a vector of 16-bit ints as a NumPy array"); 66 | m.def("vector_as_array_nocopy", &vector_as_array_nocopy, 67 | "Returns a vector of 16-bit ints as a NumPy array without making a " 68 | "copy of the data"); 69 | } 70 | 71 | } // namespace pybind11numpyexample 72 | -------------------------------------------------------------------------------- /doc/conf.py: -------------------------------------------------------------------------------- 1 | # Configuration file for the Sphinx documentation builder. 2 | # 3 | # This file only contains a selection of the most common options. For a full 4 | # list see the documentation: 5 | # https://www.sphinx-doc.org/en/master/usage/configuration.html 6 | 7 | import os 8 | import subprocess 9 | 10 | # -- Path setup -------------------------------------------------------------- 11 | 12 | # If extensions (or modules to document with autodoc) are in another directory, 13 | # add these directories to sys.path here. If the directory is relative to the 14 | # documentation root, use os.path.abspath to make it absolute, like shown here. 15 | # 16 | # import sys 17 | # sys.path.insert(0, os.path.abspath('.')) 18 | 19 | 20 | # -- Project information ----------------------------------------------------- 21 | 22 | project = "pybind11-numpy-example" 23 | copyright = "2020, Liam Keegan" 24 | author = "Liam Keegan" 25 | 26 | # -- General configuration --------------------------------------------------- 27 | 28 | # Add any Sphinx extension module names here, as strings. They can be 29 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 30 | # ones. 31 | extensions = [ 32 | "breathe", 33 | "sphinx_rtd_theme", 34 | ] 35 | 36 | # Add any paths that contain templates here, relative to this directory. 37 | templates_path = [] 38 | 39 | # List of patterns, relative to source directory, that match files and 40 | # directories to ignore when looking for source files. 41 | # This pattern also affects html_static_path and html_extra_path. 42 | exclude_patterns = [] 43 | 44 | 45 | # -- Options for HTML output ------------------------------------------------- 46 | 47 | # The theme to use for HTML and HTML Help pages. See the documentation for 48 | # a list of builtin themes. 49 | # 50 | html_theme = "sphinx_rtd_theme" 51 | 52 | # Add any paths that contain custom static files (such as style sheets) here, 53 | # relative to this directory. They are copied after the builtin static files, 54 | # so a file named "default.css" will overwrite the builtin "default.css". 55 | html_static_path = [] 56 | 57 | # Breathe Configuration: Breathe is the bridge between the information extracted 58 | # from the C++ sources by Doxygen and Sphinx. 59 | breathe_projects = {} 60 | breathe_default_project = "pybind11-numpy-example" 61 | 62 | # Check if we're running on Read the Docs' servers 63 | read_the_docs_build = os.environ.get("READTHEDOCS", None) == "True" 64 | 65 | # Implement build logic on RTD servers 66 | if read_the_docs_build: 67 | cwd = os.getcwd() 68 | os.makedirs("build-cmake", exist_ok=True) 69 | builddir = os.path.join(cwd, "build-cmake") 70 | subprocess.check_call( 71 | "cmake -DBUILD_DOCS=ON -DBUILD_TESTING=OFF -DBUILD_PYTHON=OFF ../..".split(), 72 | cwd=builddir, 73 | ) 74 | subprocess.check_call("cmake --build . --target doxygen".split(), cwd=builddir) 75 | breathe_projects["pybind11-numpy-example"] = os.path.join(builddir, "doc", "xml") 76 | -------------------------------------------------------------------------------- /doc/index.rst: -------------------------------------------------------------------------------- 1 | ---------------------- 2 | pybind11-numpy-example 3 | ---------------------- 4 | 5 | .. toctree:: 6 | :maxdepth: 2 7 | :caption: Contents: 8 | 9 | What 10 | ==== 11 | 12 | A simple example of how to use 13 | `pybind11 `__ with 14 | `numpy `__. 15 | 16 | This C++/Python library creates a ``std::vector`` of 16-bit ints, 17 | and provides a Python interface to the contents of this vector in a 18 | few different ways: 19 | 20 | - a Python 21 | `List `__ 22 | (copy the data) 23 | - a NumPy 24 | `ndarray `__ 25 | (copy the data). 26 | - a NumPy 27 | `ndarray `__ 28 | (move the data). 29 | 30 | Why 31 | === 32 | 33 | Python Lists are great! 34 | However, when storing many small elements of the same type, 35 | a Numpy array is much faster and uses a lot less memory: 36 | 37 | |Memory used vs number of elements| 38 | 39 | |Time used vs number of elements| 40 | 41 | How 42 | === 43 | 44 | The pybind11 code is in 45 | `src/pybind11\_numpy\_example\_python.cpp `__. 46 | 47 | The python project is defined in `pyproject.toml `__ 48 | and uses `scikit-build-core `__. 49 | 50 | Each tagged commit triggers a `GitHub action job `__ 51 | which uses `cibuildwheel `__ to build and upload wheels to `PyPI `__. 52 | 53 | The scripts used to generate the above plots are in 54 | `scripts `__. 55 | 56 | This repo was quickly set up using the SSC `C++ Project 57 | Cookiecutter `__. 58 | 59 | .. |License: MIT| image:: https://img.shields.io/badge/License-MIT-yellow.svg 60 | :target: https://opensource.org/licenses/MIT 61 | .. |GitHub Workflow Status| image:: https://img.shields.io/github/workflow/status/lkeegan/pybind11-numpy-example/CI 62 | :target: https://github.com/lkeegan/pybind11-numpy-example/actions?query=workflow%3ACI 63 | .. |PyPI Release| image:: https://img.shields.io/pypi/v/pybind11-numpy-example.svg 64 | :target: https://pypi.org/project/pybind11-numpy-example 65 | .. |Documentation Status| image:: https://readthedocs.org/projects/pybind11-numpy-example/badge/ 66 | :target: https://pybind11-numpy-example.readthedocs.io/ 67 | .. |Memory used vs number of elements| image:: https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/main/scripts/memory.png 68 | .. |Time used vs number of elements| image:: https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/main/scripts/time.png 69 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Prerequisites 2 | *.d 3 | 4 | # Compiled Object files 5 | *.slo 6 | *.lo 7 | *.o 8 | *.obj 9 | 10 | # Precompiled Headers 11 | *.gch 12 | *.pch 13 | 14 | # Compiled Dynamic libraries 15 | *.so 16 | *.dylib 17 | *.dll 18 | 19 | # Fortran module files 20 | *.mod 21 | *.smod 22 | 23 | # Compiled Static libraries 24 | *.lai 25 | *.la 26 | *.a 27 | *.lib 28 | 29 | # Executables 30 | *.exe 31 | *.out 32 | *.app 33 | 34 | # Byte-compiled / optimized / DLL files 35 | __pycache__/ 36 | *.py[cod] 37 | *$py.class 38 | 39 | # C extensions 40 | *.so 41 | 42 | # Distribution / packaging 43 | .Python 44 | build/ 45 | develop-eggs/ 46 | dist/ 47 | downloads/ 48 | eggs/ 49 | .eggs/ 50 | lib64/ 51 | parts/ 52 | sdist/ 53 | var/ 54 | wheels/ 55 | share/python-wheels/ 56 | *.egg-info/ 57 | .installed.cfg 58 | *.egg 59 | MANIFEST 60 | 61 | # PyInstaller 62 | # Usually these files are written by a python script from a template 63 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 64 | *.manifest 65 | *.spec 66 | 67 | # Installer logs 68 | pip-log.txt 69 | pip-delete-this-directory.txt 70 | 71 | # Unit test / coverage reports 72 | htmlcov/ 73 | .tox/ 74 | .nox/ 75 | .coverage 76 | .coverage.* 77 | .cache 78 | nosetests.xml 79 | coverage.xml 80 | *.cover 81 | *.py,cover 82 | .hypothesis/ 83 | .pytest_cache/ 84 | cover/ 85 | 86 | # Translations 87 | *.mo 88 | *.pot 89 | 90 | # Django stuff: 91 | *.log 92 | local_settings.py 93 | db.sqlite3 94 | db.sqlite3-journal 95 | 96 | # Flask stuff: 97 | instance/ 98 | .webassets-cache 99 | 100 | # Scrapy stuff: 101 | .scrapy 102 | 103 | # Sphinx documentation 104 | docs/_build/ 105 | 106 | # PyBuilder 107 | .pybuilder/ 108 | target/ 109 | 110 | # Jupyter Notebook 111 | .ipynb_checkpoints 112 | 113 | # IPython 114 | profile_default/ 115 | ipython_config.py 116 | 117 | # pyenv 118 | # For a library or package, you might want to ignore these files since the code is 119 | # intended to run in multiple environments; otherwise, check them in: 120 | # .python-version 121 | 122 | # pipenv 123 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 124 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 125 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 126 | # install all needed dependencies. 127 | #Pipfile.lock 128 | 129 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 130 | __pypackages__/ 131 | 132 | # Celery stuff 133 | celerybeat-schedule 134 | celerybeat.pid 135 | 136 | # SageMath parsed files 137 | *.sage.py 138 | 139 | # Environments 140 | .env 141 | .venv 142 | env/ 143 | venv/ 144 | ENV/ 145 | env.bak/ 146 | venv.bak/ 147 | 148 | # Spyder project settings 149 | .spyderproject 150 | .spyproject 151 | 152 | # Rope project settings 153 | .ropeproject 154 | 155 | # mkdocs documentation 156 | /site 157 | 158 | # mypy 159 | .mypy_cache/ 160 | .dmypy.json 161 | dmypy.json 162 | 163 | # Pyre type checker 164 | .pyre/ 165 | 166 | # pytype static type analyzer 167 | .pytype/ 168 | 169 | # Cython debug symbols 170 | cython_debug/ 171 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # pybind11-numpy-example 2 | 3 | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 4 | [![PyPI Release](https://img.shields.io/pypi/v/pybind11-numpy-example.svg)](https://pypi.org/project/pybind11-numpy-example) 5 | [![Conda Release](https://img.shields.io/conda/v/conda-forge/pybind11-numpy-example)](https://anaconda.org/conda-forge/pybind11-numpy-example) 6 | [![Python Versions](https://img.shields.io/pypi/pyversions/pybind11-numpy-example)](https://pypi.org/project/pybind11-numpy-example) 7 | [![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/lkeegan/pybind11-numpy-example/ci.yml?branch=main)](https://github.com/lkeegan/pybind11-numpy-example/actions/workflows/ci.yml) 8 | [![Documentation Status](https://readthedocs.org/projects/pybind11-numpy-example/badge/)](https://pybind11-numpy-example.readthedocs.io/) 9 | 10 | # What 11 | 12 | A simple example of how to use [pybind11](https://github.com/pybind/pybind11) with [numpy](https://numpy.org/) and publish this as a library on [PyPI](https://pypi.org/project/pybind11-numpy-example/) and [conda-forge](https://anaconda.org/conda-forge/pybind11-numpy-example). 13 | 14 | This C++/Python library creates a `std::vector` of 16-bit ints, 15 | and provides a Python interface to the contents of this vector in a few different ways: 16 | 17 | - a Python [List](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists) (copy the data) 18 | - a NumPy [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html) (copy the data). 19 | - a NumPy [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html) (move the data). 20 | 21 | # Why 22 | 23 | Python Lists are great! 24 | However, when storing many small elements of the same type, 25 | a Numpy array is much faster and uses a lot less memory: 26 | 27 | ![Memory used vs number of elements](https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/main/scripts/memory.png) 28 | 29 | ![Time used vs number of elements](https://raw.githubusercontent.com/ssciwr/pybind11-numpy-example/main/scripts/time.png) 30 | 31 | # How 32 | 33 | The pybind11 code is in [src/pybind11_numpy_example_python.cpp](https://github.com/ssciwr/pybind11-numpy-example/blob/main/src/pybind11_numpy_example_python.cpp). 34 | 35 | The python package is defined in [pyproject.toml](https://github.com/ssciwr/pybind11-numpy-example/blob/main/pyproject.toml) 36 | and uses [scikit-build-core](https://github.com/scikit-build/scikit-build-core). 37 | 38 | Each tagged commit triggers a [GitHub action job](https://github.com/ssciwr/pybind11-numpy-example/actions/workflows/pypi.yml) 39 | which uses [cibuildwheel](https://cibuildwheel.readthedocs.io/) to build and upload a new release including binary wheels for all platforms to [PyPI](https://pypi.org/project/pybind11-numpy-example/). 40 | 41 | The [conda-forge package](https://anaconda.org/conda-forge/pybind11-numpy-example) is generated from [this recipe](https://github.com/conda-forge/pybind11-numpy-example-feedstock/blob/main/recipe/meta.yaml), and automatically updates when a new version is uploaded to PyPI. 42 | 43 | The scripts used to generate the above plots are in [scripts](https://github.com/ssciwr/pybind11-numpy-example/tree/main/scripts). 44 | 45 | This repo was quickly set up using the SSC [C++ Project Cookiecutter](https://github.com/ssciwr/cookiecutter-cpp-project). 46 | --------------------------------------------------------------------------------