├── MANIFEST.in ├── README.rst ├── myapp ├── __init__.py ├── data_subdir │ ├── binfile.dat │ └── textfile.txt ├── data_subpackage │ ├── __init__.py │ ├── binfile.dat │ └── textfile.txt ├── example1.py ├── example2.py ├── example3.py ├── example4.py └── example5.py ├── pyproject.toml └── setup.cfg /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include pyproject.toml 2 | recursive-include myapp *.dat 3 | recursive-include myapp *.txt 4 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | This project shows how to package data files within a Python distribution, and has some example code for reading the data files. To build this distribution, create a venv with build_ installed and then execute 2 | 3 | .. code-block:: bash 4 | 5 | python -m build 6 | 7 | The distributions (an sdist .tar.gz and a bdist .whl) will be written to ./dist/ subdirectory. To test it out, install the distribution and run the console script ``resources-example``. 8 | 9 | Here's a compatibility summary of the five approaches demonstrated: 10 | 11 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 12 | | Module | Description | In stdlib? | Works on Py2? | Works on Py3? | Works in zipfile? | Run as script? [*]_ | 13 | +=============+=======================+============+===============+===============+===================+=====================+ 14 | | example1.py | os.path.join | yes | yes | yes | no | yes | 15 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 16 | | example2.py | pkgutil | yes | yes | yes | yes | no | 17 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 18 | | example3.py | pkg_resources | no | yes | deprecated | yes | yes | 19 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 20 | | example4.py | importlib.resources. | deprecated | no | yes (3.7+) | yes | yes | 21 | | | read_binary/read_text | | | | | | 22 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 23 | | example5.py | importlib.resources. | yes (3.9+) | yes [*]_ | yes | yes | yes | 24 | | | files | | | | | | 25 | +-------------+-----------------------+------------+---------------+---------------+-------------------+---------------------+ 26 | 27 | If you are interested in creating an executable zip from source, you can use stdlib `zipapp `_ utility (Python 3.5+): 28 | 29 | .. code-block:: bash 30 | 31 | python3 -m zipapp --compress /path/to/resources-example --main="myapp:main" --output=myapp.zip 32 | 33 | If this command is slow or the .zip is surprisingly large, make sure don't have any stray subdirs in the source path beforehand (e.g. ``.venv``, ``.git``, ``.idea``). 34 | 35 | Now you can run the zip directly with the interpreter (any Python version): 36 | 37 | .. code-block:: bash 38 | 39 | python myapp.zip 40 | 41 | .. _build: https://pypi.org/project/build/ 42 | 43 | .. [*] "Run as script" means executing the submodule directly, e.g. ``python myapp/example2.py``. Note that Guido considers this `an anti-pattern `_. 44 | .. [*] The same APIs are available in 2.7 by using an `importlib_resources `_ backport. 45 | -------------------------------------------------------------------------------- /myapp/__init__.py: -------------------------------------------------------------------------------- 1 | """An example project demonstrating various ways to access data files in Python package""" 2 | from __future__ import print_function 3 | 4 | import importlib 5 | import sys 6 | 7 | 8 | def main(): 9 | print("Python version:", sys.version) 10 | print("-" * 80) 11 | for i in 1, 2, 3, 4, 5: 12 | modname = "myapp.example{}".format(i) 13 | try: 14 | mod = importlib.import_module(modname) 15 | print(mod.__doc__) 16 | mod.func() 17 | except Exception as err: 18 | print("FAILED:", repr(err)) 19 | print("-" * 80) 20 | -------------------------------------------------------------------------------- /myapp/data_subdir/binfile.dat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wimglenn/resources-example/68a4b4ce6c15c3e872db8986397f8f8f074f6647/myapp/data_subdir/binfile.dat -------------------------------------------------------------------------------- /myapp/data_subdir/textfile.txt: -------------------------------------------------------------------------------- 1 | hello world 2 | -------------------------------------------------------------------------------- /myapp/data_subpackage/__init__.py: -------------------------------------------------------------------------------- 1 | # why is this file required? it's only for example4.py 2 | # that's a shortcoming of importlib.resources 3 | # see https://gitlab.com/python-devs/importlib_resources/issues/58 4 | -------------------------------------------------------------------------------- /myapp/data_subpackage/binfile.dat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wimglenn/resources-example/68a4b4ce6c15c3e872db8986397f8f8f074f6647/myapp/data_subpackage/binfile.dat -------------------------------------------------------------------------------- /myapp/data_subpackage/textfile.txt: -------------------------------------------------------------------------------- 1 | hello world 2 | -------------------------------------------------------------------------------- /myapp/example1.py: -------------------------------------------------------------------------------- 1 | """Loading resources by traversing relative paths. 2 | The naive approach. Doesn't work if loading from a zipfile.""" 3 | import os 4 | 5 | 6 | def func(): 7 | here = os.path.dirname(os.path.abspath(__file__)) 8 | with open(os.path.join(here, "data_subdir", "binfile.dat"), "rb") as f: 9 | data = f.read() 10 | with open(os.path.join(here, "data_subdir", "textfile.txt"), "rt") as f: 11 | text = f.read() 12 | print("data: " + repr(data)) 13 | print("text: " + text) 14 | -------------------------------------------------------------------------------- /myapp/example2.py: -------------------------------------------------------------------------------- 1 | """Loading resources using stdlib pkgutil APIs. 2 | https://docs.python.org/3/library/pkgutil.html""" 3 | import pkgutil 4 | 5 | 6 | def func(): 7 | data = pkgutil.get_data(__name__, "data_subdir/binfile.dat") 8 | text = pkgutil.get_data(__name__, "data_subdir/textfile.txt").decode() 9 | print("data: " + repr(data)) 10 | print("text: " + text) 11 | -------------------------------------------------------------------------------- /myapp/example3.py: -------------------------------------------------------------------------------- 1 | """Loading resources using pkg_resources APIs (setuptools). 2 | https://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access 3 | 4 | Deprecated in setuptools v67.5.0 (Mar 2023) 5 | https://github.com/pypa/setuptools/pull/3843""" 6 | import pkg_resources 7 | 8 | 9 | def func(): 10 | data = pkg_resources.resource_string(__name__, "data_subdir/binfile.dat") 11 | text = pkg_resources.resource_string(__name__, "data_subdir/textfile.txt").decode() 12 | print("data: " + repr(data)) 13 | print("text: " + text) 14 | -------------------------------------------------------------------------------- /myapp/example4.py: -------------------------------------------------------------------------------- 1 | """Loading resources using stdlib importlib.resources APIs (Python 3.7+) 2 | https://docs.python.org/3/library/importlib.resources.html 3 | 4 | Note: `read_binary` and `read_text` functions demonstrated below were 5 | deprecated in Python 3.11 (https://bugs.python.org/issue45514) 6 | 7 | You may migrate to traversable APIs avail in stdlib for newer Python versions (3.9+): 8 | https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files 9 | """ 10 | import importlib.resources 11 | 12 | 13 | def func(): 14 | data = importlib.resources.read_binary("myapp.data_subpackage", "binfile.dat") 15 | text = importlib.resources.read_text("myapp.data_subpackage", "textfile.txt") 16 | print("data: " + repr(data)) 17 | print("text: " + text) 18 | -------------------------------------------------------------------------------- /myapp/example5.py: -------------------------------------------------------------------------------- 1 | """Loading resources using importlib_resources APIs. 2 | https://importlib-resources.readthedocs.io/en/latest/ 3 | 4 | Note, the PyPI packaging importlib_resources is only required for Python <= 3.8 5 | In Python 3.9+, the same APIs are available directly in stdlib importlib.resources: 6 | https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files 7 | 8 | Resist the temptation to use importlib.resources.files() without any arguments, because 9 | omitting the anchor currently will break zipapp packaging. 10 | https://github.com/python/cpython/issues/121735 11 | """ 12 | import importlib_resources 13 | 14 | 15 | def func(): 16 | my_resources = importlib_resources.files("myapp") / "data_subdir" 17 | data = (my_resources / "binfile.dat").read_bytes() 18 | text = (my_resources / "textfile.txt").read_text() 19 | print("data: " + repr(data)) 20 | print("text: " + text) 21 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = [ 3 | "setuptools >= 40.6.0", 4 | "wheel", 5 | ] 6 | build-backend = "setuptools.build_meta" 7 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = resources-example 3 | author = Wim Glenn 4 | author_email = hey@wimglenn.com 5 | version = 0.2 6 | description = An example project demonstrating various ways to access data files in Python package 7 | long_description = file: README.rst 8 | long_description_content_type = text/x-rst 9 | keywords = packaging, resources 10 | license = MIT 11 | home_page = https://github.com/wimglenn/resources-example 12 | project_urls = 13 | Issue Tracker = https://github.com/wimglenn/resources-example/issues 14 | classifiers = 15 | Intended Audience :: Developers 16 | License :: OSI Approved :: MIT License 17 | Operating System :: OS Independent 18 | Programming Language :: Python :: 2 19 | Programming Language :: Python :: 3 20 | Topic :: Software Development :: Libraries :: Python Modules 21 | Topic :: System :: Archiving :: Packaging 22 | Topic :: System :: Systems Administration 23 | Topic :: Utilities 24 | 25 | [options] 26 | packages = find: 27 | include_package_data = true 28 | install_requires = 29 | setuptools 30 | importlib_resources 31 | 32 | 33 | [options.entry_points] 34 | console_scripts = 35 | resources-example = myapp:main 36 | 37 | [bdist_wheel] 38 | universal = 1 39 | --------------------------------------------------------------------------------