├── docs ├── authors.rst ├── readme.rst ├── requirements.txt ├── changelog.rst ├── contributing.rst ├── usage.rst ├── installation.rst ├── spelling_wordlist.txt ├── index.rst └── conf.py ├── tests ├── badmodule.py ├── badsyntax.py ├── examples.py ├── test_issue30.py ├── test_perf.py ├── test_issue65.py ├── test_tblib.py └── test_pickle_exception.py ├── .taplo.toml ├── ci ├── requirements.txt ├── templates │ └── .github │ │ └── workflows │ │ └── github-actions.yml └── bootstrap.py ├── SECURITY.md ├── .coveragerc ├── .readthedocs.yml ├── .editorconfig ├── MANIFEST.in ├── AUTHORS.rst ├── setup.py ├── .bumpversion.cfg ├── .pre-commit-config.yaml ├── pytest.ini ├── .gitignore ├── LICENSE ├── src └── tblib │ ├── decorators.py │ ├── pickling_support.py │ └── __init__.py ├── .cookiecutterrc ├── tox.ini ├── CONTRIBUTING.rst ├── pyproject.toml ├── CHANGELOG.rst ├── .github └── workflows │ └── github-actions.yml └── README.rst /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.rst 2 | -------------------------------------------------------------------------------- /docs/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.rst 2 | -------------------------------------------------------------------------------- /docs/requirements.txt: -------------------------------------------------------------------------------- 1 | sphinx>=1.3 2 | furo 3 | -------------------------------------------------------------------------------- /docs/changelog.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CHANGELOG.rst 2 | -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst 2 | -------------------------------------------------------------------------------- /tests/badmodule.py: -------------------------------------------------------------------------------- 1 | a = 1 2 | b = 2 3 | raise Exception('boom!') 4 | -------------------------------------------------------------------------------- /.taplo.toml: -------------------------------------------------------------------------------- 1 | [formatting] 2 | array_auto_collapse = false 3 | indent_string = " " 4 | -------------------------------------------------------------------------------- /tests/badsyntax.py: -------------------------------------------------------------------------------- 1 | # flake8: noqa 2 | """ 3 | bad 4 | bad bad 5 | """ 6 | is very bad 7 | -------------------------------------------------------------------------------- /ci/requirements.txt: -------------------------------------------------------------------------------- 1 | pip>=25 2 | setuptools>=80 3 | setuptools_scm>=8 4 | tox>=4 5 | virtualenv>=20.34 6 | -------------------------------------------------------------------------------- /docs/usage.rst: -------------------------------------------------------------------------------- 1 | ===== 2 | Usage 3 | ===== 4 | 5 | To use tblib in a project:: 6 | 7 | import tblib 8 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Installation 3 | ============ 4 | 5 | At the command line:: 6 | 7 | pip install tblib 8 | -------------------------------------------------------------------------------- /docs/spelling_wordlist.txt: -------------------------------------------------------------------------------- 1 | builtin 2 | builtins 3 | classmethod 4 | staticmethod 5 | classmethods 6 | staticmethods 7 | args 8 | kwargs 9 | callstack 10 | Changelog 11 | Indices 12 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | ## Security contact information 2 | 3 | To report a security vulnerability, please use the 4 | [Tidelift security contact](https://tidelift.com/security). 5 | Tidelift will coordinate the fix and disclosure. 6 | -------------------------------------------------------------------------------- /.coveragerc: -------------------------------------------------------------------------------- 1 | [paths] 2 | source = 3 | src 4 | */site-packages 5 | 6 | [run] 7 | branch = true 8 | source = 9 | tblib 10 | tests 11 | patch = subprocess 12 | 13 | [report] 14 | show_missing = true 15 | precision = 2 16 | omit = 17 | *migrations* 18 | tests/bad*.py 19 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | ======== 2 | Contents 3 | ======== 4 | 5 | .. toctree:: 6 | :maxdepth: 2 7 | 8 | readme 9 | installation 10 | usage 11 | contributing 12 | autoapi/index 13 | authors 14 | changelog 15 | 16 | Indices and tables 17 | ================== 18 | 19 | * :ref:`genindex` 20 | * :ref:`modindex` 21 | * :ref:`search` 22 | -------------------------------------------------------------------------------- /.readthedocs.yml: -------------------------------------------------------------------------------- 1 | # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details 2 | version: 2 3 | sphinx: 4 | configuration: docs/conf.py 5 | formats: all 6 | build: 7 | os: ubuntu-24.04 8 | tools: 9 | python: "3" 10 | python: 11 | install: 12 | - requirements: docs/requirements.txt 13 | - method: pip 14 | path: . 15 | -------------------------------------------------------------------------------- /tests/examples.py: -------------------------------------------------------------------------------- 1 | def func_a(_): 2 | func_b() 3 | 4 | 5 | def func_b(): 6 | func_c() 7 | 8 | 9 | def func_c(): 10 | func_d() 11 | 12 | 13 | def func_d(): 14 | raise Exception('Guessing time !') 15 | 16 | 17 | def bad_syntax(): 18 | import badsyntax # noqa: PLC0415 19 | 20 | badsyntax() 21 | 22 | 23 | def bad_module(): 24 | import badmodule # noqa: PLC0415 25 | 26 | badmodule() 27 | -------------------------------------------------------------------------------- /.editorconfig: -------------------------------------------------------------------------------- 1 | # see https://editorconfig.org/ 2 | root = true 3 | 4 | [*] 5 | # Use Unix-style newlines for most files (except Windows files, see below). 6 | end_of_line = lf 7 | trim_trailing_whitespace = true 8 | indent_style = space 9 | insert_final_newline = true 10 | indent_size = 4 11 | charset = utf-8 12 | 13 | [*.{bat,cmd,ps1}] 14 | end_of_line = crlf 15 | 16 | [*.{yml,yaml}] 17 | indent_size = 2 18 | 19 | [*.tsv] 20 | indent_style = tab 21 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | graft docs 2 | graft src 3 | graft ci 4 | graft tests 5 | 6 | include .bumpversion.cfg 7 | include .cookiecutterrc 8 | include .coveragerc 9 | include .editorconfig 10 | include .github/workflows/github-actions.yml 11 | include .pre-commit-config.yaml 12 | include .readthedocs.yml 13 | include .taplo.toml 14 | include pyproject.toml 15 | include pytest.ini 16 | include tox.ini 17 | 18 | include AUTHORS.rst 19 | include CHANGELOG.rst 20 | include CONTRIBUTING.rst 21 | include LICENSE 22 | include README.rst 23 | include SECURITY.md 24 | 25 | global-exclude *.py[cod] __pycache__/* *.so *.dylib 26 | -------------------------------------------------------------------------------- /tests/test_issue30.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | import sys 3 | 4 | import pytest 5 | 6 | from tblib import pickling_support 7 | 8 | pytest.importorskip('twisted') 9 | 10 | 11 | def test_30(): 12 | from twisted.python.failure import Failure # noqa: PLC0415 13 | 14 | pickling_support.install() 15 | 16 | try: 17 | raise ValueError 18 | except ValueError: 19 | s = pickle.dumps(sys.exc_info()) 20 | 21 | f = None 22 | try: 23 | _etype, evalue, etb = pickle.loads(s) 24 | raise evalue.with_traceback(etb) 25 | except ValueError: 26 | f = Failure() 27 | 28 | assert f is not None 29 | -------------------------------------------------------------------------------- /tests/test_perf.py: -------------------------------------------------------------------------------- 1 | from tblib import Traceback 2 | 3 | EXAMPLE = """ 4 | Traceback (most recent call last): 5 | File "file1", line 9999, in 6 | code1 7 | File "file2", line 9999, in 8 | code2 9 | File "file3", line 9999, in 10 | code3 11 | File "file4", line 9999, in 12 | code4 13 | File "file5", line 9999, in 14 | code5 15 | File "file6", line 9999, in 16 | code6 17 | File "file7", line 9999, in 18 | code7 19 | File "file8", line 9999, in 20 | code8 21 | File "file9", line 9999, in 22 | code9 23 | """ 24 | 25 | 26 | def test_perf(benchmark): 27 | @benchmark 28 | def run(): 29 | Traceback.from_string(EXAMPLE).as_traceback() 30 | -------------------------------------------------------------------------------- /tests/test_issue65.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | 3 | from tblib import pickling_support 4 | 5 | 6 | class HTTPrettyError(Exception): 7 | pass 8 | 9 | 10 | class UnmockedError(HTTPrettyError): 11 | def __init__(self): 12 | super().__init__('No mocking was registered, and real connections are not allowed (httpretty.allow_net_connect = False).') 13 | 14 | 15 | def test_65(): 16 | pickling_support.install() 17 | 18 | try: 19 | raise UnmockedError 20 | except Exception as e: 21 | exc = e 22 | 23 | exc = pickle.loads(pickle.dumps(exc)) 24 | 25 | assert isinstance(exc, UnmockedError) 26 | assert exc.args == ('No mocking was registered, and real connections are not allowed (httpretty.allow_net_connect = False).',) 27 | assert exc.__traceback__ is not None 28 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | Authors 2 | ======= 3 | 4 | * Ionel Cristian Mărieș - https://blog.ionelmc.ro 5 | * Arcadiy Ivanov - https://github.com/arcivanov 6 | * Beckjake - https://github.com/beckjake 7 | * DRayX - https://github.com/DRayX 8 | * Jason Madden - https://github.com/jamadden 9 | * Jon Dufresne - https://github.com/jdufresne 10 | * Elliott Sales de Andrade - https://github.com/QuLogic 11 | * Victor Stinner - https://github.com/vstinner 12 | * Guido Imperiale - https://github.com/crusaderky 13 | * Alisa Sireneva - https://github.com/purplesyringa 14 | * Michał Górny - https://github.com/mgorny 15 | * Tim Maxwell - https://github.com/tmaxwell-anthropic 16 | * Haoyu Weng - https://github.com/wengh 17 | * Oldřich Jedlička - https://github.com/oldium 18 | * Colin Watson - https://github.com/cjwatson 19 | * Jacob Tomlinson - https://github.com/jacobtomlinson 20 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import re 3 | from pathlib import Path 4 | 5 | from setuptools import find_namespace_packages 6 | from setuptools import setup 7 | 8 | 9 | def read(*names, **kwargs): 10 | with Path(__file__).parent.joinpath(*names).open(encoding=kwargs.get('encoding', 'utf8')) as fh: 11 | return fh.read() 12 | 13 | 14 | setup( 15 | long_description='{}\n{}'.format( 16 | re.compile('^.. start-badges.*^.. end-badges', re.M | re.S).sub('', read('README.rst')), 17 | re.sub(':[a-z]+:`~?(.*?)`', r'``\1``', read('CHANGELOG.rst')), 18 | ), 19 | long_description_content_type='text/x-rst', 20 | packages=find_namespace_packages('src'), 21 | package_dir={'': 'src'}, 22 | py_modules=[path.stem for path in Path('src').glob('*.py')], 23 | include_package_data=True, 24 | zip_safe=False, 25 | ) 26 | -------------------------------------------------------------------------------- /.bumpversion.cfg: -------------------------------------------------------------------------------- 1 | [bumpversion] 2 | current_version = 3.2.2 3 | commit = True 4 | tag = True 5 | 6 | [bumpversion:file:pyproject.toml] 7 | search = version = "{current_version}" 8 | replace = version = "{new_version}" 9 | 10 | [bumpversion:file (badge):README.rst] 11 | search = /v{current_version}.svg 12 | replace = /v{new_version}.svg 13 | 14 | [bumpversion:file (link):README.rst] 15 | search = /v{current_version}...master 16 | replace = /v{new_version}...master 17 | 18 | [bumpversion:file:docs/conf.py] 19 | search = version = release = '{current_version}' 20 | replace = version = release = '{new_version}' 21 | 22 | [bumpversion:file:src/tblib/__init__.py] 23 | search = __version__ = '{current_version}' 24 | replace = __version__ = '{new_version}' 25 | 26 | [bumpversion:file:.cookiecutterrc] 27 | search = version: {current_version} 28 | replace = version: {new_version} 29 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | # To install the git pre-commit hooks run: 2 | # pre-commit install --install-hooks 3 | # To update the versions: 4 | # pre-commit autoupdate 5 | exclude: '^(\.tox|ci/templates|\.bumpversion\.cfg|tests/badsyntax.py)(/|$)' 6 | # Note the order is intentional to avoid multiple passes of the hooks 7 | repos: 8 | - repo: https://github.com/astral-sh/ruff-pre-commit 9 | rev: v0.14.3 10 | hooks: 11 | - id: ruff 12 | args: [--fix, --exit-non-zero-on-fix, --show-fixes, --unsafe-fixes] 13 | - id: ruff-format 14 | - repo: https://github.com/ComPWA/taplo-pre-commit 15 | rev: v0.9.3 16 | hooks: 17 | - id: taplo-format 18 | - id: taplo-lint 19 | - repo: https://github.com/pre-commit/pre-commit-hooks 20 | rev: v6.0.0 21 | hooks: 22 | - id: trailing-whitespace 23 | - id: end-of-file-fixer 24 | - id: mixed-line-ending 25 | - id: debug-statements 26 | -------------------------------------------------------------------------------- /pytest.ini: -------------------------------------------------------------------------------- 1 | [pytest] 2 | # If a pytest section is found in one of the possible config files 3 | # (pytest.ini, tox.ini or setup.cfg), then pytest will not look for any others, 4 | # so if you add a pytest config section elsewhere, 5 | # you will need to delete this section from setup.cfg. 6 | norecursedirs = 7 | migrations 8 | 9 | python_files = 10 | test_*.py 11 | *_test.py 12 | tests.py 13 | addopts = 14 | -ra 15 | --strict-markers 16 | --ignore=tests/badmodule.py 17 | --ignore=tests/badsyntax.py 18 | --doctest-modules 19 | --doctest-glob=\*.rst 20 | --tb=short 21 | --benchmark-disable 22 | testpaths = 23 | tests 24 | 25 | # Idea from: https://til.simonwillison.net/pytest/treat-warnings-as-errors 26 | filterwarnings = 27 | error 28 | # You can add exclusions, some examples: 29 | # ignore:'tblib' defines default_app_config:PendingDeprecationWarning:: 30 | # ignore:The {{% if::: 31 | # ignore:Coverage disabled via --no-cov switch! 32 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | __pycache__ 3 | 4 | # Temp files 5 | .*.sw[po] 6 | *~ 7 | *.bak 8 | .DS_Store 9 | Thumbs.db 10 | 11 | # C extensions 12 | *.so 13 | 14 | # Build and package files 15 | *.egg 16 | *.egg-info 17 | .bootstrap 18 | .build 19 | .cache 20 | .eggs 21 | .env 22 | .installed.cfg 23 | .ve 24 | bin 25 | build 26 | develop-eggs 27 | dist 28 | eggs 29 | lib 30 | lib64 31 | parts 32 | pip-wheel-metadata/ 33 | pyvenv*/ 34 | sdist 35 | var 36 | venv*/ 37 | wheelhouse 38 | 39 | # Installer logs 40 | pip-log.txt 41 | 42 | # Unit test / coverage reports 43 | .benchmarks 44 | .coverage 45 | .coverage.* 46 | .pytest 47 | .pytest_cache/ 48 | .tox 49 | coverage.xml 50 | htmlcov 51 | nosetests.xml 52 | 53 | # Translations 54 | *.mo 55 | 56 | # Buildout 57 | .mr.developer.cfg 58 | 59 | # IDE project files 60 | *.iml 61 | *.komodoproject 62 | .idea 63 | .project 64 | .pydevproject 65 | .vscode 66 | 67 | # Complexity 68 | output/*.html 69 | output/*/index.html 70 | 71 | # Sphinx 72 | docs/_build 73 | 74 | # Mypy Cache 75 | .mypy_cache/ 76 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 2-Clause License 2 | 3 | Copyright (c) 2013-2025, Ionel Cristian Mărieș. All rights reserved. 4 | 5 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the 6 | following conditions are met: 7 | 8 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following 9 | disclaimer. 10 | 11 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following 12 | disclaimer in the documentation and/or other materials provided with the distribution. 13 | 14 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 15 | INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 17 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 18 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 19 | WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 20 | THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 21 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | extensions = [ 2 | 'sphinx.ext.autodoc', 3 | 'sphinx.ext.autosummary', 4 | 'sphinx.ext.coverage', 5 | 'sphinx.ext.doctest', 6 | 'sphinx.ext.extlinks', 7 | 'sphinx.ext.ifconfig', 8 | 'sphinx.ext.napoleon', 9 | 'sphinx.ext.todo', 10 | 'sphinx.ext.viewcode', 11 | ] 12 | source_suffix = '.rst' 13 | master_doc = 'index' 14 | project = 'tblib' 15 | year = '2013-2025' 16 | author = 'Ionel Cristian Mărieș' 17 | copyright = f'{year}, {author}' 18 | version = release = '3.2.2' 19 | 20 | pygments_style = 'trac' 21 | templates_path = ['.'] 22 | extlinks = { 23 | 'issue': ('https://github.com/ionelmc/python-tblib/issues/%s', '#%s'), 24 | 'pr': ('https://github.com/ionelmc/python-tblib/pull/%s', 'PR #%s'), 25 | } 26 | 27 | html_theme = 'furo' 28 | html_theme_options = { 29 | 'source_repository': 'https://github.com/ionelmc/python-tblib/', 30 | 'source_branch': 'master', 31 | 'source_directory': 'docs/', 32 | 'footer_icons': [ 33 | { 34 | 'url': 'https://github.com/ionelmc/python-tblib/', 35 | 'html': 'github.com/ionelmc/python-tblib', 36 | }, 37 | ], 38 | } 39 | 40 | html_use_smartypants = True 41 | html_last_updated_fmt = '%b %d, %Y' 42 | html_split_index = False 43 | html_short_title = f'{project}-{version}' 44 | 45 | napoleon_use_ivar = True 46 | napoleon_use_rtype = False 47 | napoleon_use_param = False 48 | -------------------------------------------------------------------------------- /src/tblib/decorators.py: -------------------------------------------------------------------------------- 1 | import sys 2 | from functools import wraps 3 | 4 | from . import Traceback 5 | 6 | 7 | def reraise(tp, value, tb=None): 8 | try: 9 | if value is None: 10 | value = tp() 11 | if value.__traceback__ is not tb: 12 | raise value.with_traceback(tb) 13 | raise value 14 | finally: 15 | value = None 16 | tb = None 17 | 18 | 19 | class Error: 20 | def __init__(self, exc_type, exc_value, traceback): 21 | self.exc_type = exc_type 22 | self.exc_value = exc_value 23 | self.__traceback = Traceback(traceback) 24 | 25 | @property 26 | def traceback(self): 27 | return self.__traceback.as_traceback() 28 | 29 | def reraise(self): 30 | reraise(self.exc_type, self.exc_value, self.traceback) 31 | 32 | 33 | def return_error(func, exc_type=Exception): 34 | @wraps(func) 35 | def return_exceptions_wrapper(*args, **kwargs): 36 | try: 37 | return func(*args, **kwargs) 38 | except exc_type: 39 | return Error(*sys.exc_info()) 40 | 41 | return return_exceptions_wrapper 42 | 43 | 44 | returns_error = return_errors = returns_errors = return_error # cause I make too many typos 45 | 46 | 47 | @return_error 48 | def apply_with_return_error(args): 49 | """ 50 | args is a tuple where the first argument is a callable. 51 | 52 | eg:: 53 | 54 | apply_with_return_error((func, 1, 2, 3)) - this will call func(1, 2, 3) 55 | 56 | """ 57 | return args[0](*args[1:]) 58 | -------------------------------------------------------------------------------- /.cookiecutterrc: -------------------------------------------------------------------------------- 1 | # Generated by cookiepatcher, a small shim around cookiecutter (pip install cookiepatcher) 2 | 3 | default_context: 4 | c_extension_optional: 'no' 5 | c_extension_support: 'no' 6 | codacy: 'no' 7 | codacy_projectid: '-' 8 | codeclimate: 'no' 9 | codecov: 'yes' 10 | command_line_interface: 'no' 11 | command_line_interface_bin_name: '-' 12 | coveralls: 'yes' 13 | distribution_name: tblib 14 | email: contact@ionelmc.ro 15 | formatter_quote_style: single 16 | full_name: Ionel Cristian Mărieș 17 | function_name: compute 18 | github_actions: 'yes' 19 | github_actions_osx: 'yes' 20 | github_actions_windows: 'yes' 21 | license: BSD 2-Clause License 22 | module_name: core 23 | package_name: tblib 24 | pre_commit: 'yes' 25 | project_name: tblib 26 | project_short_description: Traceback serialization library. 27 | pypi_badge: 'yes' 28 | pypi_disable_upload: 'no' 29 | release_date: '2020-07-24' 30 | repo_hosting: github.com 31 | repo_hosting_domain: github.com 32 | repo_main_branch: master 33 | repo_name: python-tblib 34 | repo_username: ionelmc 35 | scrutinizer: 'no' 36 | setup_py_uses_setuptools_scm: 'no' 37 | sphinx_docs: 'yes' 38 | sphinx_docs_hosting: https://python-tblib.readthedocs.io/ 39 | sphinx_doctest: 'no' 40 | sphinx_theme: furo 41 | test_matrix_separate_coverage: 'no' 42 | tests_inside_package: 'no' 43 | version: 3.2.2 44 | version_manager: bump2version 45 | website: https://blog.ionelmc.ro/ 46 | year_from: '2013' 47 | year_to: '2025' 48 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [testenv:bootstrap] 2 | deps = 3 | jinja2 4 | tox 5 | skip_install = true 6 | commands = 7 | python ci/bootstrap.py --no-env 8 | passenv = 9 | * 10 | 11 | ; a generative tox configuration, see: https://tox.wiki/en/latest/user_guide.html#generative-environments 12 | [tox] 13 | envlist = 14 | clean, 15 | check, 16 | docs, 17 | {py39,py310,py311,py312,py313,py314,pypy39,pypy310,pypy311}, 18 | report 19 | ignore_basepython_conflict = true 20 | 21 | [testenv] 22 | setenv = 23 | PYTHONPATH={toxinidir}/tests 24 | PYTHONUNBUFFERED=yes 25 | PYTHONNODEBUGRANGES=yes 26 | passenv = 27 | * 28 | package = wheel 29 | usedevelop = false 30 | dependency_groups = 31 | test 32 | deps = 33 | pytest-cov 34 | setuptools>=80 35 | commands = 36 | {posargs:pytest --cov --cov-report=term-missing --cov-report=xml -vv tests README.rst} 37 | 38 | [testenv:check] 39 | deps = 40 | docutils 41 | check-manifest 42 | pre-commit 43 | readme-renderer 44 | pygments 45 | twine 46 | uv 47 | skip_install = true 48 | commands = 49 | uv build --sdist --out-dir build 50 | twine check build/*.tar.gz 51 | check-manifest . 52 | pre-commit run --all-files --show-diff-on-failure 53 | 54 | [testenv:docs] 55 | usedevelop = true 56 | deps = 57 | -r{toxinidir}/docs/requirements.txt 58 | commands = 59 | sphinx-build {posargs:-E} -b html docs dist/docs 60 | sphinx-build -b linkcheck docs dist/docs 61 | 62 | [testenv:report] 63 | deps = 64 | coverage 65 | skip_install = true 66 | setenv = 67 | PYTHONPATH={toxinidir}/src 68 | commands = 69 | coverage report 70 | coverage html 71 | 72 | [testenv:clean] 73 | commands = 74 | python setup.py clean 75 | coverage erase 76 | skip_install = true 77 | deps = 78 | setuptools>=80 79 | coverage 80 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Contributing 3 | ============ 4 | 5 | Contributions are welcome, and they are greatly appreciated! Every 6 | little bit helps, and credit will always be given. 7 | 8 | Bug reports 9 | =========== 10 | 11 | When `reporting a bug `_ please include: 12 | 13 | * Your operating system name and version. 14 | * Any details about your local setup that might be helpful in troubleshooting. 15 | * Detailed steps to reproduce the bug. 16 | 17 | Documentation improvements 18 | ========================== 19 | 20 | tblib could always use more documentation, whether as part of the 21 | official tblib docs, in docstrings, or even on the web in blog posts, 22 | articles, and such. 23 | 24 | Feature requests and feedback 25 | ============================= 26 | 27 | The best way to send feedback is to file an issue at https://github.com/ionelmc/python-tblib/issues. 28 | 29 | If you are proposing a feature: 30 | 31 | * Explain in detail how it would work. 32 | * Keep the scope as narrow as possible, to make it easier to implement. 33 | * Remember that this is a volunteer-driven project, and that code contributions are welcome :) 34 | 35 | Development 36 | =========== 37 | 38 | To set up `python-tblib` for local development: 39 | 40 | 1. Fork `python-tblib `_ 41 | (look for the "Fork" button). 42 | 2. Clone your fork locally:: 43 | 44 | git clone git@github.com:YOURGITHUBNAME/python-tblib.git 45 | 46 | 3. Create a branch for local development:: 47 | 48 | git checkout -b name-of-your-bugfix-or-feature 49 | 50 | Now you can make your changes locally. 51 | 52 | 4. When you're done making changes run all the checks and docs builder with one command:: 53 | 54 | tox 55 | 56 | 5. Commit your changes and push your branch to GitHub:: 57 | 58 | git add . 59 | git commit -m "Your detailed description of your changes." 60 | git push origin name-of-your-bugfix-or-feature 61 | 62 | 6. Submit a pull request through the GitHub website. 63 | 64 | Pull Request Guidelines 65 | ----------------------- 66 | 67 | If you need some code review or feedback while you're developing the code just make the pull request. 68 | 69 | For merging, you should: 70 | 71 | 1. Include passing tests (run ``tox``). 72 | 2. Update documentation when there's new API, functionality etc. 73 | 3. Add a note to ``CHANGELOG.rst`` about the changes. 74 | 4. Add yourself to ``AUTHORS.rst``. 75 | 76 | Tips 77 | ---- 78 | 79 | To run a subset of tests:: 80 | 81 | tox -e envname -- pytest -k test_myfeature 82 | 83 | To run all the test environments in *parallel*:: 84 | 85 | tox -p auto 86 | -------------------------------------------------------------------------------- /ci/templates/.github/workflows/github-actions.yml: -------------------------------------------------------------------------------- 1 | name: build 2 | on: [push, pull_request, workflow_dispatch] 3 | jobs: 4 | test: 5 | name: {{ '${{ matrix.name }}' }} 6 | runs-on: {{ '${{ matrix.os }}' }} 7 | timeout-minutes: 30 8 | strategy: 9 | fail-fast: false 10 | matrix: 11 | include: 12 | - name: 'check' 13 | python: '3.13' 14 | tox_env: 'check' 15 | os: 'ubuntu-latest' 16 | - name: 'docs' 17 | python: '3.13' 18 | tox_env: 'docs' 19 | os: 'ubuntu-latest' 20 | {% for env in tox_environments %} 21 | {% set prefix = env.split('-')[0] -%} 22 | {% set freethreaded = prefix.endswith('t') %} 23 | {% if prefix.startswith('pypy') %} 24 | {% set python %}pypy-{{ prefix[4] }}.{{ prefix[5:] }}{% endset %} 25 | {% set cpython %}pp{{ prefix[4:5] }}{% endset %} 26 | {% else %} 27 | {% set python %}{{ prefix[2] }}.{{ prefix[3:].rstrip('t') }}{% endset %} 28 | {% set cpython %}cp{{ prefix[2:] }}{% endset %} 29 | {% endif %} 30 | {% for os, python_arch in [ 31 | ['ubuntu', 'x64'], 32 | ['windows', 'x64'], 33 | ['macos', 'arm64'], 34 | ] %} 35 | - name: '{{ env }} ({{ os }}/{{ python_arch }})' 36 | python: '{{ python }}' 37 | python_arch: '{{ python_arch }}' 38 | tox_env: '{{ env }}' 39 | os: '{{ os }}-latest' 40 | cover: true 41 | {% endfor %} 42 | {% endfor %} 43 | steps: 44 | - uses: actions/checkout@v5 45 | with: 46 | fetch-depth: 0 47 | - uses: actions/setup-python@v5 48 | with: 49 | python-version: {{ '${{ matrix.python }}' }} 50 | architecture: {{ '${{ matrix.python_arch }}' }} 51 | - name: install dependencies 52 | run: | 53 | python -mpip install --progress-bar=off -r ci/requirements.txt 54 | virtualenv --version 55 | pip --version 56 | tox --version 57 | pip list --format=freeze 58 | - name: test 59 | run: > 60 | tox -e {{ '${{ matrix.tox_env }}' }} -v 61 | - uses: coverallsapp/github-action@v2 62 | if: matrix.cover 63 | continue-on-error: true 64 | with: 65 | flag-name: {{ '${{ matrix.name }}' }} 66 | parallel: true 67 | - uses: codecov/codecov-action@v5 68 | if: matrix.cover 69 | with: 70 | flags: {{ '${{ matrix.name }}' }} 71 | token: {{ '${{ secrets.CODECOV_TOKEN }}' }} 72 | verbose: true 73 | finish: 74 | needs: test 75 | if: {{ '${{ always() }}' }} 76 | runs-on: ubuntu-latest 77 | steps: 78 | - uses: coverallsapp/github-action@v2 79 | with: 80 | parallel-finished: true 81 | {{- '' }} 82 | -------------------------------------------------------------------------------- /ci/bootstrap.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import pathlib 4 | import subprocess 5 | import sys 6 | 7 | base_path: pathlib.Path = pathlib.Path(__file__).resolve().parent.parent 8 | templates_path = base_path / 'ci' / 'templates' 9 | 10 | 11 | def check_call(args): 12 | print('+', *args) 13 | subprocess.check_call(args) 14 | 15 | 16 | def exec_in_env(): 17 | env_path = base_path / '.tox' / 'bootstrap' 18 | if sys.platform == 'win32': 19 | bin_path = env_path / 'Scripts' 20 | else: 21 | bin_path = env_path / 'bin' 22 | if not env_path.exists(): 23 | print(f'Making bootstrap env in: {env_path} ...') 24 | try: 25 | check_call([sys.executable, '-m', 'venv', env_path]) 26 | except subprocess.CalledProcessError: 27 | try: 28 | check_call([sys.executable, '-m', 'virtualenv', env_path]) 29 | except subprocess.CalledProcessError: 30 | check_call(['virtualenv', env_path]) 31 | print('Installing `jinja2` into bootstrap environment...') 32 | check_call([bin_path / 'pip', 'install', 'jinja2', 'tox']) 33 | python_executable = bin_path / 'python' 34 | if not python_executable.exists(): 35 | python_executable = python_executable.with_suffix('.exe') 36 | 37 | print(f'Re-executing with: {python_executable}') 38 | print('+ exec', python_executable, __file__, '--no-env') 39 | os.execv(python_executable, [python_executable, __file__, '--no-env']) 40 | 41 | 42 | def main(): 43 | import jinja2 # noqa: PLC0415 44 | 45 | print(f'Project path: {base_path}') 46 | 47 | jinja = jinja2.Environment( 48 | loader=jinja2.FileSystemLoader(str(templates_path)), 49 | trim_blocks=True, 50 | lstrip_blocks=True, 51 | keep_trailing_newline=True, 52 | ) 53 | tox_environments = [ 54 | line.strip() 55 | # 'tox' need not be installed globally, but must be importable 56 | # by the Python that is running this script. 57 | # This uses sys.executable the same way that the call in 58 | # cookiecutter-pylibrary/hooks/post_gen_project.py 59 | # invokes this bootstrap.py itself. 60 | for line in subprocess.check_output([sys.executable, '-m', 'tox', '--listenvs'], universal_newlines=True).splitlines() 61 | ] 62 | tox_environments = [line for line in tox_environments if line.startswith('py')] 63 | for template in templates_path.rglob('*'): 64 | if template.is_file(): 65 | template_path = template.relative_to(templates_path).as_posix() 66 | destination = base_path / template_path 67 | destination.parent.mkdir(parents=True, exist_ok=True) 68 | destination.write_text(jinja.get_template(template_path).render(tox_environments=tox_environments)) 69 | print(f'Wrote {template_path}') 70 | print('DONE.') 71 | 72 | 73 | if __name__ == '__main__': 74 | args = sys.argv[1:] 75 | if args == ['--no-env']: 76 | main() 77 | elif not args: 78 | exec_in_env() 79 | else: 80 | print(f'Unexpected arguments: {args}', file=sys.stderr) 81 | sys.exit(1) 82 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = [ 3 | "setuptools>=80", 4 | ] 5 | build-backend = "setuptools.build_meta" 6 | 7 | [project] 8 | dynamic = [ 9 | "readme", 10 | ] 11 | name = "tblib" 12 | version = "3.2.2" 13 | license = "BSD-2-Clause" 14 | license-files = ["LICENSE"] 15 | description = "Traceback serialization library." 16 | authors = [ 17 | { name = "Ionel Cristian Mărieș", email = "contact@ionelmc.ro" }, 18 | ] 19 | classifiers = [ 20 | # complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers 21 | "Development Status :: 5 - Production/Stable", 22 | "Intended Audience :: Developers", 23 | "Operating System :: Unix", 24 | "Operating System :: POSIX", 25 | "Operating System :: Microsoft :: Windows", 26 | "Programming Language :: Python", 27 | "Programming Language :: Python :: 3", 28 | "Programming Language :: Python :: 3 :: Only", 29 | "Programming Language :: Python :: 3.9", 30 | "Programming Language :: Python :: 3.10", 31 | "Programming Language :: Python :: 3.11", 32 | "Programming Language :: Python :: 3.12", 33 | "Programming Language :: Python :: 3.13", 34 | "Programming Language :: Python :: 3.14", 35 | "Programming Language :: Python :: Implementation :: CPython", 36 | "Programming Language :: Python :: Implementation :: PyPy", 37 | # uncomment if you test on these interpreters: 38 | # "Programming Language :: Python :: Implementation :: IronPython", 39 | # "Programming Language :: Python :: Implementation :: Jython", 40 | # "Programming Language :: Python :: Implementation :: Stackless", 41 | "Topic :: Utilities", 42 | ] 43 | keywords = [ 44 | "traceback", 45 | "debugging", 46 | "exceptions", 47 | ] 48 | requires-python = ">=3.9" 49 | dependencies = [ 50 | ] 51 | 52 | [project.optional-dependencies] 53 | # rst = ["docutils>=0.11"] 54 | 55 | [dependency-groups] 56 | test = [ 57 | "pytest", 58 | "twisted", 59 | "pytest-benchmark", 60 | ] 61 | 62 | [project.urls] 63 | "Sources" = "https://github.com/ionelmc/python-tblib" 64 | "Documentation" = "https://python-tblib.readthedocs.io/" 65 | "Changelog" = "https://python-tblib.readthedocs.io/en/latest/changelog.html" 66 | "Issue Tracker" = "https://github.com/ionelmc/python-tblib/issues" 67 | 68 | [tool.ruff] 69 | extend-exclude = ["static", "ci/templates"] 70 | line-length = 140 71 | src = ["src", "tests"] 72 | target-version = "py39" 73 | 74 | [tool.ruff.lint.per-file-ignores] 75 | "ci/*" = ["S"] 76 | 77 | [tool.ruff.lint] 78 | ignore = [ 79 | "RUF001", # ruff-specific rules ambiguous-unicode-character-string 80 | "S101", # flake8-bandit assert 81 | "S308", # flake8-bandit suspicious-mark-safe-usage 82 | "S603", # flake8-bandit subprocess-without-shell-equals-true 83 | "S607", # flake8-bandit start-process-with-partial-path 84 | "E501", # pycodestyle line-too-long 85 | "S301", 86 | ] 87 | select = [ 88 | "B", # flake8-bugbear 89 | "C4", # flake8-comprehensions 90 | "DTZ", # flake8-datetimez 91 | "E", # pycodestyle errors 92 | "EXE", # flake8-executable 93 | "F", # pyflakes 94 | "I", # isort 95 | "INT", # flake8-gettext 96 | "PIE", # flake8-pie 97 | "PLC", # pylint convention 98 | "PLE", # pylint errors 99 | "PT", # flake8-pytest-style 100 | "PTH", # flake8-use-pathlib 101 | "RSE", # flake8-raise 102 | "RUF", # ruff-specific rules 103 | "S", # flake8-bandit 104 | "UP", # pyupgrade 105 | "W", # pycodestyle warnings 106 | ] 107 | 108 | [tool.ruff.lint.flake8-pytest-style] 109 | fixture-parentheses = false 110 | mark-parentheses = false 111 | 112 | [tool.ruff.lint.isort] 113 | forced-separate = ["conftest"] 114 | force-single-line = true 115 | 116 | [tool.ruff.format] 117 | quote-style = "single" 118 | -------------------------------------------------------------------------------- /CHANGELOG.rst: -------------------------------------------------------------------------------- 1 | 2 | Changelog 3 | ========= 4 | 5 | 3.2.2 (2025-11-12) 6 | ~~~~~~~~~~~~~~~~~~ 7 | 8 | * Fixed regression occurring with ``TimeoutError`` exceptions. They should be represented now exactly as the original when unpickling. 9 | Contributed by Jacob Tomlinson in `#85 `_. 10 | 11 | 3.2.1 (2025-10-31) 12 | ~~~~~~~~~~~~~~~~~~ 13 | 14 | * Fixed regression occurring with ``ExceptionGroup`` exceptions. That exception type is now handled specifically in the new ``unpickle_exception_with_attrs`` function (just like ``OSError``). 15 | 16 | 3.2.0 (2025-10-21) 17 | ~~~~~~~~~~~~~~~~~~ 18 | 19 | * Changed ``tblib.pickling_support.install`` to support exceptions with ``__init__`` that does match the default 20 | ``BaseException.__reduce__`` (as it expects the positional arguments to ``__init__`` to match the ``args`` attribute). 21 | 22 | Special handling for OSError (and subclasses) is also included. The errno, strerror, winerror, filename and filename2 attributes will be added in the reduce structure (if set). 23 | 24 | This will support exception subclasses that do this without defining a custom ``__reduce__``: 25 | 26 | .. code-block:: python 27 | 28 | def __init__(self): 29 | super().__init__('mistery argument') 30 | 31 | def __init__(self, mistery_argument): 32 | super().__init__() 33 | self.mistery_argument = mistery_argument 34 | 35 | Tests and POC contributed by Oldřich Jedlička in `#73 `_. 36 | * Fixed some doctest and coverage config. Contributed by Colin Watson in `#79 `_. 37 | 38 | 39 | 3.1.0 (2025-03-31) 40 | ~~~~~~~~~~~~~~~~~~ 41 | 42 | * Improved performance of ``as_traceback`` by a large factor. 43 | Contributed by Haoyu Weng in `#81 `_. 44 | * Dropped support for now-EOL Python 3.8 and added 3.13 in the test grid. 45 | 46 | 3.0.0 (2023-10-22) 47 | ~~~~~~~~~~~~~~~~~~ 48 | 49 | * Added support for ``__context__``, ``__suppress_context__`` and ``__notes__``. 50 | Contributed by Tim Maxwell in `#72 `_. 51 | * Added the ``get_locals`` argument to ``tblib.pickling_support.install()``, ``tblib.Traceback`` and ``tblib.Frame``. 52 | Fixes `#41 `_. 53 | * Dropped support for now-EOL Python 3.7 and added 3.12 in the test grid. 54 | 55 | 2.0.0 (2023-06-22) 56 | ~~~~~~~~~~~~~~~~~~ 57 | 58 | * Removed support for legacy Pythons (2.7 and 3.6) and added Python 3.11 in the test grid. 59 | * Some cleanups and refactors (mostly from ruff). 60 | 61 | 1.7.0 (2020-07-24) 62 | ~~~~~~~~~~~~~~~~~~ 63 | 64 | * Add more attributes to ``Frame`` and ``Code`` objects for pytest compatibility. Contributed by Ivanq in 65 | `#58 `_. 66 | 67 | 1.6.0 (2019-12-07) 68 | ~~~~~~~~~~~~~~~~~~ 69 | 70 | * When pickling an Exception, also pickle its traceback and the Exception chain 71 | (``raise ... from ...``). Contributed by Guido Imperiale in 72 | `#53 `_. 73 | 74 | 1.5.0 (2019-10-23) 75 | ~~~~~~~~~~~~~~~~~~ 76 | 77 | * Added support for Python 3.8. Contributed by Victor Stinner in 78 | `#42 `_. 79 | * Removed support for end of life Python 3.4. 80 | * Few CI improvements and fixes. 81 | 82 | 1.4.0 (2019-05-02) 83 | ~~~~~~~~~~~~~~~~~~ 84 | 85 | * Removed support for end of life Python 3.3. 86 | * Fixed tests for Python 3.7. Contributed by Elliott Sales de Andrade in 87 | `#36 `_. 88 | * Fixed compatibility issue with Twised (``twisted.python.failure.Failure`` expected a ``co_code`` attribute). 89 | 90 | 1.3.2 (2017-04-09) 91 | ~~~~~~~~~~~~~~~~~~ 92 | 93 | * Add support for PyPy3.5-5.7.1-beta. Previously ``AttributeError: 94 | 'Frame' object has no attribute 'clear'`` could be raised. See PyPy 95 | issue `#2532 `_. 96 | 97 | 1.3.1 (2017-03-27) 98 | ~~~~~~~~~~~~~~~~~~ 99 | 100 | * Fixed handling for tracebacks due to exceeding the recursion limit. 101 | Fixes `#15 `_. 102 | 103 | 1.3.0 (2016-03-08) 104 | ~~~~~~~~~~~~~~~~~~ 105 | 106 | * Added ``Traceback.from_string``. 107 | 108 | 1.2.0 (2015-12-18) 109 | ~~~~~~~~~~~~~~~~~~ 110 | 111 | * Fixed handling for tracebacks from generators and other internal improvements 112 | and optimizations. Contributed by DRayX in `#10 `_ 113 | and `#11 `_. 114 | 115 | 1.1.0 (2015-07-27) 116 | ~~~~~~~~~~~~~~~~~~ 117 | 118 | * Added support for Python 2.6. Contributed by Arcadiy Ivanov in 119 | `#8 `_. 120 | 121 | 1.0.0 (2015-03-30) 122 | ~~~~~~~~~~~~~~~~~~ 123 | 124 | * Added ``to_dict`` method and ``from_dict`` classmethod on Tracebacks. 125 | Contributed by beckjake in `#5 `_. 126 | -------------------------------------------------------------------------------- /tests/test_tblib.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | import traceback 3 | 4 | from tblib import Traceback 5 | from tblib import pickling_support 6 | 7 | pickling_support.install() 8 | 9 | pytest_plugins = ('pytester',) 10 | 11 | 12 | def test_get_locals(): 13 | def get_locals(frame): 14 | print(frame, frame.f_locals) 15 | if 'my_variable' in frame.f_locals: 16 | return {'my_variable': int(frame.f_locals['my_variable'])} 17 | else: 18 | return {} 19 | 20 | def func(my_arg='2'): 21 | my_variable = '1' 22 | raise ValueError(my_variable) 23 | 24 | try: 25 | func() 26 | except Exception as e: 27 | exc = e 28 | else: 29 | raise AssertionError 30 | 31 | f_locals = exc.__traceback__.tb_next.tb_frame.f_locals 32 | assert 'my_variable' in f_locals 33 | assert f_locals['my_variable'] == '1' 34 | 35 | value = Traceback(exc.__traceback__, get_locals=get_locals).as_dict() 36 | lineno = exc.__traceback__.tb_lineno 37 | assert value == { 38 | 'tb_frame': { 39 | 'f_globals': {'__name__': 'test_tblib', '__file__': __file__}, 40 | 'f_locals': {}, 41 | 'f_code': {'co_filename': __file__, 'co_name': 'test_get_locals'}, 42 | 'f_lineno': lineno + 10, 43 | }, 44 | 'tb_lineno': lineno, 45 | 'tb_next': { 46 | 'tb_frame': { 47 | 'f_globals': {'__name__': 'test_tblib', '__file__': __file__}, 48 | 'f_locals': {'my_variable': 1}, 49 | 'f_code': {'co_filename': __file__, 'co_name': 'func'}, 50 | 'f_lineno': lineno - 3, 51 | }, 52 | 'tb_lineno': lineno - 3, 53 | 'tb_next': None, 54 | }, 55 | } 56 | 57 | assert Traceback.from_dict(value).tb_next.tb_frame.f_locals == {'my_variable': 1} 58 | 59 | 60 | def test_parse_traceback(): 61 | tb1 = Traceback.from_string( 62 | """ 63 | Traceback (most recent call last): 64 | File "file1", line 123, in 65 | code1 66 | File "file2", line 234, in ??? 67 | code2 68 | File "file3", line 345, in function3 69 | File "file4", line 456, in 70 | code4 71 | KeyboardInterrupt""" 72 | ) 73 | pytb = tb1.as_traceback() 74 | assert traceback.format_tb(pytb) == [ 75 | ' File "file1", line 123, in \n', 76 | ' File "file2", line 234, in ???\n', 77 | ' File "file3", line 345, in function3\n', 78 | ] 79 | tb2 = Traceback(pytb) 80 | 81 | expected_dict = { 82 | 'tb_frame': { 83 | 'f_code': {'co_filename': 'file1', 'co_name': ''}, 84 | 'f_globals': {'__file__': 'file1', '__name__': '?'}, 85 | 'f_locals': {}, 86 | 'f_lineno': 123, 87 | }, 88 | 'tb_lineno': 123, 89 | 'tb_next': { 90 | 'tb_frame': { 91 | 'f_code': {'co_filename': 'file2', 'co_name': '???'}, 92 | 'f_globals': {'__file__': 'file2', '__name__': '?'}, 93 | 'f_locals': {}, 94 | 'f_lineno': 234, 95 | }, 96 | 'tb_lineno': 234, 97 | 'tb_next': { 98 | 'tb_frame': { 99 | 'f_code': {'co_filename': 'file3', 'co_name': 'function3'}, 100 | 'f_globals': {'__file__': 'file3', '__name__': '?'}, 101 | 'f_locals': {}, 102 | 'f_lineno': 345, 103 | }, 104 | 'tb_lineno': 345, 105 | 'tb_next': None, 106 | }, 107 | }, 108 | } 109 | tb3 = Traceback.from_dict(expected_dict) 110 | tb4 = pickle.loads(pickle.dumps(tb3)) 111 | assert tb4.as_dict() == tb3.as_dict() == tb2.as_dict() == tb1.as_dict() == expected_dict 112 | 113 | 114 | def test_large_line_number(): 115 | line_number = 2**31 - 1 116 | tb1 = Traceback.from_string( 117 | f""" 118 | Traceback (most recent call last): 119 | File "file1", line {line_number}, in 120 | code1 121 | """ 122 | ).as_traceback() 123 | assert tb1.tb_lineno == line_number 124 | 125 | 126 | def test_pytest_integration(testdir): 127 | test = testdir.makepyfile( 128 | """ 129 | from tblib import Traceback 130 | 131 | def test_raise(): 132 | tb1 = Traceback.from_string(''' 133 | Traceback (most recent call last): 134 | File "file1", line 123, in 135 | code1 136 | File "file2", line 234, in ??? 137 | code2 138 | File "file3", line 345, in function3 139 | File "file4", line 456, in "" 140 | ''') 141 | pytb = tb1.as_traceback() 142 | raise RuntimeError().with_traceback(pytb) 143 | """ 144 | ) 145 | 146 | # mode(auto / long / short / line / native / no). 147 | 148 | result = testdir.runpytest_subprocess('--tb=long', '-vv', test) 149 | result.stdout.fnmatch_lines( 150 | [ 151 | '_ _ _ _ _ _ _ _ *', 152 | '', 153 | '> [?][?][?]', 154 | '', 155 | 'file1:123:*', 156 | '_ _ _ _ _ _ _ _ *', 157 | '', 158 | '> [?][?][?]', 159 | '', 160 | 'file2:234:*', 161 | '_ _ _ _ _ _ _ _ *', 162 | '', 163 | '> [?][?][?]', 164 | '', 165 | 'file3:345:*', 166 | '_ _ _ _ _ _ _ _ *', 167 | '', 168 | '> [?][?][?]', 169 | 'E RuntimeError', 170 | '', 171 | 'file4:456: RuntimeError', 172 | '===*=== 1 failed in * ===*===', 173 | ] 174 | ) 175 | 176 | result = testdir.runpytest_subprocess('--tb=short', '-vv', test) 177 | result.stdout.fnmatch_lines( 178 | [ 179 | 'test_pytest_integration.py:*: in test_raise', 180 | ' raise RuntimeError().with_traceback(pytb)', 181 | 'file1:123: in ', 182 | ' ???', 183 | 'file2:234: in ???', 184 | ' ???', 185 | 'file3:345: in function3', 186 | ' ???', 187 | 'file4:456: in ""', 188 | ' ???', 189 | 'E RuntimeError', 190 | ] 191 | ) 192 | 193 | result = testdir.runpytest_subprocess('--tb=line', '-vv', test) 194 | result.stdout.fnmatch_lines( 195 | [ 196 | '===*=== FAILURES ===*===', 197 | 'file4:456: RuntimeError', 198 | '===*=== 1 failed in * ===*===', 199 | ] 200 | ) 201 | 202 | result = testdir.runpytest_subprocess('--tb=native', '-vv', test) 203 | result.stdout.fnmatch_lines( 204 | [ 205 | 'Traceback (most recent call last):', 206 | ' File "*test_pytest_integration.py", line *, in test_raise', 207 | ' raise RuntimeError().with_traceback(pytb)', 208 | ' File "file1", line 123, in ', 209 | ' File "file2", line 234, in ???', 210 | ' File "file3", line 345, in function3', 211 | ' File "file4", line 456, in ""', 212 | 'RuntimeError', 213 | ] 214 | ) 215 | -------------------------------------------------------------------------------- /src/tblib/pickling_support.py: -------------------------------------------------------------------------------- 1 | import copyreg 2 | import sys 3 | from functools import partial 4 | from types import TracebackType 5 | 6 | from . import Frame 7 | from . import Traceback 8 | 9 | if sys.version_info < (3, 11): 10 | ExceptionGroup = None 11 | 12 | 13 | def unpickle_traceback(tb_frame, tb_lineno, tb_next): 14 | ret = object.__new__(Traceback) 15 | ret.tb_frame = tb_frame 16 | ret.tb_lineno = tb_lineno 17 | ret.tb_next = tb_next 18 | return ret.as_traceback() 19 | 20 | 21 | def pickle_traceback(tb, *, get_locals=None): 22 | return unpickle_traceback, ( 23 | Frame(tb.tb_frame, get_locals=get_locals), 24 | tb.tb_lineno, 25 | tb.tb_next and Traceback(tb.tb_next, get_locals=get_locals), 26 | ) 27 | 28 | 29 | def unpickle_exception_with_attrs(func, attrs, cause, tb, context, suppress_context, notes, args=()): 30 | inst = func.__new__(func, *args) 31 | for key, value in attrs.items(): 32 | setattr(inst, key, value) 33 | inst.__cause__ = cause 34 | inst.__traceback__ = tb 35 | inst.__context__ = context 36 | inst.__suppress_context__ = suppress_context 37 | if notes is not None: 38 | inst.__notes__ = notes 39 | return inst 40 | 41 | 42 | # Note: Older versions of tblib will generate pickle archives that call unpickle_exception() with 43 | # fewer arguments. We assign default values to some of the arguments to support this. 44 | def unpickle_exception(func, args, cause, tb, context=None, suppress_context=False, notes=None): 45 | inst = func(*args) 46 | inst.__cause__ = cause 47 | inst.__traceback__ = tb 48 | inst.__context__ = context 49 | inst.__suppress_context__ = suppress_context 50 | if notes is not None: 51 | inst.__notes__ = notes 52 | return inst 53 | 54 | 55 | def pickle_exception( 56 | obj, builtin_reducers=(OSError.__reduce__, BaseException.__reduce__), builtin_inits=(OSError.__init__, BaseException.__init__) 57 | ): 58 | reduced_value = obj.__reduce__() 59 | if isinstance(reduced_value, str): 60 | raise TypeError('Did not expect {repr(obj)}.__reduce__() to return a string!') 61 | 62 | func = type(obj) 63 | # Detect busted objects: they have a custom __init__ but no __reduce__. 64 | # This also means the resulting exceptions may be a bit "dulled" down - the args from __reduce__ are discarded. 65 | if func.__reduce__ in builtin_reducers and func.__init__ not in builtin_inits: 66 | _, args, *optionals = reduced_value 67 | attrs = { 68 | '__dict__': obj.__dict__, 69 | 'args': obj.args, 70 | } 71 | args = () 72 | if isinstance(obj, OSError): 73 | # Only set OSError-specific attributes if they are not None 74 | # Setting them to None explicitly breaks the string representation 75 | if obj.errno is not None: 76 | attrs['errno'] = obj.errno 77 | if obj.strerror is not None: 78 | attrs['strerror'] = obj.strerror 79 | if (winerror := getattr(obj, 'winerror', None)) is not None: 80 | attrs['winerror'] = winerror 81 | if obj.filename is not None: 82 | attrs['filename'] = obj.filename 83 | if obj.filename2 is not None: 84 | attrs['filename2'] = obj.filename2 85 | if ExceptionGroup is not None and isinstance(obj, ExceptionGroup): 86 | args = (obj.message, obj.exceptions) 87 | 88 | return ( 89 | unpickle_exception_with_attrs, 90 | ( 91 | func, 92 | attrs, 93 | obj.__cause__, 94 | obj.__traceback__, 95 | obj.__context__, 96 | obj.__suppress_context__, 97 | # __notes__ doesn't exist prior to Python 3.11; and even on Python 3.11 it may be absent 98 | getattr(obj, '__notes__', None), 99 | args, 100 | ), 101 | *optionals, 102 | ) 103 | else: 104 | func, args, *optionals = reduced_value 105 | 106 | return ( 107 | unpickle_exception, 108 | ( 109 | func, 110 | args, 111 | obj.__cause__, 112 | obj.__traceback__, 113 | obj.__context__, 114 | obj.__suppress_context__, 115 | # __notes__ doesn't exist prior to Python 3.11; and even on Python 3.11 it may be absent 116 | getattr(obj, '__notes__', None), 117 | ), 118 | *optionals, 119 | ) 120 | 121 | 122 | def _get_subclasses(cls): 123 | # Depth-first traversal of all direct and indirect subclasses of cls 124 | to_visit = [cls] 125 | while to_visit: 126 | this = to_visit.pop() 127 | yield this 128 | to_visit += list(this.__subclasses__()) 129 | 130 | 131 | def install(*exc_classes_or_instances, get_locals=None): 132 | """ 133 | Args: 134 | 135 | get_locals (callable): A function that take a frame argument and returns a dict. See :class:`tblib.Traceback` class for example. 136 | """ 137 | copyreg.pickle(TracebackType, partial(pickle_traceback, get_locals=get_locals)) 138 | 139 | if not exc_classes_or_instances: 140 | for exception_cls in _get_subclasses(BaseException): 141 | copyreg.pickle(exception_cls, pickle_exception) 142 | return 143 | 144 | for exc in exc_classes_or_instances: 145 | if isinstance(exc, BaseException): 146 | _install_for_instance(exc, set()) 147 | elif isinstance(exc, type) and issubclass(exc, BaseException): 148 | copyreg.pickle(exc, pickle_exception) 149 | # Allow using @install as a decorator for Exception classes 150 | if len(exc_classes_or_instances) == 1: 151 | return exc 152 | else: 153 | raise TypeError(f'Expected subclasses or instances of BaseException, got {type(exc)}') 154 | 155 | 156 | def _install_for_instance(exc, seen): 157 | assert isinstance(exc, BaseException) 158 | 159 | # Prevent infinite recursion if we somehow get a self-referential exception. (Self-referential 160 | # exceptions should never normally happen, but if it did somehow happen, we want to pickle the 161 | # exception faithfully so the developer can troubleshoot why it happened.) 162 | if id(exc) in seen: 163 | return 164 | seen.add(id(exc)) 165 | 166 | copyreg.pickle(type(exc), pickle_exception) 167 | 168 | if exc.__cause__ is not None: 169 | _install_for_instance(exc.__cause__, seen) 170 | if exc.__context__ is not None: 171 | _install_for_instance(exc.__context__, seen) 172 | 173 | # This case is meant to cover BaseExceptionGroup on Python 3.11 as well as backports like the 174 | # exceptiongroup module 175 | if hasattr(exc, 'exceptions') and isinstance(exc.exceptions, (tuple, list)): 176 | for subexc in exc.exceptions: 177 | if isinstance(subexc, BaseException): 178 | _install_for_instance(subexc, seen) 179 | -------------------------------------------------------------------------------- /.github/workflows/github-actions.yml: -------------------------------------------------------------------------------- 1 | name: build 2 | on: [push, pull_request, workflow_dispatch] 3 | jobs: 4 | test: 5 | name: ${{ matrix.name }} 6 | runs-on: ${{ matrix.os }} 7 | timeout-minutes: 30 8 | strategy: 9 | fail-fast: false 10 | matrix: 11 | include: 12 | - name: 'check' 13 | python: '3.13' 14 | tox_env: 'check' 15 | os: 'ubuntu-latest' 16 | - name: 'docs' 17 | python: '3.13' 18 | tox_env: 'docs' 19 | os: 'ubuntu-latest' 20 | - name: 'py39 (ubuntu/x64)' 21 | python: '3.9' 22 | python_arch: 'x64' 23 | tox_env: 'py39' 24 | os: 'ubuntu-latest' 25 | cover: true 26 | - name: 'py39 (windows/x64)' 27 | python: '3.9' 28 | python_arch: 'x64' 29 | tox_env: 'py39' 30 | os: 'windows-latest' 31 | cover: true 32 | - name: 'py39 (macos/arm64)' 33 | python: '3.9' 34 | python_arch: 'arm64' 35 | tox_env: 'py39' 36 | os: 'macos-latest' 37 | cover: true 38 | - name: 'py310 (ubuntu/x64)' 39 | python: '3.10' 40 | python_arch: 'x64' 41 | tox_env: 'py310' 42 | os: 'ubuntu-latest' 43 | cover: true 44 | - name: 'py310 (windows/x64)' 45 | python: '3.10' 46 | python_arch: 'x64' 47 | tox_env: 'py310' 48 | os: 'windows-latest' 49 | cover: true 50 | - name: 'py310 (macos/arm64)' 51 | python: '3.10' 52 | python_arch: 'arm64' 53 | tox_env: 'py310' 54 | os: 'macos-latest' 55 | cover: true 56 | - name: 'py311 (ubuntu/x64)' 57 | python: '3.11' 58 | python_arch: 'x64' 59 | tox_env: 'py311' 60 | os: 'ubuntu-latest' 61 | cover: true 62 | - name: 'py311 (windows/x64)' 63 | python: '3.11' 64 | python_arch: 'x64' 65 | tox_env: 'py311' 66 | os: 'windows-latest' 67 | cover: true 68 | - name: 'py311 (macos/arm64)' 69 | python: '3.11' 70 | python_arch: 'arm64' 71 | tox_env: 'py311' 72 | os: 'macos-latest' 73 | cover: true 74 | - name: 'py312 (ubuntu/x64)' 75 | python: '3.12' 76 | python_arch: 'x64' 77 | tox_env: 'py312' 78 | os: 'ubuntu-latest' 79 | cover: true 80 | - name: 'py312 (windows/x64)' 81 | python: '3.12' 82 | python_arch: 'x64' 83 | tox_env: 'py312' 84 | os: 'windows-latest' 85 | cover: true 86 | - name: 'py312 (macos/arm64)' 87 | python: '3.12' 88 | python_arch: 'arm64' 89 | tox_env: 'py312' 90 | os: 'macos-latest' 91 | cover: true 92 | - name: 'py313 (ubuntu/x64)' 93 | python: '3.13' 94 | python_arch: 'x64' 95 | tox_env: 'py313' 96 | os: 'ubuntu-latest' 97 | cover: true 98 | - name: 'py313 (windows/x64)' 99 | python: '3.13' 100 | python_arch: 'x64' 101 | tox_env: 'py313' 102 | os: 'windows-latest' 103 | cover: true 104 | - name: 'py313 (macos/arm64)' 105 | python: '3.13' 106 | python_arch: 'arm64' 107 | tox_env: 'py313' 108 | os: 'macos-latest' 109 | cover: true 110 | - name: 'py314 (ubuntu/x64)' 111 | python: '3.14' 112 | python_arch: 'x64' 113 | tox_env: 'py314' 114 | os: 'ubuntu-latest' 115 | cover: true 116 | - name: 'py314 (windows/x64)' 117 | python: '3.14' 118 | python_arch: 'x64' 119 | tox_env: 'py314' 120 | os: 'windows-latest' 121 | cover: true 122 | - name: 'py314 (macos/arm64)' 123 | python: '3.14' 124 | python_arch: 'arm64' 125 | tox_env: 'py314' 126 | os: 'macos-latest' 127 | cover: true 128 | - name: 'pypy39 (ubuntu/x64)' 129 | python: 'pypy-3.9' 130 | python_arch: 'x64' 131 | tox_env: 'pypy39' 132 | os: 'ubuntu-latest' 133 | cover: true 134 | - name: 'pypy39 (windows/x64)' 135 | python: 'pypy-3.9' 136 | python_arch: 'x64' 137 | tox_env: 'pypy39' 138 | os: 'windows-latest' 139 | cover: true 140 | - name: 'pypy39 (macos/arm64)' 141 | python: 'pypy-3.9' 142 | python_arch: 'arm64' 143 | tox_env: 'pypy39' 144 | os: 'macos-latest' 145 | cover: true 146 | - name: 'pypy310 (ubuntu/x64)' 147 | python: 'pypy-3.10' 148 | python_arch: 'x64' 149 | tox_env: 'pypy310' 150 | os: 'ubuntu-latest' 151 | cover: true 152 | - name: 'pypy310 (windows/x64)' 153 | python: 'pypy-3.10' 154 | python_arch: 'x64' 155 | tox_env: 'pypy310' 156 | os: 'windows-latest' 157 | cover: true 158 | - name: 'pypy310 (macos/arm64)' 159 | python: 'pypy-3.10' 160 | python_arch: 'arm64' 161 | tox_env: 'pypy310' 162 | os: 'macos-latest' 163 | cover: true 164 | - name: 'pypy311 (ubuntu/x64)' 165 | python: 'pypy-3.11' 166 | python_arch: 'x64' 167 | tox_env: 'pypy311' 168 | os: 'ubuntu-latest' 169 | cover: true 170 | - name: 'pypy311 (windows/x64)' 171 | python: 'pypy-3.11' 172 | python_arch: 'x64' 173 | tox_env: 'pypy311' 174 | os: 'windows-latest' 175 | cover: true 176 | - name: 'pypy311 (macos/arm64)' 177 | python: 'pypy-3.11' 178 | python_arch: 'arm64' 179 | tox_env: 'pypy311' 180 | os: 'macos-latest' 181 | cover: true 182 | steps: 183 | - uses: actions/checkout@v5 184 | with: 185 | fetch-depth: 0 186 | - uses: actions/setup-python@v5 187 | with: 188 | python-version: ${{ matrix.python }} 189 | architecture: ${{ matrix.python_arch }} 190 | - name: install dependencies 191 | run: | 192 | python -mpip install --progress-bar=off -r ci/requirements.txt 193 | virtualenv --version 194 | pip --version 195 | tox --version 196 | pip list --format=freeze 197 | - name: test 198 | run: > 199 | tox -e ${{ matrix.tox_env }} -v 200 | - uses: coverallsapp/github-action@v2 201 | if: matrix.cover 202 | continue-on-error: true 203 | with: 204 | flag-name: ${{ matrix.name }} 205 | parallel: true 206 | - uses: codecov/codecov-action@v5 207 | if: matrix.cover 208 | with: 209 | flags: ${{ matrix.name }} 210 | token: ${{ secrets.CODECOV_TOKEN }} 211 | verbose: true 212 | finish: 213 | needs: test 214 | if: ${{ always() }} 215 | runs-on: ubuntu-latest 216 | steps: 217 | - uses: coverallsapp/github-action@v2 218 | with: 219 | parallel-finished: true 220 | -------------------------------------------------------------------------------- /src/tblib/__init__.py: -------------------------------------------------------------------------------- 1 | import re 2 | import sys 3 | 4 | __version__ = '3.2.2' 5 | __all__ = 'Code', 'Frame', 'Traceback', 'TracebackParseError' 6 | 7 | FRAME_RE = re.compile(r'^\s*File "(?P.+)", line (?P\d+)(, in (?P.+))?$') 8 | 9 | 10 | class _AttrDict(dict): 11 | __slots__ = () 12 | 13 | def __getattr__(self, name): 14 | try: 15 | return self[name] 16 | except KeyError: 17 | raise AttributeError(name) from None 18 | 19 | 20 | # noinspection PyPep8Naming 21 | class __traceback_maker(Exception): 22 | pass 23 | 24 | 25 | class TracebackParseError(Exception): 26 | pass 27 | 28 | 29 | class Code: 30 | """ 31 | Class that replicates just enough of the builtin Code object to enable serialization and traceback rendering. 32 | """ 33 | 34 | co_code = None 35 | 36 | def __init__(self, code): 37 | self.co_filename = code.co_filename 38 | self.co_name = code.co_name 39 | self.co_argcount = 0 40 | self.co_kwonlyargcount = 0 41 | self.co_varnames = () 42 | self.co_nlocals = 0 43 | self.co_stacksize = 0 44 | self.co_flags = 64 45 | self.co_firstlineno = 0 46 | 47 | 48 | class Frame: 49 | """ 50 | Class that replicates just enough of the builtin Frame object to enable serialization and traceback rendering. 51 | 52 | Args: 53 | 54 | get_locals (callable): A function that take a frame argument and returns a dict. 55 | 56 | See :class:`Traceback` class for example. 57 | """ 58 | 59 | def __init__(self, frame, *, get_locals=None): 60 | self.f_locals = {} if get_locals is None else get_locals(frame) 61 | self.f_globals = {k: v for k, v in frame.f_globals.items() if k in ('__file__', '__name__')} 62 | self.f_code = Code(frame.f_code) 63 | self.f_lineno = frame.f_lineno 64 | 65 | def clear(self): 66 | """ 67 | For compatibility with PyPy 3.5; 68 | clear() was added to frame in Python 3.4 69 | and is called by traceback.clear_frames(), which 70 | in turn is called by unittest.TestCase.assertRaises 71 | """ 72 | 73 | 74 | class Traceback: 75 | """ 76 | Class that wraps builtin Traceback objects. 77 | 78 | Args: 79 | get_locals (callable): A function that take a frame argument and returns a dict. 80 | 81 | Ideally you will only return exactly what you need, and only with simple types that can be json serializable. 82 | 83 | Example: 84 | 85 | .. code:: python 86 | 87 | def get_locals(frame): 88 | if frame.f_locals.get("__tracebackhide__"): 89 | return {"__tracebackhide__": True} 90 | else: 91 | return {} 92 | """ 93 | 94 | tb_next = None 95 | 96 | def __init__(self, tb, *, get_locals=None): 97 | self.tb_frame = Frame(tb.tb_frame, get_locals=get_locals) 98 | self.tb_lineno = int(tb.tb_lineno) 99 | 100 | # Build in place to avoid exceeding the recursion limit 101 | tb = tb.tb_next 102 | prev_traceback = self 103 | cls = type(self) 104 | while tb is not None: 105 | traceback = object.__new__(cls) 106 | traceback.tb_frame = Frame(tb.tb_frame, get_locals=get_locals) 107 | traceback.tb_lineno = int(tb.tb_lineno) 108 | prev_traceback.tb_next = traceback 109 | prev_traceback = traceback 110 | tb = tb.tb_next 111 | 112 | def as_traceback(self): 113 | """ 114 | Convert to a builtin Traceback object that is usable for raising or rendering a stacktrace. 115 | """ 116 | current = self 117 | top_tb = None 118 | tb = None 119 | stub = compile( 120 | 'raise __traceback_maker', 121 | '', 122 | 'exec', 123 | ) 124 | while current: 125 | f_code = current.tb_frame.f_code 126 | code = stub.replace( 127 | co_firstlineno=current.tb_lineno, 128 | co_argcount=0, 129 | co_filename=f_code.co_filename, 130 | co_name=f_code.co_name, 131 | co_freevars=(), 132 | co_cellvars=(), 133 | ) 134 | 135 | # noinspection PyBroadException 136 | try: 137 | exec(code, dict(current.tb_frame.f_globals), dict(current.tb_frame.f_locals)) # noqa: S102 138 | except Exception: 139 | next_tb = sys.exc_info()[2].tb_next 140 | if top_tb is None: 141 | top_tb = next_tb 142 | if tb is not None: 143 | tb.tb_next = next_tb 144 | tb = next_tb 145 | del next_tb 146 | 147 | current = current.tb_next 148 | try: 149 | return top_tb 150 | finally: 151 | del top_tb 152 | del tb 153 | 154 | to_traceback = as_traceback 155 | 156 | def as_dict(self): 157 | """ 158 | Converts to a dictionary representation. You can serialize the result to JSON as it only has 159 | builtin objects like dicts, lists, ints or strings. 160 | """ 161 | if self.tb_next is None: 162 | tb_next = None 163 | else: 164 | tb_next = self.tb_next.as_dict() 165 | 166 | code = { 167 | 'co_filename': self.tb_frame.f_code.co_filename, 168 | 'co_name': self.tb_frame.f_code.co_name, 169 | } 170 | frame = { 171 | 'f_globals': self.tb_frame.f_globals, 172 | 'f_locals': self.tb_frame.f_locals, 173 | 'f_code': code, 174 | 'f_lineno': self.tb_frame.f_lineno, 175 | } 176 | return { 177 | 'tb_frame': frame, 178 | 'tb_lineno': self.tb_lineno, 179 | 'tb_next': tb_next, 180 | } 181 | 182 | to_dict = as_dict 183 | 184 | @classmethod 185 | def from_dict(cls, dct): 186 | """ 187 | Creates an instance from a dictionary with the same structure as ``.as_dict()`` returns. 188 | """ 189 | if dct['tb_next']: 190 | tb_next = cls.from_dict(dct['tb_next']) 191 | else: 192 | tb_next = None 193 | 194 | code = _AttrDict( 195 | co_filename=dct['tb_frame']['f_code']['co_filename'], 196 | co_name=dct['tb_frame']['f_code']['co_name'], 197 | ) 198 | frame = _AttrDict( 199 | f_globals=dct['tb_frame']['f_globals'], 200 | f_locals=dct['tb_frame'].get('f_locals', {}), 201 | f_code=code, 202 | f_lineno=dct['tb_frame']['f_lineno'], 203 | ) 204 | tb = _AttrDict( 205 | tb_frame=frame, 206 | tb_lineno=dct['tb_lineno'], 207 | tb_next=tb_next, 208 | ) 209 | return cls(tb, get_locals=get_all_locals) 210 | 211 | @classmethod 212 | def from_string(cls, string, strict=True): 213 | """ 214 | Creates an instance by parsing a stacktrace. Strict means that parsing stops when lines are not indented by at least two spaces 215 | anymore. 216 | """ 217 | frames = [] 218 | header = strict 219 | 220 | for line in string.splitlines(): 221 | line = line.rstrip() 222 | if header: 223 | if line == 'Traceback (most recent call last):': 224 | header = False 225 | continue 226 | frame_match = FRAME_RE.match(line) 227 | if frame_match: 228 | frames.append(frame_match.groupdict()) 229 | elif line.startswith(' '): 230 | pass 231 | elif strict: 232 | break # traceback ended 233 | 234 | if frames: 235 | previous = None 236 | for frame in reversed(frames): 237 | previous = _AttrDict( 238 | frame, 239 | tb_frame=_AttrDict( 240 | frame, 241 | f_globals=_AttrDict( 242 | __file__=frame['co_filename'], 243 | __name__='?', 244 | ), 245 | f_locals={}, 246 | f_code=_AttrDict(frame), 247 | f_lineno=int(frame['tb_lineno']), 248 | ), 249 | tb_next=previous, 250 | ) 251 | return cls(previous) 252 | else: 253 | raise TracebackParseError(f'Could not find any frames in {string!r}.') 254 | 255 | 256 | def get_all_locals(frame): 257 | return dict(frame.f_locals) 258 | -------------------------------------------------------------------------------- /tests/test_pickle_exception.py: -------------------------------------------------------------------------------- 1 | import os 2 | from traceback import format_exception 3 | 4 | try: 5 | import copyreg 6 | except ImportError: 7 | # Python 2 8 | import copy_reg as copyreg 9 | 10 | import pickle 11 | import sys 12 | 13 | import pytest 14 | 15 | import tblib.pickling_support 16 | 17 | has_python311 = sys.version_info >= (3, 11) 18 | 19 | 20 | @pytest.fixture 21 | def clear_dispatch_table(): 22 | bak = copyreg.dispatch_table.copy() 23 | copyreg.dispatch_table.clear() 24 | yield None 25 | copyreg.dispatch_table.clear() 26 | copyreg.dispatch_table.update(bak) 27 | 28 | 29 | class CustomError(Exception): 30 | pass 31 | 32 | 33 | def strip_locations(tb_text): 34 | return tb_text.replace(' ~~^~~\n', '').replace(' ^^^^^^^^^^^^^^^^^\n', '') 35 | 36 | 37 | @pytest.mark.parametrize('protocol', [None, *list(range(1, pickle.HIGHEST_PROTOCOL + 1))]) 38 | @pytest.mark.parametrize('how', ['global', 'instance', 'class']) 39 | def test_install(clear_dispatch_table, how, protocol): 40 | if how == 'global': 41 | tblib.pickling_support.install() 42 | elif how == 'class': 43 | tblib.pickling_support.install(CustomError, ValueError, ZeroDivisionError) 44 | 45 | try: 46 | try: 47 | try: 48 | 1 / 0 # noqa: B018 49 | finally: 50 | # The ValueError's __context__ will be the ZeroDivisionError 51 | raise ValueError('blah') 52 | except Exception as e: 53 | # Python 3 only syntax 54 | # raise CustomError("foo") from e 55 | new_e = CustomError('foo') 56 | new_e.__cause__ = e 57 | if has_python311: 58 | new_e.add_note('note 1') 59 | new_e.add_note('note 2') 60 | raise new_e from e 61 | except Exception as e: 62 | exc = e 63 | else: 64 | raise AssertionError 65 | 66 | expected_format_exception = strip_locations(''.join(format_exception(type(exc), exc, exc.__traceback__))) 67 | 68 | # Populate Exception.__dict__, which is used in some cases 69 | exc.x = 1 70 | exc.__cause__.x = 2 71 | exc.__cause__.__context__.x = 3 72 | 73 | if how == 'instance': 74 | tblib.pickling_support.install(exc) 75 | if protocol: 76 | exc = pickle.loads(pickle.dumps(exc, protocol=protocol)) 77 | 78 | assert isinstance(exc, CustomError) 79 | assert exc.args == ('foo',) 80 | assert exc.x == 1 81 | assert exc.__traceback__ is not None 82 | 83 | assert isinstance(exc.__cause__, ValueError) 84 | assert exc.__cause__.__traceback__ is not None 85 | assert exc.__cause__.x == 2 86 | assert exc.__cause__.__cause__ is None 87 | 88 | assert isinstance(exc.__cause__.__context__, ZeroDivisionError) 89 | assert exc.__cause__.__context__.x == 3 90 | assert exc.__cause__.__context__.__cause__ is None 91 | assert exc.__cause__.__context__.__context__ is None 92 | 93 | if has_python311: 94 | assert exc.__notes__ == ['note 1', 'note 2'] 95 | 96 | assert expected_format_exception == strip_locations(''.join(format_exception(type(exc), exc, exc.__traceback__))) 97 | 98 | 99 | @tblib.pickling_support.install 100 | class RegisteredError(Exception): 101 | pass 102 | 103 | 104 | def test_install_decorator(): 105 | with pytest.raises(RegisteredError) as ewrap: 106 | raise RegisteredError('foo') 107 | exc = ewrap.value 108 | exc.x = 1 109 | exc = pickle.loads(pickle.dumps(exc)) 110 | 111 | assert isinstance(exc, RegisteredError) 112 | assert exc.args == ('foo',) 113 | assert exc.x == 1 114 | assert exc.__traceback__ is not None 115 | 116 | 117 | @pytest.mark.skipif(not has_python311, reason='no BaseExceptionGroup before Python 3.11') 118 | def test_install_instance_recursively(clear_dispatch_table): 119 | exc = BaseExceptionGroup('test', [ValueError('foo'), CustomError('bar')]) # noqa: F821 120 | exc.exceptions[0].__cause__ = ZeroDivisionError('baz') 121 | exc.exceptions[0].__cause__.__context__ = AttributeError('quux') 122 | 123 | tblib.pickling_support.install(exc) 124 | 125 | installed = {c for c in copyreg.dispatch_table if issubclass(c, BaseException)} 126 | assert installed == {ExceptionGroup, ValueError, CustomError, ZeroDivisionError, AttributeError} # noqa: F821 127 | 128 | 129 | def test_install_typeerror(): 130 | with pytest.raises(TypeError): 131 | tblib.pickling_support.install('foo') 132 | 133 | 134 | @pytest.mark.parametrize('protocol', [None, *list(range(1, pickle.HIGHEST_PROTOCOL + 1))]) 135 | @pytest.mark.parametrize('how', ['global', 'instance', 'class']) 136 | def test_get_locals(clear_dispatch_table, how, protocol): 137 | def get_locals(frame): 138 | if 'my_variable' in frame.f_locals: 139 | return {'my_variable': int(frame.f_locals['my_variable'])} 140 | else: 141 | return {} 142 | 143 | if how == 'global': 144 | tblib.pickling_support.install(get_locals=get_locals) 145 | elif how == 'class': 146 | tblib.pickling_support.install(CustomError, ValueError, ZeroDivisionError, get_locals=get_locals) 147 | 148 | def func(my_arg='2'): 149 | my_variable = '1' 150 | raise ValueError(my_variable) 151 | 152 | try: 153 | func() 154 | except Exception as e: 155 | exc = e 156 | else: 157 | raise AssertionError 158 | 159 | f_locals = exc.__traceback__.tb_next.tb_frame.f_locals 160 | assert 'my_variable' in f_locals 161 | assert f_locals['my_variable'] == '1' 162 | 163 | if how == 'instance': 164 | tblib.pickling_support.install(exc, get_locals=get_locals) 165 | 166 | exc = pickle.loads(pickle.dumps(exc, protocol=protocol)) 167 | assert exc.__traceback__.tb_next.tb_frame.f_locals == {'my_variable': 1} 168 | 169 | 170 | class CustomWithAttributesException(Exception): 171 | def __init__(self, message, arg1, arg2, arg3): 172 | super().__init__(message) 173 | self.values12 = (arg1, arg2) 174 | self.value3 = arg3 175 | 176 | 177 | def test_custom_with_attributes(): 178 | try: 179 | raise CustomWithAttributesException('bar', 1, 2, 3) 180 | except Exception as e: 181 | exc = e 182 | 183 | tblib.pickling_support.install(exc) 184 | exc = pickle.loads(pickle.dumps(exc)) 185 | 186 | assert isinstance(exc, CustomWithAttributesException) 187 | assert exc.args == ('bar',) 188 | assert exc.values12 == (1, 2) 189 | assert exc.value3 == 3 190 | assert exc.__traceback__ is not None 191 | 192 | 193 | class CustomOSError(OSError): 194 | def __init__(self, message, errno, strerror: str, filename, none: None, filename2): 195 | super().__init__(errno, strerror, filename, none, filename2) 196 | self.message = message 197 | 198 | 199 | def test_custom_oserror(): 200 | try: 201 | raise CustomOSError('bar', 2, 'err', 3, None, 5) 202 | except Exception as e: 203 | exc = e 204 | 205 | tblib.pickling_support.install(exc) 206 | exc = pickle.loads(pickle.dumps(exc)) 207 | 208 | assert isinstance(exc, CustomOSError) 209 | assert exc.message == 'bar' 210 | assert exc.errno == 2 211 | assert exc.strerror == 'err' 212 | assert exc.filename == 3 213 | assert exc.filename2 == 5 214 | assert exc.__traceback__ is not None 215 | 216 | 217 | def test_oserror(): 218 | try: 219 | raise OSError(2, 'err', 3, None, 5) 220 | except Exception as e: 221 | exc = e 222 | 223 | tblib.pickling_support.install(exc) 224 | exc = pickle.loads(pickle.dumps(exc)) 225 | 226 | assert isinstance(exc, OSError) 227 | assert exc.errno == 2 228 | assert exc.strerror == 'err' 229 | assert exc.filename == 3 230 | assert exc.filename2 == 5 231 | assert exc.__traceback__ is not None 232 | 233 | 234 | class OpenError(Exception): 235 | pass 236 | 237 | 238 | def bad_open(): 239 | try: 240 | raise PermissionError(13, 'Booboo', 'filename', None, None) 241 | except Exception as e: 242 | raise OpenError(e) from e 243 | 244 | 245 | def test_permissionerror(): 246 | try: 247 | bad_open() 248 | except Exception as e: 249 | exc = e 250 | 251 | tblib.pickling_support.install(exc) 252 | exc = pickle.loads(pickle.dumps(exc)) 253 | 254 | assert isinstance(exc, OpenError) 255 | assert exc.__traceback__ is not None 256 | assert repr(exc) == "OpenError(PermissionError(13, 'Booboo'))" 257 | assert str(exc) == "[Errno 13] Booboo: 'filename'" 258 | assert exc.args[0].errno == 13 259 | assert exc.args[0].strerror == 'Booboo' 260 | assert exc.args[0].filename == 'filename' 261 | 262 | 263 | class BadError(Exception): 264 | def __init__(self): 265 | super().__init__('Bad Bad Bad!') 266 | 267 | 268 | def test_baderror(): 269 | try: 270 | raise BadError 271 | except Exception as e: 272 | exc = e 273 | 274 | tblib.pickling_support.install(exc) 275 | exc = pickle.loads(pickle.dumps(exc)) 276 | 277 | assert isinstance(exc, BadError) 278 | assert exc.args == ('Bad Bad Bad!',) 279 | assert exc.__traceback__ is not None 280 | 281 | 282 | class BadError2(Exception): 283 | def __init__(self, stuff): 284 | super().__init__() 285 | self.stuff = stuff 286 | 287 | 288 | def test_baderror2(): 289 | try: 290 | raise BadError2('123') 291 | except Exception as e: 292 | exc = e 293 | 294 | tblib.pickling_support.install(exc) 295 | exc = pickle.loads(pickle.dumps(exc)) 296 | 297 | assert isinstance(exc, BadError2) 298 | assert exc.args == () 299 | assert exc.stuff == '123' 300 | assert exc.__traceback__ is not None 301 | 302 | 303 | class CustomReduceException(Exception): 304 | def __init__(self, message, arg1, arg2, arg3): 305 | super().__init__(message) 306 | self.values12 = (arg1, arg2) 307 | self.value3 = arg3 308 | 309 | def __reduce__(self): 310 | return self.__class__, self.args + self.values12 + (self.value3,) 311 | 312 | 313 | def test_custom_reduce(): 314 | try: 315 | raise CustomReduceException('foo', 1, 2, 3) 316 | except Exception as e: 317 | exc = e 318 | 319 | tblib.pickling_support.install(exc) 320 | exc = pickle.loads(pickle.dumps(exc)) 321 | 322 | assert isinstance(exc, CustomReduceException) 323 | assert exc.args == ('foo',) 324 | assert exc.values12 == (1, 2) 325 | assert exc.value3 == 3 326 | assert exc.__traceback__ is not None 327 | 328 | 329 | class CustomReduceExException(Exception): 330 | def __init__(self, message, arg1, arg2, protocol): 331 | super().__init__(message) 332 | self.values12 = (arg1, arg2) 333 | self.value3 = protocol 334 | 335 | def __reduce_ex__(self, protocol): 336 | return self.__class__, self.args + self.values12 + (self.value3,) 337 | 338 | 339 | @pytest.mark.parametrize('protocol', [None, *list(range(1, pickle.HIGHEST_PROTOCOL + 1))]) 340 | def test_custom_reduce_ex(protocol): 341 | try: 342 | raise CustomReduceExException('foo', 1, 2, 3) 343 | except Exception as e: 344 | exc = e 345 | 346 | tblib.pickling_support.install(exc) 347 | exc = pickle.loads(pickle.dumps(exc, protocol=protocol)) 348 | 349 | assert isinstance(exc, CustomReduceExException) 350 | assert exc.args == ('foo',) 351 | assert exc.values12 == (1, 2) 352 | assert exc.value3 == 3 353 | assert exc.__traceback__ is not None 354 | 355 | 356 | def test_oserror_simple(): 357 | try: 358 | raise OSError(13, 'Permission denied') 359 | except Exception as e: 360 | exc = e 361 | 362 | tblib.pickling_support.install(exc) 363 | exc = pickle.loads(pickle.dumps(exc)) 364 | 365 | assert isinstance(exc, OSError) 366 | assert exc.args == (13, 'Permission denied') 367 | assert exc.errno == 13 368 | assert exc.strerror == 'Permission denied' 369 | assert exc.__traceback__ is not None 370 | 371 | 372 | def test_real_oserror(): 373 | try: 374 | os.open('non-existing-file', os.O_RDONLY) 375 | except Exception as e: 376 | exc = e 377 | else: 378 | pytest.fail('os.open should have raised an OSError') 379 | 380 | str_output = str(exc) 381 | tblib.pickling_support.install(exc) 382 | exc = pickle.loads(pickle.dumps(exc)) 383 | 384 | assert isinstance(exc, OSError) 385 | assert exc.errno == 2 386 | assert str_output == str(exc) 387 | 388 | 389 | def test_timeouterror(): 390 | try: 391 | raise TimeoutError('stuff') 392 | except Exception as e: 393 | exc = e 394 | else: 395 | pytest.fail('os.open should have raised an TimeoutError') 396 | 397 | str_output = str(exc) 398 | tblib.pickling_support.install(exc) 399 | exc = pickle.loads(pickle.dumps(exc)) 400 | 401 | assert isinstance(exc, TimeoutError) 402 | assert exc.errno is None 403 | assert str_output == str(exc) 404 | 405 | 406 | @pytest.mark.skipif(not has_python311, reason='ExceptionGroup needs Python 3.11') 407 | def test_exception_group(): 408 | errors = [] 409 | try: 410 | e = CustomWithAttributesException('bar', 1, 2, 3) 411 | e.add_note('test_custom_with_attributes') 412 | raise e 413 | except Exception as e: 414 | errors.append(e) 415 | 416 | try: 417 | e = CustomOSError('bar', 2, 'err', 3, None, 5) 418 | e.add_note('test_custom_oserror') 419 | raise e 420 | except Exception as e: 421 | errors.append(e) 422 | 423 | try: 424 | e = OSError(2, 'err', 3, None, 5) 425 | e.add_note('test_oserror') 426 | raise e 427 | except Exception as e: 428 | errors.append(e) 429 | 430 | try: 431 | bad_open() 432 | except Exception as e: 433 | e.add_note('test_permissionerror') 434 | errors.append(e) 435 | 436 | try: 437 | raise BadError 438 | except Exception as e: 439 | e.add_note('test_baderror') 440 | errors.append(e) 441 | 442 | try: 443 | e = BadError2('123') 444 | e.add_note('test_baderror2') 445 | raise e 446 | except Exception as e: 447 | errors.append(e) 448 | 449 | try: 450 | e = CustomReduceException('foo', 1, 2, 3) 451 | e.add_note('test_custom_reduce') 452 | raise e 453 | except Exception as e: 454 | errors.append(e) 455 | 456 | try: 457 | e = OSError(13, 'Permission denied') 458 | e.add_note('test_oserror_simple') 459 | raise e 460 | except Exception as e: 461 | errors.append(e) 462 | 463 | try: 464 | os.open('non-existing-file', os.O_RDONLY) 465 | except Exception as e: 466 | e.add_note('test_real_oserror') 467 | real_oserror_str = str(e) 468 | errors.append(e) 469 | else: 470 | pytest.fail('os.open should have raised an OSError') 471 | 472 | try: 473 | raise ExceptionGroup('group error', errors) # noqa: F821 474 | except Exception as e: 475 | exc = e 476 | 477 | assert len(exc.exceptions) == 9 # before pickling 478 | 479 | tblib.pickling_support.install() 480 | exc = pickle.loads(pickle.dumps(exc)) 481 | 482 | assert len(exc.exceptions) == 9 # after unpickling 483 | 484 | assert exc.exceptions[0].__notes__ == ['test_custom_with_attributes'] 485 | assert isinstance(exc.exceptions[0], CustomWithAttributesException) 486 | assert exc.exceptions[0].args == ('bar',) 487 | assert exc.exceptions[0].values12 == (1, 2) 488 | assert exc.exceptions[0].value3 == 3 489 | assert exc.exceptions[0].__traceback__ is not None 490 | 491 | assert exc.exceptions[1].__notes__ == ['test_custom_oserror'] 492 | assert isinstance(exc.exceptions[1], CustomOSError) 493 | assert exc.exceptions[1].message == 'bar' 494 | assert exc.exceptions[1].errno == 2 495 | assert exc.exceptions[1].strerror == 'err' 496 | assert exc.exceptions[1].filename == 3 497 | assert exc.exceptions[1].filename2 == 5 498 | assert exc.exceptions[1].__traceback__ is not None 499 | 500 | assert exc.exceptions[2].__notes__ == ['test_oserror'] 501 | assert isinstance(exc.exceptions[2], OSError) 502 | assert exc.exceptions[2].errno == 2 503 | assert exc.exceptions[2].strerror == 'err' 504 | assert exc.exceptions[2].filename == 3 505 | assert exc.exceptions[2].filename2 == 5 506 | assert exc.exceptions[2].__traceback__ is not None 507 | 508 | assert exc.exceptions[3].__notes__ == ['test_permissionerror'] 509 | assert isinstance(exc.exceptions[3], OpenError) 510 | assert exc.exceptions[3].__traceback__ is not None 511 | assert repr(exc.exceptions[3]) == "OpenError(PermissionError(13, 'Booboo'))" 512 | assert str(exc.exceptions[3]) == "[Errno 13] Booboo: 'filename'" 513 | assert exc.exceptions[3].args[0].errno == 13 514 | assert exc.exceptions[3].args[0].strerror == 'Booboo' 515 | assert exc.exceptions[3].args[0].filename == 'filename' 516 | 517 | assert exc.exceptions[4].__notes__ == ['test_baderror'] 518 | assert isinstance(exc.exceptions[4], BadError) 519 | assert exc.exceptions[4].args == ('Bad Bad Bad!',) 520 | assert exc.exceptions[4].__traceback__ is not None 521 | 522 | assert exc.exceptions[5].__notes__ == ['test_baderror2'] 523 | assert isinstance(exc.exceptions[5], BadError2) 524 | assert exc.exceptions[5].args == () 525 | assert exc.exceptions[5].stuff == '123' 526 | assert exc.exceptions[5].__traceback__ is not None 527 | 528 | assert exc.exceptions[6].__notes__ == ['test_custom_reduce'] 529 | assert isinstance(exc.exceptions[6], CustomReduceException) 530 | assert exc.exceptions[6].args == ('foo',) 531 | assert exc.exceptions[6].values12 == (1, 2) 532 | assert exc.exceptions[6].value3 == 3 533 | assert exc.exceptions[6].__traceback__ is not None 534 | 535 | assert exc.exceptions[7].__notes__ == ['test_oserror_simple'] 536 | assert isinstance(exc.exceptions[7], OSError) 537 | assert exc.exceptions[7].args == (13, 'Permission denied') 538 | assert exc.exceptions[7].errno == 13 539 | assert exc.exceptions[7].strerror == 'Permission denied' 540 | assert exc.exceptions[7].__traceback__ is not None 541 | 542 | assert exc.exceptions[8].__notes__ == ['test_real_oserror'] 543 | assert isinstance(exc.exceptions[8], OSError) 544 | assert exc.exceptions[8].errno == 2 545 | assert str(exc.exceptions[8]) == real_oserror_str 546 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ======== 2 | Overview 3 | ======== 4 | 5 | .. start-badges 6 | 7 | .. list-table:: 8 | :stub-columns: 1 9 | 10 | * - docs 11 | - |docs| 12 | * - tests 13 | - |github-actions| |coveralls| |codecov| 14 | * - package 15 | - |version| |wheel| |supported-versions| |supported-implementations| |commits-since| 16 | .. |docs| image:: https://readthedocs.org/projects/python-tblib/badge/?style=flat 17 | :target: https://readthedocs.org/projects/python-tblib/ 18 | :alt: Documentation Status 19 | .. |github-actions| image:: https://github.com/ionelmc/python-tblib/actions/workflows/github-actions.yml/badge.svg 20 | :alt: GitHub Actions Build Status 21 | :target: https://github.com/ionelmc/python-tblib/actions 22 | .. |coveralls| image:: https://coveralls.io/repos/github/ionelmc/python-tblib/badge.svg?branch=master 23 | :alt: Coverage Status 24 | :target: https://coveralls.io/github/ionelmc/python-tblib?branch=master 25 | .. |codecov| image:: https://codecov.io/gh/ionelmc/python-tblib/branch/master/graphs/badge.svg?branch=master 26 | :alt: Coverage Status 27 | :target: https://app.codecov.io/github/ionelmc/python-tblib 28 | .. |version| image:: https://img.shields.io/pypi/v/tblib.svg 29 | :alt: PyPI Package latest release 30 | :target: https://pypi.org/project/tblib 31 | .. |wheel| image:: https://img.shields.io/pypi/wheel/tblib.svg 32 | :alt: PyPI Wheel 33 | :target: https://pypi.org/project/tblib 34 | .. |supported-versions| image:: https://img.shields.io/pypi/pyversions/tblib.svg 35 | :alt: Supported versions 36 | :target: https://pypi.org/project/tblib 37 | .. |supported-implementations| image:: https://img.shields.io/pypi/implementation/tblib.svg 38 | :alt: Supported implementations 39 | :target: https://pypi.org/project/tblib 40 | .. |commits-since| image:: https://img.shields.io/github/commits-since/ionelmc/python-tblib/v3.2.2.svg 41 | :alt: Commits since latest release 42 | :target: https://github.com/ionelmc/python-tblib/compare/v3.2.2...master 43 | 44 | .. end-badges 45 | 46 | Serialization library for Exceptions and Tracebacks. 47 | 48 | * Free software: BSD license 49 | 50 | It allows you to: 51 | 52 | * `Pickle `_ tracebacks and raise exceptions 53 | with pickled tracebacks in different processes. This allows better error handling when running 54 | code over multiple processes (imagine multiprocessing, billiard, futures, celery etc). 55 | * Create traceback objects from strings (the ``from_string`` method). *No pickling is used*. 56 | * Serialize tracebacks to/from plain dicts (the ``from_dict`` and ``to_dict`` methods). *No pickling is used*. 57 | * Raise the tracebacks created from the aforementioned sources. 58 | * Pickle an Exception together with its traceback and exception chain 59 | (``raise ... from ...``) *(Python 3 only)* 60 | 61 | **Again, note that using the pickle support is completely optional. You are solely responsible for 62 | security problems should you decide to use the pickle support.** 63 | 64 | Installation 65 | ============ 66 | 67 | :: 68 | 69 | pip install tblib 70 | 71 | Documentation 72 | ============= 73 | 74 | .. contents:: 75 | :local: 76 | 77 | Pickling tracebacks 78 | ~~~~~~~~~~~~~~~~~~~ 79 | 80 | **Note**: The traceback objects that come out are stripped of some attributes (like variables). But you'll be able to raise exceptions with 81 | those tracebacks or print them - that should cover 99% of the usecases. 82 | 83 | :: 84 | 85 | >>> from tblib import pickling_support 86 | >>> pickling_support.install() 87 | >>> import pickle, sys 88 | >>> def inner_0(): 89 | ... raise Exception('fail') 90 | ... 91 | >>> def inner_1(): 92 | ... inner_0() 93 | ... 94 | >>> def inner_2(): 95 | ... inner_1() 96 | ... 97 | >>> try: 98 | ... inner_2() 99 | ... except: 100 | ... s1 = pickle.dumps(sys.exc_info()) 101 | ... 102 | >>> len(s1) > 1 103 | True 104 | >>> try: 105 | ... inner_2() 106 | ... except: 107 | ... s2 = pickle.dumps(sys.exc_info(), protocol=pickle.HIGHEST_PROTOCOL) 108 | ... 109 | >>> len(s2) > 1 110 | True 111 | 112 | >>> try: 113 | ... import cPickle 114 | ... except ImportError: 115 | ... import pickle as cPickle 116 | >>> try: 117 | ... inner_2() 118 | ... except: 119 | ... s3 = cPickle.dumps(sys.exc_info(), protocol=pickle.HIGHEST_PROTOCOL) 120 | ... 121 | >>> len(s3) > 1 122 | True 123 | 124 | Unpickling tracebacks 125 | ~~~~~~~~~~~~~~~~~~~~~ 126 | 127 | :: 128 | 129 | >>> pickle.loads(s1) 130 | (<...Exception'>, Exception('fail'...), ) 131 | 132 | >>> pickle.loads(s2) 133 | (<...Exception'>, Exception('fail'...), ) 134 | 135 | >>> pickle.loads(s3) 136 | (<...Exception'>, Exception('fail'...), ) 137 | 138 | Raising 139 | ~~~~~~~ 140 | 141 | :: 142 | 143 | >>> from tblib.decorators import reraise 144 | >>> reraise(*pickle.loads(s1)) 145 | Traceback (most recent call last): 146 | ... 147 | File "", line 1, in 148 | reraise(*pickle.loads(s2)) 149 | File "", line 2, in 150 | inner_2() 151 | File "", line 2, in inner_2 152 | inner_1() 153 | File "", line 2, in inner_1 154 | inner_0() 155 | File "", line 2, in inner_0 156 | raise Exception('fail') 157 | Exception: fail 158 | >>> reraise(*pickle.loads(s2)) 159 | Traceback (most recent call last): 160 | ... 161 | File "", line 1, in 162 | reraise(*pickle.loads(s2)) 163 | File "", line 2, in 164 | inner_2() 165 | File "", line 2, in inner_2 166 | inner_1() 167 | File "", line 2, in inner_1 168 | inner_0() 169 | File "", line 2, in inner_0 170 | raise Exception('fail') 171 | Exception: fail 172 | >>> reraise(*pickle.loads(s3)) 173 | Traceback (most recent call last): 174 | ... 175 | File "", line 1, in 176 | reraise(*pickle.loads(s2)) 177 | File "", line 2, in 178 | inner_2() 179 | File "", line 2, in inner_2 180 | inner_1() 181 | File "", line 2, in inner_1 182 | inner_0() 183 | File "", line 2, in inner_0 184 | raise Exception('fail') 185 | Exception: fail 186 | 187 | Pickling Exceptions together with their traceback and chain (Python 3 only) 188 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 189 | 190 | :: 191 | 192 | >>> try: # doctest: +SKIP 193 | ... try: 194 | ... 1 / 0 195 | ... except Exception as e: 196 | ... raise Exception("foo") from e 197 | ... except Exception as e: 198 | ... s = pickle.dumps(e) 199 | >>> raise pickle.loads(s) # doctest: +SKIP 200 | Traceback (most recent call last): 201 | File "", line 3, in 202 | 1 / 0 203 | ZeroDivisionError: division by zero 204 | 205 | The above exception was the direct cause of the following exception: 206 | 207 | Traceback (most recent call last): 208 | File "", line 1, in 209 | raise pickle.loads(s) 210 | File "", line 5, in 211 | raise Exception("foo") from e 212 | Exception: foo 213 | 214 | BaseException subclasses defined after calling ``pickling_support.install()`` will 215 | **not** retain their traceback and exception chain pickling. 216 | To cover custom Exceptions, there are three options: 217 | 218 | 1. Use ``@pickling_support.install`` as a decorator for each custom Exception 219 | 220 | .. code-block:: python 221 | 222 | >>> from tblib import pickling_support 223 | >>> # Declare all imports of your package's dependencies 224 | >>> import numpy # doctest: +SKIP 225 | 226 | >>> pickling_support.install() # install for all modules imported so far 227 | 228 | >>> @pickling_support.install 229 | ... class CustomError(Exception): 230 | ... pass 231 | 232 | Eventual subclasses of ``CustomError`` will need to be decorated again. 233 | 234 | 2. Invoke ``pickling_support.install()`` after all modules have been imported and all 235 | Exception subclasses have been declared 236 | 237 | .. code-block:: python 238 | 239 | >>> # Declare all imports of your package's dependencies 240 | >>> import numpy # doctest: +SKIP 241 | >>> from tblib import pickling_support 242 | 243 | >>> # Declare your own custom Exceptions 244 | >>> class CustomError(Exception): 245 | ... pass 246 | 247 | >>> # Finally, install tblib 248 | >>> pickling_support.install() 249 | 250 | 3. Selectively install tblib for Exception instances just before they are pickled 251 | 252 | .. code-block:: python 253 | 254 | pickling_support.install(, [Exception instance], ...) 255 | 256 | The above will install tblib pickling for all listed exceptions as well as any other 257 | exceptions in their exception chains. 258 | 259 | For example, one could write a wrapper to be used with 260 | `ProcessPoolExecutor `_, 261 | `Dask.distributed `_, or similar libraries: 262 | 263 | :: 264 | 265 | >>> from tblib import pickling_support 266 | >>> def wrapper(func, *args, **kwargs): 267 | ... try: 268 | ... return func(*args, **kwargs) 269 | ... except Exception as e: 270 | ... pickling_support.install(e) 271 | ... raise 272 | 273 | What if we have a local stack, does it show correctly ? 274 | ------------------------------------------------------- 275 | 276 | Yes it does:: 277 | 278 | >>> exc_info = pickle.loads(s3) 279 | >>> def local_0(): 280 | ... reraise(*exc_info) 281 | ... 282 | >>> def local_1(): 283 | ... local_0() 284 | ... 285 | >>> def local_2(): 286 | ... local_1() 287 | ... 288 | >>> local_2() 289 | Traceback (most recent call last): 290 | File "...doctest.py", line ..., in __run 291 | compileflags, 1) in test.globs 292 | File "", line 1, in 293 | local_2() 294 | File "", line 2, in local_2 295 | local_1() 296 | File "", line 2, in local_1 297 | local_0() 298 | File "", line 2, in local_0 299 | reraise(*exc_info) 300 | File "", line 2, in 301 | inner_2() 302 | File "", line 2, in inner_2 303 | inner_1() 304 | File "", line 2, in inner_1 305 | inner_0() 306 | File "", line 2, in inner_0 307 | raise Exception('fail') 308 | Exception: fail 309 | 310 | It also supports more contrived scenarios 311 | ----------------------------------------- 312 | 313 | Like tracebacks with syntax errors:: 314 | 315 | >>> from tblib import Traceback 316 | >>> from examples import bad_syntax 317 | >>> try: 318 | ... bad_syntax() 319 | ... except: 320 | ... et, ev, tb = sys.exc_info() 321 | ... tb = Traceback(tb) 322 | ... 323 | >>> reraise(et, ev, tb.as_traceback()) 324 | Traceback (most recent call last): 325 | ... 326 | File "", line 1, in 327 | reraise(et, ev, tb.as_traceback()) 328 | File "", line 2, in 329 | bad_syntax() 330 | File "...tests...examples.py", line 18, in bad_syntax 331 | import badsyntax 332 | File "...tests...badsyntax.py", line 5 333 | is very bad 334 | ^ 335 | SyntaxError: invalid syntax 336 | 337 | Or other import failures:: 338 | 339 | >>> from examples import bad_module 340 | >>> try: 341 | ... bad_module() 342 | ... except: 343 | ... et, ev, tb = sys.exc_info() 344 | ... tb = Traceback(tb) 345 | ... 346 | >>> reraise(et, ev, tb.as_traceback()) 347 | Traceback (most recent call last): 348 | ... 349 | File "", line 1, in 350 | reraise(et, ev, tb.as_traceback()) 351 | File "", line 2, in 352 | bad_module() 353 | File "...tests...examples.py", line 23, in bad_module 354 | import badmodule 355 | File "...tests...badmodule.py", line 3, in 356 | raise Exception("boom!") 357 | Exception: boom! 358 | 359 | Or a traceback that's caused by exceeding the recursion limit (here we're 360 | forcing the type and value to have consistency across platforms):: 361 | 362 | >>> def f(): f() 363 | >>> try: 364 | ... f() 365 | ... except RuntimeError: 366 | ... et, ev, tb = sys.exc_info() 367 | ... tb = Traceback(tb) 368 | ... 369 | >>> reraise(RuntimeError, RuntimeError("maximum recursion depth exceeded"), tb.as_traceback()) 370 | Traceback (most recent call last): 371 | ... 372 | File "", line 1, in f 373 | def f(): f() 374 | File "", line 1, in f 375 | def f(): f() 376 | File "", line 1, in f 377 | def f(): f() 378 | ... 379 | RuntimeError: maximum recursion depth exceeded 380 | 381 | Reference 382 | ~~~~~~~~~ 383 | 384 | tblib.Traceback 385 | --------------- 386 | 387 | It is used by the ``pickling_support``. You can use it too if you want more flexibility:: 388 | 389 | >>> from tblib import Traceback 390 | >>> try: 391 | ... inner_2() 392 | ... except: 393 | ... et, ev, tb = sys.exc_info() 394 | ... tb = Traceback(tb) 395 | ... 396 | >>> reraise(et, ev, tb.as_traceback()) 397 | Traceback (most recent call last): 398 | ... 399 | File "", line 6, in 400 | reraise(et, ev, tb.as_traceback()) 401 | File "", line 2, in 402 | inner_2() 403 | File "", line 2, in inner_2 404 | inner_1() 405 | File "", line 2, in inner_1 406 | inner_0() 407 | File "", line 2, in inner_0 408 | raise Exception('fail') 409 | Exception: fail 410 | 411 | tblib.Traceback.to_dict 412 | ``````````````````````` 413 | 414 | You can use the ``to_dict`` method and the ``from_dict`` classmethod to 415 | convert a Traceback into and from a dictionary serializable by the stdlib 416 | json.JSONDecoder:: 417 | 418 | >>> import json 419 | >>> from pprint import pprint 420 | >>> try: 421 | ... inner_2() 422 | ... except: 423 | ... et, ev, tb = sys.exc_info() 424 | ... tb = Traceback(tb) 425 | ... tb_dict = tb.to_dict() 426 | ... pprint(tb_dict) 427 | {'tb_frame': {'f_code': {'co_filename': '', 428 | 'co_name': ''}, 429 | 'f_globals': {'__name__': '__main__'}, 430 | 'f_lineno': 5, 431 | 'f_locals': {}}, 432 | 'tb_lineno': 2, 433 | 'tb_next': {'tb_frame': {'f_code': {'co_filename': ..., 434 | 'co_name': 'inner_2'}, 435 | 'f_globals': {'__name__': '__main__'}, 436 | 'f_lineno': 2, 437 | 'f_locals': {}}, 438 | 'tb_lineno': 2, 439 | 'tb_next': {'tb_frame': {'f_code': {'co_filename': ..., 440 | 'co_name': 'inner_1'}, 441 | 'f_globals': {'__name__': '__main__'}, 442 | 'f_lineno': 2, 443 | 'f_locals': {}}, 444 | 'tb_lineno': 2, 445 | 'tb_next': {'tb_frame': {'f_code': {'co_filename': ..., 446 | 'co_name': 'inner_0'}, 447 | 'f_globals': {'__name__': '__main__'}, 448 | 'f_lineno': 2, 449 | 'f_locals': {}}, 450 | 'tb_lineno': 2, 451 | 'tb_next': None}}}} 452 | 453 | tblib.Traceback.from_dict 454 | ````````````````````````` 455 | 456 | Building on the previous example:: 457 | 458 | >>> tb_json = json.dumps(tb_dict) 459 | >>> tb = Traceback.from_dict(json.loads(tb_json)) 460 | >>> reraise(et, ev, tb.as_traceback()) 461 | Traceback (most recent call last): 462 | ... 463 | File "", line 6, in 464 | reraise(et, ev, tb.as_traceback()) 465 | File "", line 2, in 466 | inner_2() 467 | File "", line 2, in inner_2 468 | inner_1() 469 | File "", line 2, in inner_1 470 | inner_0() 471 | File "", line 2, in inner_0 472 | raise Exception('fail') 473 | Exception: fail 474 | 475 | tblib.Traceback.from_string 476 | ``````````````````````````` 477 | 478 | :: 479 | 480 | >>> tb = Traceback.from_string(""" 481 | ... File "skipped.py", line 123, in func_123 482 | ... Traceback (most recent call last): 483 | ... File "tests/examples.py", line 2, in func_a 484 | ... func_b() 485 | ... File "tests/examples.py", line 6, in func_b 486 | ... func_c() 487 | ... File "tests/examples.py", line 10, in func_c 488 | ... func_d() 489 | ... File "tests/examples.py", line 14, in func_d 490 | ... Doesn't: matter 491 | ... """) 492 | >>> reraise(et, ev, tb.as_traceback()) 493 | Traceback (most recent call last): 494 | ... 495 | File "", line 6, in 496 | reraise(et, ev, tb.as_traceback()) 497 | File "...examples.py", line 2, in func_a 498 | func_b() 499 | File "...examples.py", line 6, in func_b 500 | func_c() 501 | File "...examples.py", line 10, in func_c 502 | func_d() 503 | File "...examples.py", line 14, in func_d 504 | raise Exception('Guessing time !') 505 | Exception: fail 506 | 507 | 508 | If you use the ``strict=False`` option then parsing is a bit more lax:: 509 | 510 | >>> tb = Traceback.from_string(""" 511 | ... File "bogus.py", line 123, in bogus 512 | ... Traceback (most recent call last): 513 | ... File "tests/examples.py", line 2, in func_a 514 | ... func_b() 515 | ... File "tests/examples.py", line 6, in func_b 516 | ... func_c() 517 | ... File "tests/examples.py", line 10, in func_c 518 | ... func_d() 519 | ... File "tests/examples.py", line 14, in func_d 520 | ... Doesn't: matter 521 | ... """, strict=False) 522 | >>> reraise(et, ev, tb.as_traceback()) 523 | Traceback (most recent call last): 524 | ... 525 | File "", line 6, in 526 | reraise(et, ev, tb.as_traceback()) 527 | File "bogus.py", line 123, in bogus 528 | File "...examples.py", line 2, in func_a 529 | func_b() 530 | File "...examples.py", line 6, in func_b 531 | func_c() 532 | File "...examples.py", line 10, in func_c 533 | func_d() 534 | File "...examples.py", line 14, in func_d 535 | raise Exception('Guessing time !') 536 | Exception: fail 537 | 538 | tblib.decorators.return_error 539 | ----------------------------- 540 | 541 | :: 542 | 543 | >>> from tblib.decorators import return_error 544 | >>> inner_2r = return_error(inner_2) 545 | >>> e = inner_2r() 546 | >>> e 547 | 548 | >>> e.reraise() 549 | Traceback (most recent call last): 550 | ... 551 | File "", line 1, in 552 | e.reraise() 553 | File "...tblib...decorators.py", line 19, in reraise 554 | reraise(self.exc_type, self.exc_value, self.traceback) 555 | File "...tblib...decorators.py", line 25, in return_exceptions_wrapper 556 | return func(*args, **kwargs) 557 | File "", line 2, in inner_2 558 | inner_1() 559 | File "", line 2, in inner_1 560 | inner_0() 561 | File "", line 2, in inner_0 562 | raise Exception('fail') 563 | Exception: fail 564 | 565 | How's this useful? Imagine you're using multiprocessing like this:: 566 | 567 | # Note that Python 3.4 and later will show the remote traceback (but as a string sadly) so we skip testing this. 568 | >>> import traceback 569 | >>> from multiprocessing import Pool 570 | >>> from examples import func_a 571 | >>> pool = Pool() # doctest: +SKIP 572 | >>> try: # doctest: +SKIP 573 | ... for i in pool.map(func_a, range(5)): 574 | ... print(i) 575 | ... except: 576 | ... print(traceback.format_exc()) 577 | ... 578 | Traceback (most recent call last): 579 | File "", line 2, in 580 | for i in pool.map(func_a, range(5)): 581 | File "...multiprocessing...pool.py", line ..., in map 582 | ... 583 | File "...multiprocessing...pool.py", line ..., in get 584 | ... 585 | Exception: Guessing time ! 586 | 587 | >>> pool.terminate() # doctest: +SKIP 588 | 589 | Not very useful is it? Let's sort this out:: 590 | 591 | >>> from tblib.decorators import apply_with_return_error, Error 592 | >>> from itertools import repeat 593 | >>> pool = Pool() 594 | >>> try: # doctest: +SKIP 595 | ... for i in pool.map(apply_with_return_error, zip(repeat(func_a), range(5))): 596 | ... if isinstance(i, Error): 597 | ... i.reraise() 598 | ... else: 599 | ... print(i) 600 | ... except: 601 | ... print(traceback.format_exc()) 602 | ... 603 | Traceback (most recent call last): 604 | File "", line 4, in 605 | i.reraise() 606 | File "...tblib...decorators.py", line ..., in reraise 607 | reraise(self.exc_type, self.exc_value, self.traceback) 608 | File "...tblib...decorators.py", line ..., in reraise 609 | raise value.with_traceback(tb) 610 | File "...tblib...decorators.py", line ..., in return_exceptions_wrapper 611 | return func(*args, **kwargs) 612 | File "...tblib...decorators.py", line ..., in apply_with_return_error 613 | return args[0](*args[1:]) 614 | File "...examples.py", line 2, in func_a 615 | func_b() 616 | File "...examples.py", line 6, in func_b 617 | func_c() 618 | File "...examples.py", line 10, in func_c 619 | func_d() 620 | File "...examples.py", line 14, in func_d 621 | raise Exception('Guessing time !') 622 | Exception: Guessing time ! 623 | 624 | >>> pool.terminate() 625 | 626 | Much better ! 627 | 628 | What if we have a local call stack ? 629 | ```````````````````````````````````` 630 | 631 | :: 632 | 633 | >>> def local_0(): 634 | ... pool = Pool() 635 | ... try: 636 | ... for i in pool.map(apply_with_return_error, zip(repeat(func_a), range(5))): 637 | ... if isinstance(i, Error): 638 | ... i.reraise() 639 | ... else: 640 | ... print(i) 641 | ... finally: 642 | ... pool.close() 643 | ... 644 | >>> def local_1(): 645 | ... local_0() 646 | ... 647 | >>> def local_2(): 648 | ... local_1() 649 | ... 650 | >>> try: # doctest: +SKIP 651 | ... local_2() 652 | ... except: 653 | ... print(traceback.format_exc()) 654 | ... 655 | Traceback (most recent call last): 656 | File "", line 2, in 657 | local_2() 658 | File "", line 2, in local_2 659 | local_1() 660 | File "", line 2, in local_1 661 | local_0() 662 | File "", line 6, in local_0 663 | i.reraise() 664 | File "...tblib...decorators.py", line ..., in reraise 665 | reraise(self.exc_type, self.exc_value, self.traceback) 666 | File "...tblib...decorators.py", line ..., in reraise 667 | raise value.with_traceback(tb) 668 | File "...tblib...decorators.py", line ..., in return_exceptions_wrapper 669 | return func(*args, **kwargs) 670 | File "...tblib...decorators.py", line ..., in apply_with_return_error 671 | return args[0](*args[1:]) 672 | File "...tests...examples.py", line 2, in func_a 673 | func_b() 674 | File "...tests...examples.py", line 6, in func_b 675 | func_c() 676 | File "...tests...examples.py", line 10, in func_c 677 | func_d() 678 | File "...tests...examples.py", line 14, in func_d 679 | raise Exception('Guessing time !') 680 | Exception: Guessing time ! 681 | 682 | 683 | Other weird stuff 684 | ````````````````` 685 | 686 | Clearing traceback works (Python 3.4 and up):: 687 | 688 | >>> tb = Traceback.from_string(""" 689 | ... File "skipped.py", line 123, in func_123 690 | ... Traceback (most recent call last): 691 | ... File "tests/examples.py", line 2, in func_a 692 | ... func_b() 693 | ... File "tests/examples.py", line 6, in func_b 694 | ... func_c() 695 | ... File "tests/examples.py", line 10, in func_c 696 | ... func_d() 697 | ... File "tests/examples.py", line 14, in func_d 698 | ... Doesn't: matter 699 | ... """) 700 | >>> import traceback, sys 701 | >>> if sys.version_info > (3, 4): 702 | ... traceback.clear_frames(tb) 703 | 704 | Credits 705 | ======= 706 | 707 | * `mitsuhiko/jinja2 `_ for figuring a way to create traceback objects. 708 | --------------------------------------------------------------------------------