├── .github └── FUNDING.yml ├── .gitignore ├── .pre-commit-config.yaml ├── LICENSE ├── README.md ├── azure-pipelines.yml ├── future_annotations.py ├── requirements-dev.txt ├── setup.cfg ├── setup.py ├── testing ├── fix_coverage.py └── remove_pycdir.py ├── tests ├── __init__.py └── future_annotations_test.py └── tox.ini /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | github: asottile 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.egg-info 2 | *.pyc 3 | /.coverage 4 | /.tox 5 | /venv* 6 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v4.0.1 4 | hooks: 5 | - id: trailing-whitespace 6 | - id: end-of-file-fixer 7 | - id: check-yaml 8 | - id: name-tests-test 9 | - id: requirements-txt-fixer 10 | - repo: https://github.com/PyCQA/flake8 11 | rev: 4.0.1 12 | hooks: 13 | - id: flake8 14 | - repo: https://github.com/pre-commit/mirrors-autopep8 15 | rev: v1.5.7 16 | hooks: 17 | - id: autopep8 18 | - repo: https://github.com/asottile/setup-cfg-fmt 19 | rev: v1.19.0 20 | hooks: 21 | - id: setup-cfg-fmt 22 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2020 Anthony Sottile 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DEPRECATED 2 | 3 | with python3.6 reaching end of life, there is no need for this 4 | ___ 5 | 6 | [![Build Status](https://asottile.visualstudio.com/asottile/_apis/build/status/asottile.future-annotations?branchName=master)](https://asottile.visualstudio.com/asottile/_build/latest?definitionId=66&branchName=master) 7 | [![Azure DevOps coverage](https://img.shields.io/azure-devops/coverage/asottile/asottile/66/master.svg)](https://dev.azure.com/asottile/asottile/_build/latest?definitionId=66&branchName=master) 8 | [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/asottile/future-annotations/master.svg)](https://results.pre-commit.ci/latest/github/asottile/future-annotations/master) 9 | 10 | future-annotations 11 | ================== 12 | 13 | A backport of \_\_future\_\_ annotations to python<3.7. 14 | 15 | 16 | ## Installation 17 | 18 | `pip install future-annotations` 19 | 20 | 21 | ## Usage 22 | 23 | Include the following encoding cookie at the top of your file (this replaces 24 | the utf-8 cookie if you already have it): 25 | 26 | ```python 27 | # -*- coding: future_annotations -*- 28 | ``` 29 | 30 | And then write python3.7+ forward-annotation code as usual! 31 | 32 | ```python 33 | # -*- coding: future_annotations -*- 34 | class C: 35 | @classmethod 36 | def make(cls) -> C: 37 | return cls() 38 | 39 | print(C.make()) 40 | ``` 41 | 42 | ```console 43 | $ python3.6 main.py 44 | <__main__.C object at 0x7fb50825dd90> 45 | $ mypy main.py 46 | Success: no issues found in 1 source file 47 | ``` 48 | 49 | ## Showing transformed source 50 | 51 | `future-annotations` also includes a cli to show transformed source. 52 | 53 | ```console 54 | $ future-annotations-show main.py 55 | # ****************************** -*- 56 | class C: 57 | @classmethod 58 | def make(cls) -> 'C': 59 | return cls() 60 | 61 | print(C.make()) 62 | ``` 63 | 64 | ## How does this work? 65 | 66 | `future-annotations` has two parts: 67 | 68 | 1. A utf-8 compatible `codec` which performs source manipulation 69 | - The `codec` first decodes the source bytes using the UTF-8 codec 70 | - The `codec` then leverages 71 | [tokenize-rt](https://github.com/asottile/tokenize-rt) to rewrite 72 | annotations. 73 | 2. A `.pth` file which registers a codec on interpreter startup. 74 | 75 | ## when you aren't using normal `site` registration 76 | 77 | in setups (such as aws lambda) where you utilize `PYTHONPATH` or `sys.path` 78 | instead of truly installed packages, the `.pth` magic above will not take. 79 | 80 | for those circumstances, you'll need to manually initialize `future-annotations` 81 | in a non-annotations wrapper. for instance: 82 | 83 | ```python 84 | import future_annotations 85 | 86 | future_annotations.register() 87 | 88 | from actual_main import main 89 | 90 | if __name__ == '__main__': 91 | raise SystemExit(main()) 92 | ``` 93 | 94 | ## you may also like 95 | 96 | - [future-breakpoint](https://github.com/asottile/future-breakpoint) 97 | - [future-fstrings](https://github.com/asottile/future-fstrings) 98 | -------------------------------------------------------------------------------- /azure-pipelines.yml: -------------------------------------------------------------------------------- 1 | trigger: 2 | branches: 3 | include: [master, test-me-*] 4 | tags: 5 | include: ['*'] 6 | 7 | resources: 8 | repositories: 9 | - repository: asottile 10 | type: github 11 | endpoint: github 12 | name: asottile/azure-pipeline-templates 13 | ref: refs/tags/v2.1.0 14 | 15 | jobs: 16 | - template: job--python-tox.yml@asottile 17 | parameters: 18 | toxenvs: [pypy3, py36, py37, py38] 19 | os: linux 20 | -------------------------------------------------------------------------------- /future_annotations.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import ast 3 | import codecs 4 | import encodings 5 | import io 6 | import sys 7 | import tokenize 8 | import warnings 9 | from typing import Match 10 | from typing import Optional 11 | from typing import Sequence 12 | from typing import Set 13 | from typing import Tuple 14 | 15 | import tokenize_rt 16 | 17 | 18 | def _ast_parse(contents_text: str) -> ast.Module: 19 | # intentionally ignore warnings, we might be fixing warning-ridden syntax 20 | with warnings.catch_warnings(): 21 | warnings.simplefilter('ignore') 22 | return ast.parse(contents_text.encode()) 23 | 24 | 25 | def _ast_to_offset(node: ast.expr) -> tokenize_rt.Offset: 26 | return tokenize_rt.Offset(node.lineno, node.col_offset) 27 | 28 | 29 | class Visitor(ast.NodeVisitor): 30 | def __init__(self) -> None: 31 | self.offsets: Set[tokenize_rt.Offset] = set() 32 | 33 | def visit_AnnAssign(self, node: ast.AnnAssign) -> None: 34 | self.offsets.add(_ast_to_offset(node.annotation)) 35 | self.generic_visit(node) 36 | 37 | def visit_FunctionDef(self, node: ast.FunctionDef) -> None: 38 | args = [] 39 | if hasattr(node.args, 'posonlyargs'): # pragma: no cover (py38+) 40 | args.extend(node.args.posonlyargs) 41 | args.extend(node.args.args) 42 | if node.args.vararg is not None: 43 | args.append(node.args.vararg) 44 | args.extend(node.args.kwonlyargs) 45 | if node.args.kwarg is not None: 46 | args.append(node.args.kwarg) 47 | 48 | for arg in args: 49 | if arg.annotation is not None: 50 | self.offsets.add(_ast_to_offset(arg.annotation)) 51 | 52 | if node.returns is not None: 53 | self.offsets.add(_ast_to_offset(node.returns)) 54 | 55 | self.generic_visit(node) 56 | 57 | 58 | utf_8 = encodings.search_function('utf8') 59 | 60 | 61 | def _new_coding_cookie(match: Match[str]) -> str: 62 | s = match[0] 63 | i = 0 64 | while s[i].isspace(): 65 | i += 1 66 | ret = f'{s[:i]}# {"*" * (len(s) - 2 - i)}' 67 | assert len(ret) == len(s), (len(ret), len(s)) 68 | return ret 69 | 70 | 71 | def decode(b: bytes, errors: str = 'strict') -> Tuple[str, int]: 72 | u, length = utf_8.decode(b, errors) 73 | 74 | # replace encoding cookie so there isn't a recursion problem 75 | lines = u.splitlines(True) 76 | for idx in (0, 1): 77 | if idx >= len(lines): 78 | break 79 | lines[idx] = tokenize.cookie_re.sub(_new_coding_cookie, lines[idx]) 80 | u = ''.join(lines) 81 | 82 | visitor = Visitor() 83 | visitor.visit(_ast_parse(u)) 84 | 85 | tokens = tokenize_rt.src_to_tokens(u) 86 | for i, token in tokenize_rt.reversed_enumerate(tokens): 87 | if token.offset in visitor.offsets: 88 | # look forward for a `:`, `,`, `=`, ')' 89 | depth = 0 90 | j = i + 1 91 | while depth or tokens[j].src not in {':', ',', '=', ')', '\n'}: 92 | if tokens[j].src in {'(', '{', '['}: 93 | depth += 1 94 | elif tokens[j].src in {')', '}', ']'}: 95 | depth -= 1 96 | j += 1 97 | j -= 1 98 | 99 | # look backward to delete whitespace / comments / etc. 100 | while tokens[j].name in tokenize_rt.NON_CODING_TOKENS: 101 | j -= 1 102 | 103 | quoted = repr(tokenize_rt.tokens_to_src(tokens[i:j + 1])) 104 | tokens[i:j + 1] = [tokenize_rt.Token('STRING', quoted)] 105 | 106 | return tokenize_rt.tokens_to_src(tokens), length 107 | 108 | 109 | class IncrementalDecoder(codecs.BufferedIncrementalDecoder): 110 | def _buffer_decode(self, input, errors, final): # pragma: no cover 111 | if final: 112 | return decode(input, errors) 113 | else: 114 | return '', 0 115 | 116 | 117 | class StreamReader(utf_8.streamreader): 118 | """decode is deferred to support better error messages""" 119 | _stream = None 120 | _decoded = False 121 | 122 | @property 123 | def stream(self): 124 | if not self._decoded: 125 | text, _ = decode(self._stream.read()) 126 | self._stream = io.BytesIO(text.encode('UTF-8')) 127 | self._decoded = True 128 | return self._stream 129 | 130 | @stream.setter 131 | def stream(self, stream): 132 | self._stream = stream 133 | self._decoded = False 134 | 135 | 136 | # codec api 137 | 138 | codec_map = { 139 | name: codecs.CodecInfo( 140 | name=name, 141 | encode=utf_8.encode, 142 | decode=decode, 143 | incrementalencoder=utf_8.incrementalencoder, 144 | incrementaldecoder=IncrementalDecoder, 145 | streamreader=StreamReader, 146 | streamwriter=utf_8.streamwriter, 147 | ) 148 | for name in ('future-annotations', 'future_annotations') 149 | } 150 | 151 | 152 | def register() -> None: # pragma: no cover 153 | codecs.register(codec_map.get) 154 | 155 | 156 | def main(argv: Optional[Sequence[str]] = None) -> int: 157 | parser = argparse.ArgumentParser(description='Prints transformed source.') 158 | parser.add_argument('filename') 159 | args = parser.parse_args(argv) 160 | 161 | with open(args.filename, 'rb') as f: 162 | text, _ = decode(f.read()) 163 | getattr(sys.stdout, 'buffer', sys.stdout).write(text.encode('UTF-8')) 164 | return 0 165 | 166 | 167 | if __name__ == '__main__': 168 | raise SystemExit(main()) 169 | -------------------------------------------------------------------------------- /requirements-dev.txt: -------------------------------------------------------------------------------- 1 | covdefaults>=1.1.1 2 | coverage>=5,<6 3 | pre-commit 4 | pytest 5 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = future_annotations 3 | version = 1.0.0 4 | description = A backport of __future__ annotations to python<3.7 5 | long_description = file: README.md 6 | long_description_content_type = text/markdown 7 | url = https://github.com/asottile/future-annotations 8 | author = Anthony Sottile 9 | author_email = asottile@umich.edu 10 | license = MIT 11 | license_file = LICENSE 12 | classifiers = 13 | License :: OSI Approved :: MIT License 14 | Programming Language :: Python :: 3 15 | Programming Language :: Python :: 3 :: Only 16 | Programming Language :: Python :: 3.6 17 | Programming Language :: Python :: 3.7 18 | Programming Language :: Python :: 3.8 19 | Programming Language :: Python :: 3.9 20 | Programming Language :: Python :: 3.10 21 | Programming Language :: Python :: Implementation :: CPython 22 | Programming Language :: Python :: Implementation :: PyPy 23 | 24 | [options] 25 | py_modules = future_annotations 26 | install_requires = 27 | tokenize-rt>=3 28 | python_requires = >=3.6.1 29 | 30 | [options.entry_points] 31 | console_scripts = 32 | future-annotations-show=future_annotations:main 33 | 34 | [options.extras_require] 35 | rewrite = 36 | tokenize-rt>=3 37 | 38 | [bdist_wheel] 39 | universal = True 40 | 41 | [coverage:run] 42 | plugins = covdefaults 43 | 44 | [coverage:covdefaults] 45 | subtract_omit = */.tox/* 46 | 47 | [mypy] 48 | check_untyped_defs = true 49 | disallow_any_generics = true 50 | disallow_incomplete_defs = true 51 | disallow_untyped_defs = true 52 | no_implicit_optional = true 53 | warn_redundant_casts = true 54 | warn_unused_ignores = true 55 | 56 | [mypy-testing.*] 57 | disallow_untyped_defs = false 58 | 59 | [mypy-tests.*] 60 | disallow_untyped_defs = false 61 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import distutils 2 | import os.path 3 | 4 | from setuptools import setup 5 | from setuptools.command.install import install as _install 6 | 7 | 8 | PTH = ( 9 | 'try:\n' 10 | ' import future_annotations\n' 11 | 'except ImportError:\n' 12 | ' pass\n' 13 | 'else:\n' 14 | ' future_annotations.register()\n' 15 | ) 16 | 17 | 18 | class install(_install): 19 | def initialize_options(self): 20 | _install.initialize_options(self) 21 | # Use this prefix to get loaded as early as possible 22 | name = 'aaaaa_' + self.distribution.metadata.name 23 | 24 | contents = f'import sys; exec({PTH!r})\n' 25 | self.extra_path = (name, contents) 26 | 27 | def finalize_options(self): 28 | _install.finalize_options(self) 29 | 30 | install_suffix = os.path.relpath( 31 | self.install_lib, self.install_libbase, 32 | ) 33 | if install_suffix == '.': 34 | distutils.log.info('skipping install of .pth during easy-install') 35 | elif install_suffix == self.extra_path[1]: 36 | self.install_lib = self.install_libbase 37 | distutils.log.info( 38 | "will install .pth to '%s.pth'", 39 | os.path.join(self.install_lib, self.extra_path[0]), 40 | ) 41 | else: 42 | raise AssertionError( 43 | 'unexpected install_suffix', 44 | self.install_lib, self.install_libbase, install_suffix, 45 | ) 46 | 47 | 48 | setup(cmdclass={'install': install}) 49 | -------------------------------------------------------------------------------- /testing/fix_coverage.py: -------------------------------------------------------------------------------- 1 | import sqlite3 2 | 3 | QUERY_SITE_PACKAGES = '''\ 4 | SELECT id FROM file WHERE path LIKE '%site-packages/future_annotations.py' 5 | ''' 6 | QUERY_NOT_SITE_PACKAGES = '''\ 7 | SELECT id FROM file 8 | WHERE ( 9 | path LIKE '%/future_annotations.py' AND 10 | path NOT LIKE '%site-packages/future_annotations.py' 11 | ) 12 | ''' 13 | QUERY_TEST = '''\ 14 | SELECT id FROM file WHERE path LIKE '%/future_annotations_test.py' 15 | ''' 16 | MERGE_FILE_IN_ARC = 'UPDATE arc SET file_id = ? WHERE file_id = ?' 17 | DELETE_FROM_ARC = 'DELETE FROM arc WHERE file_id NOT IN (?, ?)' 18 | DELETE_FROM_FILE = 'DELETE FROM file WHERE id NOT IN (?, ?)' 19 | 20 | 21 | def main() -> int: 22 | with sqlite3.connect('.coverage') as db: 23 | (site_packages,) = db.execute(QUERY_SITE_PACKAGES).fetchone() 24 | (src,) = db.execute(QUERY_NOT_SITE_PACKAGES).fetchone() 25 | (test,) = db.execute(QUERY_TEST).fetchone() 26 | db.execute(MERGE_FILE_IN_ARC, (src, site_packages)) 27 | db.execute(DELETE_FROM_ARC, (src, test)) 28 | db.execute(DELETE_FROM_FILE, (src, test)) 29 | return 0 30 | 31 | 32 | if __name__ == '__main__': 33 | raise SystemExit(main()) 34 | -------------------------------------------------------------------------------- /testing/remove_pycdir.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import shutil 3 | 4 | 5 | def main() -> int: 6 | pth = 'tests/__pycache__' 7 | if os.path.exists(pth): 8 | shutil.rmtree(pth) 9 | return 0 10 | 11 | 12 | if __name__ == '__main__': 13 | raise SystemExit(main()) 14 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/asottile-archive/future-annotations/741a20704066a6cc10e1780b7d22d5933f74f594/tests/__init__.py -------------------------------------------------------------------------------- /tests/future_annotations_test.py: -------------------------------------------------------------------------------- 1 | # -*- coding: future_annotations -*- 2 | import importlib 3 | import io 4 | import tokenize 5 | from typing import Optional 6 | from typing import TYPE_CHECKING 7 | 8 | import pytest 9 | 10 | import future_annotations 11 | 12 | if TYPE_CHECKING: 13 | from typing import Protocol 14 | else: 15 | Protocol = object 16 | 17 | 18 | @pytest.mark.parametrize( 19 | ('s', 'expected'), 20 | ( 21 | ('# coding: wat', '# ***********'), 22 | (' # coding: wat', ' # ***********'), 23 | ), 24 | ) 25 | def test_new_coding_cookie(s, expected): 26 | matched = tokenize.cookie_re.match(s) 27 | assert matched 28 | replaced = tokenize.cookie_re.sub(future_annotations._new_coding_cookie, s) 29 | assert replaced == expected 30 | 31 | 32 | def test_empty_file(): 33 | assert future_annotations.decode(b'') == ('', 0) 34 | 35 | 36 | def test_multiple_tokens_until_stopping(): 37 | b_src = b'''\ 38 | def f(x: str # ohai 39 | ,): ... 40 | ''' 41 | expected = '''\ 42 | def f(x: 'str' # ohai 43 | ,): ... 44 | ''' 45 | src, _ = future_annotations.decode(b_src) 46 | assert src == expected 47 | 48 | 49 | def test_streamreader_read(): 50 | reader = future_annotations.StreamReader(io.BytesIO(b'def f(x: str): ...')) 51 | assert reader.read() == "def f(x: 'str'): ..." 52 | 53 | 54 | class C(Protocol): 55 | c: C # noqa: F821 56 | def make_arg(self, a: C) -> C: ... # noqa: F821 57 | def make_stararg(self, *a: C) -> C: ... # noqa: F821 58 | def make_namedonlyarg(self, *, a: C) -> C: ... # noqa: F821 59 | def make_starstararg(self, **a: C) -> C: ... # noqa: F821 60 | def make_opt(self, a: Optional[C]) -> C: ... # noqa: F821 61 | 62 | 63 | def test_it_works(): 64 | assert C.__annotations__ == {'c': 'C'} 65 | assert C.make_arg.__annotations__ == {'a': 'C', 'return': 'C'} 66 | assert C.make_stararg.__annotations__ == {'a': 'C', 'return': 'C'} 67 | assert C.make_namedonlyarg.__annotations__ == {'a': 'C', 'return': 'C'} 68 | assert C.make_starstararg.__annotations__ == {'a': 'C', 'return': 'C'} 69 | assert C.make_opt.__annotations__ == {'a': 'Optional[C]', 'return': 'C'} 70 | 71 | 72 | def test_main(tmpdir, capsys): 73 | f = tmpdir.join('f.py') 74 | f.write( 75 | '# -*- coding: future-annotations -*-\n' 76 | 'class C:\n' 77 | ' c: C\n' 78 | ) 79 | assert not future_annotations.main((str(f),)) 80 | out, _ = capsys.readouterr() 81 | assert out == ( 82 | '# ****************************** -*-\n' 83 | 'class C:\n' 84 | " c: 'C'\n" 85 | ) 86 | 87 | 88 | def test_fix_coverage(): 89 | """Because our module is loaded so early in python startup, coverage 90 | doesn't have a chance to instrument the module-level scope. 91 | 92 | Run this last so it doesn't interfere with tests in any way. 93 | """ 94 | importlib.reload(future_annotations) 95 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py36,py37,pypy3,pre-commit 3 | 4 | [testenv] 5 | deps = -rrequirements-dev.txt 6 | extras = rewrite 7 | commands = 8 | # Since our encoding modifies the source, clear the pyc files 9 | python testing/remove_pycdir.py 10 | coverage erase 11 | coverage run -m pytest {posargs:tests} 12 | python testing/fix_coverage.py 13 | coverage report --fail-under 100 14 | 15 | [testenv:pre-commit] 16 | skip_install = true 17 | deps = pre-commit 18 | commands = pre-commit run --all-files --show-diff-on-failure 19 | 20 | [pep8] 21 | ignore = E265,E501,W504 22 | --------------------------------------------------------------------------------