├── .github └── workflows │ └── main.yml ├── .gitignore ├── .pre-commit-config.yaml ├── LICENSE ├── README.md ├── classify_imports.py ├── requirements-dev.txt ├── setup.cfg ├── setup.py ├── tests ├── __init__.py └── classify_imports_test.py └── tox.ini /.github/workflows/main.yml: -------------------------------------------------------------------------------- 1 | name: main 2 | 3 | on: 4 | push: 5 | branches: [main, test-me-*] 6 | tags: '*' 7 | pull_request: 8 | 9 | jobs: 10 | main-windows: 11 | uses: asottile/workflows/.github/workflows/tox.yml@v1.8.1 12 | with: 13 | env: '["py39"]' 14 | os: windows-latest 15 | main-linux: 16 | uses: asottile/workflows/.github/workflows/tox.yml@v1.8.1 17 | with: 18 | env: '["py39", "py310", "py311", "py312"]' 19 | os: ubuntu-latest 20 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.egg-info 2 | *.py[co] 3 | /.coverage 4 | /.tox 5 | /dist 6 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v5.0.0 4 | hooks: 5 | - id: trailing-whitespace 6 | - id: end-of-file-fixer 7 | - id: check-yaml 8 | - id: debug-statements 9 | - id: double-quote-string-fixer 10 | - id: name-tests-test 11 | - id: requirements-txt-fixer 12 | - repo: https://github.com/asottile/setup-cfg-fmt 13 | rev: v2.8.0 14 | hooks: 15 | - id: setup-cfg-fmt 16 | - repo: https://github.com/asottile/reorder-python-imports 17 | rev: v3.15.0 18 | hooks: 19 | - id: reorder-python-imports 20 | args: [--py39-plus, --add-import, 'from __future__ import annotations'] 21 | - repo: https://github.com/asottile/add-trailing-comma 22 | rev: v3.2.0 23 | hooks: 24 | - id: add-trailing-comma 25 | - repo: https://github.com/asottile/pyupgrade 26 | rev: v3.20.0 27 | hooks: 28 | - id: pyupgrade 29 | args: [--py39-plus] 30 | - repo: https://github.com/hhatto/autopep8 31 | rev: v2.3.2 32 | hooks: 33 | - id: autopep8 34 | - repo: https://github.com/PyCQA/flake8 35 | rev: 7.2.0 36 | hooks: 37 | - id: flake8 38 | - repo: https://github.com/pre-commit/mirrors-mypy 39 | rev: v1.15.0 40 | hooks: 41 | - id: mypy 42 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014 Anthony Sottile 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![build status](https://github.com/asottile/classify-imports/actions/workflows/main.yml/badge.svg)](https://github.com/asottile/classify-imports/actions/workflows/main.yml) 2 | [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/asottile/classify-imports/main.svg)](https://results.pre-commit.ci/latest/github/asottile/classify-imports/main) 3 | 4 | classify-imports 5 | ================ 6 | 7 | Utilities for refactoring imports in python-like syntax. 8 | 9 | ## installation 10 | 11 | ```bash 12 | pip install classify-imports 13 | ``` 14 | 15 | ## examples 16 | 17 | ### splitting an import object 18 | 19 | ```pycon 20 | >>> from classify_imports import import_obj_from_str 21 | >>> obj = import_obj_from_str('import foo, bar, baz') 22 | >>> [str(i) for i in obj.split()] 23 | ['import foo\n', 'import bar\n', 'import baz\n'] 24 | ``` 25 | 26 | ### sorting import objects 27 | 28 | ```pycon 29 | # Or to partition into blocks (even with mixed imports) 30 | >>> import pprint 31 | >>> from classify_imports import import_obj_from_str, sort 32 | >>> partitioned = sort( 33 | [ 34 | import_obj_from_str('from classify_imports import sort'), 35 | import_obj_from_str('import sys'), 36 | import_obj_from_str('from pyramid.view import view_config'), 37 | import_obj_from_str('import cached_property'), 38 | ], 39 | ) 40 | >>> pprint.pprint(partitioned) 41 | ( 42 | (import_obj_from_str('import sys\n'),), 43 | ( 44 | import_obj_from_str('import cached_property\n'), 45 | import_obj_from_str('from pyramid.view import view_config\n'), 46 | ), 47 | (import_obj_from_str('from classify_imports import sort\n'),), 48 | ) 49 | 50 | ``` 51 | 52 | ### classify a module 53 | 54 | ```pycon 55 | >>> from classify_imports import classify_base, import_obj_from_str, Classified 56 | >>> classify_base('__future__') 57 | 'FUTURE' 58 | >>> classify_base('classify_imports') 59 | 'APPLICATION' 60 | >>> classify_base('pyramid') 61 | 'THIRD_PARTY' 62 | >>> classify_base('os') 63 | 'BUILTIN' 64 | >>> classify_base(import_obj_from_str('import os.path').module_base) 65 | 'BUILTIN' 66 | >>> Classified.APPLICATION 67 | 'APPLICATION' 68 | >>> Classified.order 69 | ('FUTURE', 'BUILTIN', 'THIRD_PARTY', 'APPLICATION') 70 | ``` 71 | -------------------------------------------------------------------------------- /classify_imports.py: -------------------------------------------------------------------------------- 1 | from __future__ import annotations 2 | 3 | import ast 4 | import collections 5 | import functools 6 | import operator 7 | import os.path 8 | import stat 9 | import sys 10 | from collections.abc import Generator 11 | from collections.abc import Iterable 12 | from typing import Callable 13 | from typing import NamedTuple 14 | 15 | 16 | class Classified: 17 | FUTURE = 'FUTURE' 18 | BUILTIN = 'BUILTIN' 19 | THIRD_PARTY = 'THIRD_PARTY' 20 | APPLICATION = 'APPLICATION' 21 | 22 | order = (FUTURE, BUILTIN, THIRD_PARTY, APPLICATION) 23 | 24 | 25 | _STATIC_CLASSIFICATIONS = { 26 | '__future__': Classified.FUTURE, 27 | '__main__': Classified.APPLICATION, 28 | # force distutils to be "third party" after being gobbled by setuptools 29 | 'distutils': Classified.THIRD_PARTY, 30 | # relative imports: `from .foo import bar` 31 | '': Classified.APPLICATION, 32 | } 33 | 34 | 35 | class Settings(NamedTuple): 36 | application_directories: tuple[str, ...] = ('.',) 37 | unclassifiable_application_modules: frozenset[str] = frozenset() 38 | 39 | 40 | def _path_key(path: str) -> tuple[str, tuple[int, int]]: 41 | path = path or '.' # '' in sys.path is the current directory 42 | # os.path.samestat uses (st_ino, st_dev) to determine equality 43 | st = os.stat(path) 44 | return path, (st.st_ino, st.st_dev) 45 | 46 | 47 | def _find_local(path: tuple[str, ...], base: str) -> bool: 48 | for p in path: 49 | p_dir = os.path.join(p, base) 50 | try: 51 | stat_dir = os.lstat(p_dir) 52 | except OSError: 53 | pass 54 | else: 55 | if stat.S_ISDIR(stat_dir.st_mode) and os.listdir(p_dir): 56 | return True 57 | try: 58 | stat_file = os.lstat(os.path.join(p, f'{base}.py')) 59 | except OSError: 60 | pass 61 | else: 62 | return stat.S_ISREG(stat_file.st_mode) 63 | else: 64 | return False 65 | 66 | 67 | if sys.version_info >= (3, 10): # pragma: >=3.10 cover 68 | @functools.cache 69 | def _get_app(app_dirs: tuple[str, ...]) -> tuple[str, ...]: 70 | app_dirs_ret = [] 71 | filtered_stats = set() 72 | for p in app_dirs: 73 | try: 74 | p, key = _path_key(p) 75 | except OSError: 76 | continue 77 | else: 78 | if key not in filtered_stats: 79 | app_dirs_ret.append(p) 80 | filtered_stats.add(key) 81 | 82 | return tuple(app_dirs_ret) 83 | 84 | @functools.cache 85 | def classify_base(base: str, settings: Settings = Settings()) -> str: 86 | try: 87 | return _STATIC_CLASSIFICATIONS[base] 88 | except KeyError: 89 | pass 90 | 91 | if base in sys.stdlib_module_names: 92 | return Classified.BUILTIN 93 | elif ( 94 | base in settings.unclassifiable_application_modules or 95 | _find_local(_get_app(settings.application_directories), base) 96 | ): 97 | return Classified.APPLICATION 98 | else: 99 | return Classified.THIRD_PARTY 100 | 101 | else: # pragma: <3.10 cover 102 | import importlib.machinery 103 | 104 | _BUILTIN_MODS = frozenset(sys.builtin_module_names) 105 | 106 | @functools.cache 107 | def _get_path( 108 | sys_path: tuple[str, ...], 109 | app_dirs: tuple[str, ...], 110 | pythonpath: str | None, 111 | ) -> tuple[Callable[[str], object | None], tuple[str, ...]]: 112 | app_dirs_ret = [] 113 | filtered_stats = set() 114 | for p in app_dirs: 115 | try: 116 | p, key = _path_key(p) 117 | except OSError: 118 | continue 119 | else: 120 | if key not in filtered_stats: 121 | app_dirs_ret.append(p) 122 | filtered_stats.add(key) 123 | 124 | if pythonpath: # subtract out pythonpath from sys.path 125 | for p in pythonpath.split(os.pathsep): 126 | try: 127 | filtered_stats.add(_path_key(p)[1]) 128 | except OSError: 129 | pass 130 | 131 | sys_path_ret = [] 132 | for p in sys_path: 133 | # subtract out site-packages 134 | if p.rstrip('/\\').endswith('-packages'): 135 | continue 136 | 137 | try: 138 | p, key = _path_key(p) 139 | except OSError: 140 | continue 141 | else: 142 | if key not in filtered_stats: 143 | sys_path_ret.append(p) 144 | filtered_stats.add(key) 145 | 146 | finder = functools.partial( 147 | importlib.machinery.PathFinder.find_spec, 148 | path=sys_path_ret, 149 | ) 150 | return finder, tuple(app_dirs_ret) 151 | 152 | @functools.cache 153 | def classify_base(base: str, settings: Settings = Settings()) -> str: 154 | try: 155 | return _STATIC_CLASSIFICATIONS[base] 156 | except KeyError: 157 | pass 158 | 159 | if base in settings.unclassifiable_application_modules: 160 | return Classified.APPLICATION 161 | elif base in _BUILTIN_MODS: 162 | return Classified.BUILTIN 163 | 164 | find_stdlib, app = _get_path( 165 | tuple(sys.path), 166 | settings.application_directories, 167 | os.environ.get('PYTHONPATH'), 168 | ) 169 | 170 | if _find_local(app, base): 171 | return Classified.APPLICATION 172 | elif find_stdlib(base) is not None: 173 | return Classified.BUILTIN 174 | else: 175 | return Classified.THIRD_PARTY 176 | 177 | 178 | def _ast_alias_to_s(node: ast.alias) -> str: 179 | if node.asname: 180 | return f'{node.name} as {node.asname}' 181 | else: 182 | return node.name 183 | 184 | 185 | class ImportKey(NamedTuple): 186 | module: str 187 | asname: str 188 | 189 | 190 | class Import: 191 | def __init__(self, node: ast.Import) -> None: 192 | self.node = node 193 | self.is_multiple = len(node.names) > 1 194 | 195 | @property 196 | def module(self) -> str: 197 | return self.node.names[0].name 198 | 199 | @property 200 | def module_base(self) -> str: 201 | return self.module.partition('.')[0] 202 | 203 | @functools.cached_property 204 | def key(self) -> ImportKey: 205 | alias = self.node.names[0] 206 | return ImportKey(alias.name, alias.asname or '') 207 | 208 | def __hash__(self) -> int: 209 | return hash(self.key) 210 | 211 | def __eq__(self, other: object) -> bool: 212 | return isinstance(other, Import) and self.key == other.key 213 | 214 | @property 215 | def sort_key(self) -> tuple[str, str, str, str, str]: 216 | name, asname = self.key 217 | return ('0', name.lower(), asname.lower(), name, asname) 218 | 219 | def split(self) -> Generator[Import]: 220 | if not self.is_multiple: 221 | yield self 222 | else: 223 | for name in self.node.names: 224 | yield type(self)(ast.Import(names=[name])) 225 | 226 | def __str__(self) -> str: 227 | assert not self.is_multiple 228 | return f'import {_ast_alias_to_s(self.node.names[0])}\n' 229 | 230 | def __repr__(self) -> str: 231 | return f'import_obj_from_str({str(self)!r})' 232 | 233 | 234 | class ImportFromKey(NamedTuple): 235 | module: str 236 | symbol: str 237 | asname: str 238 | 239 | 240 | class ImportFrom: 241 | def __init__(self, node: ast.ImportFrom) -> None: 242 | self.node = node 243 | self.is_multiple = len(node.names) > 1 244 | 245 | @property 246 | def module(self) -> str: 247 | level = '.' * self.node.level # local imports 248 | mod = self.node.module or '' # from . import bar makes module `None` 249 | return f'{level}{mod}' 250 | 251 | @property 252 | def module_base(self) -> str: 253 | return self.module.partition('.')[0] 254 | 255 | @functools.cached_property 256 | def key(self) -> ImportFromKey: 257 | alias = self.node.names[0] 258 | return ImportFromKey(self.module, alias.name, alias.asname or '') 259 | 260 | def __hash__(self) -> int: 261 | return hash(self.key) 262 | 263 | def __eq__(self, other: object) -> bool: 264 | return isinstance(other, ImportFrom) and self.key == other.key 265 | 266 | @property 267 | def sort_key(self) -> tuple[str, str, str, str, str, str, str]: 268 | mod, name, asname = self.key 269 | return ( 270 | '1', 271 | mod.lower(), name.lower(), asname.lower(), 272 | mod, name, asname, 273 | ) 274 | 275 | def split(self) -> Generator[ImportFrom]: 276 | if not self.is_multiple: 277 | yield self 278 | else: 279 | for name in self.node.names: 280 | node = ast.ImportFrom( 281 | module=self.node.module, 282 | names=[name], 283 | level=self.node.level, 284 | ) 285 | yield type(self)(node) 286 | 287 | def __str__(self) -> str: 288 | assert not self.is_multiple 289 | return ( 290 | f'from {self.module} ' 291 | f'import {_ast_alias_to_s(self.node.names[0])}\n' 292 | ) 293 | 294 | def __repr__(self) -> str: 295 | return f'import_obj_from_str({str(self)!r})' 296 | 297 | 298 | _import_type = {ast.Import: Import, ast.ImportFrom: ImportFrom} 299 | 300 | 301 | @functools.cache 302 | def import_obj_from_str(s: str) -> Import | ImportFrom: 303 | node = ast.parse(s, mode='single').body[0] 304 | return _import_type[type(node)](node) 305 | 306 | 307 | def sort( 308 | imports: Iterable[Import | ImportFrom], 309 | settings: Settings = Settings(), 310 | ) -> tuple[tuple[Import | ImportFrom, ...], ...]: 311 | # Partition the imports 312 | imports_partitioned = collections.defaultdict(list) 313 | for obj in imports: 314 | tp = classify_base(obj.module_base, settings=settings) 315 | if tp is Classified.FUTURE and isinstance(obj, Import): 316 | tp = Classified.BUILTIN 317 | 318 | imports_partitioned[tp].append(obj) 319 | 320 | # sort each of the segments 321 | sortkey = operator.attrgetter('sort_key') 322 | for val in imports_partitioned.values(): 323 | val.sort(key=sortkey) 324 | 325 | return tuple( 326 | tuple(imports_partitioned[key]) 327 | for key in Classified.order if key in imports_partitioned 328 | ) 329 | -------------------------------------------------------------------------------- /requirements-dev.txt: -------------------------------------------------------------------------------- 1 | covdefaults 2 | coverage 3 | pytest 4 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = classify_imports 3 | version = 4.2.0 4 | description = Utilities for refactoring imports in python-like syntax. 5 | long_description = file: README.md 6 | long_description_content_type = text/markdown 7 | url = https://github.com/asottile/classify-imports 8 | author = Anthony Sottile 9 | author_email = asottile@umich.edu 10 | license = MIT 11 | license_files = LICENSE 12 | classifiers = 13 | Programming Language :: Python :: 3 14 | Programming Language :: Python :: 3 :: Only 15 | Programming Language :: Python :: Implementation :: CPython 16 | Programming Language :: Python :: Implementation :: PyPy 17 | 18 | [options] 19 | py_modules = classify_imports 20 | python_requires = >=3.9 21 | 22 | [bdist_wheel] 23 | universal = True 24 | 25 | [coverage:run] 26 | plugins = covdefaults 27 | 28 | [mypy] 29 | check_untyped_defs = true 30 | disallow_any_generics = true 31 | disallow_incomplete_defs = true 32 | disallow_untyped_defs = true 33 | warn_redundant_casts = true 34 | warn_unused_ignores = true 35 | 36 | [mypy-testing.*] 37 | disallow_untyped_defs = false 38 | 39 | [mypy-tests.*] 40 | disallow_untyped_defs = false 41 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from __future__ import annotations 2 | 3 | from setuptools import setup 4 | setup() 5 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/asottile/classify-imports/35eb01977f0cb0e187cc7a1853addeaef99e23bf/tests/__init__.py -------------------------------------------------------------------------------- /tests/classify_imports_test.py: -------------------------------------------------------------------------------- 1 | from __future__ import annotations 2 | 3 | import ast 4 | import contextlib 5 | import os.path 6 | import subprocess 7 | import sys 8 | import zipfile 9 | from unittest import mock 10 | 11 | import pytest 12 | 13 | from classify_imports import Classified 14 | from classify_imports import classify_base 15 | from classify_imports import import_obj_from_str 16 | from classify_imports import ImportFromKey 17 | from classify_imports import ImportKey 18 | from classify_imports import Settings 19 | from classify_imports import sort 20 | 21 | if sys.version_info >= (3, 10): # pragma: >=3.10 cover 22 | from classify_imports import _get_app 23 | else: # pragma: <3.10 cover 24 | from classify_imports import _get_path 25 | 26 | def _get_app(dirs): 27 | return _get_path((), dirs, None)[1] 28 | 29 | 30 | @pytest.fixture(autouse=True) 31 | def no_warnings(recwarn): 32 | yield 33 | assert len(recwarn) == 0 34 | 35 | 36 | @pytest.fixture(autouse=True) 37 | def no_empty_path(): 38 | # Some of our tests check things based on their pwd where things aren't 39 | # necessarily importable. Let's make them not actually importable. 40 | with contextlib.suppress(ValueError): 41 | sys.path.remove('') 42 | 43 | 44 | @pytest.fixture(autouse=True) 45 | def reset_caches(): 46 | classify_base.cache_clear() 47 | if sys.version_info >= (3, 10): # pragma: >=3.10 cover 48 | _get_app.cache_clear() 49 | else: # pragma: <3.10 cover 50 | _get_path.cache_clear() 51 | 52 | 53 | @pytest.fixture 54 | def in_tmpdir(tmpdir): 55 | with tmpdir.as_cwd(): 56 | yield tmpdir 57 | 58 | 59 | def test_get_app_removes_duplicate_app_dirs(tmpdir): 60 | d1 = tmpdir.join('d1').ensure_dir() 61 | app = _get_app((str(d1), str(d1))) 62 | assert app == (str(d1),) 63 | 64 | 65 | def test_get_app_removes_non_existent_app_dirs(tmpdir): 66 | d1 = tmpdir.join('d1').ensure_dir() 67 | d2 = tmpdir.join('d2') 68 | app = _get_app((str(d1), str(d2))) 69 | assert app == (str(d1),) 70 | 71 | 72 | @pytest.mark.parametrize( 73 | ('module', 'expected'), 74 | ( 75 | ('__future__', Classified.FUTURE), 76 | ('os', Classified.BUILTIN), 77 | ('random', Classified.BUILTIN), 78 | ('sys', Classified.BUILTIN), 79 | ('cached_property', Classified.THIRD_PARTY), 80 | ('pyramid', Classified.THIRD_PARTY), 81 | ('classify_imports', Classified.APPLICATION), 82 | ('', Classified.APPLICATION), 83 | ('__main__', Classified.APPLICATION), 84 | ('tests', Classified.APPLICATION), 85 | pytest.param( 86 | 'distutils', Classified.THIRD_PARTY, 87 | id='force setuptools-distutils detection', 88 | ), 89 | ), 90 | ) 91 | def test_classify_base(module, expected): 92 | ret = classify_base(module) 93 | assert ret is expected 94 | 95 | 96 | def test_spec_is_none(): 97 | """for __main__ in a subprocess, spec is None and raises an error""" 98 | prog = '''\ 99 | import __main__ 100 | from classify_imports import classify_base, Classified 101 | assert __main__.__spec__ is None, __main__.__spec__ 102 | tp = classify_base('__main__') 103 | assert tp == Classified.APPLICATION, tp 104 | ''' 105 | subprocess.check_call((sys.executable, '-c', prog)) 106 | 107 | # simulate this situation for coverage 108 | with mock.patch.object(sys.modules['__main__'], '__spec__', None): 109 | assert classify_base('__main__') == Classified.APPLICATION 110 | 111 | 112 | def test_true_namespace_package(tmpdir): 113 | site_packages = tmpdir.join('site-packages') 114 | site_packages.join('a').ensure_dir() 115 | sys_path = [site_packages.strpath] + sys.path 116 | with mock.patch.object(sys, 'path', sys_path): 117 | # while this is a py3+ feature, classify_imports happens to get 118 | # this correct anyway! 119 | assert classify_base('a') == Classified.THIRD_PARTY 120 | 121 | 122 | xfail_win32 = pytest.mark.xfail(sys.platform == 'win32', reason='symlinks') 123 | 124 | 125 | @xfail_win32 # pragma: win32 no cover 126 | def test_symlink_path_different(in_tmpdir): 127 | # symlink a file, these are likely to not be application files 128 | in_tmpdir.join('dest_file.py').ensure() 129 | in_tmpdir.join('src_file.py').mksymlinkto('dest-file.py') 130 | ret = classify_base('src_file') 131 | assert ret is Classified.THIRD_PARTY 132 | 133 | 134 | @xfail_win32 # pragma: win32 no cover 135 | def test_symlink_path_directory(in_tmpdir): 136 | # symlink a dir, these are likely to not be application files 137 | in_tmpdir.join('dest').ensure_dir() 138 | in_tmpdir.join('dest/t.py').ensure() 139 | in_tmpdir.join('srcmod').mksymlinkto('dest') 140 | ret = classify_base('srcmod') 141 | assert ret is Classified.THIRD_PARTY 142 | 143 | 144 | @contextlib.contextmanager 145 | def in_sys_path(pth): 146 | paths = [os.path.abspath(p) for p in pth.split(os.pathsep)] 147 | 148 | path_before = sys.path[:] 149 | sys.path[:] = paths + path_before 150 | try: 151 | yield 152 | finally: 153 | sys.path[:] = path_before 154 | 155 | 156 | @contextlib.contextmanager 157 | def in_sys_path_and_pythonpath(pth): 158 | with in_sys_path(pth), mock.patch.dict(os.environ, {'PYTHONPATH': pth}): 159 | yield 160 | 161 | 162 | def test_classify_pythonpath_third_party(in_tmpdir): 163 | in_tmpdir.join('ppth').ensure_dir().join('f.py').ensure() 164 | with in_sys_path_and_pythonpath('ppth'): 165 | assert classify_base('f') is Classified.THIRD_PARTY 166 | 167 | 168 | def test_classify_pythonpath_dot_app(in_tmpdir): 169 | in_tmpdir.join('f.py').ensure() 170 | with in_sys_path_and_pythonpath('.'): 171 | assert classify_base('f') is Classified.APPLICATION 172 | 173 | 174 | def test_classify_pythonpath_multiple(in_tmpdir): 175 | in_tmpdir.join('ppth').ensure_dir().join('f.py').ensure() 176 | with in_sys_path_and_pythonpath(os.pathsep.join(('ppth', 'foo'))): 177 | assert classify_base('f') is Classified.THIRD_PARTY 178 | 179 | 180 | def test_classify_pythonpath_zipimport(in_tmpdir): 181 | path_zip = in_tmpdir.join('ppth').ensure_dir().join('fzip.zip') 182 | with zipfile.ZipFile(str(path_zip), 'w') as fzip: 183 | fzip.writestr('fzip.py', '') 184 | with in_sys_path_and_pythonpath('ppth/fzip.zip'): 185 | assert classify_base('fzip') is Classified.THIRD_PARTY 186 | 187 | 188 | @pytest.mark.xfail(sys.version_info >= (3, 10), reason='3.10 we know directly') 189 | def test_classify_embedded_builtin(in_tmpdir): 190 | path_zip = in_tmpdir.join('ppth').ensure_dir().join('fzip.zip') 191 | with zipfile.ZipFile(str(path_zip), 'w') as fzip: 192 | fzip.writestr('fzip.py', '') 193 | with in_sys_path('ppth/fzip.zip'): 194 | assert classify_base('fzip') is Classified.BUILTIN 195 | 196 | 197 | def test_file_existing_is_application_level(in_tmpdir): 198 | in_tmpdir.join('my_file.py').ensure() 199 | ret = classify_base('my_file') 200 | assert ret is Classified.APPLICATION 201 | 202 | 203 | def test_package_existing_is_application_level(in_tmpdir): 204 | in_tmpdir.join('my_package').ensure_dir().join('__init__.py').ensure() 205 | ret = classify_base('my_package') 206 | assert ret is Classified.APPLICATION 207 | 208 | 209 | def test_empty_directory_is_not_package(in_tmpdir): 210 | in_tmpdir.join('my_package').ensure_dir() 211 | ret = classify_base('my_package') 212 | assert ret is Classified.THIRD_PARTY 213 | 214 | 215 | def test_application_directories(in_tmpdir): 216 | # Similar to @bukzor's testing setup 217 | in_tmpdir.join('tests/testing').ensure_dir().join('__init__.py').ensure() 218 | # Should be classified 3rd party without argument 219 | ret = classify_base('testing') 220 | assert ret is Classified.THIRD_PARTY 221 | # Should be application with extra directories 222 | ret = classify_base( 223 | 'testing', 224 | settings=Settings(application_directories=('.', 'tests')), 225 | ) 226 | assert ret is Classified.APPLICATION 227 | 228 | 229 | def test_application_directory_case(in_tmpdir): 230 | srcdir = in_tmpdir.join('SRC').ensure_dir() 231 | srcdir.join('my_package').ensure_dir().join('__init__.py').ensure() 232 | with in_sys_path('src'): 233 | ret = classify_base( 234 | 'my_package', 235 | settings=Settings(application_directories=('SRC',)), 236 | ) 237 | assert ret is Classified.APPLICATION 238 | 239 | 240 | def test_unclassifiable_application_modules(): 241 | # Should be classified 3rd party without argument 242 | ret = classify_base('c_module') 243 | assert ret is Classified.THIRD_PARTY 244 | # Should be classified application with the override 245 | ret = classify_base( 246 | 'c_module', 247 | settings=Settings( 248 | unclassifiable_application_modules=frozenset(('c_module',)), 249 | ), 250 | ) 251 | assert ret is Classified.APPLICATION 252 | 253 | 254 | def test_unclassifiable_application_modules_ignores_future(): 255 | # Trying to force __future__ to be APPLICATION shouldn't have any effect 256 | ret = classify_base( 257 | '__future__', 258 | settings=Settings( 259 | unclassifiable_application_modules=frozenset(('__future__',)), 260 | ), 261 | ) 262 | assert ret is Classified.FUTURE 263 | 264 | 265 | @pytest.mark.parametrize( 266 | ('input_str', 'expected'), 267 | ( 268 | ('from foo import bar', ImportFromKey('foo', 'bar', '')), 269 | ('from foo import bar as baz', ImportFromKey('foo', 'bar', 'baz')), 270 | ('from . import bar', ImportFromKey('.', 'bar', '')), 271 | ('from .foo import bar', ImportFromKey('.foo', 'bar', '')), 272 | ('from .. import bar', ImportFromKey('..', 'bar', '')), 273 | ('from ..foo import bar', ImportFromKey('..foo', 'bar', '')), 274 | ), 275 | ) 276 | def test_from_import_key_from_python_ast(input_str, expected): 277 | assert import_obj_from_str(input_str).key == expected 278 | 279 | 280 | @pytest.mark.parametrize( 281 | ('input_str', 'expected'), 282 | ( 283 | ('import foo', ImportKey('foo', '')), 284 | ('import foo as bar', ImportKey('foo', 'bar')), 285 | ), 286 | ) 287 | def test_import_import_sort_key_from_python_ast(input_str, expected): 288 | assert import_obj_from_str(input_str).key == expected 289 | 290 | 291 | @pytest.fixture 292 | def import_import(): 293 | yield import_obj_from_str('import Foo as bar') 294 | 295 | 296 | def test_import_import_node(import_import): 297 | assert type(import_import.node) is ast.Import 298 | 299 | 300 | def test_import_import_key(import_import): 301 | assert import_import.key == ImportKey('Foo', 'bar') 302 | 303 | 304 | def test_import_import_sort_key(import_import): 305 | assert import_import.sort_key == ('0', 'foo', 'bar', 'Foo', 'bar') 306 | 307 | 308 | def test_import_import_equality_casing(): 309 | assert ( 310 | import_obj_from_str('import herp.DERP') != 311 | import_obj_from_str('import herp.derp') 312 | ) 313 | 314 | 315 | @pytest.mark.parametrize( 316 | ('input_str', 'expected'), 317 | ( 318 | ('import foo', False), 319 | ('import foo, bar', True), 320 | ), 321 | ) 322 | def test_import_import_is_multiple(input_str, expected): 323 | assert import_obj_from_str(input_str).is_multiple is expected 324 | 325 | 326 | @pytest.mark.parametrize( 327 | ('input_str', 'expected'), 328 | ( 329 | ('import foo', [import_obj_from_str('import foo')]), 330 | ( 331 | 'import foo, bar', 332 | [ 333 | import_obj_from_str('import foo'), 334 | import_obj_from_str('import bar'), 335 | ], 336 | ), 337 | ), 338 | ) 339 | def test_import_import_split(input_str, expected): 340 | assert list(import_obj_from_str(input_str).split()) == expected 341 | 342 | 343 | @pytest.mark.parametrize( 344 | 'import_str', 345 | ( 346 | 'import foo\n', 347 | 'import foo.bar\n', 348 | 'import foo as bar\n', 349 | ), 350 | ) 351 | def test_import_import_str(import_str): 352 | assert str(import_obj_from_str(import_str)) == import_str 353 | 354 | 355 | def test_import_import_repr(import_import): 356 | expected = "import_obj_from_str('import Foo as bar\\n')" 357 | assert repr(import_import) == expected 358 | 359 | 360 | @pytest.mark.parametrize( 361 | ('import_str', 'expected'), 362 | ( 363 | ('import foo', 'import foo\n'), 364 | ('import foo as bar', 'import foo as bar\n'), 365 | ('import foo as bar', 'import foo as bar\n'), 366 | ), 367 | ) 368 | def test_import_import_str_normalizes_whitespace(import_str, expected): 369 | assert str(import_obj_from_str(import_str)) == expected 370 | 371 | 372 | @pytest.fixture 373 | def from_import(): 374 | yield import_obj_from_str('from Foo import bar as baz') 375 | 376 | 377 | def test_from_import_node(from_import): 378 | assert type(from_import.node) is ast.ImportFrom 379 | 380 | 381 | def test_from_import_key(from_import): 382 | ret = from_import.key 383 | assert ret == ImportFromKey('Foo', 'bar', 'baz') 384 | 385 | 386 | def test_from_import_sort_key(from_import): 387 | ret = from_import.sort_key 388 | assert ret == ('1', 'foo', 'bar', 'baz', 'Foo', 'bar', 'baz') 389 | 390 | 391 | @pytest.mark.parametrize( 392 | ('input_str', 'expected'), 393 | ( 394 | ('from foo import bar', False), 395 | ('from foo import bar, baz', True), 396 | ), 397 | ) 398 | def test_from_import_is_multiple(input_str, expected): 399 | assert import_obj_from_str(input_str).is_multiple is expected 400 | 401 | 402 | @pytest.mark.parametrize( 403 | ('input_str', 'expected'), 404 | ( 405 | ('from foo import bar', [import_obj_from_str('from foo import bar')]), 406 | ( 407 | 'from foo import bar, baz', 408 | [ 409 | import_obj_from_str('from foo import bar'), 410 | import_obj_from_str('from foo import baz'), 411 | ], 412 | ), 413 | ( 414 | 'from .foo import bar, baz', 415 | [ 416 | import_obj_from_str('from .foo import bar'), 417 | import_obj_from_str('from .foo import baz'), 418 | ], 419 | ), 420 | ), 421 | ) 422 | def test_from_import_split(input_str, expected): 423 | assert list(import_obj_from_str(input_str).split()) == expected 424 | 425 | 426 | @pytest.mark.parametrize( 427 | 'import_str', 428 | ( 429 | 'from foo import bar\n', 430 | 'from foo.bar import baz\n', 431 | 'from foo.bar import baz as buz\n', 432 | ), 433 | ) 434 | def test_from_import_str(import_str): 435 | assert str(import_obj_from_str(import_str)) == import_str 436 | 437 | 438 | @pytest.mark.parametrize( 439 | ('import_str', 'expected'), 440 | ( 441 | ('from foo import bar', 'from foo import bar\n'), 442 | ('from foo import bar', 'from foo import bar\n'), 443 | ('from foo import bar', 'from foo import bar\n'), 444 | ('from foo import bar as baz', 'from foo import bar as baz\n'), 445 | ('from foo import bar as baz', 'from foo import bar as baz\n'), 446 | ), 447 | ) 448 | def test_from_import_str_normalizes_whitespace(import_str, expected): 449 | assert str(import_obj_from_str(import_str)) == expected 450 | 451 | 452 | def test_from_import_repr(from_import): 453 | expected = "import_obj_from_str('from Foo import bar as baz\\n')" 454 | assert repr(from_import) == expected 455 | 456 | 457 | def test_from_import_hashable(): 458 | my_set = set() 459 | my_set.add(import_obj_from_str('from foo import bar')) 460 | my_set.add(import_obj_from_str('from foo import bar')) 461 | assert len(my_set) == 1 462 | 463 | 464 | def test_import_import_hashable(): 465 | my_set = set() 466 | my_set.add(import_obj_from_str('import foo')) 467 | my_set.add(import_obj_from_str('import foo')) 468 | assert len(my_set) == 1 469 | 470 | 471 | @pytest.mark.parametrize( 472 | 'input_str', 473 | ( 474 | 'from . import bar\n', 475 | 'from .foo import bar\n', 476 | 'from .. import bar\n', 477 | 'from ..foo import bar\n', 478 | ), 479 | ) 480 | def test_local_imports(input_str): 481 | assert str(import_obj_from_str(input_str)) == input_str 482 | 483 | 484 | @pytest.mark.parametrize( 485 | ('input_str', 'expected'), 486 | ( 487 | ('from foo import bar', import_obj_from_str('from foo import bar')), 488 | ( 489 | 'from foo import bar, baz', 490 | import_obj_from_str('from foo import bar, baz'), 491 | ), 492 | ('import bar', import_obj_from_str('import bar')), 493 | ('import bar, baz', import_obj_from_str('import bar, baz')), 494 | ), 495 | ) 496 | def test_import_obj_from_str(input_str, expected): 497 | assert import_obj_from_str(input_str) == expected 498 | 499 | 500 | IMPORTS = ( 501 | import_obj_from_str('from os import path'), 502 | import_obj_from_str('from classify_imports import classify_base'), 503 | import_obj_from_str('import sys'), 504 | import_obj_from_str('import pyramid'), 505 | ) 506 | 507 | 508 | def test_separate_import_before_from(): 509 | ret = sort(IMPORTS) 510 | assert ret == ( 511 | ( 512 | import_obj_from_str('import sys'), 513 | import_obj_from_str('from os import path'), 514 | ), 515 | ( 516 | import_obj_from_str('import pyramid'), 517 | ), 518 | ( 519 | import_obj_from_str('from classify_imports import classify_base'), 520 | ), 521 | ) 522 | 523 | 524 | def test_future_from_always_first(): 525 | ret = sort( 526 | ( 527 | import_obj_from_str('from __future__ import absolute_import'), 528 | import_obj_from_str('import __future__'), 529 | ), 530 | ) 531 | assert ret == ( 532 | (import_obj_from_str('from __future__ import absolute_import'),), 533 | (import_obj_from_str('import __future__'),), 534 | ) 535 | 536 | 537 | def test_passes_through_kwargs_to_classify(in_tmpdir): 538 | # Make a module 539 | in_tmpdir.join('my_module.py').ensure() 540 | 541 | imports = ( 542 | import_obj_from_str('import my_module'), 543 | import_obj_from_str('import pyramid'), 544 | ) 545 | # Without kwargs, my_module should get classified as application (in a 546 | # separate group). 547 | ret = sort(imports) 548 | assert ret == ( 549 | (import_obj_from_str('import pyramid'),), 550 | (import_obj_from_str('import my_module'),), 551 | ) 552 | # But when we put the application at a nonexistent directory 553 | # it'll be third party (and in the same group as pyramid) 554 | ret = sort(imports, Settings(application_directories=('dne',))) 555 | assert ret == (imports,) 556 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py,pre-commit 3 | 4 | [testenv] 5 | deps = -rrequirements-dev.txt 6 | commands = 7 | coverage erase 8 | coverage run -m pytest {posargs:tests} 9 | coverage report 10 | 11 | [testenv:pre-commit] 12 | skip_install = true 13 | deps = pre-commit 14 | commands = pre-commit run --all-files --show-diff-on-failure 15 | 16 | [pep8] 17 | ignore = E265,E501,W504 18 | --------------------------------------------------------------------------------