├── .coveragerc ├── .gitignore ├── .gitmodules ├── LICENSE.txt ├── MANIFEST.in ├── README.rst ├── circle.yml ├── pundle.py ├── setup.cfg ├── setup.py ├── tests ├── __init__.py ├── lib.py ├── test_install.py ├── test_pipfile.py ├── test_setup_py.py ├── test_simple.py └── test_vcs_requirements.py ├── tox.ini └── virtualenv.md /.coveragerc: -------------------------------------------------------------------------------- 1 | [run] 2 | omit = 3 | autoscaler/run.py 4 | tests/* 5 | 6 | [report] 7 | fail_under = 42 8 | show_missing = True 9 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | *.pyc 3 | build 4 | dist 5 | .coverage 6 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Deepwalker/pundler/dba19d6b7b8c2bc602d1ac9c5612a5cb8125cdb4/.gitmodules -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Mikhail Krivushin. 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are 6 | met: 7 | 8 | 1. Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 11 | 2. Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions and the following disclaimer in the 13 | documentation and/or other materials provided with the distribution. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 16 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 17 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 18 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 19 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 20 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 21 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 22 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 23 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 24 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include *.md 2 | include *.rst 3 | include *.txt 4 | include tox.ini 5 | recursive-include tests *.py 6 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ====== 2 | Pundle 3 | ====== 4 | 5 | |circleci_build| |pypi_version| |pypi_license| 6 | 7 | Changelog 8 | --------- 9 | 10 | - PUNDLEENV can include comma separated envs 11 | - Pipfile initial support. Only strings as versions now. 12 | Do not calculates hashes and do not use it yet. 13 | - New setup.py support with mocking of setuptools.setup 14 | - Added python shell ``try package`` feature. To use it use 15 | ``pundle.use("package_name==0.1")``. Version is optional. 16 | - Added environments support. To use it just make files like 17 | ``requirement.txt``, ``requirements_dev.txt``, 18 | ``requirements_test.txt``. To activate env use like 19 | ``PUNDLEENV=dev pundle ...`` 20 | - Added VCS support for urls like 21 | ``git+https://github.com/mitsuhiko/jinja2.git@85820fceb83569df62fa5e6b9b0f2f76b7c6a3cf#egg=jinja2-2.8.0``. 22 | Push exactly like this formatted str to requirements.txt 23 | - Added initial support for setup.py requirements. Helpful for package 24 | development. 25 | 26 | 27 | What is it all about? 28 | --------------------- 29 | 30 | Pundle get rid of virtualenv, because I think that virtualenv directory is pile of 31 | garbage and we must get rid of it. 32 | 33 | - Pundle installs all packages and their versions to special folder. And 34 | mount pinned, frozen versions on activate step. 35 | - After that you program will use exactly this versions that were 36 | pinned in ``frozen.txt``. 37 | - If you change branch or edit requirements.txt or frozen.txt, pundle 38 | will note you about you need make install new packages or freeze 39 | newly added packages. It will not let you use packages that have not 40 | bin pinned. You will never fall in situation where you test old 41 | version of package. 42 | 43 | 44 | Why not Pipenv, I heard it is "for humans"? 45 | ------------------------------------------- 46 | 47 | I don't think that anything that is based on virtualenv can be "for humans". 48 | 49 | 50 | How to 51 | ------ 52 | 53 | Install: 54 | 55 | .. code-block:: bash 56 | 57 | > pip install pundle 58 | 59 | or just place ``pundle.py`` where python can find it 60 | 61 | Create ``requirements.txt`` or ``setup.py`` (we will support .toml a bit 62 | later). You can pin versions if you need it, or just place package name. 63 | Pundle will pin it anyway as well as all of it dependencies. 64 | 65 | Reveal all dependencies, pin versions, download and install everything: 66 | 67 | .. code-block:: bash 68 | 69 | > python -m pundle 70 | 71 | Where actually it will install? Pundle use special folder 72 | ``.pundledir/python-version/package-name-version`` for every seperate 73 | package and version. 74 | 75 | To make it short create alias: 76 | 77 | .. code-block:: bash 78 | 79 | alias pundle='/usr/bin/env python -m pundle' 80 | pundle install 81 | 82 | After packages install, frozen/pinned, we want to use them, you know, 83 | import, right? 84 | 85 | .. code-block:: python 86 | 87 | import pundle; pundle.activate() 88 | 89 | Or we can try to use pundle features: 90 | 91 | .. code-block:: bash 92 | 93 | # to execute entry point 94 | pundle exec some_package_entry_point 95 | # to run python script 96 | pundle run my_script.py 97 | # run module like python -m 98 | pundle module some.my.module 99 | 100 | To add VCS to ``requirements.txt`` use ``git+url#egg=my_package-0.1.11`` 101 | form. 102 | 103 | 104 | Pundle console 105 | -------------- 106 | 107 | To start console with Pundle activated use 108 | 109 | .. code-block:: bash 110 | 111 | > pundle console [ipython|ptpython|bpython] 112 | 113 | You will have ``pundle_suite`` object inserted to environment. You can use it 114 | to call ``pundle_suite.use("trafaret_schema")`` for example. 115 | 116 | 117 | Python shell usage 118 | ------------------ 119 | 120 | You can use pundle to expirement in python shell: 121 | 122 | .. code-block:: python 123 | 124 | >>> import pundle 125 | >>> pundle.use('django==1.11.1') # will download and install django 126 | >>> import django 127 | 128 | Or you can use it in script: 129 | 130 | .. code-block:: python 131 | 132 | >>> import pundle 133 | >>> pundle.use('django') 134 | >>> pundle.use('arrow') 135 | >>> pundle.use('trafaret') 136 | >>> 137 | >>> import django 138 | >>> import arrow 139 | >>> import trafaret 140 | 141 | Environments 142 | ------------ 143 | 144 | Pundle support environments. You can create seperate requirements file 145 | with suffix like ``requirements_dev.txt``. Pundle will create 146 | ``frozen_dev.txt`` that will track common requirements + dev 147 | requirements. 148 | 149 | To use ``dev`` environment use ``PUNDLEENV=dev`` environment variable: 150 | 151 | .. code-block:: bash 152 | 153 | bash> PUNDLEENV=dev pundle run myscript.py 154 | 155 | or common usage: 156 | 157 | .. code-block:: bash 158 | 159 | bash> PUNDLEENV=test pundle exec pytest 160 | 161 | For ``setup.py`` file pundle uses ``extras_require`` as environments. For example if 162 | you have ``extras_require = {'test': ['pylint', 'pyflakes']}`` then you can use 163 | ``pylint`` with ``PUNDLEENV=test pundle exec pylint``. 164 | 165 | More usage info 166 | --------------- 167 | 168 | Upgrade package: 169 | 170 | .. code-block:: bash 171 | 172 | pundle upgrade django 173 | 174 | Upgrade all packages: 175 | 176 | .. code-block:: bash 177 | 178 | pundle upgrade 179 | 180 | List of all entry points: 181 | 182 | .. code-block:: bash 183 | 184 | pundle entry_points 185 | 186 | Do not hesitate to ``pundle help`` ;) 187 | 188 | Howto 189 | ----- 190 | 191 | Q: How to use custom index url or extra index? 192 | 193 | A: use PIP_EXTRA_INDEX_URL or any other ``pip`` environment variables. 194 | 195 | .. |circleci_build| image:: https://circleci.com/gh/Deepwalker/pundler.svg?style=svg 196 | :target: https://circleci.com/gh/Deepwalker/pundler 197 | .. |pypi_version| image:: https://img.shields.io/pypi/v/pundle.svg?style=flat-square 198 | :target: https://pypi.python.org/pypi/pundle 199 | .. |pypi_license| image:: https://img.shields.io/pypi/l/pundle.svg?style=flat-square 200 | :target: https://pypi.python.org/pypi/pundle 201 | -------------------------------------------------------------------------------- /circle.yml: -------------------------------------------------------------------------------- 1 | version: 2 # use CircleCI 2.0 2 | jobs: # A basic unit of work in a run 3 | build: # runs not using Workflows must have a `build` job as entry point 4 | # directory where steps are run 5 | working_directory: ~/pundle 6 | docker: # run the steps with Docker 7 | # CircleCI Python images available at: https://hub.docker.com/r/circleci/python/ 8 | - image: circleci/python:3.6.4 9 | environment: # environment variables for primary container 10 | PIPENV_VENV_IN_PROJECT: true 11 | DATABASE_URL: postgresql://root@localhost/circle_test?sslmode=disable 12 | steps: # steps that comprise the `build` job 13 | - checkout # check out source code to working directory 14 | - run: sudo chown -R circleci:circleci /usr/local/bin 15 | - run: sudo chown -R circleci:circleci /usr/local/lib/python3.6/site-packages 16 | - restore_cache: 17 | # Read about caching dependencies: https://circleci.com/docs/2.0/caching/ 18 | key: deps9-{{ .Branch }}-{{ checksum "setup.py" }} 19 | - run: 20 | command: | 21 | sudo pip install pipenv tox 22 | pipenv install 23 | - save_cache: # cache Python dependencies using checksum of Pipfile as the cache-key 24 | key: deps9-{{ .Branch }}-{{ checksum "setup.py" }} 25 | paths: 26 | - ".venv" 27 | - "/usr/local/bin" 28 | - "/usr/local/lib/python3.6/site-packages" 29 | - run: 30 | command: | 31 | pipenv run tox 32 | - store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/ 33 | path: test-results 34 | - store_artifacts: # Upload test summary for display in Artifacts: https://circleci.com/docs/2.0/artifacts/ 35 | path: test-results 36 | destination: tr1 37 | -------------------------------------------------------------------------------- /pundle.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | Data Model, start here to not get mad 4 | ===================================== 5 | 6 | Main entity will be distribution, like Flask. Per key 7 | Pundle tracks three state parts: 8 | 1. requirement, like Flask>0.12.2. 9 | 2. frozen version, like ==0.12.2 10 | 3. installed distributions, like [flask==0.12.2, flask==0.10.0] 11 | 12 | Requirement basically is from file, like requirements.txt, setup.py or Pipfile. This requirements 13 | have source like `requirements.txt`. And base requirements can have dependencies. This 14 | dependencies are turned to requirements too with source like `Flask-Admin << requirements.txt`. 15 | To track requirements we use `CustomReq` class. It can work with PyPi and VCS requirements. 16 | CustomReq can have `self.req = pkg_resources.Requirement` or custom vcs line. 17 | 18 | Distribution is one of VCSDist or pkg_resources.DistInfoDistribution. VCSDist is to 19 | track installed VCS packages and pkg_resources.DistInfoDistribution is for PyPi packages. 20 | 21 | All three states of distribution are tracked inside `RequirementState` object that includes info 22 | about requirement, frozen version and installed versions. 23 | 24 | `Suite` keeps state of all distributions like a dictionary of RequirentStates. 25 | 26 | To populate Suite and to parse all requirements, frozen versions and what we have installed pundle 27 | uses `Parser`. There is plenty of them - `Parser` that works with `requirements.txt`, 28 | `SetupParser` that works with `setup.py`, PipfileParser that works with Pipfile and Pipfile.lock. 29 | """ 30 | 31 | from __future__ import print_function 32 | import re 33 | try: 34 | from urllib.parse import urlparse, parse_qsl 35 | except ImportError: # pragma: no cover 36 | from urlparse import urlparse, parse_qsl 37 | from collections import defaultdict 38 | from base64 import b64encode, b64decode 39 | import platform 40 | import os.path as op 41 | import os 42 | from os import makedirs 43 | import stat 44 | import tempfile 45 | import shutil 46 | import subprocess 47 | import sys 48 | import shlex 49 | import json 50 | import hashlib 51 | import pkg_resources 52 | try: 53 | from pip import main as pip_exec 54 | except ImportError: 55 | from pip._internal import main as pip_exec 56 | from types import ModuleType 57 | 58 | if isinstance(pip_exec, ModuleType): 59 | pip_exec = pip_exec.main 60 | 61 | # TODO bundle own version of distlib. Perhaps 62 | try: 63 | from pip._vendor.distlib import locators 64 | except ImportError: # pragma: no cover 65 | from pip.vendor.distlib import locators 66 | 67 | try: 68 | str_types = (basestring,) 69 | except NameError: # pragma: no cover 70 | str_types = (str, bytes) 71 | 72 | try: 73 | pkg_resources_parse_version = pkg_resources.SetuptoolsVersion 74 | except AttributeError: # pragma: no cover 75 | pkg_resources_parse_version = pkg_resources.packaging.version.Version 76 | 77 | 78 | def print_message(*a, **kw): 79 | print(*a, **kw) 80 | 81 | 82 | class PundleException(Exception): 83 | pass 84 | 85 | 86 | def python_version_string(): 87 | """We use it to generate per python folder name, where 88 | we will install all packages. 89 | """ 90 | if platform.python_implementation() == 'PyPy': 91 | version_info = sys.pypy_version_info 92 | else: 93 | version_info = sys.version_info 94 | version_string = '{v.major}.{v.minor}.{v.micro}'.format(v=version_info) 95 | build, _ = platform.python_build() 96 | build = build.replace(':', '_') # windows do not understand `:` in path 97 | return '{}-{}-{}'.format(platform.python_implementation(), version_string, build) 98 | 99 | 100 | def parse_file(filename): 101 | """Helper to parse requirements.txt or frozen.txt. 102 | """ 103 | res = [] 104 | with open(filename) as f: 105 | for line in f: 106 | s = line.strip() 107 | if s and not s.startswith('#'): 108 | if s.startswith('-r'): 109 | continue 110 | if s.startswith('-e '): 111 | s = s[3:].strip() 112 | if parse_vcs_requirement(s): 113 | res.append(s) 114 | else: 115 | req = shlex.split(s, comments=True) 116 | res.append(req[0]) 117 | return res 118 | 119 | 120 | def test_vcs(req): 121 | """Checks if requirement line is for VCS. 122 | """ 123 | return '+' in req and req.index('+') == 3 124 | 125 | 126 | def parse_vcs_requirement(req): 127 | """Parses VCS line to egg name, version etc. 128 | """ 129 | if '+' not in req: 130 | return None 131 | vcs, url = req.split('+', 1) 132 | if vcs not in ('git', 'svn', 'hg'): 133 | return None 134 | parsed_url = urlparse(url) 135 | parsed = dict(parse_qsl(parsed_url.fragment)) 136 | if 'egg' not in parsed: 137 | return None 138 | egg = parsed['egg'].rsplit('-', 1) 139 | if len(egg) > 1: 140 | try: 141 | pkg_resources_parse_version(egg[1]) 142 | except pkg_resources._vendor.packaging.version.InvalidVersion: 143 | return parsed['egg'].lower(), req, None 144 | return egg[0].lower(), req, egg[1] 145 | else: 146 | return parsed['egg'].lower(), req, None 147 | 148 | 149 | def parse_frozen_vcs(req): 150 | res = parse_vcs_requirement(req) 151 | if not res: 152 | return 153 | return res[0], res[1] 154 | 155 | 156 | class VCSDist(object): 157 | """ Represents installed VCS distribution. 158 | """ 159 | def __init__(self, directory): 160 | self.dir = directory 161 | name = op.split(directory)[-1] 162 | key, encoded = name.split('+', 1) 163 | self.key = key.lower() 164 | self.line = b64decode(encoded).decode('utf-8') 165 | egg, req, version = parse_vcs_requirement(self.line) 166 | version = version or '0.0.0' 167 | self.hashcmp = (pkg_resources_parse_version(version), -1, egg, self.dir) 168 | self.version = self.line 169 | self.pkg_resource = next(iter(pkg_resources.find_distributions(self.dir, True)), None) 170 | self.location = self.pkg_resource.location 171 | 172 | def requires(self, extras=[]): 173 | return self.pkg_resource.requires(extras=extras) 174 | 175 | def activate(self): 176 | return self.pkg_resource.activate() 177 | 178 | def __lt__(self, other): 179 | return self.hashcmp.__lt__(other.hashcmp) 180 | 181 | 182 | class CustomReq(object): 183 | """Represents PyPi or VCS requirement. 184 | Can locate and install it. 185 | """ 186 | def __init__(self, line, env, source=None): 187 | self.line = line 188 | self.egg = None 189 | if isinstance(line, pkg_resources.Requirement): 190 | self.req = line 191 | elif test_vcs(line): 192 | res = parse_vcs_requirement(line) 193 | if not res: 194 | raise PundleException('Bad url %r' % line) 195 | egg, req, version = res 196 | self.egg = egg 197 | self.req = None # pkg_resources.Requirement.parse(res) 198 | else: 199 | self.req = pkg_resources.Requirement.parse(line) 200 | self.sources = set([source]) 201 | self.envs = set() 202 | self.add_env(env) 203 | 204 | def __contains__(self, something): 205 | if self.req: 206 | return (something in self.req) 207 | elif self.egg: 208 | return something == self.line 209 | else: 210 | return False 211 | 212 | def __repr__(self): 213 | return '' % self.__dict__ 214 | 215 | def why_str(self): 216 | if len(self.sources) < 2: 217 | return '{} << {}'.format(self.line, self.why_str_one(list(self.sources)[0])) 218 | causes = list(sorted(self.why_str_one(source) for source in self.sources)) 219 | return '{} << [{}]'.format(self.line, ' | '.join(causes)) 220 | 221 | def why_str_one(self, source): 222 | if isinstance(source, str_types): 223 | return source 224 | elif isinstance(source, CustomReq): 225 | return source.why_str() 226 | return '?' 227 | 228 | def adjust_with_req(self, req): 229 | if not self.req: 230 | return 231 | raise PundleException('VCS') 232 | versions = ','.join(''.join(t) for t in set(self.req.specs + req.req.specs)) 233 | self.requirement = pkg_resources.Requirement.parse('{} {}'.format( 234 | self.req.project_name, versions 235 | )) 236 | self.sources.update(req.sources) 237 | self.add_env(req.envs) 238 | 239 | @property 240 | def key(self): 241 | return self.req.key if self.req else self.egg 242 | 243 | @property 244 | def extras(self): 245 | return self.req.extras if self.req else [] 246 | 247 | def locate(self, suite, prereleases=False): 248 | # requirements can have something after `;` symbol that `locate` does not understand 249 | req = str(self.req).split(';', 1)[0] 250 | dist = suite.locate(req, prereleases=prereleases) 251 | if not dist: 252 | # have not find any releases so search for pre 253 | dist = suite.locate(req, prereleases=True) 254 | if not dist: 255 | raise PundleException('%s can not be located' % self.req) 256 | return dist 257 | 258 | def locate_and_install(self, suite, installed=None, prereleases=False): 259 | if self.egg: 260 | key = b64encode(self.line.encode('utf-8')).decode() 261 | target_dir = op.join(suite.parser.directory, '{}+{}'.format(self.egg, key)) 262 | target_req = self.line 263 | ready = [ 264 | installation 265 | for installation in (installed or []) 266 | if getattr(installation, 'line', None) == self.line 267 | ] 268 | else: 269 | loc_dist = self.locate(suite, prereleases=prereleases) 270 | ready = [ 271 | installation 272 | for installation in (installed or []) 273 | if installation.version == loc_dist.version 274 | ] 275 | target_dir = op.join(suite.parser.directory, '{}-{}'.format(loc_dist.key, loc_dist.version)) 276 | # DEL? target_req = '%s==%s' % (loc_dist.name, loc_dist.version) 277 | # If we use custom index, then we want not to configure PIP with it 278 | # and just give it URL 279 | target_req = loc_dist.download_url 280 | if ready: 281 | return ready[0] 282 | try: 283 | makedirs(target_dir) 284 | except OSError: 285 | pass 286 | tmp_dir = tempfile.mkdtemp() 287 | print('Use temp dir', tmp_dir) 288 | try: 289 | print('pip install --no-deps -t %s %s' % (tmp_dir, target_req)) 290 | pip_exec([ 291 | 'install', 292 | '--no-deps', 293 | '-t', tmp_dir, 294 | '-v', 295 | target_req 296 | ]) 297 | for item in os.listdir(tmp_dir): 298 | shutil.move(op.join(tmp_dir, item), op.join(target_dir, item)) 299 | except Exception as exc: 300 | raise PundleException('%s was not installed due error %s' % (self.egg or loc_dist.name, exc)) 301 | finally: 302 | shutil.rmtree(tmp_dir, ignore_errors=True) 303 | return next(iter(pkg_resources.find_distributions(target_dir, True)), None) 304 | 305 | def add_env(self, env): 306 | if isinstance(env, str): 307 | self.envs.add(env) 308 | else: 309 | self.envs.update(env) 310 | 311 | 312 | class RequirementState(object): 313 | """Holds requirement state, like what version do we have installed, is 314 | some version frozen or not, what requirement constrains do we have. 315 | """ 316 | def __init__(self, key, req=None, frozen=None, installed=None, hashes=None): 317 | self.key = key 318 | self.requirement = req 319 | self.frozen = frozen 320 | self.hashes = hashes 321 | self.installed = installed or [] 322 | self.installed.sort() 323 | self.installed.reverse() 324 | 325 | def __repr__(self): 326 | return '' % self.__dict__ 327 | 328 | def adjust_with_req(self, req): 329 | if self.requirement: 330 | self.requirement.adjust_with_req(req) 331 | else: 332 | self.requirement = req 333 | 334 | def has_correct_freeze(self): 335 | return self.requirement and self.frozen and self.frozen in self.requirement 336 | 337 | def check_installed_version(self, suite, install=False): 338 | # install version of package if not installed 339 | dist = None 340 | if self.has_correct_freeze(): 341 | dist = [ 342 | installation 343 | for installation in self.installed 344 | if pkg_resources.parse_version(installation.version) == pkg_resources.parse_version(self.frozen) 345 | ] 346 | dist = dist[0] if dist else None 347 | if install and not dist: 348 | dist = self.install_frozen(suite) 349 | if install and not dist: 350 | dist = self.requirement.locate_and_install(suite, installed=self.get_installed()) 351 | if dist is None: 352 | raise PundleException('Package %s was not installed due some error' % self.key) 353 | self.frozen = dist.version 354 | self.installed.append(dist) 355 | self.frozen = dist.version 356 | return dist 357 | 358 | def get_installed(self): 359 | return [installation for installation in self.installed if installation.version in self.requirement] 360 | 361 | def upgrade(self, suite, prereleases=False): 362 | # check if we have fresh packages on PIPY 363 | dists = self.get_installed() 364 | dist = dists[0] if dists else None 365 | latest = self.requirement.locate(suite, prereleases=prereleases) 366 | if not dist or pkg_resources.parse_version(latest.version) > pkg_resources.parse_version(dist.version): 367 | print_message('Upgrade to', latest) 368 | dist = self.requirement.locate_and_install(suite, installed=self.get_installed(), prereleases=prereleases) 369 | # Anyway use latest available dist 370 | self.frozen = dist.version 371 | self.installed.append(dist) 372 | return dist 373 | 374 | def reveal_requirements(self, suite, install=False, upgrade=False, already_revealed=None, prereleases=False): 375 | already_revealed = already_revealed.copy() if already_revealed is not None else set() 376 | if self.key in already_revealed: 377 | return 378 | if upgrade: 379 | dist = self.upgrade(suite, prereleases=prereleases) 380 | else: 381 | dist = self.check_installed_version(suite, install=install) 382 | if not dist: 383 | return 384 | already_revealed.add(self.key) 385 | for req in dist.requires(extras=self.requirement.extras): 386 | suite.adjust_with_req( 387 | CustomReq(str(req), self.requirement.envs, source=self.requirement), 388 | install=install, 389 | upgrade=upgrade, 390 | already_revealed=already_revealed, 391 | ) 392 | 393 | def frozen_dump(self): 394 | if self.requirement.egg: 395 | return self.requirement.line 396 | main = '{}=={}'.format(self.key, self.frozen) 397 | comment = self.requirement.why_str() 398 | return '{:20s} # {}'.format(main, comment) 399 | 400 | def frozen_dist(self): 401 | if not self.frozen: 402 | return 403 | for dist in self.installed: 404 | if pkg_resources.parse_version(dist.version) == pkg_resources.parse_version(self.frozen): 405 | return dist 406 | 407 | def install_frozen(self, suite): 408 | if self.frozen_dist() or not self.frozen: 409 | return 410 | envs = self.requirement.envs if self.requirement else '' 411 | if test_vcs(self.frozen): 412 | frozen_req = CustomReq(self.frozen, envs) 413 | else: 414 | frozen_req = CustomReq("{}=={}".format(self.key, self.frozen), envs) 415 | dist = frozen_req.locate_and_install(suite) 416 | self.installed.append(dist) 417 | return dist 418 | 419 | def activate(self): 420 | dist = self.frozen_dist() 421 | if not dist: 422 | raise PundleException('Distribution is not installed %s' % self.key) 423 | dist.activate() 424 | pkg_resources.working_set.add_entry(dist.location) 425 | # find end execute *.pth 426 | sitedir = dist.location # noqa some PTH search for sitedir 427 | for filename in os.listdir(dist.location): 428 | if not filename.endswith('.pth'): 429 | continue 430 | try: 431 | for line in open(op.join(dist.location, filename)): 432 | if line.startswith('import '): 433 | exec(line.strip()) 434 | except Exception as e: 435 | print('Erroneous pth file %r' % op.join(dist.location, filename)) 436 | print(e) 437 | 438 | 439 | class AggregatingLocator(object): 440 | def __init__(self, locators): 441 | self.locators = locators 442 | 443 | def locate(self, req, **kw): 444 | for locator in self.locators: 445 | print_message('try', locator, 'for', req) 446 | revealed = locator.locate(req, **kw) 447 | if revealed: 448 | return revealed 449 | 450 | 451 | class Suite(object): 452 | """Main object that represents current directory pundle state. 453 | It tracks RequirementStates, envs, urls for package locator. 454 | """ 455 | def __init__(self, parser, envs=[], urls=None): 456 | self.states = {} 457 | self.parser = parser 458 | self.envs = envs 459 | self.urls = urls or ['https://pypi.python.org/simple/'] 460 | if 'PIP_EXTRA_INDEX_URL' in os.environ: 461 | self.urls.append(os.environ['PIP_EXTRA_INDEX_URL']) 462 | self.locators = [] 463 | for url in self.urls: 464 | self.locators.append( 465 | locators.SimpleScrapingLocator(url, timeout=3.0, scheme='legacy') 466 | ) 467 | self.locators.append(locators.JSONLocator(scheme='legacy')) 468 | self.locator = AggregatingLocator(self.locators) 469 | 470 | def use(self, key): 471 | """For single mode 472 | You can call suite.use('arrow') and then `import arrow` 473 | 474 | :key: package name 475 | """ 476 | self.adjust_with_req(CustomReq(key, '')) 477 | self.install() 478 | self.activate_all() 479 | 480 | def locate(self, *a, **kw): 481 | return self.locator.locate(*a, **kw) 482 | 483 | def add(self, key, state): 484 | self.states[key] = state 485 | 486 | def __repr__(self): 487 | return '' % self.states 488 | 489 | def required_states(self): 490 | return [state for state in self.states.values() if state.requirement] 491 | 492 | def need_freeze(self, verbose=False): 493 | self.install(install=False) 494 | not_correct = not all(state.has_correct_freeze() for state in self.required_states()) 495 | if not_correct and verbose: 496 | for state in self.required_states(): 497 | if not state.has_correct_freeze(): 498 | print( 499 | state.key, 500 | 'Need', 501 | state.requirement, 502 | 'have not been frozen', 503 | state.frozen, 504 | ', installed', 505 | state.installed 506 | ) 507 | # TODO 508 | # unneeded = any(state.frozen for state in self.states.values() if not state.requirement) 509 | # if unneeded: 510 | # print('!!! Unneeded', [state.key for state in self.states.values() if not state.requirement]) 511 | return not_correct # or unneeded 512 | 513 | def adjust_with_req(self, req, install=False, upgrade=False, already_revealed=None): 514 | state = self.states.get(req.key) 515 | if not state: 516 | state = RequirementState(req.key, req=req) 517 | self.add(req.key, state) 518 | else: 519 | state.adjust_with_req(req) 520 | state.reveal_requirements(self, install=install, upgrade=upgrade, already_revealed=already_revealed or set()) 521 | 522 | def install(self, install=True): 523 | for state in self.required_states(): 524 | state.reveal_requirements(self, install=install) 525 | 526 | def upgrade(self, key=None, prereleases=False): 527 | states = [self.states[key]] if key else self.required_states() 528 | for state in states: 529 | print('Check', state.requirement.req) 530 | state.reveal_requirements(self, upgrade=True, prereleases=prereleases) 531 | 532 | def get_frozen_states(self, env): 533 | return [ 534 | state 535 | for state in self.required_states() 536 | if state.requirement and env in state.requirement.envs 537 | ] 538 | 539 | def need_install(self): 540 | return not all(state.frozen_dist() for state in self.states.values() if state.frozen) 541 | 542 | def install_frozen(self): 543 | for state in self.states.values(): 544 | state.install_frozen(self) 545 | 546 | def activate_all(self, envs=('',)): 547 | for state in self.required_states(): 548 | if '' in state.requirement.envs or any(env in state.requirement.envs for env in envs): 549 | state.activate() 550 | 551 | def save_frozen(self): 552 | "Saves frozen files to disk" 553 | states_by_env = dict( 554 | (env, self.get_frozen_states(env)) 555 | for env in self.parser.envs() 556 | ) 557 | self.parser.save_frozen(states_by_env) 558 | 559 | 560 | class Parser(object): 561 | """Gather environment info, requirements, 562 | frozen packages and create Suite object 563 | """ 564 | def __init__( 565 | self, 566 | base_path=None, 567 | directory='Pundledir', 568 | requirements_files=None, 569 | frozen_files=None, 570 | package=None, 571 | ): 572 | self.base_path = base_path or '.' 573 | self.directory = directory 574 | self.requirements_files = requirements_files 575 | if frozen_files is None: 576 | self.frozen_files = {'': 'frozen.txt'} 577 | else: 578 | self.frozen_files = frozen_files 579 | self.package = package 580 | self.package_envs = set(['']) 581 | 582 | def envs(self): 583 | if self.requirements_files: 584 | return list(self.requirements_files.keys()) or [''] 585 | elif self.package: 586 | return self.package_envs 587 | return [''] 588 | 589 | def get_frozen_file(self, env): 590 | if env in self.frozen_files: 591 | return self.frozen_files[env] 592 | else: 593 | return os.path.join(self.base_path, 'frozen_%s.txt' % env) 594 | 595 | def create_suite(self): 596 | reqs = self.parse_requirements() 597 | freezy = self.parse_frozen() 598 | hashes = self.parse_frozen_hashes() 599 | diry = self.parse_directory() 600 | state_keys = set(list(reqs.keys()) + list(freezy.keys()) + list(diry.keys())) 601 | suite = Suite(self, envs=self.envs()) 602 | for key in state_keys: 603 | suite.add( 604 | key, 605 | RequirementState( 606 | key, 607 | req=reqs.get(key), 608 | frozen=freezy.get(key), 609 | installed=diry.get(key, []), 610 | hashes=hashes.get(key), 611 | ), 612 | ) 613 | return suite 614 | 615 | def parse_directory(self): 616 | if not op.exists(self.directory): 617 | return {} 618 | dists = [ 619 | # this magic takes first element or None 620 | next(iter( 621 | pkg_resources.find_distributions(op.join(self.directory, item), True) 622 | ), None) 623 | for item in os.listdir(self.directory) if '-' in item 624 | ] 625 | dists.extend( 626 | VCSDist(op.join(self.directory, item)) 627 | for item in os.listdir(self.directory) if '+' in item 628 | ) 629 | dists = filter(None, dists) 630 | result = defaultdict(list) 631 | for dist in dists: 632 | result[dist.key].append(dist) 633 | return result 634 | 635 | def parse_frozen(self): 636 | frozen_versions = {} 637 | for env in self.envs(): 638 | frozen_file = self.get_frozen_file(env) 639 | if op.exists(frozen_file): 640 | frozen = [ 641 | (parse_frozen_vcs(line) or line.split('==')) 642 | for line in parse_file(frozen_file) 643 | ] 644 | else: 645 | frozen = [] 646 | for name, version in frozen: 647 | frozen_versions[name.lower()] = version 648 | return frozen_versions 649 | 650 | def parse_frozen_hashes(self): 651 | """This implementation does not support hashes yet 652 | """ 653 | return {} 654 | 655 | def parse_requirements(self): 656 | all_requirements = {} 657 | for env, req_file in self.requirements_files.items(): 658 | requirements = parse_file(req_file) 659 | if env: 660 | source = 'requirements %s file' % env 661 | else: 662 | source = 'requirements file' 663 | for line in requirements: 664 | req = CustomReq(line, env, source=source) 665 | if req.key in all_requirements: 666 | # if requirements exists in other env, then add this env too 667 | all_requirements[req.key].add_env(env) 668 | else: 669 | all_requirements[req.key] = req 670 | return all_requirements 671 | 672 | def save_frozen(self, states_by_env): 673 | for env, states in states_by_env.items(): 674 | data = '\n'.join(sorted( 675 | state.frozen_dump() 676 | for state in states 677 | )) + '\n' 678 | frozen_file = self.get_frozen_file(env) 679 | with open(frozen_file, 'w') as f: 680 | f.write(data) 681 | 682 | 683 | class SingleParser(Parser): 684 | """Parser for console mode. 685 | """ 686 | def parse_requirements(self): 687 | return {} 688 | 689 | 690 | class SetupParser(Parser): 691 | """Parser for `setup.py`. Because it mostly used to develop package, we 692 | do not freeze packages to setup.py. We use `frozen.txt`. 693 | """ 694 | def parse_requirements(self): 695 | setup_info = get_info_from_setup(self.package) 696 | if setup_info is None: 697 | raise PundleException('There is no requirements.txt nor setup.py') 698 | install_requires = setup_info.get('install_requires') or [] 699 | reqs = [ 700 | CustomReq(str(req), '', source='setup.py') 701 | for req in install_requires 702 | ] 703 | requirements = dict((req.key, req) for req in reqs) 704 | # we use `feature` as environment for pundle 705 | extras_require = (setup_info.get('extras_require') or {}) 706 | for feature, feature_requires in extras_require.items(): 707 | for req_line in feature_requires: 708 | req = CustomReq(req_line, feature, source='setup.py') 709 | # if this req already in dict, then add our feature as env 710 | if req.key in requirements: 711 | requirements[req.key].add_env(feature) 712 | else: 713 | requirements[req.key] = req 714 | self.package_envs.add(feature) 715 | return requirements 716 | 717 | 718 | class PipfileParser(Parser): 719 | """Parser for Pipfile and Pipfile.lock. 720 | """ 721 | DEFAULT_PIPFILE_SOURCES = [ 722 | { 723 | 'name': 'pypi', 724 | 'url': 'https://pypi.python.org/simple', 725 | 'verify_ssl': True, 726 | }, 727 | ] 728 | 729 | def __init__(self, **kw): 730 | self.pipfile = kw.pop('pipfile') 731 | self.pipfile_envs = set(['']) 732 | super(PipfileParser, self).__init__(**kw) 733 | # cache 734 | self.loaded_pipfile = None 735 | self.loaded_pipfile_lock = None 736 | 737 | def envs(self): 738 | return self.pipfile_envs 739 | 740 | def pipfile_content(self): 741 | import toml 742 | if self.loaded_pipfile: 743 | return self.loaded_pipfile 744 | self.loaded_pipfile = toml.load(open(self.pipfile)) 745 | return self.loaded_pipfile 746 | 747 | def pipfile_lock_content(self): 748 | if self.loaded_pipfile_lock: 749 | return self.loaded_pipfile_lock 750 | try: 751 | self.loaded_pipfile_lock = json.load(open(self.pipfile + '.lock')) 752 | except Exception: 753 | pass 754 | return self.loaded_pipfile_lock 755 | 756 | def parse_requirements(self): 757 | info = self.pipfile_content() 758 | all_requirements = {} 759 | for info_key in info: 760 | if not info_key.endswith('packages'): 761 | continue 762 | if '-' in info_key: 763 | env, _ = info_key.split('-', 1) 764 | else: 765 | env = '' 766 | self.pipfile_envs.add(env) 767 | for key, details in info[info_key].items(): 768 | if isinstance(details, str_types): 769 | if details != '*': 770 | key = key + details # details is a version requirement 771 | req = CustomReq(key, env, source='Pipfile') 772 | else: 773 | # a dict 774 | if 'file' in details or 'path' in details: 775 | raise PundleException('Unsupported Pipfile feature yet %s: %r' % (key, details)) 776 | if 'git' in details: 777 | # wow, this as a git package! 778 | req = CustomReq('git+%s#egg=%s' % (details['git'], key), env, source='Pipfile') 779 | else: 780 | # else just simple requirement 781 | req = CustomReq(key + details['version'], env, source='Pipfile') 782 | if req.key in all_requirements: 783 | # if requirements exists in other env, then add this env too 784 | all_requirements[req.key].add_env(env) 785 | else: 786 | all_requirements[req.key] = req 787 | return all_requirements 788 | 789 | def parse_frozen(self): 790 | parsed_frozen = self.pipfile_lock_content() 791 | if parsed_frozen is None: 792 | return {} 793 | frozen_versions = {} 794 | for env in parsed_frozen: 795 | if env.startswith('_'): 796 | # this is not an env 797 | continue 798 | for key, details in parsed_frozen[env].items(): 799 | if 'vcs' in details: 800 | frozen_versions[key] = details['vcs'] 801 | else: 802 | frozen_versions[key] = details.get('version', '0.0.0').lstrip('=') 803 | return frozen_versions 804 | 805 | def parse_frozen_hashes(self): 806 | parsed_frozen = self.pipfile_lock_content() 807 | if parsed_frozen is None: 808 | return {} 809 | frozen_versions = {} 810 | for env in parsed_frozen: 811 | if env.startswith('_'): 812 | # this is not an env 813 | continue 814 | for key, details in parsed_frozen[env].items(): 815 | frozen_versions[key] = details.get('hashes', []) 816 | return frozen_versions 817 | 818 | def hash(self): 819 | """Returns the SHA256 of the pipfile's data. 820 | From pipfile. 821 | """ 822 | pipfile_content = self.pipfile_content() 823 | data = { 824 | '_meta': { 825 | 'sources': pipfile_content.get('sources') or self.DEFAULT_PIPFILE_SOURCES, 826 | 'requires': pipfile_content.get('requires') or {}, 827 | }, 828 | 'default': pipfile_content.get('packages') or {}, 829 | 'develop': pipfile_content.get('dev-packages') or {}, 830 | } 831 | content = json.dumps(data, sort_keys=True, separators=(",", ":")) 832 | return hashlib.sha256(content.encode("utf8")).hexdigest() 833 | 834 | def save_frozen(self, states_by_env): 835 | """Implementation is not complete. 836 | """ 837 | data = self.pipfile_lock_content() or {} 838 | data.setdefault('_meta', { 839 | 'pipfile-spec': 5, 840 | 'requires': {}, 841 | 'sources': self.DEFAULT_PIPFILE_SOURCES, 842 | }) 843 | data.setdefault('_meta', {}).setdefault('hash', {})['sha256'] = self.hash() 844 | for env, states in states_by_env.items(): 845 | if env == '': 846 | env_key = 'default' 847 | elif env == 'dev': 848 | env_key = 'develop' 849 | else: 850 | env_key = env 851 | reqs = data.setdefault(env_key, {}) 852 | for state in states: 853 | if state.requirement.egg: 854 | egg, url, version = parse_vcs_requirement(state.requirement.line) 855 | reqs[state.key] = { 856 | 'vcs': url, 857 | } 858 | else: 859 | reqs[state.key] = { 860 | 'version': '==' + state.frozen, 861 | 'hashes': state.hashes or [], 862 | } 863 | with open(self.pipfile + '.lock', 'w') as f: 864 | f.write(json.dumps(data, sort_keys=True, indent=4)) 865 | 866 | 867 | def create_parser(**parser_args): 868 | """Utility function that tried to figure out what Parser to use 869 | in current directory. 870 | """ 871 | if parser_args.get('requirements_files'): 872 | return Parser(**parser_args) 873 | elif parser_args.get('package'): 874 | return SetupParser(**parser_args) 875 | elif parser_args.get('pipfile'): 876 | return PipfileParser(**parser_args) 877 | return SingleParser(**parser_args) 878 | 879 | 880 | # Utilities 881 | def get_info_from_setup(path): 882 | """Mock setuptools.setup(**kargs) to get 883 | package information about requirements and extras 884 | """ 885 | preserve = {} 886 | 887 | def _save_info(**setup_args): 888 | preserve['args'] = setup_args 889 | 890 | import setuptools 891 | original_setup = setuptools.setup 892 | setuptools.setup = _save_info 893 | import runpy 894 | runpy.run_path(os.path.join(path, 'setup.py'), run_name='__main__') 895 | setuptools.setup = original_setup 896 | return preserve.get('args') 897 | 898 | 899 | def search_files_upward(start_path=None): 900 | "Search for requirements.txt, setup.py or Pipfile upward" 901 | if not start_path: 902 | start_path = op.abspath(op.curdir) 903 | if any( 904 | op.exists(op.join(start_path, filename)) 905 | for filename in ('requirements.txt', 'setup.py', 'Pipfile') 906 | ): 907 | return start_path 908 | up_path = op.abspath(op.join(start_path, '..')) 909 | if op.samefile(start_path, up_path): 910 | return None 911 | return search_files_upward(start_path=up_path) 912 | 913 | 914 | def find_all_prefixed_files(directory, prefix): 915 | "find all requirements_*.txt files" 916 | envs = {} 917 | for entry in os.listdir(directory): 918 | if not entry.startswith(prefix): 919 | continue 920 | name, ext = op.splitext(entry) 921 | env = name[len(prefix):].lstrip('_') 922 | envs[env] = op.join(directory, entry) 923 | return envs 924 | 925 | 926 | def create_parser_parameters(): 927 | base_path = search_files_upward() 928 | if not base_path: 929 | raise PundleException('Can not find requirements.txt nor setup.py nor Pipfile') 930 | py_version_path = python_version_string() 931 | pundledir_base = os.environ.get('PUNDLEDIR') or op.join(op.expanduser('~'), '.pundledir') 932 | if op.exists(op.join(base_path, 'requirements.txt')): 933 | requirements_files = find_all_prefixed_files(base_path, 'requirements') 934 | else: 935 | requirements_files = {} 936 | envs = list(requirements_files.keys()) or [''] 937 | params = { 938 | 'base_path': base_path, 939 | 'frozen_files': { 940 | env: op.join(base_path, 'frozen_%s.txt' % env if env else 'frozen.txt') 941 | for env in envs 942 | }, 943 | 'directory': op.join(pundledir_base, py_version_path), 944 | } 945 | if requirements_files: 946 | params['requirements_files'] = requirements_files 947 | elif op.exists(op.join(base_path, 'setup.py')): 948 | params['package'] = base_path 949 | elif op.exists(op.join(base_path, 'Pipfile')): 950 | params['pipfile'] = op.join(base_path, 'Pipfile') 951 | else: 952 | return 953 | return params 954 | 955 | 956 | def create_parser_or_exit(): 957 | parser_kw = create_parser_parameters() 958 | if not parser_kw: 959 | print_message('You have not requirements.txt. Create it and run again.') 960 | exit(1) 961 | return parser_kw 962 | 963 | 964 | # Commands 965 | def upgrade_all(**kw): 966 | key = kw.pop('key') 967 | prereleases = kw.pop('prereleases') 968 | suite = create_parser(**kw).create_suite() 969 | suite.need_freeze() 970 | suite.upgrade(key=key, prereleases=prereleases) 971 | suite.install() 972 | suite.save_frozen() 973 | 974 | 975 | def install_all(**kw): 976 | suite = create_parser(**kw).create_suite() 977 | if suite.need_freeze() or suite.need_install(): 978 | print_message('Install some packages') 979 | suite.install() 980 | else: 981 | print_message('Nothing to do, all packages installed') 982 | suite.save_frozen() 983 | return suite 984 | 985 | 986 | def activate(): 987 | parser_kw = create_parser_parameters() 988 | if not parser_kw: 989 | raise PundleException('Can`t create parser parameters') 990 | suite = create_parser(**parser_kw).create_suite() 991 | if suite.need_freeze(verbose=True): 992 | raise PundleException('frozen file is outdated') 993 | if suite.need_install(): 994 | raise PundleException('Some dependencies not installed') 995 | envs = (os.environ.get('PUNDLEENV') or '').split(',') 996 | suite.activate_all(envs=envs) 997 | return suite 998 | 999 | 1000 | FIXATE_TEMPLATE = """ 1001 | # pundle user customization start 1002 | import pundle; pundle.activate() 1003 | # pundle user customization end 1004 | """ 1005 | 1006 | 1007 | def fixate(): 1008 | "puts activation code to usercustomize.py for user" 1009 | print_message('Fixate') 1010 | import site 1011 | userdir = site.getusersitepackages() 1012 | if not userdir: 1013 | raise PundleException('Can`t fixate due user have not site package directory') 1014 | try: 1015 | makedirs(userdir) 1016 | except OSError: 1017 | pass 1018 | template = FIXATE_TEMPLATE.replace('op.dirname(__file__)', "'%s'" % op.abspath(op.dirname(__file__))) 1019 | usercustomize_file = op.join(userdir, 'usercustomize.py') 1020 | print_message('Will edit %s file' % usercustomize_file) 1021 | if op.exists(usercustomize_file): 1022 | content = open(usercustomize_file).read() 1023 | if '# pundle user customization start' in content: 1024 | regex = re.compile(r'\n# pundle user customization start.*# pundle user customization end\n', re.DOTALL) 1025 | content, res = regex.subn(template, content) 1026 | open(usercustomize_file, 'w').write(content) 1027 | else: 1028 | open(usercustomize_file, 'a').write(content) 1029 | else: 1030 | open(usercustomize_file, 'w').write(template) 1031 | link_file = op.join(userdir, 'pundle.py') 1032 | if op.lexists(link_file): 1033 | print_message('Remove exist link to pundle') 1034 | os.unlink(link_file) 1035 | print_message('Create link to pundle %s' % link_file) 1036 | os.symlink(op.abspath(__file__), link_file) 1037 | print_message('Complete') 1038 | 1039 | 1040 | def entry_points(): 1041 | suite = activate() 1042 | entries = {} 1043 | for r in suite.states.values(): 1044 | d = r.frozen_dist() 1045 | if not d: 1046 | continue 1047 | if isinstance(d, VCSDist): 1048 | continue 1049 | scripts = d.get_entry_map().get('console_scripts', {}) 1050 | for name in scripts: 1051 | entries[name] = d 1052 | return entries 1053 | 1054 | 1055 | class CmdRegister: 1056 | commands = {} 1057 | ordered = [] 1058 | 1059 | @classmethod 1060 | def cmdline(cls, *cmd_aliases): 1061 | def wrap(func): 1062 | for alias in cmd_aliases: 1063 | cls.commands[alias] = func 1064 | cls.ordered.append(alias) 1065 | return wrap 1066 | 1067 | @classmethod 1068 | def help(cls): 1069 | for alias in cls.ordered: 1070 | if not alias: 1071 | continue 1072 | print("{:15s} {}".format(alias, cls.commands[alias].__doc__)) 1073 | 1074 | @classmethod 1075 | def main(cls): 1076 | alias = '' if len(sys.argv) == 1 else sys.argv[1] 1077 | if alias == 'help': 1078 | cls.help() 1079 | return 1080 | if alias not in cls.commands: 1081 | print('Unknown command\nTry this:') 1082 | cls.help() 1083 | sys.exit(1) 1084 | cls.commands[alias]() 1085 | 1086 | 1087 | @CmdRegister.cmdline('', 'install') 1088 | def cmd_install(): 1089 | "Install packages by frozen.txt and resolve ones that was not frozen" 1090 | install_all(**create_parser_or_exit()) 1091 | 1092 | 1093 | @CmdRegister.cmdline('upgrade') 1094 | def cmd_upgrade(): 1095 | """ 1096 | [package [pre]] if package provided will upgrade it and dependencies or all packages from PyPI. 1097 | If `pre` provided will look for prereleases. 1098 | """ 1099 | key = sys.argv[2] if len(sys.argv) > 2 else None 1100 | prereleases = sys.argv[3] == 'pre' if len(sys.argv) > 3 else False 1101 | upgrade_all(key=key, prereleases=prereleases, **create_parser_or_exit()) 1102 | 1103 | 1104 | CmdRegister.cmdline('fixate')(fixate) 1105 | 1106 | 1107 | @CmdRegister.cmdline('exec') 1108 | def cmd_exec(): 1109 | "executes setuptools entry" 1110 | cmd = sys.argv[2] 1111 | args = sys.argv[3:] 1112 | entries = entry_points() 1113 | if cmd not in entries: 1114 | print_message('Script is not found. Check if package is installed, or look at the `pundle entry_points`') 1115 | sys.exit(1) 1116 | exc = entries[cmd].get_entry_info('console_scripts', cmd).load() 1117 | sys.path.insert(0, '') 1118 | sys.argv = [cmd] + args 1119 | exc() 1120 | 1121 | 1122 | @CmdRegister.cmdline('entry_points') 1123 | def cmd_entry_points(): 1124 | "prints available setuptools entries" 1125 | for entry, package in entry_points().items(): 1126 | print('%s (%s)' % (entry, package)) 1127 | 1128 | 1129 | @CmdRegister.cmdline('edit') 1130 | def cmd_edit(): 1131 | "prints directory path to package" 1132 | parser_kw = create_parser_parameters() 1133 | suite = create_parser(**parser_kw).create_suite() 1134 | if suite.need_freeze(): 1135 | raise PundleException('%s file is outdated' % suite.parser.frozen_file) 1136 | print(suite.states[sys.argv[2]].frozen_dist().location) 1137 | 1138 | 1139 | @CmdRegister.cmdline('info') 1140 | def cmd_info(): 1141 | "prints info about Pundle state" 1142 | parser_kw = create_parser_parameters() 1143 | suite = create_parser(**parser_kw).create_suite() 1144 | if suite.need_freeze(): 1145 | print('frozen.txt is outdated') 1146 | else: 1147 | print('frozen.txt is up to date') 1148 | for state in suite.required_states(): 1149 | print( 1150 | 'Requirement "{}", frozen {}, {}'.format( 1151 | state.key, 1152 | state.frozen, 1153 | state.requirement.line if state.requirement else 'None' 1154 | ) 1155 | ) 1156 | print('Installed versions:') 1157 | for dist in state.installed: 1158 | print(' ', repr(dist)) 1159 | if not state.installed: 1160 | print(' None') 1161 | 1162 | 1163 | def run_console(glob): 1164 | import readline 1165 | import rlcompleter 1166 | import atexit 1167 | import code 1168 | 1169 | history_path = os.path.expanduser("~/.python_history") 1170 | 1171 | def save_history(history_path=history_path): 1172 | readline.write_history_file(history_path) 1173 | if os.path.exists(history_path): 1174 | readline.read_history_file(history_path) 1175 | 1176 | atexit.register(save_history) 1177 | 1178 | readline.set_completer(rlcompleter.Completer(glob).complete) 1179 | readline.parse_and_bind("tab: complete") 1180 | code.InteractiveConsole(locals=glob).interact() 1181 | 1182 | 1183 | @CmdRegister.cmdline('console') 1184 | def cmd_console(): 1185 | "[ipython|bpython|ptpython] starts python console with activated pundle environment" 1186 | suite = activate() 1187 | glob = { 1188 | 'pundle_suite': suite, 1189 | } 1190 | interpreter = sys.argv[2] if len(sys.argv) > 2 else None 1191 | if not interpreter: 1192 | run_console(glob) 1193 | elif interpreter == 'ipython': 1194 | from IPython import embed 1195 | embed() 1196 | elif interpreter == 'ptpython': 1197 | from ptpython.repl import embed 1198 | embed(glob, {}) 1199 | elif interpreter == 'bpython': 1200 | from bpython import embed 1201 | embed(glob) 1202 | else: 1203 | raise PundleException('Unknown interpreter: {}. Choose one of None, ipython, bpython, ptpython.') 1204 | 1205 | 1206 | @CmdRegister.cmdline('run') 1207 | def cmd_run(): 1208 | "executes given script" 1209 | activate() 1210 | import runpy 1211 | sys.path.insert(0, '') 1212 | script = sys.argv[2] 1213 | sys.argv = [sys.argv[2]] + sys.argv[3:] 1214 | runpy.run_path(script, run_name='__main__') 1215 | 1216 | 1217 | @CmdRegister.cmdline('module') 1218 | def cmd_module(): 1219 | "executes module like `python -m`" 1220 | activate() 1221 | import runpy 1222 | sys.path.insert(0, '') 1223 | module = sys.argv[2] 1224 | sys.argv = [sys.argv[2]] + sys.argv[3:] 1225 | runpy.run_module(module, run_name='__main__') 1226 | 1227 | 1228 | @CmdRegister.cmdline('env') 1229 | def cmd_env(): 1230 | "populates PYTHONPATH with packages paths and executes command line in subprocess" 1231 | activate() 1232 | aug_env = os.environ.copy() 1233 | aug_env['PYTHONPATH'] = ':'.join(sys.path) 1234 | subprocess.call(sys.argv[2:], env=aug_env) 1235 | 1236 | 1237 | @CmdRegister.cmdline('print_env') 1238 | def cmd_print_env(): 1239 | "Prints PYTHONPATH. For usage with mypy and MYPYPATH" 1240 | suite = activate() 1241 | path = ':'.join( 1242 | state.frozen_dist().location 1243 | for state in suite.states.values() 1244 | if state.frozen_dist() 1245 | ) 1246 | print(path) 1247 | 1248 | 1249 | ENTRY_POINT_TEMPLATE = '''#! /usr/bin/env python 1250 | import pundle; pundle.activate() 1251 | pundle.entry_points()['{entry_point}'].get_entry_info('console_scripts', '{entry_point}').load(require=False)() 1252 | ''' 1253 | 1254 | 1255 | @CmdRegister.cmdline('linkall') 1256 | def link_all(): 1257 | "links all packages to `.pundle_local` dir" 1258 | local_dir = '.pundle_local' 1259 | suite = activate() 1260 | 1261 | try: 1262 | makedirs(local_dir) 1263 | except OSError: 1264 | pass 1265 | local_dir_info = {de.name: de for de in os.scandir(local_dir)} 1266 | for r in suite.states.values(): 1267 | d = r.frozen_dist() 1268 | if not d: 1269 | continue 1270 | for dir_entry in os.scandir(d.location): 1271 | if dir_entry.name.startswith('__') or dir_entry.name.startswith('.') or dir_entry.name == 'bin': 1272 | continue 1273 | dest_path = os.path.join(local_dir, dir_entry.name) 1274 | if dir_entry.name in local_dir_info: 1275 | sym = local_dir_info.pop(dir_entry.name) 1276 | existed = op.realpath(sym.path) 1277 | if existed == dir_entry.path: 1278 | continue 1279 | os.remove(sym.path) 1280 | os.symlink(dir_entry.path, dest_path) 1281 | # create entry_points binaries 1282 | try: 1283 | makedirs(os.path.join(local_dir, 'bin')) 1284 | except OSError: 1285 | pass 1286 | for bin_name, entry_point in entry_points().items(): 1287 | bin_filename = os.path.join(local_dir, 'bin', bin_name) 1288 | open(bin_filename, 'w').write(ENTRY_POINT_TEMPLATE.format(entry_point=bin_name)) 1289 | file_stat = os.stat(bin_filename) 1290 | os.chmod(bin_filename, file_stat.st_mode | stat.S_IEXEC) 1291 | local_dir_info.pop('bin') 1292 | 1293 | # remove extra links 1294 | for de in local_dir_info: 1295 | os.remove(de.path) 1296 | 1297 | 1298 | @CmdRegister.cmdline('show_requirements') 1299 | def show_requirements(): 1300 | "shows details requirements info" 1301 | suite = activate() 1302 | for name, state in suite.states.items(): 1303 | if state.requirement: 1304 | print( 1305 | name, 1306 | 'frozen:', 1307 | state.frozen, 1308 | 'required:', 1309 | state.requirement.req if state.requirement.req else 'VCS', 1310 | state.requirement.envs, 1311 | ) 1312 | 1313 | 1314 | # Single mode that you can use in console 1315 | _single_mode_suite = {} # cache variable to keep current suite for single_mode 1316 | 1317 | 1318 | def single_mode(): 1319 | """ Create, cache and return Suite instance for single_mode. 1320 | """ 1321 | if not _single_mode_suite: 1322 | py_version_path = python_version_string() 1323 | pundledir_base = os.environ.get('PUNDLEDIR') or op.join(op.expanduser('~'), '.pundledir') 1324 | directory = op.join(pundledir_base, py_version_path) 1325 | _single_mode_suite['cache'] = create_parser(directory=directory).create_suite() 1326 | return _single_mode_suite['cache'] 1327 | 1328 | 1329 | def use(key): 1330 | """ Installs `key` requirement, like `django==1.11` or just `django` 1331 | """ 1332 | suite = single_mode() 1333 | suite.use(key) 1334 | 1335 | 1336 | if __name__ == '__main__': 1337 | CmdRegister.main() 1338 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [bdist_wheel] 2 | universal = 1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from setuptools import setup 4 | import os.path 5 | 6 | 7 | def read(fname): 8 | try: 9 | return open(os.path.join(os.path.dirname(__file__), fname)).read() 10 | except IOError: 11 | return '' 12 | 13 | 14 | setupconf = dict( 15 | name='pundle', 16 | version='0.9.2', 17 | license='BSD', 18 | url='https://github.com/Deepwalker/pundler/', 19 | author='Deepwalker', 20 | author_email='krivushinme@gmail.com', 21 | description=('Requirements management tool.'), 22 | long_description=read('README.rst'), 23 | keywords='bundler virtualenv pip install package setuptools', 24 | 25 | py_modules=['pundle'], 26 | entry_points=dict( 27 | console_scripts=[ 28 | 'pundle = pundle:CmdRegister.main' 29 | ] 30 | ), 31 | classifiers=[ 32 | 'Intended Audience :: Developers', 33 | 'License :: OSI Approved :: BSD License', 34 | 'Operating System :: OS Independent', 35 | 'Programming Language :: Python', 36 | 'Programming Language :: Python :: 3', 37 | ] 38 | ) 39 | 40 | if __name__ == '__main__': 41 | setup(**setupconf) 42 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Deepwalker/pundler/dba19d6b7b8c2bc602d1ac9c5612a5cb8125cdb4/tests/__init__.py -------------------------------------------------------------------------------- /tests/lib.py: -------------------------------------------------------------------------------- 1 | def fake_parse(files): 2 | def parse_file(filename): 3 | print('Parse', filename) 4 | return files[filename] 5 | return parse_file 6 | -------------------------------------------------------------------------------- /tests/test_install.py: -------------------------------------------------------------------------------- 1 | from pundle import Parser 2 | 3 | from .lib import fake_parse 4 | 5 | 6 | PARSER_ARGS = { 7 | 'requirements_files': {'': 'requirements.txt'}, 8 | 'frozen_files': {'': 'frozen.txt'}, 9 | } 10 | 11 | 12 | def test_need_freeze(mocker): 13 | parse_file = fake_parse({ 14 | 'requirements.txt': ['trafaret'], 15 | 'frozen.txt': [], 16 | }) 17 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 18 | mocker.patch('pundle.op.exists') 19 | mocker.patch('pundle.os.listdir') 20 | parser = Parser(**PARSER_ARGS) 21 | suite = parser.create_suite() 22 | print(suite) 23 | assert suite.need_freeze() == True 24 | 25 | 26 | def test_frozen(mocker): 27 | parse_file = fake_parse({ 28 | 'requirements.txt': ['trafaret'], 29 | 'frozen.txt': ['trafaret==0.1'], 30 | }) 31 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 32 | mocker.patch('pundle.op.exists') 33 | mocker.patch('pundle.os.listdir') 34 | parser = Parser(**PARSER_ARGS) 35 | suite = parser.create_suite() 36 | print(suite) 37 | assert suite.need_freeze() == False 38 | assert suite.need_install() == True 39 | 40 | 41 | def test_vcs(mocker): 42 | parse_file = fake_parse({ 43 | 'requirements.txt': ['git+https://github.com/karanlyons/django-save-the-change@e48502d2568d76bd9c7093f4c002a5b0061bc468#egg=django-save-the-change'], 44 | 'frozen.txt': [], 45 | }) 46 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 47 | mocker.patch('pundle.op.exists') 48 | mocker.patch('pundle.os.listdir') 49 | parser = Parser(**PARSER_ARGS) 50 | suite = parser.create_suite() 51 | print(suite) 52 | assert suite.need_freeze() == True 53 | 54 | 55 | def test_vcs_frozen(mocker): 56 | parse_file = fake_parse({ 57 | 'requirements.txt': ['git+https://github.com/karanlyons/django-save-the-change@e48502d2568d76bd9c7093f4c002a5b0061bc468#egg=django-save-the-change'], 58 | 'frozen.txt': ['git+https://github.com/karanlyons/django-save-the-change@e48502d2568d76bd9c7093f4c002a5b0061bc468#egg=django-save-the-change'], 59 | }) 60 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 61 | mocker.patch('pundle.op.exists') 62 | mocker.patch('pundle.os.listdir') 63 | parser = Parser(**PARSER_ARGS) 64 | suite = parser.create_suite() 65 | print(suite) 66 | assert suite.need_freeze() == False 67 | assert suite.need_install() == True 68 | -------------------------------------------------------------------------------- /tests/test_pipfile.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | from pundle import create_parser 4 | from .lib import fake_parse 5 | 6 | 7 | PARSER_ARGS = { 8 | 'pipfile': '...../bla/blu/Pipfile', 9 | } 10 | 11 | 12 | PIPFILE = ''' 13 | [[source]] 14 | url = "https://pypi.python.org/simple" 15 | verify_ssl = true 16 | name = "pypi" 17 | 18 | [packages] 19 | requests = "*" 20 | 21 | 22 | [dev-packages] 23 | pytest = "*" 24 | ''' 25 | 26 | 27 | PIPFILE_LOCK = '''{ 28 | "_meta": { 29 | "hash": { 30 | "sha256": "8d14434df45e0ef884d6c3f6e8048ba72335637a8631cc44792f52fd20b6f97a" 31 | }, 32 | "host-environment-markers": { 33 | "implementation_name": "cpython", 34 | "implementation_version": "3.6.1", 35 | "os_name": "posix", 36 | "platform_machine": "x86_64", 37 | "platform_python_implementation": "CPython", 38 | "platform_release": "16.7.0", 39 | "platform_system": "Darwin", 40 | "platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64", 41 | "python_full_version": "3.6.1", 42 | "python_version": "3.6", 43 | "sys_platform": "darwin" 44 | }, 45 | "pipfile-spec": 5, 46 | "requires": {}, 47 | "sources": [ 48 | { 49 | "name": "pypi", 50 | "url": "https://pypi.python.org/simple", 51 | "verify_ssl": true 52 | } 53 | ] 54 | }, 55 | "default": { 56 | "certifi": { 57 | "hashes": [ 58 | "sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704", 59 | "sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5" 60 | ], 61 | "version": "==2017.7.27.1" 62 | }, 63 | "chardet": { 64 | "hashes": [ 65 | "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691", 66 | "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae" 67 | ], 68 | "version": "==3.0.4" 69 | }, 70 | "idna": { 71 | "hashes": [ 72 | "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4", 73 | "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f" 74 | ], 75 | "version": "==2.6" 76 | }, 77 | "requests": { 78 | "hashes": [ 79 | "sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b", 80 | "sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e" 81 | ], 82 | "version": "==2.18.4" 83 | }, 84 | "urllib3": { 85 | "hashes": [ 86 | "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b", 87 | "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f" 88 | ], 89 | "version": "==1.22" 90 | } 91 | }, 92 | "develop": { 93 | "py": { 94 | "hashes": [ 95 | "sha256:2ccb79b01769d99115aa600d7eed99f524bf752bba8f041dc1c184853514655a", 96 | "sha256:0f2d585d22050e90c7d293b6451c83db097df77871974d90efd5a30dc12fcde3" 97 | ], 98 | "version": "==1.4.34" 99 | }, 100 | "pytest": { 101 | "hashes": [ 102 | "sha256:b84f554f8ddc23add65c411bf112b2d88e2489fd45f753b1cae5936358bdf314", 103 | "sha256:f46e49e0340a532764991c498244a60e3a37d7424a532b3ff1a6a7653f1a403a" 104 | ], 105 | "version": "==3.2.2" 106 | } 107 | } 108 | }''' 109 | 110 | 111 | def test_parse_pipfile(mocker): 112 | open_mock = mocker.patch('pundle.open') 113 | open_mock.side_effect = [ 114 | mocker.mock_open(read_data=PIPFILE).return_value, 115 | mocker.mock_open(read_data=PIPFILE_LOCK).return_value, 116 | ] 117 | mocker.patch('pundle.op.exists') 118 | mocker.patch('pundle.os.listdir') 119 | parser = create_parser(**PARSER_ARGS) 120 | suite = parser.create_suite() 121 | assert suite.need_freeze() == False 122 | assert 'requests' in suite.states 123 | 124 | 125 | def test_parse_pipfile_no_lock(mocker): 126 | open_mock = mocker.patch('pundle.open') 127 | open_mock.side_effect = [ 128 | mocker.mock_open(read_data=PIPFILE).return_value, 129 | Exception('bam!'), 130 | Exception('bam!'), 131 | Exception('bam!'), 132 | ] 133 | mocker.patch('pundle.op.exists') 134 | mocker.patch('pundle.os.listdir') 135 | parser = create_parser(**PARSER_ARGS) 136 | suite = parser.create_suite() 137 | assert suite.need_freeze() == True 138 | assert 'requests' in suite.states 139 | 140 | 141 | def test_save_pipfile_lock(mocker): 142 | open_mock = mocker.patch('pundle.open') 143 | write_mock = mocker.MagicMock() 144 | open_mock.side_effect = [ 145 | mocker.mock_open(read_data=PIPFILE).return_value, 146 | mocker.mock_open(read_data=PIPFILE_LOCK).return_value, 147 | write_mock, 148 | ] 149 | mocker.patch('pundle.op.exists') 150 | mocker.patch('pundle.os.listdir') 151 | parser = create_parser(**PARSER_ARGS) 152 | suite = parser.create_suite() 153 | assert suite.need_freeze() == False 154 | assert 'requests' in suite.states 155 | suite.save_frozen() 156 | assert open_mock.call_count == 3 157 | assert open_mock.call_args == mocker.call('...../bla/blu/Pipfile.lock', 'w') 158 | assert write_mock.__enter__().write.called 159 | # FIXME? this line produces error and it is right - we does not dump any dependencies here 160 | # assert json.loads(write_mock.__enter__().write.call_args[0][0]) == json.loads(PIPFILE_LOCK) 161 | -------------------------------------------------------------------------------- /tests/test_setup_py.py: -------------------------------------------------------------------------------- 1 | from pundle import create_parser 2 | 3 | from .lib import fake_parse 4 | 5 | 6 | PARSER_ARGS = { 7 | 'requirements_files': None, 8 | 'frozen_files': {'': 'frozen.txt'}, 9 | 'package': '.', 10 | } 11 | 12 | 13 | def test_parse_setup_need_freeze(mocker): 14 | parse_file = fake_parse({ 15 | 'frozen.txt': [], 16 | './frozen_objectid.txt': [], 17 | }) 18 | setup_data = { 19 | 'install_requires': ['trafaret'], 20 | 'extras_require': { 21 | 'objectid': ['mongodb'] 22 | }, 23 | } 24 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 25 | mocker.patch('pundle.op.exists') 26 | mocker.patch('pundle.os.listdir') 27 | mocker.patch('pundle.get_info_from_setup', new_callable=lambda: (lambda x: setup_data)) 28 | parser = create_parser(**PARSER_ARGS) 29 | suite = parser.create_suite() 30 | print(suite) 31 | assert suite.need_freeze() == True 32 | 33 | 34 | def test_parse_setup_frozen(mocker): 35 | parse_file = fake_parse({ 36 | 'frozen.txt': ['trafaret==0.1.1'], 37 | './frozen_objectid.txt': ['mongodb==0.1.0'], 38 | }) 39 | setup_data = { 40 | 'install_requires': ['trafaret'], 41 | 'extras_require': { 42 | 'objectid': ['mongodb'] 43 | }, 44 | } 45 | mocker.patch('pundle.parse_file', new_callable=lambda: parse_file) 46 | mocker.patch('pundle.op.exists') 47 | mocker.patch('pundle.os.listdir') 48 | mocker.patch('pundle.get_info_from_setup', new_callable=lambda: (lambda x: setup_data)) 49 | parser = create_parser(**PARSER_ARGS) 50 | suite = parser.create_suite() 51 | print(suite) 52 | assert suite.need_freeze() == False 53 | assert suite.need_install() == True 54 | -------------------------------------------------------------------------------- /tests/test_simple.py: -------------------------------------------------------------------------------- 1 | import pundle 2 | 3 | 4 | def test_pypy_python_version(mocker): 5 | sys_mock = mocker.patch('pundle.sys') 6 | platform_mock = mocker.patch('pundle.platform') 7 | platform_mock.python_implementation.return_value = 'PyPy' 8 | platform_mock.python_build.return_value = ('build1', 'blabla') 9 | class VersionInfo: 10 | major = 1 11 | minor = 2 12 | micro = 3 13 | sys_mock.pypy_version_info = VersionInfo 14 | assert pundle.python_version_string() == 'PyPy-1.2.3-build1' 15 | 16 | 17 | def test_cpython_python_version(mocker): 18 | sys_mock = mocker.patch('pundle.sys') 19 | platform_mock = mocker.patch('pundle.platform') 20 | platform_mock.python_implementation.return_value = 'CPython' 21 | platform_mock.python_build.return_value = ('build1', 'blabla') 22 | class VersionInfo: 23 | major = 1 24 | minor = 2 25 | micro = 3 26 | sys_mock.version_info = VersionInfo 27 | assert pundle.python_version_string() == 'CPython-1.2.3-build1' 28 | -------------------------------------------------------------------------------- /tests/test_vcs_requirements.py: -------------------------------------------------------------------------------- 1 | from pundle import parse_vcs_requirement 2 | 3 | 4 | def test_parse_vcs_requirement(): 5 | assert parse_vcs_requirement('git+https://github.com/pampam/PKG.git@master#egg=PKG') == \ 6 | ('pkg', 'git+https://github.com/pampam/PKG.git@master#egg=PKG', None) 7 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py27,py38 3 | 4 | [testenv] 5 | deps= 6 | unittest2 7 | pymongo 8 | flake8 9 | pylint 10 | pytest 11 | pytest-mock 12 | pytest-cov 13 | coverage 14 | toml 15 | commands= 16 | pytest --cov=pundle -s -vv {toxinidir}/tests 17 | flake8 pundle.py 18 | 19 | [flake8] 20 | exclude = .tox,*.egg,build 21 | max-line-length = 120 22 | -------------------------------------------------------------------------------- /virtualenv.md: -------------------------------------------------------------------------------- 1 | Virtualenv – the worst thing ever happened to python. 2 | ==================================================== 3 | 4 | Typo-typa, click-clack, `venv ve` etc 5 | `pip install` and `pip remove` 6 | You don't know completely friend 7 | What is a state of your virtualenv 8 | 9 | 10 | Virtualenv is a pile of grabage. We can stop here, but you damn don't understand 11 | what I'm talking about every damn time when I'm talking about it. Shit. 12 | Anyway, we need to have a talk, fellow python programmer. 13 | 14 | 15 | Falsehoods about python packages and virtualenv 16 | ----------------------------------------------- 17 | 18 | 1. If I write package to requirements.txt it will be in virtualenv 19 | 2. if I run `pip install -r requirements.txt` all packages with 20 | right versions will be in virtualenv 21 | 3. I can manage all packages versions myself 22 | 4. Ok, at least if I will pin versions for packages like Django 23 | I'm safe 24 | 3. If I use Pipenv then I'm safe and my virtualenv will regard 25 | what pipenv use to keep packages 26 | 4. I will not forget to keep my virtualenv up to date 27 | 5. Some people can forget, but not me 28 | 6. Virtualenv can not be broken 29 | 7. Package maintainers can not broke my virtualenv 30 | 8. Package maintainers are sane people and do not do any meaningless shit 31 | on package installation that can breake my virtualenv or 32 | will require recreate virtualenv to install this damn piece of shit 33 | because it broke itself 34 | 35 | 36 | Why? 37 | ---- 38 | 39 | Virtualenv is dynamic. I mean like python, ruby, javascript. It is 40 | dynamic and you don't know what state it is. Do you know about 41 | shiny `pip freeze`? Do you belive that it is always correct and can actually 42 | figure out what packages installed in your virtualenv? Yep? Sure? 43 | One more falsehood. 44 | 45 | The only way to have some confidence about virtualenv is recreate it from scratch, 46 | install all packages in same order. Did you pinned versions, by the way? 47 | 48 | But when to recreate it? Did you check that requirements.txt is changed on 49 | `git checkout` or `git pull`? Oh, sorry, virtualenv are not to save your ass from 50 | hours of debugging, pure little coder, common, world are not owe you anything, grow up. 51 | 52 | But why it is this way? Because pile of grabage is dynamic and is not depend on your 53 | warm, soft requirements.txt or what do you use for yourself. Even something 54 | funny .toml are not any interest for virtualenv, because it is a pile of garbage and 55 | piles of garbage are not have real interest for anything. 56 | 57 | 58 | How virtualenv mess with python packaging 59 | ----------------------------------------- 60 | 61 | Virtualenv is bad for python packaging. Because if it is a pile of 62 | garbage, then it does not matter to keep yourself sane and don't try 63 | to be stateless, to not interfere with explicitness of importing 64 | machinery. 65 | 66 | Package creators do insane things that are meaningless. Just open this link 67 | and find about \*.pth files. 68 | https://github.com/search?utf8=%E2%9C%93&q=extension%3Apth+import&type=Code 69 | And you have noticed that setup.py is actual python script, yes? 70 | 71 | 72 | Pipenv 73 | ------ 74 | 75 | It is not a Bundler. Man, if you use pile of garbage that virtualenv is, 76 | you did not get what bundler is about. It is not about "I'v write packages 77 | version", "I'v added shiny sha256's, but hell I know if they are actually 78 | was correct when I pinned them". 79 | It is about "I'v enforced damn packages versions, so you have not 80 | a chance to use wrong version, 'hole in the head' bastard!". 81 | 82 | What I dislike about pipenv, it is try to mask that virtualenv is shitty 83 | pile of garbage with nice decorations around it. 84 | 85 | Not on my watch, not on my watch. 86 | 87 | 88 | How it must to be actually? 89 | --------------------------- 90 | 91 | We can make it in different way, I will show you. Look carefully. 92 | 93 | We want to have requests==2.18.4 for our code service. We write it to some 94 | file, `requirements.txt` or `setup.py` or funny `.toml` thing. 95 | Then we install this package somewhere in isolation. I mean not to pile of 96 | garbage, listen here! In isolation, you know, not with other packages. 97 | 98 | When we start our programm, we tell python where to find this `requests` thing when 99 | it will issue `import` statement. 100 | 101 | So what we have in this case? If funny requirements file changes, then we will 102 | tell python interpreter that we have not installed version that we need, so 103 | no luck, we are doomed. We install it then and start program again. Now it is ok, 104 | we have right version of package. Really, without right package version you can not 105 | start your damn dear switty program. You have to have right version beforehand 106 | and this is good. This is how it supposed to be. 107 | 108 | Be a good man, don't use pile of garbage anymore. 109 | Use pundle instead. Guys, really. Stop this shit now. 110 | --------------------------------------------------------------------------------