├── .gitignore ├── .travis.yml ├── LICENSE ├── MANIFEST.in ├── README.md ├── appveyor.yml ├── get-pipsi.py ├── pipsi ├── __init__.py ├── __main__.py └── scripts │ ├── find_scripts.py │ └── get_version.py ├── setup.py ├── testing ├── conftest.py ├── test_command_line.py ├── test_install_script.py └── test_repo.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.pyc 3 | *.pyo 4 | dist 5 | build 6 | *.egg 7 | *.egg-info 8 | .tox/ 9 | .env/ 10 | .cache/ 11 | .pytest_cache/ 12 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | matrix: 3 | include: 4 | - env: TOXENV=py27 5 | python: "2.7" 6 | - env: TOXENV=pypy 7 | python: "pypy" 8 | - env: TOXENV=py34 9 | python: "3.4" 10 | - env: TOXENV=py35 11 | python: "3.5" 12 | - env: TOXENV=py36 13 | python: "3.6" 14 | 15 | install: pip install "tox<3" 16 | script: tox 17 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014 by Armin Ronacher. 2 | 3 | Some rights reserved. 4 | 5 | Redistribution and use in source and binary forms of the software as well 6 | as documentation, with or without modification, are permitted provided 7 | that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright 10 | notice, this list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above 13 | copyright notice, this list of conditions and the following 14 | disclaimer in the documentation and/or other materials provided 15 | with the distribution. 16 | 17 | * The names of the contributors may not be used to endorse or 18 | promote products derived from this software without specific 19 | prior written permission. 20 | 21 | THIS SOFTWARE AND DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND 22 | CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT 23 | NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 24 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER 25 | OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 26 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 27 | PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 28 | PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 29 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 30 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 31 | SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH 32 | DAMAGE. 33 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.md LICENSE tox.ini 2 | include get-pipsi.py 3 | recursive-include testing *.py 4 | recursive-include pipsi *.py 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # pipsi 2 | 3 | **⚠️ pipsi is no longer maintained. See [pipx](https://github.com/pipxproject/pipx) for an actively maintained alternative. https://github.com/pipxproject/pipx** 4 | 5 | --- 6 | 7 | pipsi = **pip** **s**cript **i**nstaller 8 | 9 | ## What does it do? 10 | **pipsi** makes installing python packages with global entry points painless. These are Python packages that expose an entry point through the command line such as [Pygments](https://pypi.org/project/Pygments/). 11 | 12 | If you are installing Python packages globally for cli access, you almost certainly want to use pipsi instead of running `sudo pip ...`. so that you get 13 | * Isolated dependencies to guarantee no version conflicts 14 | * The ability to install packages globally without using sudo 15 | * The ability to uninstall a package and its dependencies without affecting other globally installed Python programs 16 | 17 | pipsi is not meant for installing libraries that will be imported by other Python modules. 18 | 19 | ## How do I get it? 20 | 21 | ```bash 22 | curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python 23 | ``` 24 | 25 | to see installation options, including not automatically modifying the PATH environment variable 26 | 27 | ```bash 28 | curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python - --help 29 | ``` 30 | 31 | ## How does it work? 32 | 33 | pipsi is a wrapper around virtualenv and pip which installs scripts provided by python packages into isolated virtualenvs so they do not pollute your system's Python packages. 34 | 35 | pipsi installs each package into `~/.local/venvs/PKGNAME` and then symlinks all new scripts into `~/.local/bin` (these can be changed by `PIPSI_HOME` and `PIPSI_BIN_DIR` environment variables respectively). 36 | 37 | Here is a tree view into the directory structure created by pipsi after installing pipsi and running `pipsi install Pygments`. 38 | 39 | ``` 40 | /Users/user/.local 41 | ├── bin 42 | │   ├── pipsi -> /Users/user/.local/venvs/pipsi/bin/pipsi 43 | │   └── pygmentize -> /Users/user/.local/venvs/pygments/bin/pygmentize 44 | ├── share 45 | │   └── virtualenvs 46 | └── venvs 47 | ├── pipsi 48 | └── pygments 49 | ``` 50 | 51 | Compared to `pip install --user` each `PKGNAME` is installed into its own virtualenv, so you don't have to worry about different packages having conflicting dependencies. As long as `~/.local/bin` is on your PATH, you can run any of these scripts directly. 52 | 53 | ### Installing scripts from a package: 54 | 55 | ```bash 56 | $ pipsi install Pygments 57 | ``` 58 | 59 | ### Installing scripts from a package using a particular version of python: 60 | 61 | ```bash 62 | $ pipsi install --python /usr/bin/python3.5 hovercraft 63 | ``` 64 | 65 | ### Uninstalling packages and their scripts: 66 | 67 | ```bash 68 | $ pipsi uninstall Pygments 69 | ``` 70 | 71 | ### Upgrading a package: 72 | 73 | ```bash 74 | $ pipsi upgrade Pygments 75 | ``` 76 | 77 | ### Showing what's installed: 78 | 79 | ```bash 80 | $ pipsi list 81 | ``` 82 | 83 | ### How do I get rid of pipsi? 84 | 85 | ```bash 86 | $ pipsi uninstall pipsi 87 | ``` 88 | 89 | ### How do I upgrade pipsi? 90 | 91 | With 0.5 and later just do this: 92 | 93 | ```bash 94 | $ pipsi upgrade pipsi 95 | ``` 96 | 97 | On older versions just uninstall and reinstall. 98 | -------------------------------------------------------------------------------- /appveyor.yml: -------------------------------------------------------------------------------- 1 | build: false # Not a C# project, build stuff at the test step instead. 2 | environment: 3 | matrix: 4 | - PYTHON: "C:/Python27" 5 | - PYTHON: "C:/Python33" 6 | - PYTHON: "C:/Python34" 7 | 8 | init: 9 | - "ECHO %PYTHON%" 10 | - ps: "ls C:/Python*" 11 | 12 | install: 13 | - ps: (new-object net.webclient).DownloadFile('https://raw.github.com/pypa/pip/master/contrib/get-pip.py', 'C:/get-pip.py') 14 | - "%PYTHON%/python.exe C:/get-pip.py" 15 | - "%PYTHON%/Scripts/pip.exe install tox" 16 | 17 | test_script: 18 | - "%PYTHON%/Scripts/tox.exe" 19 | -------------------------------------------------------------------------------- /get-pipsi.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import argparse 3 | import os 4 | import shutil 5 | import sys 6 | from subprocess import call, check_output 7 | import textwrap 8 | 9 | 10 | try: 11 | WindowsError 12 | except NameError: 13 | IS_WIN = False 14 | PIP = '/bin/pip' 15 | PIPSI = '/bin/pipsi' 16 | else: 17 | IS_WIN = True 18 | PIP = '/Scripts/pip.exe' 19 | PIPSI = '/Scripts/pipsi.exe' 20 | 21 | if sys.version_info.major < 3: 22 | try: 23 | import virtualenv # NOQA 24 | venv_pkg = 'virtualenv' 25 | del virtualenv 26 | except ImportError: 27 | venv_pkg = None 28 | else: 29 | venv_pkg = 'venv' 30 | 31 | DEFAULT_PIPSI_HOME = os.path.expanduser('~/.local/venvs') 32 | DEFAULT_PIPSI_BIN_DIR = os.path.expanduser('~/.local/bin') 33 | 34 | 35 | def echo(msg=''): 36 | sys.stdout.write(msg + '\n') 37 | sys.stdout.flush() 38 | 39 | 40 | def fail(msg): 41 | sys.stderr.write(msg + '\n') 42 | sys.stderr.flush() 43 | sys.exit(1) 44 | 45 | 46 | def succeed(msg): 47 | echo(msg) 48 | sys.exit(0) 49 | 50 | 51 | def command_exists(cmd): 52 | with open(os.devnull, 'w') as null: 53 | try: 54 | return call( 55 | [cmd, '--version'], 56 | stdout=null, stderr=null) == 0 57 | except OSError: 58 | return False 59 | 60 | 61 | def publish_script(venv, bin_dir): 62 | if IS_WIN: 63 | for name in os.listdir(venv + '/Scripts'): 64 | if 'pipsi' in name.lower(): 65 | shutil.copy(venv + '/Scripts/' + name, bin_dir) 66 | else: 67 | os.symlink(venv + '/bin/pipsi', bin_dir + '/pipsi') 68 | echo('Installed pipsi binary in ' + bin_dir) 69 | 70 | 71 | def install_files(venv, bin_dir, install): 72 | try: 73 | os.makedirs(bin_dir) 74 | except OSError: 75 | pass 76 | 77 | def _cleanup(): 78 | try: 79 | shutil.rmtree(venv) 80 | except (OSError, IOError): 81 | pass 82 | 83 | 84 | if sys.version_info.major < 3: 85 | executable = sys.executable 86 | else: 87 | executable = get_real_python(sys.executable) 88 | print('sys.executable={} sys.real_prefix={} executable={}'.format(sys.executable, getattr(sys, 'real_prefix', None), executable)) 89 | venv_cmd = [executable, '-m', venv_pkg] 90 | if venv_pkg == 'virtualenv': 91 | venv_cmd += ['-p', executable] 92 | venv_cmd += [venv] 93 | 94 | if call(venv_cmd) != 0: 95 | _cleanup() 96 | fail('Could not create virtualenv for pipsi :(') 97 | 98 | if call([venv + PIP, 'install', install]) != 0: 99 | _cleanup() 100 | fail('Could not install pipsi :(') 101 | 102 | publish_script(venv, bin_dir) 103 | 104 | 105 | def parse_options(argv): 106 | bin_dir = os.environ.get('PIPSI_BIN_DIR', DEFAULT_PIPSI_BIN_DIR) 107 | home_dir = os.environ.get('PIPSI_HOME', DEFAULT_PIPSI_HOME) 108 | 109 | parser = argparse.ArgumentParser() 110 | parser.add_argument( 111 | '--bin-dir', 112 | default=bin_dir, 113 | help=( 114 | 'Executables will be installed into this folder. ' 115 | 'Default: %(default)s' 116 | ), 117 | ) 118 | parser.add_argument( 119 | '--home', 120 | dest='home_dir', 121 | default=home_dir, 122 | help='Virtualenvs are created in this folder. Default: %(default)s', 123 | ) 124 | parser.add_argument( 125 | '--src', 126 | default='pipsi', 127 | help=( 128 | 'The specific version of pipsi to install. This value is passed ' 129 | 'to "pip install ". For example, to install from master ' 130 | 'use "git+https://github.com/mitsuhiko/pipsi.git#egg=pipsi". ' 131 | 'Default: %(default)s' 132 | ), 133 | ) 134 | parser.add_argument( 135 | '--no-modify-path', 136 | action='store_true', 137 | help='Don\'t configure the PATH environment variable' 138 | ) 139 | parser.add_argument( 140 | '--ignore-existing', 141 | action='store_true', 142 | help=( 143 | "ignore versions of pipsi already installed. " 144 | "Use this to ignore a package manager based install or for testing" 145 | ), 146 | ) 147 | return parser.parse_args(argv) 148 | 149 | 150 | code_for_get_real_python = ( 151 | 'import sys; print("{},{}".format(' 152 | 'getattr(sys, "real_prefix", ""), ' 153 | 'sys.version_info.major))' 154 | ) 155 | 156 | 157 | def get_real_python(python): 158 | cmd = [python, '-c', code_for_get_real_python] 159 | out = check_output(cmd) 160 | if not isinstance(out, str): 161 | out = out.decode() 162 | real_prefix, major = out.strip().split(',') 163 | if not real_prefix: 164 | return python 165 | 166 | for i in [major, '']: 167 | real_python = os.path.join(real_prefix, 'bin', 'python' + i) 168 | if os.path.exists(real_python): 169 | return real_python 170 | raise ValueError('Can not find real python under {}'.format(real_prefix)) 171 | 172 | 173 | def ensure_pipsi_on_path(bin_dir, modify_path): 174 | if not command_exists('pipsi'): 175 | shell = os.environ.get('SHELL', '') 176 | if 'bash' in shell: 177 | config_file = '~/.bashrc' 178 | elif 'zsh' in shell: 179 | config_file = '~/.zshrc' 180 | elif 'fish' in shell: 181 | config_file = '~/.config/fish/config.fish' 182 | else: 183 | config_file = None 184 | 185 | if config_file: 186 | config_file = os.path.expanduser(config_file) 187 | 188 | if modify_path and os.path.exists(config_file): 189 | with open(config_file, 'a') as f: 190 | f.write('\n# added by pipsi (https://github.com/mitsuhiko/pipsi)\n') 191 | if 'fish' in shell: 192 | f.write('set -x PATH %s $PATH\n\n' % bin_dir) 193 | else: 194 | f.write('export PATH="%s:$PATH"\n' % bin_dir) 195 | echo( 196 | 'Added %s to the PATH environment variable in %s' % 197 | (bin_dir, config_file) 198 | ) 199 | echo('Open a new terminal to use pipsi') 200 | else: 201 | echo(textwrap.dedent( 202 | ''' 203 | %(sep)s 204 | 205 | Note: 206 | To finish installation, %(bin_dir)s must be added to your PATH. 207 | This can be done by adding the following line to your shell 208 | config file: 209 | 210 | export PATH=%(bin_dir)s:$PATH 211 | 212 | %(sep)s 213 | ''' % dict(sep='=' * 60, bin_dir=bin_dir) 214 | )) 215 | 216 | 217 | def main(argv=sys.argv[1:]): 218 | args = parse_options(argv) 219 | 220 | if command_exists('pipsi') and not args.ignore_existing: 221 | succeed('You already have pipsi installed') 222 | elif os.path.exists(os.path.join(args.bin_dir, 'pipsi')): 223 | ensure_pipsi_on_path(args.bin_dir, not args.no_modify_path) 224 | succeed('pipsi is now installed') 225 | 226 | echo('Installing pipsi') 227 | if venv_pkg is None: 228 | fail('You need to have virtualenv installed to bootstrap pipsi.') 229 | 230 | venv = os.path.join(args.home_dir, 'pipsi') 231 | install_files(venv, args.bin_dir, args.src) 232 | ensure_pipsi_on_path(args.bin_dir, not args.no_modify_path) 233 | succeed('pipsi is now installed.') 234 | 235 | 236 | if __name__ == '__main__': 237 | main() 238 | -------------------------------------------------------------------------------- /pipsi/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import json 3 | import os 4 | import pkgutil 5 | import sys 6 | import shutil 7 | import subprocess 8 | import glob 9 | from collections import namedtuple 10 | from os.path import join, realpath, dirname, normpath, normcase 11 | from operator import methodcaller 12 | import distutils.spawn 13 | import re 14 | try: 15 | subprocess.run 16 | 17 | def run(*args, **kw): 18 | kw.update(stdout=subprocess.PIPE, stderr=subprocess.PIPE) 19 | r = subprocess.run(*args, **kw) 20 | r.stdout, r.stderr = map(proc_output, (r.stdout, r.stderr)) 21 | return r 22 | except AttributeError: # no `subprocess.run`, py < 3.5 23 | CompletedProcess = namedtuple('CompletedProcess', 24 | ('args', 'returncode', 'stdout', 'stderr')) 25 | 26 | def run(argv, **kw): 27 | p = subprocess.Popen( 28 | argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kw) 29 | out, err = map(proc_output, p.communicate()) 30 | return CompletedProcess(argv, p.returncode, out, err) 31 | try: 32 | from urlparse import urlparse 33 | except ImportError: 34 | from urllib.parse import urlparse 35 | 36 | import click 37 | from pkg_resources import Requirement 38 | 39 | 40 | try: 41 | WindowsError 42 | except NameError: 43 | IS_WIN = False 44 | BIN_DIR = 'bin' 45 | else: 46 | IS_WIN = True 47 | BIN_DIR = 'Scripts' 48 | 49 | FIND_SCRIPTS_SCRIPT = pkgutil.get_data('pipsi', 'scripts/find_scripts.py').decode('utf-8') 50 | GET_VERSION_SCRIPT = pkgutil.get_data('pipsi', 'scripts/get_version.py').decode('utf-8') 51 | 52 | # The `click` custom context settings 53 | CONTEXT_SETTINGS = dict( 54 | help_option_names=['-h', '--help'], 55 | ) 56 | 57 | 58 | def debugp(*args): 59 | if os.environ.get('PIPSI_DEBUG'): 60 | print(*args) 61 | 62 | 63 | def proc_output(s): 64 | s = s.strip() 65 | if isinstance(s, bytes): 66 | s = s.decode('utf-8', 'replace') 67 | return s 68 | 69 | 70 | def normalize_package(value): 71 | # Strips the version and normalizes name 72 | requirement = Requirement.parse(value) 73 | return requirement.project_name.lower() 74 | 75 | 76 | def normalize(path): 77 | return normcase(normpath(realpath(path))) 78 | 79 | 80 | def real_readlink(filename): 81 | try: 82 | target = os.readlink(filename) 83 | except (OSError, IOError, AttributeError): 84 | return None 85 | return normpath(realpath(join(dirname(filename), target))) 86 | 87 | 88 | def publish_script(src, dst): 89 | if IS_WIN: 90 | # always copy new exe on windows 91 | shutil.copy(src, dst) 92 | click.echo(' Copied Executable ' + dst) 93 | return True 94 | else: 95 | old_target = real_readlink(dst) 96 | if old_target == src: 97 | return True 98 | try: 99 | os.remove(dst) 100 | except OSError: 101 | pass 102 | try: 103 | os.symlink(src, dst) 104 | except OSError: 105 | pass 106 | else: 107 | click.echo(' Linked script ' + dst) 108 | return True 109 | 110 | 111 | def extract_package_version(virtualenv, package): 112 | prefix = normalize(join(virtualenv, BIN_DIR, '')) 113 | 114 | return run([ 115 | join(prefix, 'python'), '-c', GET_VERSION_SCRIPT, 116 | package, 117 | ]).stdout.strip() 118 | 119 | 120 | def find_scripts(virtualenv, package): 121 | prefix = normalize(join(virtualenv, BIN_DIR, '')) 122 | 123 | files = run([ 124 | join(prefix, 'python'), '-c', FIND_SCRIPTS_SCRIPT, 125 | package, prefix 126 | ]).stdout.splitlines() 127 | 128 | files = map(normalize, files) 129 | files = filter( 130 | methodcaller('startswith', prefix), 131 | files, 132 | ) 133 | 134 | def valid(filename): 135 | return os.path.isfile(filename) and \ 136 | IS_WIN or os.access(filename, os.X_OK) 137 | 138 | result = list(filter(valid, files)) 139 | 140 | if IS_WIN: 141 | for filename in files: 142 | globed = glob.glob(filename + '*') 143 | result.extend(filter(valid, globed)) 144 | return result 145 | 146 | 147 | class UninstallInfo(object): 148 | 149 | def __init__(self, package, paths=None, installed=True): 150 | self.package = package 151 | self.paths = paths or [] 152 | self.installed = installed 153 | 154 | def perform(self): 155 | for path in self.paths: 156 | try: 157 | os.remove(path) 158 | except OSError: 159 | shutil.rmtree(path) 160 | 161 | 162 | python_semver_regex = re.compile(r'^Python (\d)\.(\d+)\.(\d+)') 163 | 164 | 165 | def get_python_semver(python_bin): 166 | cmd = [python_bin, '--version'] 167 | r = run(cmd) 168 | if r.returncode != 0: 169 | raise ValueError( 170 | 'Failed to run {}: {}, {}, {}'.format(cmd, r.returncode, r.stdout, r.stderr)) 171 | raw_version = r.stdout.strip() 172 | if not raw_version: 173 | raw_version = r.stderr.strip() 174 | r = python_semver_regex.search(raw_version) 175 | if not r: 176 | raise ValueError( 177 | 'Could not match {} out of {}'.format( 178 | python_semver_regex.pattern, repr(raw_version))) 179 | return tuple(int(i) for i in r.groups()) 180 | 181 | 182 | code_for_get_real_python = ( 183 | 'import sys; print("{},{}".format(' 184 | 'getattr(sys, "real_prefix", ""), ' 185 | 'sys.version_info.major))' 186 | ) 187 | 188 | 189 | # `venv` for python 3 has the problem that `venv` cannot 190 | # add pip in virtualenv if it is executed under a virtualenv, 191 | # use this function to avoid this problem 192 | def get_real_python(python): 193 | cmd = [python, '-c', code_for_get_real_python] 194 | r = run(cmd) 195 | if r.returncode != 0: 196 | raise ValueError( 197 | 'Failed to run {}: {}, {}, {}'.format(cmd, r.returncode, r.stdout, r.stderr)) 198 | debugp('get_real_python run {}: {}, {}, {}'.format( 199 | cmd, r.returncode, r.stdout, r.stderr)) 200 | 201 | real_prefix, major = r.stdout.strip().split(',') 202 | if not real_prefix: 203 | return python 204 | 205 | for i in [major, '']: 206 | real_python = os.path.join(real_prefix, 'bin', 'python' + i) 207 | if os.path.exists(real_python): 208 | return real_python 209 | raise ValueError('Can not find real python under {}'.format(real_prefix)) 210 | 211 | 212 | class Repo(object): 213 | 214 | def __init__(self, home, bin_dir): 215 | self.home = realpath(home) 216 | self.bin_dir = bin_dir 217 | 218 | def resolve_package(self, spec, python=None): 219 | url = urlparse(spec) 220 | if url.netloc == 'file': 221 | location = url.path 222 | elif url.netloc != '': 223 | if not url.fragment.startswith('egg='): 224 | raise click.UsageError('When installing from URLs you need ' 225 | 'to add an egg at the end. For ' 226 | 'instance git+https://.../#egg=Foo') 227 | return url.fragment[4:], [spec] 228 | elif os.path.isdir(spec): 229 | location = spec 230 | else: 231 | return spec, [spec] 232 | 233 | if not os.path.exists(join(location, 'setup.py')): 234 | raise click.UsageError('%s does not appear to be a local ' 235 | 'Python package.' % spec) 236 | 237 | res = run( 238 | [python or sys.executable, 'setup.py', '--name'], 239 | cwd=location) 240 | if res.returncode: 241 | raise click.UsageError( 242 | '%s does not appear to be a valid ' 243 | 'package. Error from setup.py: %s' % (spec, res.stderr) 244 | ) 245 | name = res.stdout 246 | 247 | return name, [location] 248 | 249 | def get_package_path(self, package): 250 | return join(self.home, normalize_package(package)) 251 | 252 | def find_installed_executables(self, path): 253 | prefix = join(realpath(normpath(path)), '') 254 | try: 255 | for filename in os.listdir(self.bin_dir): 256 | exe = os.path.join(self.bin_dir, filename) 257 | target = real_readlink(exe) 258 | if target is None: 259 | continue 260 | if target.startswith(prefix): 261 | yield exe 262 | except OSError: 263 | pass 264 | 265 | def get_package_scripts(self, path): 266 | """Get the scripts installed for PATH 267 | 268 | Looks for package metadata listing which scripts were 269 | installed. If there is no metadata (package was installed 270 | with an older version of pipsi) then fall back to the old 271 | find_installed_executables method. 272 | """ 273 | info = self.get_package_info(path) 274 | if 'scripts' in info: 275 | return info['scripts'] 276 | # No script metadata - fall back to older method of searching for executables 277 | return self.find_installed_executables(path) 278 | 279 | def link_scripts(self, scripts): 280 | rv = [] 281 | for script in scripts: 282 | script_dst = os.path.join( 283 | self.bin_dir, os.path.basename(script)) 284 | if publish_script(script, script_dst): 285 | rv.append((script, script_dst)) 286 | 287 | return rv 288 | 289 | def save_package_info(self, venv_path, package, scripts): 290 | package_info_file_path = join(venv_path, 'package_info.json') 291 | package_name = Requirement.parse(package).project_name 292 | version = extract_package_version(venv_path, package_name) 293 | 294 | package_info = { 295 | 'name': package_name, 296 | 'version': version, 297 | 'scripts': [script for target, script in scripts], 298 | } 299 | with open(package_info_file_path, 'w') as fh: 300 | json.dump(package_info, fh) 301 | 302 | def get_package_info(self, venv_path): 303 | package_info_file_path = join(venv_path, 'package_info.json') 304 | with open(package_info_file_path, 'r') as fh: 305 | return json.load(fh) 306 | 307 | def install(self, package, python=None, editable=False, system_site_packages=False): 308 | # `python` could be int as major version, or str as absolute bin path, 309 | # if it's int, then we will try to find the executable `python2` or `python3` in PATH 310 | if isinstance(python, int): 311 | python_exe = 'python{}'.format(python) 312 | python = distutils.spawn.find_executable(python_exe) 313 | if not python: 314 | raise ValueError('Can not find {} in PATH'.format(python_exe)) 315 | if not python: 316 | python = sys.executable 317 | python_semver = get_python_semver(python) 318 | debugp('python: {}, python_bin_semver: {}'.format(python, python_semver)) 319 | 320 | package, install_args = self.resolve_package(package, python) 321 | 322 | venv_path = self.get_package_path(package) 323 | if os.path.isdir(venv_path): 324 | click.echo('%s is already installed' % package) 325 | return 326 | 327 | if not os.path.exists(self.bin_dir): 328 | os.makedirs(self.bin_dir) 329 | 330 | from subprocess import Popen 331 | 332 | def _cleanup(): 333 | try: 334 | shutil.rmtree(venv_path) 335 | except (OSError, IOError): 336 | pass 337 | return False 338 | 339 | # Install virtualenv, use the pipsi used python version by default 340 | args = [sys.executable, '-m', 'virtualenv', '-p', python, venv_path] 341 | 342 | if python_semver[0] == 3: 343 | # if target python is 3, use its builtin `venv` module to create virtualenv 344 | real_python = get_real_python(python) 345 | args = [real_python, '-m', 'venv', venv_path] 346 | 347 | if system_site_packages: 348 | args.append('--system-site-packages') 349 | 350 | try: 351 | debugp('Popen: {}'.format(args)) 352 | if Popen(args).wait() != 0: 353 | click.echo('Failed to create virtualenv. Aborting.') 354 | return _cleanup() 355 | 356 | args = [os.path.join(venv_path, BIN_DIR, 'python'), '-m', 'pip', 'install'] 357 | if editable: 358 | args.append('--editable') 359 | 360 | debugp('Popen: {}'.format(args + install_args)) 361 | if Popen(args + install_args).wait() != 0: 362 | click.echo('Failed to pip install. Aborting.') 363 | return _cleanup() 364 | except Exception: 365 | _cleanup() 366 | raise 367 | 368 | # Find all the scripts 369 | scripts = find_scripts(venv_path, package) 370 | 371 | # And link them 372 | linked_scripts = self.link_scripts(scripts) 373 | 374 | self.save_package_info(venv_path, package, linked_scripts) 375 | 376 | # We did not link any, rollback. 377 | if not linked_scripts: 378 | click.echo('Did not find any scripts. Uninstalling.') 379 | return _cleanup() 380 | return True 381 | 382 | def uninstall(self, package): 383 | path = self.get_package_path(package) 384 | if not os.path.isdir(path): 385 | return UninstallInfo(package, installed=False) 386 | paths = [path] 387 | paths.extend(self.get_package_scripts(path)) 388 | return UninstallInfo(package, paths) 389 | 390 | def upgrade(self, package, editable=False): 391 | package, install_args = self.resolve_package(package) 392 | 393 | venv_path = self.get_package_path(package) 394 | if not os.path.isdir(venv_path): 395 | click.echo('%s is not installed' % package) 396 | return 397 | 398 | from subprocess import Popen 399 | 400 | old_scripts = set(self.get_package_scripts(venv_path)) 401 | 402 | args = [os.path.join(venv_path, BIN_DIR, 'python'), '-m', 'pip', 'install', 403 | '--upgrade'] 404 | if editable: 405 | args.append('--editable') 406 | 407 | if Popen(args + install_args).wait() != 0: 408 | click.echo('Failed to upgrade through pip. Aborting.') 409 | return 410 | 411 | scripts = find_scripts(venv_path, package) 412 | linked_scripts = self.link_scripts(scripts) 413 | to_delete = old_scripts - set(script for target, script in linked_scripts) 414 | 415 | for script in to_delete: 416 | try: 417 | click.echo(' Removing old script %s' % script) 418 | os.remove(script) 419 | except (IOError, OSError): 420 | pass 421 | 422 | self.save_package_info(venv_path, package, linked_scripts) 423 | 424 | return True 425 | 426 | def list_everything(self, versions=False): 427 | venvs = {} 428 | python = '/Scripts/python.exe' if IS_WIN else '/bin/python' 429 | if os.path.isdir(self.home): 430 | for venv in os.listdir(self.home): 431 | venv_path = os.path.join(self.home, venv) 432 | if os.path.isdir(venv_path) and \ 433 | os.path.isfile(venv_path + python): 434 | info = self.get_package_info(venv_path) 435 | version = None 436 | if versions: 437 | version = info.get('version') 438 | venvs[venv] = [info.get('scripts', []), version] 439 | 440 | return sorted(venvs.items()) 441 | 442 | 443 | @click.group(context_settings=CONTEXT_SETTINGS) 444 | @click.option( 445 | '--home', type=click.Path(),envvar='PIPSI_HOME', 446 | default=os.path.join(os.path.expanduser('~'), '.local', 'venvs'), 447 | help='The folder that contains the virtualenvs.') 448 | @click.option( 449 | '--bin-dir', type=click.Path(), 450 | envvar='PIPSI_BIN_DIR', 451 | default=os.path.join(os.path.expanduser('~'), '.local', 'bin'), 452 | help='The path where the scripts are symlinked to.') 453 | @click.version_option( 454 | message='%(prog)s, version %(version)s, python ' + str(sys.executable)) 455 | @click.pass_context 456 | def cli(ctx, home, bin_dir): 457 | """pipsi is a tool that uses virtualenv and pip to install shell 458 | tools that are separated from each other. 459 | """ 460 | ctx.obj = Repo(home, bin_dir) 461 | 462 | 463 | @cli.command() 464 | @click.argument('package') 465 | @click.option( 466 | '--python', type=str, 467 | envvar='PIPSI_PYTHON', 468 | default=sys.executable, 469 | help=('The python interpreter to use, could be major version or path. ' 470 | 'By default it would be `sys.executable`')) 471 | @click.option('--editable', '-e', is_flag=True, 472 | help='Enable editable installation. This only works for ' 473 | 'locally installed packages.') 474 | @click.option('--system-site-packages', is_flag=True, 475 | help='Give the virtual environment access to the global ' 476 | 'site-packages.') 477 | @click.pass_obj 478 | def install(repo, package, python, editable, system_site_packages): 479 | """Installs scripts from a Python package. 480 | 481 | Given a package this will install all the scripts and their dependencies 482 | of the given Python package into a new virtualenv and symlinks the 483 | discovered scripts into BIN_DIR (defaults to ~/.local/bin). 484 | """ 485 | if re.search(r'^\d$', python): 486 | python = int(python) 487 | if repo.install(package, python, editable, system_site_packages): 488 | click.echo('Done.') 489 | else: 490 | sys.exit(1) 491 | 492 | 493 | @cli.command() 494 | @click.argument('package') 495 | @click.option('--editable', '-e', is_flag=True, 496 | help='Enable editable installation. This only works for ' 497 | 'locally installed packages.') 498 | @click.pass_obj 499 | def upgrade(repo, package, editable): 500 | """Upgrades an already installed package.""" 501 | if repo.upgrade(package, editable): 502 | click.echo('Done.') 503 | else: 504 | sys.exit(1) 505 | 506 | 507 | @cli.command(short_help='Uninstalls scripts of a package.') 508 | @click.argument('package') 509 | @click.option('--yes', is_flag=True, help='Skips all prompts.') 510 | @click.pass_obj 511 | def uninstall(repo, package, yes): 512 | """Uninstalls all scripts of a Python package and cleans up the 513 | virtualenv. 514 | """ 515 | uinfo = repo.uninstall(package) 516 | if not uinfo.installed: 517 | click.echo('%s is not installed' % package) 518 | else: 519 | click.echo('The following paths will be removed:') 520 | for path in uinfo.paths: 521 | click.echo(' %s' % click.format_filename(path)) 522 | click.echo() 523 | if yes or click.confirm('Do you want to uninstall %s?' % package): 524 | uinfo.perform() 525 | click.echo('Done!') 526 | else: 527 | click.echo('Aborted!') 528 | sys.exit(1) 529 | 530 | 531 | @cli.command('list') 532 | @click.option('--versions', is_flag=True, 533 | help='Show packages version') 534 | @click.pass_obj 535 | def list_cmd(repo, versions): 536 | """Lists all scripts installed through pipsi.""" 537 | list_of_non_empty_venv = [(venv, scripts) 538 | for venv, scripts in repo.list_everything() 539 | if scripts] 540 | if list_of_non_empty_venv: 541 | click.echo('Packages and scripts installed through pipsi:') 542 | for venv, (scripts, version) in repo.list_everything(versions): 543 | if versions: 544 | click.echo(' Package "%s" (%s):' % (venv, version or 'unknown')) 545 | else: 546 | click.echo(' Package "%s":' % venv) 547 | for script in scripts: 548 | click.echo(' ' + script) 549 | else: 550 | click.echo('There are no scripts installed through pipsi') 551 | 552 | 553 | if __name__ == '__main__': 554 | cli() 555 | -------------------------------------------------------------------------------- /pipsi/__main__.py: -------------------------------------------------------------------------------- 1 | from pipsi import cli 2 | cli() 3 | -------------------------------------------------------------------------------- /pipsi/scripts/find_scripts.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import pkg_resources 4 | pkg = sys.argv[1] 5 | prefix = sys.argv[2] 6 | dist = pkg_resources.get_distribution(pkg) 7 | if dist.has_metadata('RECORD'): 8 | for line in dist.get_metadata_lines('RECORD'): 9 | print(os.path.join(dist.location, line.split(',')[0])) 10 | elif dist.has_metadata('installed-files.txt'): 11 | for line in dist.get_metadata_lines('installed-files.txt'): 12 | print(os.path.join(dist.egg_info, line.split(',')[0])) 13 | elif dist.has_metadata('entry_points.txt'): 14 | try: 15 | from ConfigParser import SafeConfigParser 16 | from StringIO import StringIO 17 | except ImportError: 18 | from configparser import SafeConfigParser 19 | from io import StringIO 20 | parser = SafeConfigParser() 21 | parser.readfp(StringIO( 22 | '\n'.join(dist.get_metadata_lines('entry_points.txt')))) 23 | if parser.has_section('console_scripts'): 24 | for name, _ in parser.items('console_scripts'): 25 | print(os.path.join(prefix, name)) 26 | -------------------------------------------------------------------------------- /pipsi/scripts/get_version.py: -------------------------------------------------------------------------------- 1 | import sys, pkg_resources 2 | pkg = sys.argv[1] 3 | dist = pkg_resources.get_distribution(pkg) 4 | print(dist.version) 5 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | with open('README.md', 'rb') as f: 4 | readme = f.read().decode('utf-8') 5 | 6 | 7 | setup( 8 | name='pipsi', 9 | version='0.10.dev', 10 | description='Wraps pip and virtualenv to install scripts', 11 | long_description=readme, 12 | long_description_content_type="text/markdown", 13 | license='BSD', 14 | author='Armin Ronacher', 15 | author_email='armin.ronacher@active-4.com', 16 | url='http://github.com/mitsuhiko/pipsi/', 17 | packages=['pipsi'], 18 | package_data={ 19 | 'pipsi': ['scripts/*.py'], 20 | }, 21 | include_package_data=True, 22 | install_requires=[ 23 | 'Click', 24 | 'virtualenv', 25 | ], 26 | python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*", 27 | entry_points=''' 28 | [console_scripts] 29 | pipsi=pipsi:cli 30 | ''' 31 | ) 32 | -------------------------------------------------------------------------------- /testing/conftest.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | 4 | @pytest.fixture(params=['normal', 'MixedCase']) 5 | def mix(request): 6 | return request.param 7 | 8 | 9 | @pytest.fixture 10 | def bin(tmpdir, mix): 11 | return tmpdir.ensure(mix, 'bin', dir=1) 12 | 13 | 14 | @pytest.fixture 15 | def home(tmpdir, mix): 16 | return tmpdir.ensure(mix, 'venvs', dir=1) 17 | -------------------------------------------------------------------------------- /testing/test_command_line.py: -------------------------------------------------------------------------------- 1 | import subprocess 2 | 3 | import pytest 4 | import sys 5 | 6 | 7 | def test_list_command(home): 8 | assert not home.listdir() 9 | output = subprocess.check_output([ 10 | 'pipsi', '--home', home.strpath, 'list' 11 | ]) 12 | assert output.strip() == b'There are no scripts installed through pipsi' 13 | -------------------------------------------------------------------------------- /testing/test_install_script.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import sys 3 | import subprocess 4 | from pipsi import IS_WIN 5 | 6 | 7 | def test_create_env(tmpdir): 8 | subprocess.check_call([ 9 | sys.executable, 'get-pipsi.py', 10 | '--home', str(tmpdir.join('venv')), 11 | '--bin-dir', str(tmpdir.join('test_bin')), 12 | '--src', '.', 13 | '--ignore-existing', 14 | ]) 15 | pipsi_bin = str(tmpdir.join('test_bin/pipsi' + ('.exe' if IS_WIN else ''))) 16 | 17 | subprocess.check_call([pipsi_bin]) 18 | -------------------------------------------------------------------------------- /testing/test_repo.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import pytest 4 | import click 5 | from pipsi import Repo, find_scripts 6 | 7 | 8 | @pytest.fixture 9 | def repo(home, bin): 10 | return Repo(str(home), str(bin)) 11 | 12 | 13 | @pytest.mark.resolve 14 | def test_resolve_local_package(repo, tmpdir): 15 | pkgdir = tmpdir.ensure('foopkg', dir=True) 16 | pkgdir.join('setup.py').write_text( 17 | u'\n'.join([ 18 | u'from setuptools import setup', 19 | u'setup(name="foopkg", version="0.0.1", py_modules=["foo"])' 20 | ]), 21 | 'utf-8' 22 | ) 23 | pkgdir.join('foo.py').write_text(u'print("hello world")\n', 'utf-8') 24 | 25 | assert repo.resolve_package(str(pkgdir)) == ('foopkg', [str(pkgdir)]) 26 | 27 | 28 | @pytest.mark.resolve 29 | def test_resolve_local_fails_when_invalid_package(repo, tmpdir): 30 | pkgdir = tmpdir.ensure('foopkg', dir=True) 31 | pkgdir.join('setup.py').write_text(u'raise Exception("EXCMSG")', 'utf-8') 32 | pkgdir.join('foo.py').ensure() 33 | 34 | with pytest.raises(click.UsageError) as excinfo: 35 | repo.resolve_package(str(pkgdir)) 36 | assert 'does not appear to be a valid package' in str(excinfo.value) 37 | assert 'EXCMSG' in str(excinfo.value) 38 | 39 | 40 | @pytest.mark.resolve 41 | def test_resolve_local_fails_when_no_package(repo, tmpdir): 42 | pkgdir = tmpdir.ensure('foopkg', dir=True) 43 | 44 | with pytest.raises(click.UsageError) as excinfo: 45 | repo.resolve_package(str(pkgdir)) 46 | assert 'does not appear to be a local Python package' in str(excinfo.value) 47 | 48 | 49 | @pytest.mark.parametrize('package, glob', [ 50 | ('grin', 'grin*'), 51 | pytest.param('pipsi', 'pipsi*', 52 | marks=pytest.mark.xfail(reason="Clashes with local pipsi directory")), 53 | ]) 54 | def test_simple_install(repo, home, bin, package, glob): 55 | assert not home.listdir() 56 | assert not bin.listdir() 57 | repo.install(package) 58 | assert home.join(package).check() 59 | assert bin.listdir(glob) 60 | assert repo.upgrade(package) 61 | 62 | 63 | @pytest.mark.xfail( 64 | sys.version_info[0] != 3, 65 | reason='attic is python3 only', run=False) 66 | @pytest.mark.xfail( 67 | 'TRAVIS' in os.environ, 68 | reason='attic won\'t build on travis', run=False) 69 | def test_simple_install_attic(repo, home, bin): 70 | test_simple_install(repo, home, bin, 'attic', 'attic*') 71 | 72 | 73 | def test_list_everything(repo, home, bin): 74 | assert not home.listdir() 75 | assert not bin.listdir() 76 | assert repo.list_everything() == [] 77 | 78 | 79 | def test_find_scripts(): 80 | print('executable ' + sys.executable) 81 | env = os.path.dirname( 82 | os.path.dirname(sys.executable)) 83 | print('env %r' % env) 84 | print('listdir %r' % os.listdir(env)) 85 | scripts = list(find_scripts(env, 'pipsi')) 86 | print('scripts %r' % scripts) 87 | assert scripts 88 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist=py27,py36 3 | [testenv] 4 | passenv = LANG TRAVIS 5 | deps= 6 | pytest<3.3 7 | commands= 8 | py.test [] 9 | --------------------------------------------------------------------------------