├── .travis.yml ├── MANIFEST.in ├── README.rst ├── callisto ├── __init__.py ├── __main__.py └── callisto.py ├── setup.cfg ├── setup.py ├── test └── test_callisto.py └── tox.ini /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | language: python 3 | python: 4 | - "2.7" 5 | - "3.5" 6 | - "3.6" 7 | - "nightly" 8 | install: pip install tox-travis coveralls 9 | script: tox 10 | after_success: 11 | - coveralls 12 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE 2 | include README.rst 3 | 4 | recursive-include tests * 5 | recursive-exclude * __pycache__ 6 | recursive-exclude * *.py[co] 7 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | |Build Status| |Coverage Status| 2 | 3 | ======== 4 | Callisto 5 | ======== 6 | 7 | 8 | *The fourth Galilean moon of Jupyter.* 9 | 10 | A command line utility to create kernels in Jupyter from virtual environments. 11 | 12 | Installation 13 | ============ 14 | Callisto may be installed `from pypi `_: 15 | :: 16 | 17 | pip install callisto 18 | 19 | Tested against python 2.7, 3.4, 3.5, 3.6. 20 | 21 | 22 | Basic Usage. 23 | ============ 24 | Typical use is to just activate it inside a virtual environment: 25 | :: 26 | 27 | $ virtualenv venv 28 | ... 29 | $ source venv/bin/activate 30 | (venv) $ callisto 31 | Successfully installed a new jupyter kernel "venv": 32 | { 33 | "env": {}, 34 | "language": "python", 35 | "display_name": "venv", 36 | "argv": [ 37 | "/Users/colin/venv/bin/python", 38 | "-m", 39 | "ipykernel", 40 | "-f", 41 | "{connection_file}" 42 | ] 43 | } 44 | See /Users/colin/Library/Jupyter/kernels/venv/kernel.json to edit. 45 | 46 | Jupyter servers will now have an option for a kernel called `venv`. 47 | 48 | .. image:: https://colindcarroll.com/img/venv.png 49 | 50 | Naming the kernel. 51 | ================== 52 | You may also give kernels a more descriptive name: 53 | :: 54 | 55 | (venv) $ callisto -n pete 56 | Successfully installed a new jupyter kernel "pete": 57 | { 58 | "env": {}, 59 | "display_name": "pete", 60 | "argv": [ 61 | "/Users/colin/venv/bin/python", 62 | "-m", 63 | "ipykernel", 64 | "-f", 65 | "{connection_file}" 66 | ], 67 | "language": "python" 68 | } 69 | See /Users/colin/Library/Jupyter/kernels/pete/kernel.json to edit. 70 | 71 | Jupyter servers will now have an option for a kernel called `venv`, and `pete`. 72 | 73 | .. image:: https://colindcarroll.com/img/venv_and_pete.png 74 | 75 | Deleting kernels. 76 | ================= 77 | Sometimes you may want to tidy kernels up a bit. 78 | :: 79 | 80 | (venv) $ callisto -d 81 | Deleted jupyter kernel "venv" from /Users/colin/Library/Jupyter/kernels/venv/kernel.json: 82 | { 83 | "argv": [ 84 | "/Users/colin/venv/bin/python", 85 | "-m", 86 | "ipykernel", 87 | "-f", 88 | "{connection_file}" 89 | ], 90 | "env": {}, 91 | "language": "python", 92 | "display_name": "venv" 93 | } 94 | 95 | Jupyter servers will no longer have a kernel named `venv`. 96 | 97 | 98 | 99 | Lacking courage. 100 | ================ 101 | Callisto doesn't try to be too clever. 102 | :: 103 | 104 | (venv) $ deactivate 105 | 106 | $ callisto 107 | Usage: callisto [OPTIONS] 108 | 109 | Error: The environment variable VIRTUAL_ENV is not set (usually this is set 110 | automatically activating a virtualenv). Please make sure you are in a 111 | virtual environment! 112 | 113 | Viewing existing kernels. 114 | ========================= 115 | If you forgot the informative message about the kernel information, you can see it later. 116 | :: 117 | 118 | $ source venv/bin/activate 119 | 120 | (venv) $ callisto --list 121 | No kernel found at /Users/colin/Library/Jupyter/kernels/venv/kernel.json 122 | 123 | (venv) $ callisto -l --name pete 124 | Found kernel "pete" at /Users/colin/Library/Jupyter/kernels/pete/kernel.json: 125 | { 126 | "display_name": "pete", 127 | "language": "python", 128 | "argv": [ 129 | "/Users/colin/venv/bin/python", 130 | "-m", 131 | "ipykernel", 132 | "-f", 133 | "{connection_file}" 134 | ], 135 | "env": {} 136 | } 137 | 138 | 139 | 140 | Adjusting the `PYTHONPATH`. 141 | =========================== 142 | With isolated kernels, you may wish to run all your notebooks from a single directory, 143 | but using code from the project directories. 144 | :: 145 | 146 | (venv) $ callisto -n pete --path=$(pwd) 147 | Successfully installed a new jupyter kernel "pete": 148 | 149 | { 150 | "argv": [ 151 | "/Users/colin/venv/bin/python", 152 | "-m", 153 | "ipykernel", 154 | "-f", 155 | "{connection_file}" 156 | ], 157 | "language": "python", 158 | "env": { 159 | "PYTHONPATH": "/Users/colin/projects/pete:PYTHONPATH" 160 | }, 161 | "display_name": "pete" 162 | } 163 | See /Users/colin/Library/Jupyter/kernels/pete/kernel.json to edit. 164 | 165 | Now the `pete` kernel will be able to import from the folder `/Users/colin/projects/pete`. 166 | 167 | .. |Build Status| image:: https://travis-ci.org/ColCarroll/callisto.svg?branch=master 168 | :target: https://travis-ci.org/ColCarroll/callisto 169 | .. |Coverage Status| image:: https://coveralls.io/repos/github/ColCarroll/callisto/badge.svg?branch=master 170 | :target: https://coveralls.io/github/ColCarroll/callisto?branch=master 171 | -------------------------------------------------------------------------------- /callisto/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = 0.7 # noqa 2 | from .callisto import cli # noqa 3 | -------------------------------------------------------------------------------- /callisto/__main__.py: -------------------------------------------------------------------------------- 1 | import sys 2 | from callisto.callisto import cli 3 | 4 | sys.exit(cli()) 5 | -------------------------------------------------------------------------------- /callisto/callisto.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import shutil 4 | import subprocess 5 | 6 | import click 7 | from jupyter_client import kernelspec 8 | 9 | VIRTUAL_ENV_VAR = 'VIRTUAL_ENV' 10 | CONDA_PREFIX = 'CONDA_PREFIX' 11 | CONDA_NAME = 'CONDA_DEFAULT_ENV' 12 | 13 | 14 | @click.command() 15 | @click.option('-n', '--name', 16 | help='Name of kernel. Must provide a kernel name or run in a virtual environment.', 17 | default='') 18 | @click.option('-p', '--path', type=click.Path(), 19 | help='Path to add to the start of PYTHONPATH.', default='') 20 | @click.option('-l', '--list', is_flag=True, help='Get information about the current kernel') 21 | @click.option('-d', '--delete', is_flag=True, help='Delete the existing Jupyter kernel') 22 | def cli(name, path, list, delete): 23 | """Manage jupyter kernel for this virtual environment.""" 24 | if not in_virtual_env() and not name: 25 | raise click.UsageError('The environment variables {} and {} are not set (usually this is ' 26 | 'set automatically activating a conda environment or virtualenv, ' 27 | 'respectively). Please activate one of these and ' 28 | 'try again!'.format(CONDA_PREFIX, VIRTUAL_ENV_VAR)) 29 | if list: 30 | success, kernel, kernel_path = read_kernel(name) 31 | if success: 32 | click.secho('Found kernel "{display_name}" at {kernel_path}:'.format( 33 | kernel_path=kernel_path, 34 | **kernel), fg='green') 35 | click.secho(json.dumps(kernel, indent=2)) 36 | else: 37 | click.secho('No kernel found at {}'.format(kernel_path), fg='red') 38 | elif delete: 39 | success, kernel, kernel_path = delete_kernel(name) 40 | if success: 41 | click.secho('Deleted jupyter kernel "{display_name}" from {kernel_path}:'.format( 42 | kernel_path=kernel_path, **kernel), fg='green') 43 | click.secho(json.dumps(kernel, indent=2)) 44 | else: 45 | click.secho('No kernel found to delete (checked {})'.format(kernel_path), fg='red') 46 | else: 47 | success, kernel, kernel_path = install_kernel(name, click.format_filename(path)) 48 | if success: 49 | click.secho( 50 | 'Successfully installed a new jupyter kernel "{display_name}":'.format(**kernel), 51 | fg='green') 52 | click.secho(json.dumps(kernel, indent=2)) 53 | click.secho('See {} to edit.'.format(kernel_path), fg='green') 54 | if need_to_install_ipykernel(): 55 | click.secho('Remember to install ipykernel before starting a jupyter notebook!', 56 | fg='red') 57 | else: 58 | click.secho('Failed to install a new jupyter kernel "{display_name}".\n' 59 | 'See {kernel_path} to confirm it isn\'t already there.'.format( 60 | kernel_path=kernel_path, **kernel), 61 | fg='red') 62 | 63 | 64 | def in_virtual_env(): 65 | """Check whether script is being run in a virtualenv.""" 66 | return bool(os.getenv(VIRTUAL_ENV_VAR) or os.getenv(CONDA_PREFIX)) 67 | 68 | 69 | def delete_kernel(name): 70 | """Delete the jupyter kernel for the current virtualenv.""" 71 | success, kernel, kernel_path = read_kernel(name) 72 | if success: 73 | shutil.rmtree(os.path.dirname(kernel_path)) 74 | return success, kernel, kernel_path 75 | 76 | 77 | def read_kernel(name): 78 | """Get information about the jupyter kernel for the current virtualenv.""" 79 | display_name = get_display_name(name) 80 | kernel_path = get_kernel_path(display_name) 81 | if os.path.exists(kernel_path): 82 | with open(kernel_path, 'r') as buff: 83 | return True, json.load(buff), kernel_path 84 | else: 85 | return False, {}, kernel_path 86 | 87 | 88 | def install_kernel(name, path): 89 | """Creates a jupyter kernel using the activated virtual environment.""" 90 | display_name = get_display_name(name) 91 | executable = get_executable() 92 | env = get_env(path) 93 | kernel = get_kernel(display_name, executable, env) 94 | kernel_path = get_kernel_path(display_name) 95 | if confirm_kernel_path_is_safe(kernel_path): 96 | with open(kernel_path, 'w') as buff: 97 | json.dump(kernel, buff) 98 | return True, kernel, kernel_path 99 | return False, kernel, kernel_path 100 | 101 | 102 | def get_kernel(display_name, executable, env): 103 | """Formats kernel in necessary format""" 104 | return { 105 | 'argv': [ 106 | executable, 107 | '-m', 108 | 'ipykernel', 109 | '-f', 110 | '{connection_file}'], 111 | 'display_name': display_name, 112 | 'env': env, 113 | 'language': 'python' 114 | } 115 | 116 | 117 | def get_display_name(name): 118 | """Display name for kernel""" 119 | if name: 120 | return name 121 | conda_name = os.getenv(CONDA_NAME) 122 | if conda_name: 123 | return conda_name 124 | return os.path.basename(os.getenv(VIRTUAL_ENV_VAR)) 125 | 126 | 127 | def get_executable(): 128 | """Get the python executable path""" 129 | base_path = os.getenv(CONDA_PREFIX, os.getenv(VIRTUAL_ENV_VAR)) 130 | return os.path.join(base_path, 'bin', 'python') 131 | 132 | 133 | def get_env(path): 134 | """Get any environment variables""" 135 | if path: 136 | return {'PYTHONPATH': path} 137 | else: 138 | return {} 139 | 140 | 141 | def get_kernel_path(display_name): 142 | """Convert display name into file-safe name""" 143 | safe_display = "".join([s if s.isalnum() else "_" for s in display_name]) 144 | filename = os.path.join(safe_display, 'kernel.json') 145 | return os.path.join(kernelspec.jupyter_data_dir(), 'kernels', filename) 146 | 147 | 148 | def confirm_kernel_path_is_safe(kernel_path): 149 | """Make sure necessary directory exists and a kernel with the same name does not exist""" 150 | if os.path.exists(kernel_path): 151 | return False 152 | directory = os.path.dirname(kernel_path) 153 | if not os.path.isdir(directory): 154 | os.makedirs(directory) 155 | return True 156 | 157 | 158 | def need_to_install_ipykernel(): 159 | """Check if ipykernel is installed""" 160 | status = subprocess.call([get_executable(), '-c', 'import ipykernel'], 161 | stderr=subprocess.STDOUT, 162 | stdout=open(os.devnull, 'w'), 163 | close_fds=True) 164 | return status == 1 165 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [tool:pytest] 2 | norecursedirs = .tox 3 | testpaths = test 4 | 5 | [bdist_wheel] 6 | universal=1 7 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from codecs import open 2 | from os import path 3 | from setuptools import setup, find_packages 4 | 5 | here = path.abspath(path.dirname(__file__)) 6 | 7 | # Get the long description from the README file 8 | with open(path.join(here, 'README.rst'), encoding='utf-8') as buff: 9 | long_description = buff.read() 10 | 11 | setup( 12 | name='callisto', 13 | version='0.7', 14 | description='Create jupyter kernels from virtual environments', 15 | long_description=long_description, 16 | author='Colin Carroll', 17 | author_email='colcarroll@gmail.com', 18 | url='https://github.com/ColCarroll/callisto', 19 | license='MIT', 20 | classifiers=[ 21 | 'Development Status :: 3 - Alpha', 22 | 'Intended Audience :: Developers', 23 | 'License :: OSI Approved :: MIT License', 24 | 'Programming Language :: Python :: 2', 25 | 'Programming Language :: Python :: 2.7', 26 | 'Programming Language :: Python :: 3', 27 | 'Programming Language :: Python :: 3.4', 28 | 'Programming Language :: Python :: 3.5', 29 | 'Programming Language :: Python :: 3.6', 30 | ], 31 | packages=find_packages(exclude=['test']), 32 | install_requires=[ 33 | 'Click', 34 | 'ipykernel' 35 | ], 36 | include_package_data=True, 37 | entry_points=''' 38 | [console_scripts] 39 | callisto=callisto:cli 40 | ''', 41 | ) 42 | -------------------------------------------------------------------------------- /test/test_callisto.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import os 3 | import shutil 4 | import tempfile 5 | import unittest 6 | try: 7 | from unittest.mock import patch # py3 8 | except ImportError: 9 | from mock import patch 10 | 11 | from click.testing import CliRunner 12 | 13 | from callisto import callisto 14 | 15 | 16 | @patch('callisto.callisto.kernelspec') 17 | class TestCallisto(unittest.TestCase): 18 | test_cases = ( 19 | 'easy', 20 | 'hard/one with spaces', 21 | '\o/', 22 | '¯\\_(ツ)_/¯', 23 | u'\\u041c\\u0430\\u043a\\u0435\\u0434\\u043e\\u043d\\u0438\\u0458\\u0430', 24 | ) 25 | 26 | @classmethod 27 | def setUpClass(cls): 28 | cls.tempdir = tempfile.mkdtemp() 29 | 30 | def setUp(self): 31 | os.environ[callisto.VIRTUAL_ENV_VAR] = self.tempdir 32 | 33 | @classmethod 34 | def tearDownClass(cls): 35 | shutil.rmtree(cls.tempdir) 36 | 37 | def tearDown(self): 38 | os.environ.pop(callisto.VIRTUAL_ENV_VAR) 39 | 40 | def test_get_kernel_path(self, mock_kernelspec): 41 | mock_kernelspec.jupyter_data_dir.return_value = self.tempdir 42 | for test_case in self.test_cases: 43 | filename = callisto.get_kernel_path(test_case) 44 | # All filenames end with kernel.json 45 | self.assertTrue(filename.endswith('kernel.json')) 46 | 47 | @patch('callisto.callisto.get_display_name') 48 | def test_install_kernel(self, mock_get_display_name, mock_kernelspec): 49 | mock_kernelspec.jupyter_data_dir.return_value = self.tempdir 50 | for test_case in self.test_cases: 51 | mock_get_display_name.return_value = test_case 52 | kernel_path = callisto.get_kernel_path(test_case) 53 | # Path is safe to write to 54 | self.assertTrue(callisto.confirm_kernel_path_is_safe(kernel_path)) 55 | 56 | fake_env_path = 'fake_env_path' 57 | success, kernel, actual_kernel_path = callisto.install_kernel('', fake_env_path) 58 | 59 | # Patching worked 60 | self.assertEqual(kernel_path, actual_kernel_path) 61 | 62 | # Great success 63 | self.assertTrue(success) 64 | 65 | # Added to python path 66 | self.assertIn(fake_env_path, kernel['env']['PYTHONPATH']) 67 | 68 | # Path isn't safe any more 69 | self.assertFalse(callisto.confirm_kernel_path_is_safe(kernel_path)) 70 | 71 | # Data is sort of correct 72 | self.assertEqual(kernel['display_name'], test_case) 73 | 74 | # No more success 75 | success, _, __ = callisto.install_kernel('', fake_env_path) 76 | self.assertFalse(success) 77 | 78 | @patch('callisto.callisto.get_executable') 79 | def test_cli(self, mock_executable, mock_kernelspec): 80 | mock_executable.return_value = 'python' 81 | mock_kernelspec.jupyter_data_dir.return_value = self.tempdir 82 | runner = CliRunner() 83 | 84 | result = runner.invoke(callisto.cli) 85 | self.assertEqual(result.exit_code, 0) 86 | 87 | success, kernel, path = callisto.read_kernel('') 88 | self.assertTrue(success) 89 | self.assertIn(self.tempdir, path) 90 | 91 | # Can also confirm this with '-l' 92 | result = runner.invoke(callisto.cli, ["-l"]) 93 | self.assertEqual(result.exit_code, 0) 94 | self.assertIn(self.tempdir, result.output) 95 | 96 | result = runner.invoke(callisto.cli, ["-d"]) 97 | self.assertEqual(result.exit_code, 0) 98 | # Deleting also echoes the kernel 99 | self.assertIn(self.tempdir, result.output) 100 | 101 | # Deleting twice is handled elegantly 102 | result = runner.invoke(callisto.cli, ["-d"]) 103 | self.assertEqual(result.exit_code, 0) 104 | 105 | result = runner.invoke(callisto.cli, ["-l"]) 106 | self.assertEqual(result.exit_code, 0) 107 | # No kernel installed 108 | self.assertIn('No kernel found', result.output) 109 | 110 | venv_name = 'pete' 111 | result = runner.invoke(callisto.cli, ["-n", venv_name]) 112 | self.assertEqual(result.exit_code, 0) 113 | self.assertIn(self.tempdir, result.output) 114 | self.assertIn(venv_name, result.output) 115 | 116 | result = runner.invoke(callisto.cli, ["-n", venv_name]) 117 | self.assertEqual(result.exit_code, 0) 118 | self.assertIn("Failed", result.output) 119 | 120 | # Exit without status code 0 if used outside of virtual environment 121 | tmp = os.environ.pop(callisto.VIRTUAL_ENV_VAR) 122 | result = runner.invoke(callisto.cli) 123 | self.assertNotEqual(result.exit_code, 0) 124 | os.environ[callisto.VIRTUAL_ENV_VAR] = tmp 125 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py27, py35, py36 3 | 4 | [testenv] 5 | deps= 6 | pytest 7 | pytest-cov 8 | py27: mock 9 | commands=py.test --cov={envsitepackagesdir}/callisto --cov-report=html --cov-report=term 10 | --------------------------------------------------------------------------------