└── README.md


/README.md:
--------------------------------------------------------------------------------
 1 | # Singularity container in Jupyter Notebooks
 2 | 
 3 | These are notes on how to run singularity containers
 4 | inside Jupyter Notebooks.
 5 | 
 6 | ## Example use case
 7 | 
 8 | We provide `tensorflow` on the cluster from a singularity container:
 9 | 
10 | ~~~
11 | [atrikut@node1669 ~]$ module add tensorflow/1.0
12 | [atrikut@node1669 ~]$ module list
13 | Currently Loaded Modulefiles:
14 |   1) singularity/2.2.1   2) /tensorflow/1.0
15 |   [atrikut@node1669 ~]$ which python
16 |   alias python='singularity /path/to/ubuntu_tensorflow_v1.0_gpu.img /usr/bin/python'
17 | ~~~
18 | 
19 | The `python` command is now aliased to `singularity exec ... python`;
20 | so that when users run `python`, they are actually running a singularity container under the hood.
21 | 
22 | ~~~
23 | [atrikut@node1669 ~]$ python
24 | Python 2.7.6 (default, Mar 22 2014, 22:59:56)
25 | [GCC 4.8.2] on linux2
26 | Type "help", "copyright", "credits" or "license" for more information.
27 | >>> import tensorflow
28 | '/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.pyc'
29 | >>>
30 | ~~~
31 | 
32 | We would like to do the same from Jupyter Notebook, i.e., we'd like to enable
33 | users to create Notebooks which use the container Python.
34 | 
35 | ## The `kernel.json` file
36 | 
37 | Jupyter Notebook allows you to create notebooks
38 | with various backends for executing code, called "kernels".
39 | Thus you can have, for example a "Python 2" kernel for notebooks containing Python 2 code,
40 | a "Python 3" kernel for notebooks containing Python 3 code,
41 | and a "MATLAB" kernel for notebooks containing MATLAB code.
42 | 
43 | Kernels are defined in `.json` files, which can be located in
44 | `~/.local/share/jupyter/kernels` (among other places).
45 | Here is an example of a `kernel.json` file for Python 2:
46 | 
47 | ~~~
48 | {
49 |  "display_name": "Python 2",
50 |  "language": "python",
51 |  "argv": [
52 |   "/software/anaconda/4.2.0/bin/python",
53 |   "-m",
54 |   "ipykernel",
55 |   "-f",
56 |   "{connection_file}"
57 |  ],
58 |  "display_name": "Python 2 (Anaconda)"
59 | }
60 | ~~~
61 | 
62 | ## `kernel.json` for Python in a singularity container
63 | 
64 | You can similarly define a kernel
65 | that uses Python from a singularity container.
66 | Make sure that you install `ipykernel` inside the container:
67 | 
68 | ~~~
69 | $ pip install ipykernel
70 | ~~~
71 | 
72 | And create a `kernel.json` similar to the following:
73 | 
74 | ~~~
75 | {
76 |  "language": "python",
77 |  "argv": ["/path/to/singularity",
78 |    "exec",
79 |    "/path/to/ubuntu_tensorflow_v1.0_gpu.img",
80 |    "/usr/bin/python",
81 |    "-m",
82 |    "ipykernel",
83 |    "-f",
84 |    "{connection_file}"
85 |  ],
86 |  "display_name": "Python 2 (Tensorflow)"
87 | }
88 | ~~~
89 | 
90 | Now you will see a "Python 2 (Tensorflow)" option
91 | in the list of Jupyter Notebook kernels.
92 | 


--------------------------------------------------------------------------------