├── .gitignore ├── 0_dask_tutorial ├── 01-dask.delayed.ipynb ├── 02-dask-arrays.ipynb ├── README.md ├── prep_data.py ├── solutions │ ├── 01-delayed-control-flow.py │ ├── 01-delayed-loop.py │ ├── 02-dask-arrays-blocked-mean.py │ ├── 02-dask-arrays-make-arrays.py │ ├── 02-dask-arrays-stacked.py │ ├── 02-dask-arrays-store.py │ ├── 02-dask-arrays-weather-difference.py │ └── 02-dask-arrays-weather-mean.py └── static │ ├── fail-case.gif │ ├── ml-dimensions-color.png │ ├── ml-dimensions.png │ ├── sklearn-parallel-dask.png │ └── sklearn-parallel.png ├── 1_dask_array ├── 1_multicore_array.ipynb ├── 2_multicore_array_outofcore.ipynb ├── 3_multinode_distributed_array.ipynb ├── 4_dask_graphs.ipynb ├── dask.delayed.ipynb ├── mydask.png ├── processes.ipynb └── threads.ipynb ├── 2_digits_of_pi ├── 1_serial_digits_of_pi.py ├── 2_multicore_digits_of_pi.py ├── 3_dask_multicore_digits_of_pi.py ├── 4_dask_multinode_digits_of_pi.py └── run_digits_of_pi_scripts.ipynb ├── GIL_counter.py ├── README.md ├── dask_slurm ├── dask_workers.slrm ├── launch_scheduler.sh └── launch_worker.sh ├── intro.ipynb ├── notebook_singularity.slrm └── numba_groupby_pixels.ipynb /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/.gitignore -------------------------------------------------------------------------------- /0_dask_tutorial/01-dask.delayed.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/01-dask.delayed.ipynb -------------------------------------------------------------------------------- /0_dask_tutorial/02-dask-arrays.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/02-dask-arrays.ipynb -------------------------------------------------------------------------------- /0_dask_tutorial/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/README.md -------------------------------------------------------------------------------- /0_dask_tutorial/prep_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/prep_data.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/01-delayed-control-flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/01-delayed-control-flow.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/01-delayed-loop.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/01-delayed-loop.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-blocked-mean.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/02-dask-arrays-blocked-mean.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-make-arrays.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/02-dask-arrays-make-arrays.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-stacked.py: -------------------------------------------------------------------------------- 1 | x = da.stack(arrays, axis=0) 2 | x -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-store.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/02-dask-arrays-store.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-weather-difference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/02-dask-arrays-weather-difference.py -------------------------------------------------------------------------------- /0_dask_tutorial/solutions/02-dask-arrays-weather-mean.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/solutions/02-dask-arrays-weather-mean.py -------------------------------------------------------------------------------- /0_dask_tutorial/static/fail-case.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/static/fail-case.gif -------------------------------------------------------------------------------- /0_dask_tutorial/static/ml-dimensions-color.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/static/ml-dimensions-color.png -------------------------------------------------------------------------------- /0_dask_tutorial/static/ml-dimensions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/static/ml-dimensions.png -------------------------------------------------------------------------------- /0_dask_tutorial/static/sklearn-parallel-dask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/static/sklearn-parallel-dask.png -------------------------------------------------------------------------------- /0_dask_tutorial/static/sklearn-parallel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/0_dask_tutorial/static/sklearn-parallel.png -------------------------------------------------------------------------------- /1_dask_array/1_multicore_array.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/1_multicore_array.ipynb -------------------------------------------------------------------------------- /1_dask_array/2_multicore_array_outofcore.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/2_multicore_array_outofcore.ipynb -------------------------------------------------------------------------------- /1_dask_array/3_multinode_distributed_array.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/3_multinode_distributed_array.ipynb -------------------------------------------------------------------------------- /1_dask_array/4_dask_graphs.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/4_dask_graphs.ipynb -------------------------------------------------------------------------------- /1_dask_array/dask.delayed.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/dask.delayed.ipynb -------------------------------------------------------------------------------- /1_dask_array/mydask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/mydask.png -------------------------------------------------------------------------------- /1_dask_array/processes.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/processes.ipynb -------------------------------------------------------------------------------- /1_dask_array/threads.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/1_dask_array/threads.ipynb -------------------------------------------------------------------------------- /2_digits_of_pi/1_serial_digits_of_pi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/2_digits_of_pi/1_serial_digits_of_pi.py -------------------------------------------------------------------------------- /2_digits_of_pi/2_multicore_digits_of_pi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/2_digits_of_pi/2_multicore_digits_of_pi.py -------------------------------------------------------------------------------- /2_digits_of_pi/3_dask_multicore_digits_of_pi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/2_digits_of_pi/3_dask_multicore_digits_of_pi.py -------------------------------------------------------------------------------- /2_digits_of_pi/4_dask_multinode_digits_of_pi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/2_digits_of_pi/4_dask_multinode_digits_of_pi.py -------------------------------------------------------------------------------- /2_digits_of_pi/run_digits_of_pi_scripts.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/2_digits_of_pi/run_digits_of_pi_scripts.ipynb -------------------------------------------------------------------------------- /GIL_counter.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/GIL_counter.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/README.md -------------------------------------------------------------------------------- /dask_slurm/dask_workers.slrm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/dask_slurm/dask_workers.slrm -------------------------------------------------------------------------------- /dask_slurm/launch_scheduler.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/dask_slurm/launch_scheduler.sh -------------------------------------------------------------------------------- /dask_slurm/launch_worker.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/dask_slurm/launch_worker.sh -------------------------------------------------------------------------------- /intro.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/intro.ipynb -------------------------------------------------------------------------------- /notebook_singularity.slrm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/notebook_singularity.slrm -------------------------------------------------------------------------------- /numba_groupby_pixels.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zonca/python_hpc_tutorial/HEAD/numba_groupby_pixels.ipynb --------------------------------------------------------------------------------