├── README.md ├── ch01-intro.html ├── ch01-intro.ipynb ├── ch01-intro.pdf ├── ch01-intro.py ├── ch02-vectors-matrices-ndarrays.ipynb ├── ch03-symbolic-math-with-sympy.ipynb ├── ch04-plotting-visualization.ipynb ├── ch05-equation-solving.ipynb ├── ch06-optimization.ipynb ├── ch07-interpolation.ipynb ├── ch08-integration.ipynb ├── ch09-ordinary-differential-equations.ipynb ├── ch10-sparse-matrices-graphs.ipynb ├── ch11-fdm-2d-ex1.png ├── ch11-partial-differential-equations.ipynb ├── ch12-data-analysis-pandas-seaborn.ipynb ├── ch13-statistics.ipynb ├── ch14-statistical-modeling.ipynb ├── ch15-machine-learning.ipynb ├── ch16-bayes-statistics.ipynb ├── ch17-signal-processing.ipynb ├── ch18-data-input-output.ipynb ├── ch18-data.csv ├── ch18-data.h5 ├── ch18-playerstats-2013-2014-subset.csv ├── ch18-playerstats-2013-2014-top30.csv ├── ch18-playerstats-2013-2014.csv ├── ch18-playerstats-2013-2014.h5 ├── ch19-optimization.ipynb ├── cy_cumsum.pyx ├── cy_sum.c ├── cy_sum.cpython-36m-x86_64-linux-gnu.so ├── cy_sum.pyx ├── data.csv ├── data.json ├── european_cities.csv ├── fib.py ├── file1.py ├── file2.py ├── file3.py ├── guitar-echo.wav ├── hello.py ├── pics ├── aggregate-funcs-illustrated.png ├── aggregate-funcs.png ├── arithmetic-ops.png ├── array-funcs.png ├── array-gen-funcs.png ├── array-gen-funcs2.png ├── array-slice-funcs.png ├── autocompletion.png ├── bookcover-2nd-ed.png ├── bookcover-reduced.png ├── bookcover.png ├── broadcasting.png ├── ch08-numerical-integration-example.png ├── ch4-figure-1.pdf ├── ch4-figure-2.pdf ├── conditional-logical-funcs.png ├── constants-specials.png ├── convex-vs-nonconvex.png ├── coupled-damped-oscilators.png ├── dense-vs-sparse.png ├── direction-field-numint.png ├── direction-fields.png ├── direction-graphs-2nd-ex.png ├── dolfin-api-p1.png ├── dolfin-api-p2.png ├── dolfin-mshr-funcs.png ├── element-wise-math-functs.png ├── element-wise-math.png ├── expression-tree.png ├── figures-axes.png ├── font-properties.png ├── graph.pdf ├── graph.png ├── graphtypes-2d.png ├── hypothesis-test-cases.png ├── indexing-viz.png ├── lineprops1.png ├── lineprops2.png ├── lorentz-odes.png ├── matrix-funcs.png ├── networkx-ops.png ├── numerical-error-and-condition-num-vs-p.png ├── numpy-array-index-methods.png ├── patsy-syntax.png ├── plot-types-1.pdf ├── plot-types-2.pdf ├── plot-types-3.pdf ├── plot-types-4.pdf ├── plot-types-5.pdf ├── plot-types-6.pdf ├── plot-types-7.pdf ├── plot-types-8.pdf ├── plot-types-9.pdf ├── predator-prey-populations.png ├── random-var-funcs.png ├── rcm-ordering.png ├── rectangular-linear-equation.png ├── reshape-ops.png ├── set-funcs.png ├── simplifications.png ├── software-stack-overview.png ├── sparse-a-b-tensorprod.png ├── sparse-matrix-representations.png ├── sparse-nz2diags.png ├── sum-by-axis.png ├── symbol-arguments.png ├── symbol-attributes.png ├── sympy-matrix-ops.png └── tokyo-metro.png ├── setup.py ├── store.h5 ├── temperature_indoor_2014.tsv ├── temperature_outdoor_2014.tsv ├── tokyo-metro.json ├── tokyo-metro.msgpack └── tokyo-metro.pickle /README.md: -------------------------------------------------------------------------------- 1 | # Numeric Computing with Python, 2nd edition 2 | ### [Original ebook on Apress](https://www.apress.com/us/book/9781484242452) 3 | ![book-cover](pics/bookcover-2nd-ed.png) 4 | 5 | ### TODO: 6 | - Fix FeniCS installation bugs (Chapter 11) 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 |
ChapterTopics
IntroPython, iPython, Jupyter
Intro to NumPyArrays: Create, Index, Slice, Reshape, Resize, Vectorized ops, Matrix & Vector ops
Intro to SymPySymbols, Expressions, Manipulations, Calculus, Equations, Linear Algebra
Plotting & Visualization
Intro to Matplotlib		Plots, Steps, Bargraphs, Histograms, ErrorBars, ScatterPlots, Fill_Between, QuiverPlots, ColorMaps, 3D Plots
Equation Solvers with SciPyLinear Equations (square, rectangular), Eigenvalues, NonLinear Equations
Optimization with SciPyUnivariate, Multivariate (unconstrained), Nonlinear Least Squares, Constrained
Interpolation with SciPyPolynomials, Splines, Multivariate
Integration with SciPy & Scikit-monacoNumerical Methods, Multiple integration, Symbolic & arbitrary precision, Integral transforms
Ordinary Differential Equations (ODEs)Direction fields, Laplace transforms, Numerical methods
Sparse Matrices & GraphsMatrix ops, Linear equation systems incl. Eigenvalues, Graphs/Networks
Partial Differential Equations (PDEs)Finite-difference methods, Finite element methods & libraries, PDE solvers with FEniCS
Data Analysis with Pandas & SeabornSeries, DataFrames, TimeSeries
Statistics with SciPy & NumPyProbability, Random numbers, Distributions, Hypothesis testing, Nonparametric methods
Statistical Modeling with Stasmodels & PatsyModel definitions, Linear regression, Logistic regression, Poisson models, Time series
Intro to Machine Learning with scikit-learnConcepts, Regression, Classification, Clustering
Bayesian Statistics with pyMCConcepts, Sampling posterior distributions, Linear regression
Signal Processing with SciPy, fftpack, signal, wavfile & ioSpectral analysis (Fourier transforms, windows, spectrograms), Signal filters (Convolution, FIR/IIR filters)
Data I/OCSV, HDF5 (h5py files, groups, datasets, attributes, PyTables, HDFStore), JSON, Serialization
Code optimizationNumba, Cython
89 | 90 | -------------------------------------------------------------------------------- /ch01-intro.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Chapter 1: Computing with Python" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### Overview: a typical Python-based scientific computing stack.\n", 15 | "![software stack](pics/software-stack-overview.png)\n", 16 | "\n", 17 | "### Resources:\n", 18 | "- [Intel MKL (Math Kernel Library)](https://software.intel.com/en-us/intel-mkl)\n", 19 | "- [openBLAS](https://www.openblas.net)\n", 20 | "- [ATLAS](https://math-atlas.sourceforge.net)\n", 21 | "- [SciPy](http://www.scipy.org)\n", 22 | "- [Python Numeric & Scientific topics](http://wiki.python.org/moin/NumericAndScientific)" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "### Interpreter\n", 30 | "- The easist way to execute Python code: run the program directly.\n", 31 | "- Use Jupyter magic command to write Python source file to disk:" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": 1, 37 | "metadata": {}, 38 | "outputs": [ 39 | { 40 | "name": "stdout", 41 | "output_type": "stream", 42 | "text": [ 43 | "Overwriting hello.py\n" 44 | ] 45 | } 46 | ], 47 | "source": [ 48 | "%%writefile hello.py\n", 49 | "print(\"Hello from Python!\")" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "* Use the ! system shell command (included in the Python Jupyter kernel) to interactively run Python with hello.py as its argument." 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 2, 62 | "metadata": {}, 63 | "outputs": [ 64 | { 65 | "name": "stdout", 66 | "output_type": "stream", 67 | "text": [ 68 | "Hello from Python!\n" 69 | ] 70 | } 71 | ], 72 | "source": [ 73 | "!python3 hello.py" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": 3, 79 | "metadata": {}, 80 | "outputs": [ 81 | { 82 | "name": "stdout", 83 | "output_type": "stream", 84 | "text": [ 85 | "Python 3.11.4\n" 86 | ] 87 | } 88 | ], 89 | "source": [ 90 | "!python3 --version" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "### Input and output caching\n", 98 | "\n", 99 | "* Input & output history can be accessed using __In__ (a list) & __Out__ (a dictionary). Both can be indexed with a cell number. " 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "execution_count": 4, 105 | "metadata": {}, 106 | "outputs": [ 107 | { 108 | "data": { 109 | "text/plain": [ 110 | "'get_ipython().run_cell_magic(\\'writefile\\', \\'hello.py\\', \\'print(\"Hello from Python!\")\\\\n\\')'" 111 | ] 112 | }, 113 | "execution_count": 4, 114 | "metadata": {}, 115 | "output_type": "execute_result" 116 | } 117 | ], 118 | "source": [ 119 | "3*3\n", 120 | "In[1]" 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "metadata": {}, 126 | "source": [ 127 | "* A single underscore = the most recent output; \n", 128 | "* A double underscore = the _next_ most recent output." 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": 5, 134 | "metadata": {}, 135 | "outputs": [ 136 | { 137 | "data": { 138 | "text/plain": [ 139 | "2" 140 | ] 141 | }, 142 | "execution_count": 5, 143 | "metadata": {}, 144 | "output_type": "execute_result" 145 | } 146 | ], 147 | "source": [ 148 | "1+1" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 6, 154 | "metadata": {}, 155 | "outputs": [ 156 | { 157 | "data": { 158 | "text/plain": [ 159 | "4" 160 | ] 161 | }, 162 | "execution_count": 6, 163 | "metadata": {}, 164 | "output_type": "execute_result" 165 | } 166 | ], 167 | "source": [ 168 | "2+2" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": 7, 174 | "metadata": {}, 175 | "outputs": [ 176 | { 177 | "data": { 178 | "text/plain": [ 179 | "(4, 2)" 180 | ] 181 | }, 182 | "execution_count": 7, 183 | "metadata": {}, 184 | "output_type": "execute_result" 185 | } 186 | ], 187 | "source": [ 188 | "_, __" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": 8, 194 | "metadata": {}, 195 | "outputs": [ 196 | { 197 | "data": { 198 | "text/plain": [ 199 | "['',\n", 200 | " 'get_ipython().run_cell_magic(\\'writefile\\', \\'hello.py\\', \\'print(\"Hello from Python!\")\\\\n\\')',\n", 201 | " \"get_ipython().system('python3 hello.py')\",\n", 202 | " \"get_ipython().system('python3 --version')\",\n", 203 | " '3*3\\nIn[1]',\n", 204 | " '1+1',\n", 205 | " '2+2',\n", 206 | " '_, __',\n", 207 | " '# In = a list\\nIn']" 208 | ] 209 | }, 210 | "execution_count": 8, 211 | "metadata": {}, 212 | "output_type": "execute_result" 213 | } 214 | ], 215 | "source": [ 216 | "# In = a list\n", 217 | "In" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": 9, 223 | "metadata": {}, 224 | "outputs": [ 225 | { 226 | "data": { 227 | "text/plain": [ 228 | "{4: 'get_ipython().run_cell_magic(\\'writefile\\', \\'hello.py\\', \\'print(\"Hello from Python!\")\\\\n\\')',\n", 229 | " 5: 2,\n", 230 | " 6: 4,\n", 231 | " 7: (4, 2),\n", 232 | " 8: ['',\n", 233 | " 'get_ipython().run_cell_magic(\\'writefile\\', \\'hello.py\\', \\'print(\"Hello from Python!\")\\\\n\\')',\n", 234 | " \"get_ipython().system('python3 hello.py')\",\n", 235 | " \"get_ipython().system('python3 --version')\",\n", 236 | " '3*3\\nIn[1]',\n", 237 | " '1+1',\n", 238 | " '2+2',\n", 239 | " '_, __',\n", 240 | " '# In = a list\\nIn',\n", 241 | " '# Out = a dictionary\\nOut']}" 242 | ] 243 | }, 244 | "execution_count": 9, 245 | "metadata": {}, 246 | "output_type": "execute_result" 247 | } 248 | ], 249 | "source": [ 250 | "# Out = a dictionary\n", 251 | "Out" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": 10, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "# Suppress output results by ending statement with a semicolon\n", 261 | "1+2;" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "### Autocompletion\n", 269 | "\n", 270 | "* The __Tab__ key activates autocompletion (displays list of symbol names that are valid completions of what has been typed thus far.)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": 11, 276 | "metadata": {}, 277 | "outputs": [], 278 | "source": [ 279 | "import os" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "* Results of typing \"os.w\", followed by \\t:\n", 287 | "\n", 288 | "![autocompletion](pics/autocompletion.png)" 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "### Documentation\n", 296 | "\n", 297 | "* \"Docstrings\" provide a built-in reference manual for most Python modules. Display the docstring by appending a Python object with \"?\"." 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": 12, 303 | "metadata": {}, 304 | "outputs": [ 305 | { 306 | "data": { 307 | "text/plain": [ 308 | "\u001b[0;31mSignature:\u001b[0m \u001b[0mmath\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcos\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m/\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 309 | "\u001b[0;31mDocstring:\u001b[0m Return the cosine of x (measured in radians).\n", 310 | "\u001b[0;31mType:\u001b[0m builtin_function_or_method" 311 | ] 312 | }, 313 | "metadata": {}, 314 | "output_type": "display_data" 315 | } 316 | ], 317 | "source": [ 318 | "import math\n", 319 | "math.cos?" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "### Interaction with System Shell\n", 327 | "\n", 328 | "* Anything after ! is evaluated using the system shell, such as bash.\n", 329 | "* (I use Ubuntu Linux as my laptop OS. Your Windows equivalents will vary.)" 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": 13, 335 | "metadata": {}, 336 | "outputs": [ 337 | { 338 | "name": "stdout", 339 | "output_type": "stream", 340 | "text": [ 341 | "file1.py file2.py file3.py\n" 342 | ] 343 | } 344 | ], 345 | "source": [ 346 | "!touch file1.py file2.py file3.py\n", 347 | "!ls file*" 348 | ] 349 | }, 350 | { 351 | "cell_type": "code", 352 | "execution_count": 14, 353 | "metadata": {}, 354 | "outputs": [ 355 | { 356 | "name": "stdout", 357 | "output_type": "stream", 358 | "text": [ 359 | "3\n", 360 | "['file1.py', 'file2.py', 'file3.py']\n" 361 | ] 362 | } 363 | ], 364 | "source": [ 365 | "# output of a system shell command can be captured in a Python variable\n", 366 | "files = !ls file*\n", 367 | "print(len(files))\n", 368 | "print(files)" 369 | ] 370 | }, 371 | { 372 | "cell_type": "code", 373 | "execution_count": 15, 374 | "metadata": {}, 375 | "outputs": [ 376 | { 377 | "name": "stdout", 378 | "output_type": "stream", 379 | "text": [ 380 | "-rw-rw-r-- 1 bjpcjp bjpcjp 0 Nov 14 08:18 file1.py\n" 381 | ] 382 | } 383 | ], 384 | "source": [ 385 | "# pass Python variable values to shell commands\n", 386 | "# by prefixing the variable name with $.\n", 387 | "file = \"file1.py\"\n", 388 | "!ls -l $file" 389 | ] 390 | }, 391 | { 392 | "cell_type": "markdown", 393 | "metadata": {}, 394 | "source": [ 395 | "### IPython Extensions\n", 396 | "\n", 397 | "* Commands start with one or two \"%\" characters. A single % is used for single-line commands; dual %% is used for cells (multiple lines).\n", 398 | "\n", 399 | "* `%lsmagic` returns a list of available commands." 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 16, 405 | "metadata": {}, 406 | "outputs": [ 407 | { 408 | "data": { 409 | "application/json": { 410 | "cell": { 411 | "!": "OSMagics", 412 | "HTML": "Other", 413 | "SVG": "Other", 414 | "bash": "Other", 415 | "capture": "ExecutionMagics", 416 | "code_wrap": "ExecutionMagics", 417 | "debug": "ExecutionMagics", 418 | "file": "Other", 419 | "html": "DisplayMagics", 420 | "javascript": "DisplayMagics", 421 | "js": "DisplayMagics", 422 | "latex": "DisplayMagics", 423 | "markdown": "DisplayMagics", 424 | "perl": "Other", 425 | "prun": "ExecutionMagics", 426 | "pypy": "Other", 427 | "python": "Other", 428 | "python2": "Other", 429 | "python3": "Other", 430 | "ruby": "Other", 431 | "script": "ScriptMagics", 432 | "sh": "Other", 433 | "svg": "DisplayMagics", 434 | "sx": "OSMagics", 435 | "system": "OSMagics", 436 | "time": "ExecutionMagics", 437 | "timeit": "ExecutionMagics", 438 | "writefile": "OSMagics" 439 | }, 440 | "line": { 441 | "alias": "OSMagics", 442 | "alias_magic": "BasicMagics", 443 | "autoawait": "AsyncMagics", 444 | "autocall": "AutoMagics", 445 | "automagic": "AutoMagics", 446 | "autosave": "KernelMagics", 447 | "bookmark": "OSMagics", 448 | "cat": "Other", 449 | "cd": "OSMagics", 450 | "clear": "KernelMagics", 451 | "code_wrap": "ExecutionMagics", 452 | "colors": "BasicMagics", 453 | "conda": "PackagingMagics", 454 | "config": "ConfigMagics", 455 | "connect_info": "KernelMagics", 456 | "cp": "Other", 457 | "debug": "ExecutionMagics", 458 | "dhist": "OSMagics", 459 | "dirs": "OSMagics", 460 | "doctest_mode": "BasicMagics", 461 | "ed": "Other", 462 | "edit": "KernelMagics", 463 | "env": "OSMagics", 464 | "gui": "BasicMagics", 465 | "hist": "Other", 466 | "history": "HistoryMagics", 467 | "killbgscripts": "ScriptMagics", 468 | "ldir": "Other", 469 | "less": "KernelMagics", 470 | "lf": "Other", 471 | "lk": "Other", 472 | "ll": "Other", 473 | "load": "CodeMagics", 474 | "load_ext": "ExtensionMagics", 475 | "loadpy": "CodeMagics", 476 | "logoff": "LoggingMagics", 477 | "logon": "LoggingMagics", 478 | "logstart": "LoggingMagics", 479 | "logstate": "LoggingMagics", 480 | "logstop": "LoggingMagics", 481 | "ls": "Other", 482 | "lsmagic": "BasicMagics", 483 | "lx": "Other", 484 | "macro": "ExecutionMagics", 485 | "magic": "BasicMagics", 486 | "mamba": "PackagingMagics", 487 | "man": "KernelMagics", 488 | "matplotlib": "PylabMagics", 489 | "micromamba": "PackagingMagics", 490 | "mkdir": "Other", 491 | "more": "KernelMagics", 492 | "mv": "Other", 493 | "notebook": "BasicMagics", 494 | "page": "BasicMagics", 495 | "pastebin": "CodeMagics", 496 | "pdb": "ExecutionMagics", 497 | "pdef": "NamespaceMagics", 498 | "pdoc": "NamespaceMagics", 499 | "pfile": "NamespaceMagics", 500 | "pinfo": "NamespaceMagics", 501 | "pinfo2": "NamespaceMagics", 502 | "pip": "PackagingMagics", 503 | "popd": "OSMagics", 504 | "pprint": "BasicMagics", 505 | "precision": "BasicMagics", 506 | "prun": "ExecutionMagics", 507 | "psearch": "NamespaceMagics", 508 | "psource": "NamespaceMagics", 509 | "pushd": "OSMagics", 510 | "pwd": "OSMagics", 511 | "pycat": "OSMagics", 512 | "pylab": "PylabMagics", 513 | "qtconsole": "KernelMagics", 514 | "quickref": "BasicMagics", 515 | "recall": "HistoryMagics", 516 | "rehashx": "OSMagics", 517 | "reload_ext": "ExtensionMagics", 518 | "rep": "Other", 519 | "rerun": "HistoryMagics", 520 | "reset": "NamespaceMagics", 521 | "reset_selective": "NamespaceMagics", 522 | "rm": "Other", 523 | "rmdir": "Other", 524 | "run": "ExecutionMagics", 525 | "save": "CodeMagics", 526 | "sc": "OSMagics", 527 | "set_env": "OSMagics", 528 | "store": "StoreMagics", 529 | "sx": "OSMagics", 530 | "system": "OSMagics", 531 | "tb": "ExecutionMagics", 532 | "time": "ExecutionMagics", 533 | "timeit": "ExecutionMagics", 534 | "unalias": "OSMagics", 535 | "unload_ext": "ExtensionMagics", 536 | "who": "NamespaceMagics", 537 | "who_ls": "NamespaceMagics", 538 | "whos": "NamespaceMagics", 539 | "xdel": "NamespaceMagics", 540 | "xmode": "BasicMagics" 541 | } 542 | }, 543 | "text/plain": [ 544 | "Available line magics:\n", 545 | "%alias %alias_magic %autoawait %autocall %automagic %autosave %bookmark %cat %cd %clear %code_wrap %colors %conda %config %connect_info %cp %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history %killbgscripts %ldir %less %lf %lk %ll %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %lx %macro %magic %mamba %man %matplotlib %micromamba %mkdir %more %mv %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %pip %popd %pprint %precision %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %rm %rmdir %run %save %sc %set_env %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode\n", 546 | "\n", 547 | "Available cell magics:\n", 548 | "%%! %%HTML %%SVG %%bash %%capture %%code_wrap %%debug %%file %%html %%javascript %%js %%latex %%markdown %%perl %%prun %%pypy %%python %%python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile\n", 549 | "\n", 550 | "Automagic is ON, % prefix IS NOT needed for line magics." 551 | ] 552 | }, 553 | "execution_count": 16, 554 | "metadata": {}, 555 | "output_type": "execute_result" 556 | } 557 | ], 558 | "source": [ 559 | "%lsmagic" 560 | ] 561 | }, 562 | { 563 | "cell_type": "markdown", 564 | "metadata": {}, 565 | "source": [ 566 | "### Running scripts\n", 567 | "\n", 568 | "- `%run` executes an external Python source file within an interactive IPython session." 569 | ] 570 | }, 571 | { 572 | "cell_type": "code", 573 | "execution_count": 17, 574 | "metadata": {}, 575 | "outputs": [ 576 | { 577 | "name": "stdout", 578 | "output_type": "stream", 579 | "text": [ 580 | "Overwriting fib.py\n" 581 | ] 582 | } 583 | ], 584 | "source": [ 585 | "%%writefile fib.py\n", 586 | "\n", 587 | "def fib(N): \n", 588 | " \"\"\" \n", 589 | " Return a list of the first N Fibonacci numbers.\n", 590 | " \"\"\" \n", 591 | " f0, f1 = 0, 1\n", 592 | " f = [1] * N\n", 593 | " for n in range(1, N):\n", 594 | " f[n] = f0 + f1\n", 595 | " f0, f1 = f1, f[n]\n", 596 | "\n", 597 | " return f\n", 598 | "\n", 599 | "print(fib(10))" 600 | ] 601 | }, 602 | { 603 | "cell_type": "code", 604 | "execution_count": 18, 605 | "metadata": {}, 606 | "outputs": [ 607 | { 608 | "name": "stdout", 609 | "output_type": "stream", 610 | "text": [ 611 | "[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]\n" 612 | ] 613 | } 614 | ], 615 | "source": [ 616 | "!python3 fib.py" 617 | ] 618 | }, 619 | { 620 | "cell_type": "code", 621 | "execution_count": 19, 622 | "metadata": {}, 623 | "outputs": [ 624 | { 625 | "name": "stdout", 626 | "output_type": "stream", 627 | "text": [ 628 | "[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]\n" 629 | ] 630 | } 631 | ], 632 | "source": [ 633 | "%run fib.py" 634 | ] 635 | }, 636 | { 637 | "cell_type": "code", 638 | "execution_count": 20, 639 | "metadata": {}, 640 | "outputs": [ 641 | { 642 | "data": { 643 | "text/plain": [ 644 | "[1, 1, 2, 3, 5, 8]" 645 | ] 646 | }, 647 | "execution_count": 20, 648 | "metadata": {}, 649 | "output_type": "execute_result" 650 | } 651 | ], 652 | "source": [ 653 | "fib(6)" 654 | ] 655 | }, 656 | { 657 | "cell_type": "markdown", 658 | "metadata": {}, 659 | "source": [ 660 | "## Listing all defined symbols\n", 661 | "\n", 662 | "* `%who` lists all defined symbols\n", 663 | "* `%whos` provides more detailed info." 664 | ] 665 | }, 666 | { 667 | "cell_type": "code", 668 | "execution_count": 21, 669 | "metadata": {}, 670 | "outputs": [ 671 | { 672 | "name": "stdout", 673 | "output_type": "stream", 674 | "text": [ 675 | "fib\t file\t files\t math\t os\t \n" 676 | ] 677 | } 678 | ], 679 | "source": [ 680 | "%who" 681 | ] 682 | }, 683 | { 684 | "cell_type": "code", 685 | "execution_count": 22, 686 | "metadata": {}, 687 | "outputs": [ 688 | { 689 | "name": "stdout", 690 | "output_type": "stream", 691 | "text": [ 692 | "Variable Type Data/Info\n", 693 | "--------------------------------\n", 694 | "fib function \n", 695 | "file str file1.py\n", 696 | "files SList ['file1.py', 'file2.py', 'file3.py']\n", 697 | "math module \n", 698 | "os module \n" 699 | ] 700 | } 701 | ], 702 | "source": [ 703 | "%whos" 704 | ] 705 | }, 706 | { 707 | "cell_type": "markdown", 708 | "metadata": {}, 709 | "source": [ 710 | "## Debugger\n", 711 | "\n", 712 | "* Use `%debug` to step directly into the Python debugger." 713 | ] 714 | }, 715 | { 716 | "cell_type": "code", 717 | "execution_count": 23, 718 | "metadata": {}, 719 | "outputs": [ 720 | { 721 | "name": "stdout", 722 | "output_type": "stream", 723 | "text": [ 724 | "nope. can't do that.\n" 725 | ] 726 | } 727 | ], 728 | "source": [ 729 | "# fib function fails - can't use floating point numbers.\n", 730 | "try:\n", 731 | " fib(1.0)\n", 732 | "except TypeError:\n", 733 | " print(\"nope. can't do that.\")" 734 | ] 735 | }, 736 | { 737 | "cell_type": "code", 738 | "execution_count": 24, 739 | "metadata": {}, 740 | "outputs": [], 741 | "source": [ 742 | "#%debug" 743 | ] 744 | }, 745 | { 746 | "cell_type": "markdown", 747 | "metadata": {}, 748 | "source": [ 749 | "### Resetting the Python namespace\n", 750 | "- Ensures a program is run in a pristine environment, uncluttered by existing variables and functions. Although it is necessary to reimport modules, it is important to know that even if the modules have changed since the last\n", 751 | "import, a new import after a %reset will not import the new module but rather reenable a cached version from the previous import. \n", 752 | "\n", 753 | "- When developing Python modules, this is usually not the desired behavior. In that case, a reimport of a previously imported (and since updated) module can often be achieved by using the reload function from `IPython.lib.deepreload`. However, this method does not always work, as\n", 754 | "some libraries run code at import time that is only intended to run once. In this case, the only option might be to terminate and restart the IPython interpreter." 755 | ] 756 | }, 757 | { 758 | "cell_type": "code", 759 | "execution_count": 26, 760 | "metadata": {}, 761 | "outputs": [ 762 | { 763 | "name": "stdin", 764 | "output_type": "stream", 765 | "text": [ 766 | "Once deleted, variables cannot be recovered. Proceed (y/[n])? \n" 767 | ] 768 | }, 769 | { 770 | "name": "stdout", 771 | "output_type": "stream", 772 | "text": [ 773 | "Nothing done.\n" 774 | ] 775 | } 776 | ], 777 | "source": [ 778 | "%reset" 779 | ] 780 | }, 781 | { 782 | "cell_type": "markdown", 783 | "metadata": {}, 784 | "source": [ 785 | "## Timing and profiling code\n", 786 | "\n", 787 | "* `%timeit` and `%time` provide simple benchmarking utilities." 788 | ] 789 | }, 790 | { 791 | "cell_type": "code", 792 | "execution_count": 27, 793 | "metadata": {}, 794 | "outputs": [], 795 | "source": [ 796 | "# first, re-define fibonacci code used above.\n", 797 | "def fib(N): \n", 798 | " \"\"\" \n", 799 | " Return a list of the first N Fibonacci numbers.\n", 800 | " \"\"\" \n", 801 | " f0, f1 = 0, 1\n", 802 | " f = [1] * N\n", 803 | " for n in range(1, N):\n", 804 | " f[n] = f0 + f1\n", 805 | " f0, f1 = f1, f[n]\n", 806 | "\n", 807 | " return f" 808 | ] 809 | }, 810 | { 811 | "cell_type": "code", 812 | "execution_count": 28, 813 | "metadata": {}, 814 | "outputs": [ 815 | { 816 | "name": "stdout", 817 | "output_type": "stream", 818 | "text": [ 819 | "1.38 µs ± 12.2 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)\n" 820 | ] 821 | } 822 | ], 823 | "source": [ 824 | "# timeit does not return expression's resulting value.\n", 825 | "%timeit fib(50)" 826 | ] 827 | }, 828 | { 829 | "cell_type": "code", 830 | "execution_count": 29, 831 | "metadata": {}, 832 | "outputs": [ 833 | { 834 | "name": "stdout", 835 | "output_type": "stream", 836 | "text": [ 837 | "CPU times: user 8 µs, sys: 0 ns, total: 8 µs\n", 838 | "Wall time: 8.82 µs\n" 839 | ] 840 | } 841 | ], 842 | "source": [ 843 | "# %time only runs once. less accurate estimate.\n", 844 | "result = %time fib(100)" 845 | ] 846 | }, 847 | { 848 | "cell_type": "code", 849 | "execution_count": 30, 850 | "metadata": {}, 851 | "outputs": [ 852 | { 853 | "data": { 854 | "text/plain": [ 855 | "100" 856 | ] 857 | }, 858 | "execution_count": 30, 859 | "metadata": {}, 860 | "output_type": "execute_result" 861 | } 862 | ], 863 | "source": [ 864 | "len(result)" 865 | ] 866 | }, 867 | { 868 | "cell_type": "markdown", 869 | "metadata": {}, 870 | "source": [ 871 | "* The __cProfile__ module provides __%prun__ (for statements) and __%run__ (for external scripts) profiling commands." 872 | ] 873 | }, 874 | { 875 | "cell_type": "code", 876 | "execution_count": 31, 877 | "metadata": {}, 878 | "outputs": [], 879 | "source": [ 880 | "import numpy as np\n", 881 | "\n", 882 | "def random_walker_max_distance(M, N):\n", 883 | " \"\"\"\n", 884 | " Simulate N random walkers taking M steps\n", 885 | " Return the largest distance from the starting point.\n", 886 | " \"\"\"\n", 887 | " trajectories = [np.random.randn(M).cumsum() \n", 888 | " for _ in range(N)]\n", 889 | " return np.max(np.abs(trajectories))" 890 | ] 891 | }, 892 | { 893 | "cell_type": "code", 894 | "execution_count": 32, 895 | "metadata": {}, 896 | "outputs": [ 897 | { 898 | "name": "stdout", 899 | "output_type": "stream", 900 | "text": [ 901 | " " 902 | ] 903 | }, 904 | { 905 | "data": { 906 | "text/plain": [ 907 | " 20013 function calls in 0.100 seconds\n", 908 | "\n", 909 | " Ordered by: internal time\n", 910 | "\n", 911 | " ncalls tottime percall cumtime percall filename:lineno(function)\n", 912 | " 10000 0.058 0.000 0.058 0.000 {method 'randn' of 'numpy.random.mtrand.RandomState' objects}\n", 913 | " 10000 0.022 0.000 0.022 0.000 {method 'cumsum' of 'numpy.ndarray' objects}\n", 914 | " 1 0.013 0.013 0.099 0.099 220825864.py:3(random_walker_max_distance)\n", 915 | " 1 0.003 0.003 0.084 0.084 220825864.py:8()\n", 916 | " 1 0.002 0.002 0.002 0.002 {method 'reduce' of 'numpy.ufunc' objects}\n", 917 | " 1 0.001 0.001 0.100 0.100 :1()\n", 918 | " 1 0.000 0.000 0.100 0.100 {built-in method builtins.exec}\n", 919 | " 1 0.000 0.000 0.002 0.002 fromnumeric.py:69(_wrapreduction)\n", 920 | " 1 0.000 0.000 0.002 0.002 <__array_function__ internals>:177(amax)\n", 921 | " 1 0.000 0.000 0.002 0.002 fromnumeric.py:2703(amax)\n", 922 | " 1 0.000 0.000 0.002 0.002 {built-in method numpy.core._multiarray_umath.implement_array_function}\n", 923 | " 1 0.000 0.000 0.000 0.000 fromnumeric.py:70()\n", 924 | " 1 0.000 0.000 0.000 0.000 fromnumeric.py:2698(_amax_dispatcher)\n", 925 | " 1 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}\n", 926 | " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}" 927 | ] 928 | }, 929 | "metadata": {}, 930 | "output_type": "display_data" 931 | } 932 | ], 933 | "source": [ 934 | "# returns call counts, runtime & cume runtime for each function.\n", 935 | "%prun random_walker_max_distance(400, 10000)" 936 | ] 937 | }, 938 | { 939 | "cell_type": "markdown", 940 | "metadata": {}, 941 | "source": [ 942 | "### Jupyter: External image rendering" 943 | ] 944 | }, 945 | { 946 | "cell_type": "code", 947 | "execution_count": 33, 948 | "metadata": {}, 949 | "outputs": [], 950 | "source": [ 951 | "from IPython.display import display, Image, HTML, Math" 952 | ] 953 | }, 954 | { 955 | "cell_type": "code", 956 | "execution_count": 34, 957 | "metadata": {}, 958 | "outputs": [ 959 | { 960 | "data": { 961 | "text/html": [ 962 | "" 963 | ], 964 | "text/plain": [ 965 | "" 966 | ] 967 | }, 968 | "execution_count": 34, 969 | "metadata": {}, 970 | "output_type": "execute_result" 971 | } 972 | ], 973 | "source": [ 974 | "Image(url='http://python.org/images/python-logo.gif')" 975 | ] 976 | }, 977 | { 978 | "cell_type": "markdown", 979 | "metadata": {}, 980 | "source": [ 981 | "### Jupyter: HTML rendering" 982 | ] 983 | }, 984 | { 985 | "cell_type": "code", 986 | "execution_count": 35, 987 | "metadata": {}, 988 | "outputs": [], 989 | "source": [ 990 | "import scipy, numpy, matplotlib\n", 991 | "modules = [numpy, matplotlib, scipy]\n", 992 | "\n", 993 | "row = \"%s%s\"\n", 994 | "rows = \"\\n\".join(\n", 995 | " [row % \n", 996 | " (module.__name__, module.__version__) \n", 997 | " for module in modules])\n", 998 | "table = \" %s
LibraryVersion
\" % rows" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "code", 1003 | "execution_count": 36, 1004 | "metadata": {}, 1005 | "outputs": [ 1006 | { 1007 | "data": { 1008 | "text/html": [ 1009 | "\n", 1010 | "\n", 1011 | "
LibraryVersion
numpy1.24.2
matplotlib3.5.2
scipy1.10.1
" 1012 | ], 1013 | "text/plain": [ 1014 | "" 1015 | ] 1016 | }, 1017 | "execution_count": 36, 1018 | "metadata": {}, 1019 | "output_type": "execute_result" 1020 | } 1021 | ], 1022 | "source": [ 1023 | "HTML(table)" 1024 | ] 1025 | }, 1026 | { 1027 | "cell_type": "code", 1028 | "execution_count": 37, 1029 | "metadata": {}, 1030 | "outputs": [ 1031 | { 1032 | "data": { 1033 | "text/html": [ 1034 | "\n", 1035 | "\n", 1036 | "
LibraryVersion
numpy1.24.2
matplotlib3.5.2
scipy1.10.1
" 1037 | ], 1038 | "text/plain": [ 1039 | "<__main__.HTMLdisplayer at 0x7f3940100fd0>" 1040 | ] 1041 | }, 1042 | "execution_count": 37, 1043 | "metadata": {}, 1044 | "output_type": "execute_result" 1045 | } 1046 | ], 1047 | "source": [ 1048 | "# another method\n", 1049 | "class HTMLdisplayer(object):\n", 1050 | " def __init__(self,code):\n", 1051 | " self.code = code\n", 1052 | " def _repr_html_(self):\n", 1053 | " return self.code\n", 1054 | " \n", 1055 | "HTMLdisplayer(table)" 1056 | ] 1057 | }, 1058 | { 1059 | "cell_type": "markdown", 1060 | "metadata": {}, 1061 | "source": [ 1062 | "### Jupyter: Formula rendering using Latex" 1063 | ] 1064 | }, 1065 | { 1066 | "cell_type": "code", 1067 | "execution_count": 38, 1068 | "metadata": {}, 1069 | "outputs": [ 1070 | { 1071 | "data": { 1072 | "text/latex": [ 1073 | "$\\displaystyle \\hat{H} = -\\frac{1}{2}\\epsilon \\hat{\\sigma}_z-\\frac{1}{2}\\delta \\hat{\\sigma}_x$" 1074 | ], 1075 | "text/plain": [ 1076 | "" 1077 | ] 1078 | }, 1079 | "execution_count": 38, 1080 | "metadata": {}, 1081 | "output_type": "execute_result" 1082 | } 1083 | ], 1084 | "source": [ 1085 | "Math(r'\\hat{H} = -\\frac{1}{2}\\epsilon \\hat{\\sigma}_z-\\frac{1}{2}\\delta \\hat{\\sigma}_x')" 1086 | ] 1087 | }, 1088 | { 1089 | "cell_type": "markdown", 1090 | "metadata": {}, 1091 | "source": [ 1092 | "### Jupyter: UI Widgets" 1093 | ] 1094 | }, 1095 | { 1096 | "cell_type": "code", 1097 | "execution_count": 39, 1098 | "metadata": {}, 1099 | "outputs": [], 1100 | "source": [ 1101 | "import matplotlib.pyplot as plt\n", 1102 | "import numpy as np\n", 1103 | "from scipy import stats\n", 1104 | "\n", 1105 | "def f(mu):\n", 1106 | " X = stats.norm(loc=mu, scale=np.sqrt(mu))\n", 1107 | " N = stats.poisson(mu)\n", 1108 | " x = np.linspace(0, X.ppf(0.999))\n", 1109 | " n = np.arange(0, x[-1])\n", 1110 | " \n", 1111 | " fig, ax = plt.subplots()\n", 1112 | " ax.plot(x, X.pdf(x), color='black', lw=2, label=\"Normal($\\mu=%d, \\sigma^2=%d$)\" % (mu,mu))\n", 1113 | " ax.bar(n, N.pmf(n), align='edge', label=r\"Poisson($\\lambda=%d$)\" % mu)\n", 1114 | " ax.set_ylim(0, X.pdf(x).max() * 1.25)\n", 1115 | " ax.legend(loc=2, ncol=2)\n", 1116 | " plt.close(fig)\n", 1117 | " return fig" 1118 | ] 1119 | }, 1120 | { 1121 | "cell_type": "code", 1122 | "execution_count": 40, 1123 | "metadata": {}, 1124 | "outputs": [], 1125 | "source": [ 1126 | "from ipywidgets import interact\n", 1127 | "import ipywidgets as widgets" 1128 | ] 1129 | }, 1130 | { 1131 | "cell_type": "code", 1132 | "execution_count": 42, 1133 | "metadata": {}, 1134 | "outputs": [ 1135 | { 1136 | "data": { 1137 | "application/vnd.jupyter.widget-view+json": { 1138 | "model_id": "49a1fee515fe4fa7864a75d1b241114e", 1139 | "version_major": 2, 1140 | "version_minor": 0 1141 | }, 1142 | "text/plain": [ 1143 | "interactive(children=(FloatSlider(value=1.0, description='mu', max=20.0, min=1.0, step=1.0), Output()), _dom_c…" 1144 | ] 1145 | }, 1146 | "metadata": {}, 1147 | "output_type": "display_data" 1148 | } 1149 | ], 1150 | "source": [ 1151 | "interact(f, mu=widgets.FloatSlider(min=1.0, max=20.0, step=1.0));" 1152 | ] 1153 | }, 1154 | { 1155 | "cell_type": "markdown", 1156 | "metadata": {}, 1157 | "source": [ 1158 | "### nbconvert to HTML" 1159 | ] 1160 | }, 1161 | { 1162 | "cell_type": "code", 1163 | "execution_count": 43, 1164 | "metadata": {}, 1165 | "outputs": [ 1166 | { 1167 | "name": "stdout", 1168 | "output_type": "stream", 1169 | "text": [ 1170 | "[NbConvertApp] Converting notebook ch01-intro.ipynb to html\n", 1171 | "[NbConvertApp] WARNING | Alternative text is missing on 1 image(s).\n", 1172 | "[NbConvertApp] Writing 348064 bytes to ch01-intro.html\n" 1173 | ] 1174 | } 1175 | ], 1176 | "source": [ 1177 | "!jupyter nbconvert --to html ch01-intro.ipynb" 1178 | ] 1179 | }, 1180 | { 1181 | "cell_type": "markdown", 1182 | "metadata": {}, 1183 | "source": [ 1184 | "### nbconvert to PDF\n", 1185 | "* [Requires a LaTeX environment](https://nbconvert.readthedocs.io/en/latest/install.html#installing-tex) to be installed.\n", 1186 | "* On this system (Ubuntu Linux): ```sudo apt-get install texlive-xetex```" 1187 | ] 1188 | }, 1189 | { 1190 | "cell_type": "code", 1191 | "execution_count": 45, 1192 | "metadata": {}, 1193 | "outputs": [ 1194 | { 1195 | "name": "stdout", 1196 | "output_type": "stream", 1197 | "text": [ 1198 | "[NbConvertApp] Converting notebook ch01-intro.ipynb to pdf\n", 1199 | "[NbConvertApp] Writing 63377 bytes to notebook.tex\n", 1200 | "[NbConvertApp] Building PDF\n", 1201 | "[NbConvertApp] Running xelatex 3 times: ['xelatex', 'notebook.tex', '-quiet']\n", 1202 | "[NbConvertApp] Running bibtex 1 time: ['bibtex', 'notebook']\n", 1203 | "[NbConvertApp] WARNING | bibtex had problems, most likely because there were no citations\n", 1204 | "[NbConvertApp] PDF successfully created\n", 1205 | "[NbConvertApp] Writing 128251 bytes to ch01-intro.pdf\n" 1206 | ] 1207 | } 1208 | ], 1209 | "source": [ 1210 | "# be sure to install pandoc first.\n", 1211 | "!jupyter nbconvert --to pdf ch01-intro.ipynb;" 1212 | ] 1213 | }, 1214 | { 1215 | "cell_type": "code", 1216 | "execution_count": 46, 1217 | "metadata": {}, 1218 | "outputs": [ 1219 | { 1220 | "name": "stdout", 1221 | "output_type": "stream", 1222 | "text": [ 1223 | "ch01-intro.html ch01-intro.ipynb ch01-intro.pdf ch01-intro.py\n" 1224 | ] 1225 | } 1226 | ], 1227 | "source": [ 1228 | "!ls ch01*" 1229 | ] 1230 | } 1231 | ], 1232 | "metadata": { 1233 | "kernelspec": { 1234 | "display_name": "Python 3 (ipykernel)", 1235 | "language": "python", 1236 | "name": "python3" 1237 | }, 1238 | "language_info": { 1239 | "codemirror_mode": { 1240 | "name": "ipython", 1241 | "version": 3 1242 | }, 1243 | "file_extension": ".py", 1244 | "mimetype": "text/x-python", 1245 | "name": "python", 1246 | "nbconvert_exporter": "python", 1247 | "pygments_lexer": "ipython3", 1248 | "version": "3.11.4" 1249 | } 1250 | }, 1251 | "nbformat": 4, 1252 | "nbformat_minor": 4 1253 | } 1254 | -------------------------------------------------------------------------------- /ch01-intro.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/ch01-intro.pdf -------------------------------------------------------------------------------- /ch01-intro.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # coding: utf-8 3 | 4 | # ### Chapter 1: Computing with Python 5 | 6 | # ### Overview: a typical Python-based scientific computing stack. 7 | # ![software stack](pics/software-stack-overview.png) 8 | # 9 | # ### Resources: 10 | # - [Intel MKL (Math Kernel Library)](https://software.intel.com/en-us/intel-mkl) 11 | # - [openBLAS](https://www.openblas.net) 12 | # - [ATLAS](https://math-atlas.sourceforge.net) 13 | # - [SciPy](http://www.scipy.org) 14 | # - [Python Numeric & Scientific topics](http://wiki.python.org/moin/NumericAndScientific) 15 | 16 | # ### Interpreter 17 | # - The easist way to execute Python code: run the program directly. 18 | # - Use Jupyter magic command to write Python source file to disk: 19 | 20 | # In[1]: 21 | 22 | 23 | get_ipython().run_cell_magic('writefile', 'hello.py', 'print("Hello from Python!")') 24 | 25 | 26 | # * Use the ! system shell command (included in the Python Jupyter kernel) to interactively run Python with hello.py as its argument. 27 | 28 | # In[2]: 29 | 30 | 31 | get_ipython().system('python hello.py') 32 | 33 | 34 | # In[3]: 35 | 36 | 37 | get_ipython().system('python --version') 38 | 39 | 40 | # ### Input and output caching 41 | # 42 | # * Input & output history can be accessed using __In__ (a list) & __Out__ (a dictionary). Both can be indexed with a cell number. 43 | 44 | # In[4]: 45 | 46 | 47 | 3*3 48 | In[1] 49 | 50 | 51 | # * A single underscore = the most recent output; 52 | # * A double underscore = the _next_ most recent output. 53 | 54 | # In[5]: 55 | 56 | 57 | 1+1 58 | 59 | 60 | # In[6]: 61 | 62 | 63 | 2+2 64 | 65 | 66 | # In[7]: 67 | 68 | 69 | _, __ 70 | 71 | 72 | # In[8]: 73 | 74 | 75 | # In = a list 76 | In 77 | 78 | 79 | # In[9]: 80 | 81 | 82 | # Out = a dictionary 83 | Out 84 | 85 | 86 | # In[10]: 87 | 88 | 89 | # Suppress output results by ending statement with a semicolon 90 | 1+2; 91 | 92 | 93 | # ### Autocompletion 94 | # 95 | # * The __Tab__ key activates autocompletion (displays list of symbol names that are valid completions of what has been typed thus far.) 96 | 97 | # In[11]: 98 | 99 | 100 | import os 101 | 102 | 103 | # * Results of typing "os.w", followed by \t: 104 | # 105 | # ![autocompletion](pics/autocompletion.png) 106 | 107 | # ### Documentation 108 | # 109 | # * "Docstrings" provide a built-in reference manual for most Python modules. Display the docstring by appending a Python object with "?". 110 | 111 | # In[12]: 112 | 113 | 114 | import math 115 | get_ipython().run_line_magic('pinfo', 'math.cos') 116 | 117 | 118 | # ### Interaction with System Shell 119 | # 120 | # * Anything after ! is evaluated using the system shell, such as bash. 121 | # * (I use Ubuntu Linux as my laptop OS. Your Windows equivalents will vary.) 122 | 123 | # In[13]: 124 | 125 | 126 | get_ipython().system('touch file1.py file2.py file3.py') 127 | get_ipython().system('ls file*') 128 | 129 | 130 | # In[14]: 131 | 132 | 133 | # output of a system shell command can be captured in a Python variable 134 | files = get_ipython().getoutput('ls file*') 135 | print(len(files)) 136 | print(files) 137 | 138 | 139 | # In[15]: 140 | 141 | 142 | # pass Python variable values to shell commands 143 | # by prefixing the variable name with $. 144 | file = "file1.py" 145 | get_ipython().system('ls -l $file') 146 | 147 | 148 | # ### IPython Extensions 149 | # 150 | # * Commands start with one or two "%" characters. A single % is used for single-line commands; dual %% is used for cells (multiple lines). 151 | # 152 | # * `%lsmagic` returns a list of available commands. 153 | 154 | # In[16]: 155 | 156 | 157 | get_ipython().run_line_magic('lsmagic', '') 158 | 159 | 160 | # ### Running scripts 161 | # 162 | # - `%run` executes an external Python source file within an interactive IPython session. 163 | 164 | # In[17]: 165 | 166 | 167 | get_ipython().run_cell_magic('writefile', 'fib.py', '\ndef fib(N): \n """ \n Return a list of the first N Fibonacci numbers.\n """ \n f0, f1 = 0, 1\n f = [1] * N\n for n in range(1, N):\n f[n] = f0 + f1\n f0, f1 = f1, f[n]\n\n return f\n\nprint(fib(10))') 168 | 169 | 170 | # In[18]: 171 | 172 | 173 | get_ipython().system('python fib.py') 174 | 175 | 176 | # In[19]: 177 | 178 | 179 | get_ipython().run_line_magic('run', 'fib.py') 180 | 181 | 182 | # In[20]: 183 | 184 | 185 | fib(6) 186 | 187 | 188 | # ## Listing all defined symbols 189 | # 190 | # * `%who` lists all defined symbols 191 | # * `%whos` provides more detailed info. 192 | 193 | # In[21]: 194 | 195 | 196 | get_ipython().run_line_magic('who', '') 197 | 198 | 199 | # In[22]: 200 | 201 | 202 | get_ipython().run_line_magic('whos', '') 203 | 204 | 205 | # ## Debugger 206 | # 207 | # * Use `%debug` to step directly into the Python debugger. 208 | 209 | # In[23]: 210 | 211 | 212 | # fib function fails - can't use floating point numbers. 213 | try: 214 | fib(1.0) 215 | except TypeError: 216 | print("nope. can't do that.") 217 | 218 | 219 | # In[24]: 220 | 221 | 222 | #%debug 223 | 224 | 225 | # ### Resetting the Python namespace 226 | # - Ensures a program is run in a pristine environment, uncluttered by existing variables and functions. Although it is necessary to reimport modules, it is important to know that even if the modules have changed since the last 227 | # import, a new import after a %reset will not import the new module but rather reenable a cached version from the previous import. 228 | # 229 | # - When developing Python modules, this is usually not the desired behavior. In that case, a reimport of a previously imported (and since updated) module can often be achieved by using the reload function from `IPython.lib.deepreload`. However, this method does not always work, as 230 | # some libraries run code at import time that is only intended to run once. In this case, the only option might be to terminate and restart the IPython interpreter. 231 | 232 | # In[26]: 233 | 234 | 235 | get_ipython().run_line_magic('reset', '') 236 | 237 | 238 | # ## Timing and profiling code 239 | # 240 | # * `%timeit` and `%time` provide simple benchmarking utilities. 241 | 242 | # In[25]: 243 | 244 | 245 | # first, re-define fibonacci code used above. 246 | def fib(N): 247 | """ 248 | Return a list of the first N Fibonacci numbers. 249 | """ 250 | f0, f1 = 0, 1 251 | f = [1] * N 252 | for n in range(1, N): 253 | f[n] = f0 + f1 254 | f0, f1 = f1, f[n] 255 | 256 | return f 257 | 258 | 259 | # In[26]: 260 | 261 | 262 | # timeit does not return expression's resulting value. 263 | get_ipython().run_line_magic('timeit', 'fib(50)') 264 | 265 | 266 | # In[27]: 267 | 268 | 269 | # %time only runs once. less accurate estimate. 270 | result = get_ipython().run_line_magic('time', 'fib(100)') 271 | 272 | 273 | # In[28]: 274 | 275 | 276 | len(result) 277 | 278 | 279 | # * The __cProfile__ module provides __%prun__ (for statements) and __%run__ (for external scripts) profiling commands. 280 | 281 | # In[29]: 282 | 283 | 284 | import numpy as np 285 | 286 | def random_walker_max_distance(M, N): 287 | """ 288 | Simulate N random walkers taking M steps 289 | Return the largest distance from the starting point. 290 | """ 291 | trajectories = [np.random.randn(M).cumsum() 292 | for _ in range(N)] 293 | return np.max(np.abs(trajectories)) 294 | 295 | 296 | # In[30]: 297 | 298 | 299 | # returns call counts, runtime & cume runtime for 300 | # each function. 301 | get_ipython().run_line_magic('prun', 'random_walker_max_distance(400, 10000)') 302 | 303 | 304 | # ### Jupyter: External image rendering 305 | 306 | # In[31]: 307 | 308 | 309 | from IPython.display import display, Image, HTML, Math 310 | 311 | 312 | # In[32]: 313 | 314 | 315 | Image(url='http://python.org/images/python-logo.gif') 316 | 317 | 318 | # ### Jupyter: HTML rendering 319 | 320 | # In[33]: 321 | 322 | 323 | import scipy, numpy, matplotlib 324 | modules = [numpy, matplotlib, scipy] 325 | 326 | row = "%s%s" 327 | rows = "\n".join( 328 | [row % 329 | (module.__name__, module.__version__) 330 | for module in modules]) 331 | table = " %s
LibraryVersion
" % rows 332 | 333 | 334 | # In[34]: 335 | 336 | 337 | HTML(table) 338 | 339 | 340 | # In[35]: 341 | 342 | 343 | # another method 344 | class HTMLdisplayer(object): 345 | def __init__(self,code): 346 | self.code = code 347 | def _repr_html_(self): 348 | return self.code 349 | 350 | HTMLdisplayer(table) 351 | 352 | 353 | # ### Jupyter: Formula rendering using Latex 354 | 355 | # In[36]: 356 | 357 | 358 | Math(r'\hat{H} = -\frac{1}{2}\epsilon \hat{\sigma}_z-\frac{1}{2}\delta \hat{\sigma}_x') 359 | 360 | 361 | # ### Jupyter: UI Widgets 362 | # 363 | # ** Needs debugging: slider widget doesn't appear. ** 364 | 365 | # In[37]: 366 | 367 | 368 | import matplotlib.pyplot as plt 369 | import numpy as np 370 | from scipy import stats 371 | 372 | def f(mu): 373 | X = stats.norm(loc=mu, scale=np.sqrt(mu)) 374 | N = stats.poisson(mu) 375 | x = np.linspace(0, X.ppf(0.999)) 376 | n = np.arange(0, x[-1]) 377 | 378 | fig, ax = plt.subplots() 379 | ax.plot(x, X.pdf(x), color='black', lw=2, label="Normal($\mu=%d, \sigma^2=%d$)" % (mu,mu)) 380 | ax.bar(n, N.pmf(n), align='edge', label=r"Poisson($\lambda=%d$)" % mu) 381 | ax.set_ylim(0, X.pdf(x).max() * 1.25) 382 | ax.legend(loc=2, ncol=2) 383 | plt.close(fig) 384 | return fig 385 | 386 | 387 | # In[38]: 388 | 389 | 390 | from ipywidgets import interact 391 | import ipywidgets as widgets 392 | 393 | 394 | # In[39]: 395 | 396 | 397 | interact(f, mu=widgets.FloatSlider(min=1.0, max=20.0, step=1.0)); 398 | 399 | 400 | # ### nbconvert to HTML 401 | 402 | # In[40]: 403 | 404 | 405 | get_ipython().system('jupyter nbconvert --to html ch01-intro.ipynb') 406 | 407 | 408 | # ### nbconvert to PDF 409 | # * [Requires a LaTeX environment](https://nbconvert.readthedocs.io/en/latest/install.html#installing-tex) to be installed. 410 | # * On this system (Ubuntu Linux): ```sudo apt-get install texlive-xetex``` 411 | 412 | # In[41]: 413 | 414 | 415 | get_ipython().system('jupyter nbconvert --to pdf ch01-intro.ipynb;') 416 | 417 | 418 | # ## nbconvert to pure Python source code 419 | 420 | # In[70]: 421 | 422 | 423 | get_ipython().system('jupyter nbconvert ch01-intro.ipynb --to python') 424 | 425 | 426 | # In[71]: 427 | 428 | 429 | get_ipython().system('ls ch01*') 430 | 431 | 432 | # In[ ]: 433 | 434 | 435 | 436 | 437 | -------------------------------------------------------------------------------- /ch11-fdm-2d-ex1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/ch11-fdm-2d-ex1.png -------------------------------------------------------------------------------- /ch18-data-input-output.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Data Input/Output Options\n", 8 | "\n", 9 | "- __Comma-Separated Values (CSV)__\n", 10 | "- __Hierarchical Data Format (HDF5)__\n", 11 | "- __h5py__ (files, groups, datasets, attributes)\n", 12 | "- __PyTables__\n", 13 | "- __Pandas HDFStore__\n", 14 | "- __JSON__\n", 15 | "- __Serialization__\n", 16 | "\n", 17 | "* data classes:\n", 18 | " * structured vs unstructured\n", 19 | " * categorical (finite set) vs ordinal (ordered) vs numerical (continuous/discrete)\n", 20 | "* should consider: [Blaze](http://blaze.pydata.org/en/latest) for high-level, multi-format API for data I/O" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "## Imports" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 2, 33 | "metadata": { 34 | "collapsed": true 35 | }, 36 | "outputs": [], 37 | "source": [ 38 | "from __future__ import print_function" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 3, 44 | "metadata": { 45 | "collapsed": false 46 | }, 47 | "outputs": [], 48 | "source": [ 49 | "import numpy as np\n", 50 | "np.random.seed(0)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 4, 56 | "metadata": { 57 | "collapsed": false 58 | }, 59 | "outputs": [], 60 | "source": [ 61 | "import pandas as pd" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 5, 67 | "metadata": { 68 | "collapsed": false 69 | }, 70 | "outputs": [ 71 | { 72 | "name": "stderr", 73 | "output_type": "stream", 74 | "text": [ 75 | "/home/bjpcjp/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", 76 | " from ._conv import register_converters as _register_converters\n" 77 | ] 78 | } 79 | ], 80 | "source": [ 81 | "import csv\n", 82 | "import json\n", 83 | "import h5py\n", 84 | "import tables\n", 85 | "import pickle\n", 86 | "\n", 87 | "# python3: import _pickle as cPickle\n", 88 | "import _pickle as cPickle\n", 89 | "\n", 90 | "# conda install msgpack-python\n", 91 | "import msgpack" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "### CSV\n", 99 | "\n", 100 | "- first: create some example CSV data (hockey player stats) & save it to disk" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": 6, 106 | "metadata": { 107 | "collapsed": false 108 | }, 109 | "outputs": [ 110 | { 111 | "name": "stdout", 112 | "output_type": "stream", 113 | "text": [ 114 | "Overwriting ch18-playerstats-2013-2014.csv\n" 115 | ] 116 | } 117 | ], 118 | "source": [ 119 | "%%writefile ch18-playerstats-2013-2014.csv\n", 120 | "# 2013-2014 / Regular Season / All Skaters / Summary / Points\n", 121 | "Rank,Player,Team,Pos,GP,G,A,P,+/-,PIM,PPG,PPP,SHG,SHP,GW,OT,S,S%,TOI/GP,Shift/GP,FO%\n", 122 | "1,Sidney Crosby,PIT,C,80,36,68,104,+18,46,11,38,0,0,5,1,259,13.9,21:58,24.0,52.5\n", 123 | "2,Ryan Getzlaf,ANA,C,77,31,56,87,+28,31,5,23,0,0,7,1,204,15.2,21:17,25.2,49.0\n", 124 | "3,Claude Giroux,PHI,C,82,28,58,86,+7,46,7,37,0,0,7,1,223,12.6,20:26,25.1,52.9\n", 125 | "4,Tyler Seguin,DAL,C,80,37,47,84,+16,18,11,25,0,0,8,0,294,12.6,19:20,23.4,41.5\n", 126 | "5,Corey Perry,ANA,R,81,43,39,82,+32,65,8,18,0,0,9,1,280,15.4,19:28,23.2,36.0" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 7, 132 | "metadata": { 133 | "collapsed": false 134 | }, 135 | "outputs": [ 136 | { 137 | "name": "stdout", 138 | "output_type": "stream", 139 | "text": [ 140 | "Overwriting ch18-playerstats-2013-2014-top30.csv\n" 141 | ] 142 | } 143 | ], 144 | "source": [ 145 | "%%writefile ch18-playerstats-2013-2014-top30.csv\n", 146 | "# 2013-2014 / Regular Season / All Skaters / Summary / Points\n", 147 | "Rank,Player,Team,Pos,GP,G,A,P,+/-,PIM,PPG,PPP,SHG,SHP,GW,OT,S,S%,TOI/GP,Shift/GP,FO%\n", 148 | "1,Sidney Crosby,PIT,C,80,36,68,104,+18,46,11,38,0,0,5,1,259,13.9,21:58,24.0,52.5\n", 149 | "2,Ryan Getzlaf,ANA,C,77,31,56,87,+28,31,5,23,0,0,7,1,204,15.2,21:17,25.2,49.0\n", 150 | "3,Claude Giroux,PHI,C,82,28,58,86,+7,46,7,37,0,0,7,1,223,12.6,20:26,25.1,52.9\n", 151 | "4,Tyler Seguin,DAL,C,80,37,47,84,+16,18,11,25,0,0,8,0,294,12.6,19:20,23.4,41.5\n", 152 | "5,Corey Perry,ANA,R,81,43,39,82,+32,65,8,18,0,0,9,1,280,15.4,19:28,23.2,36.0\n", 153 | "6,Phil Kessel,TOR,R,82,37,43,80,-5,27,8,20,0,0,6,0,305,12.1,20:39,24.5,14.3\n", 154 | "7,Taylor Hall,EDM,L,75,27,53,80,-15,44,7,17,0,1,1,1,250,10.8,20:00,25.4,45.7\n", 155 | "8,Alex Ovechkin,WSH,L,78,51,28,79,-35,48,24,39,0,1,10,3,386,13.2,20:32,21.8,66.7\n", 156 | "9,Joe Pavelski,SJS,C,82,41,38,79,+23,32,16,31,1,2,3,0,225,18.2,19:51,27.1,56.0\n", 157 | "10,Jamie Benn,DAL,L,81,34,45,79,+21,64,5,19,1,3,3,1,279,12.2,19:09,25.0,52.8\n", 158 | "11,Nicklas Backstrom,WSH,C,82,18,61,79,-20,54,6,44,1,1,1,0,196,9.2,19:48,23.3,50.4\n", 159 | "12,Patrick Sharp,CHI,L,82,34,44,78,+13,40,10,25,0,0,3,1,313,10.9,18:53,22.7,54.6\n", 160 | "13,Joe Thornton,SJS,C,82,11,65,76,+20,32,2,19,0,1,3,1,122,9.0,18:55,26.3,56.1\n", 161 | "14,Erik Karlsson,OTT,D,82,20,54,74,-15,36,5,31,0,0,1,0,257,7.8,27:04,28.6,0.0\n", 162 | "15,Evgeni Malkin,PIT,C,60,23,49,72,+10,62,7,30,0,0,3,0,191,12.0,20:03,21.4,48.8\n", 163 | "16,Patrick Marleau,SJS,L,82,33,37,70,+0,18,11,23,2,2,4,0,285,11.6,20:31,27.3,52.9\n", 164 | "17,Anze Kopitar,LAK,C,82,29,41,70,+34,24,10,23,0,0,9,2,200,14.5,20:53,25.4,53.3\n", 165 | "18,Matt Duchene,COL,C,71,23,47,70,+8,19,5,17,0,0,6,1,217,10.6,18:29,22.0,50.3\n", 166 | "19,Martin St. Louis,\"TBL, NYR\",R,81,30,39,69,+13,10,9,21,1,2,5,1,204,14.7,20:56,25.7,40.7\n", 167 | "20,Patrick Kane,CHI,R,69,29,40,69,+7,22,10,25,0,0,6,0,227,12.8,19:36,22.9,50.0\n", 168 | "21,Blake Wheeler,WPG,R,82,28,41,69,+4,63,8,19,0,0,4,2,225,12.4,18:41,24.0,37.5\n", 169 | "22,Kyle Okposo,NYI,R,71,27,42,69,-9,51,5,15,0,0,4,1,195,13.8,20:26,22.2,47.5\n", 170 | "23,David Krejci,BOS,C,80,19,50,69,+39,28,3,19,0,0,6,1,169,11.2,19:07,21.3,51.2\n", 171 | "24,Chris Kunitz,PIT,L,78,35,33,68,+25,66,13,22,0,0,8,0,218,16.1,19:09,22.2,75.0\n", 172 | "25,Jonathan Toews,CHI,C,76,28,40,68,+26,34,5,15,3,5,5,0,193,14.5,20:28,25.9,57.2\n", 173 | "26,Thomas Vanek,\"BUF, NYI, MTL\",L,78,27,41,68,+7,46,8,18,0,0,4,0,248,10.9,19:21,21.6,43.5\n", 174 | "27,Jaromir Jagr,NJD,R,82,24,43,67,+16,46,5,17,0,0,6,1,231,10.4,19:09,22.8,0.0\n", 175 | "28,John Tavares,NYI,C,59,24,42,66,-6,40,8,25,0,0,4,0,188,12.8,21:14,22.3,49.1\n", 176 | "29,Jason Spezza,OTT,C,75,23,43,66,-26,46,9,22,0,0,5,0,223,10.3,18:12,23.8,54.0\n", 177 | "30,Jordan Eberle,EDM,R,80,28,37,65,-11,18,7,20,1,1,4,1,200,14.0,19:32,25.4,38.1" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": 8, 183 | "metadata": { 184 | "collapsed": false 185 | }, 186 | "outputs": [ 187 | { 188 | "name": "stdout", 189 | "output_type": "stream", 190 | "text": [ 191 | "# 2013-2014 / Regular Season / All Skaters / Summary / Points\n", 192 | "Rank,Player,Team,Pos,GP,G,A,P,+/-,PIM,PPG,PPP,SHG,SHP,GW,OT,S,S%,TOI/GP,Shift/GP,FO%\n", 193 | "1,Sidney Crosby,PIT,C,80,36,68,104,+18,46,11,38,0,0,5,1,259,13.9,21:58,24.0,52.5\n", 194 | "2,Ryan Getzlaf,ANA,C,77,31,56,87,+28,31,5,23,0,0,7,1,204,15.2,21:17,25.2,49.0\n", 195 | "3,Claude Giroux,PHI,C,82,28,58,86,+7,46,7,37,0,0,7,1,223,12.6,20:26,25.1,52.9\n" 196 | ] 197 | } 198 | ], 199 | "source": [ 200 | "# let's see if file contents are as expected\n", 201 | "!head -n 5 ch18-playerstats-2013-2014-top30.csv" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": {}, 207 | "source": [ 208 | "* Parsed row values will be read as strings, even if values represent numbers.\n", 209 | "* Numpy __loadtxt__ and __savetxt__ are good for handling numerical arrays on disk." 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 9, 215 | "metadata": { 216 | "collapsed": false 217 | }, 218 | "outputs": [], 219 | "source": [ 220 | "data = np.random.randn(100,3)\n", 221 | "np.savetxt(\"data.csv\", data, delimiter=\",\", header=\"x,y,z\", comments=\"random x,y,z coords\\n\")" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 10, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "name": "stdout", 231 | "output_type": "stream", 232 | "text": [ 233 | "random x,y,z coords\n", 234 | "x,y,z\n", 235 | "1.764052345967664026e+00,4.001572083672232938e-01,9.787379841057392005e-01\n", 236 | "2.240893199201457797e+00,1.867557990149967484e+00,-9.772778798764110153e-01\n", 237 | "9.500884175255893682e-01,-1.513572082976978872e-01,-1.032188517935578448e-01\n" 238 | ] 239 | } 240 | ], 241 | "source": [ 242 | "!head -n 5 data.csv" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": 11, 248 | "metadata": {}, 249 | "outputs": [ 250 | { 251 | "data": { 252 | "text/plain": [ 253 | "True" 254 | ] 255 | }, 256 | "execution_count": 11, 257 | "metadata": {}, 258 | "output_type": "execute_result" 259 | } 260 | ], 261 | "source": [ 262 | "# Read data back into NumPy array\n", 263 | "data_load = np.loadtxt(\"data.csv\", skiprows=2, delimiter=\",\")\n", 264 | "# and check for equality\n", 265 | "(data == data_load).all()" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": 12, 271 | "metadata": {}, 272 | "outputs": [ 273 | { 274 | "data": { 275 | "text/plain": [ 276 | "array([ 2.2408932 , 1.86755799, -0.97727788])" 277 | ] 278 | }, 279 | "execution_count": 12, 280 | "metadata": {}, 281 | "output_type": "execute_result" 282 | } 283 | ], 284 | "source": [ 285 | "# by default, loadtxt converts all fields into float64 values\n", 286 | "data_load[1,:]" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 13, 292 | "metadata": {}, 293 | "outputs": [ 294 | { 295 | "data": { 296 | "text/plain": [ 297 | "dtype('float64')" 298 | ] 299 | }, 300 | "execution_count": 13, 301 | "metadata": {}, 302 | "output_type": "execute_result" 303 | } 304 | ], 305 | "source": [ 306 | "data_load.dtype" 307 | ] 308 | }, 309 | { 310 | "cell_type": "markdown", 311 | "metadata": {}, 312 | "source": [ 313 | "* Need to explicitly set a dtype if reading non-numerical CSV data.\n", 314 | "* Otherwise NumPy will barf.\n", 315 | "* dtype=bytes, or str, or object, will return unparsed values." 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": 14, 321 | "metadata": { 322 | "collapsed": false 323 | }, 324 | "outputs": [ 325 | { 326 | "data": { 327 | "text/plain": [ 328 | "array([b'Sidney Crosby', b'PIT', b'C', b'80', b'36'], dtype='|S13')" 329 | ] 330 | }, 331 | "execution_count": 14, 332 | "metadata": {}, 333 | "output_type": "execute_result" 334 | } 335 | ], 336 | "source": [ 337 | "data = np.loadtxt(\n", 338 | " \"ch18-playerstats-2013-2014.csv\", \n", 339 | " skiprows=2, delimiter=\",\", dtype=bytes)\n", 340 | "\n", 341 | "data[0][1:6]" 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "execution_count": 15, 347 | "metadata": { 348 | "collapsed": false 349 | }, 350 | "outputs": [ 351 | { 352 | "data": { 353 | "text/plain": [ 354 | "array([[ 68., 104., 18.],\n", 355 | " [ 56., 87., 28.],\n", 356 | " [ 58., 86., 7.],\n", 357 | " [ 47., 84., 16.],\n", 358 | " [ 39., 82., 32.]])" 359 | ] 360 | }, 361 | "execution_count": 15, 362 | "metadata": {}, 363 | "output_type": "execute_result" 364 | } 365 | ], 366 | "source": [ 367 | "# read selected columns:\n", 368 | "np.loadtxt(\"ch18-playerstats-2013-2014.csv\", \n", 369 | " skiprows=2, delimiter=\",\", usecols=[6,7,8])" 370 | ] 371 | }, 372 | { 373 | "cell_type": "code", 374 | "execution_count": 16, 375 | "metadata": { 376 | "collapsed": false 377 | }, 378 | "outputs": [], 379 | "source": [ 380 | "# A 3rd method: Pandas read_csv()\n", 381 | "df = pd.read_csv(\"ch18-playerstats-2013-2014.csv\", \n", 382 | " skiprows=1)" 383 | ] 384 | }, 385 | { 386 | "cell_type": "code", 387 | "execution_count": 17, 388 | "metadata": { 389 | "collapsed": false 390 | }, 391 | "outputs": [], 392 | "source": [ 393 | "df = df.set_index(\"Rank\")" 394 | ] 395 | }, 396 | { 397 | "cell_type": "code", 398 | "execution_count": 18, 399 | "metadata": { 400 | "collapsed": false 401 | }, 402 | "outputs": [ 403 | { 404 | "data": { 405 | "text/html": [ 406 | "
\n", 407 | "\n", 420 | "\n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | " \n", 434 | " \n", 435 | " \n", 436 | " \n", 437 | " \n", 438 | " \n", 439 | " \n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | "
PlayerGPGAP
Rank
1Sidney Crosby803668104
2Ryan Getzlaf77315687
3Claude Giroux82285886
4Tyler Seguin80374784
5Corey Perry81433982
\n", 482 | "
" 483 | ], 484 | "text/plain": [ 485 | " Player GP G A P\n", 486 | "Rank \n", 487 | "1 Sidney Crosby 80 36 68 104\n", 488 | "2 Ryan Getzlaf 77 31 56 87\n", 489 | "3 Claude Giroux 82 28 58 86\n", 490 | "4 Tyler Seguin 80 37 47 84\n", 491 | "5 Corey Perry 81 43 39 82" 492 | ] 493 | }, 494 | "execution_count": 18, 495 | "metadata": {}, 496 | "output_type": "execute_result" 497 | } 498 | ], 499 | "source": [ 500 | "df[[\"Player\", \"GP\", \"G\", \"A\", \"P\"]]" 501 | ] 502 | }, 503 | { 504 | "cell_type": "code", 505 | "execution_count": 19, 506 | "metadata": { 507 | "collapsed": false 508 | }, 509 | "outputs": [ 510 | { 511 | "name": "stdout", 512 | "output_type": "stream", 513 | "text": [ 514 | "\n", 515 | "Int64Index: 5 entries, 1 to 5\n", 516 | "Data columns (total 20 columns):\n", 517 | "Player 5 non-null object\n", 518 | "Team 5 non-null object\n", 519 | "Pos 5 non-null object\n", 520 | "GP 5 non-null int64\n", 521 | "G 5 non-null int64\n", 522 | "A 5 non-null int64\n", 523 | "P 5 non-null int64\n", 524 | "+/- 5 non-null int64\n", 525 | "PIM 5 non-null int64\n", 526 | "PPG 5 non-null int64\n", 527 | "PPP 5 non-null int64\n", 528 | "SHG 5 non-null int64\n", 529 | "SHP 5 non-null int64\n", 530 | "GW 5 non-null int64\n", 531 | "OT 5 non-null int64\n", 532 | "S 5 non-null int64\n", 533 | "S% 5 non-null float64\n", 534 | "TOI/GP 5 non-null object\n", 535 | "Shift/GP 5 non-null float64\n", 536 | "FO% 5 non-null float64\n", 537 | "dtypes: float64(3), int64(13), object(4)\n", 538 | "memory usage: 840.0+ bytes\n" 539 | ] 540 | } 541 | ], 542 | "source": [ 543 | "# use info() to see the dtype of each parsed column\n", 544 | "df.info()" 545 | ] 546 | }, 547 | { 548 | "cell_type": "code", 549 | "execution_count": 20, 550 | "metadata": { 551 | "collapsed": true 552 | }, 553 | "outputs": [], 554 | "source": [ 555 | "# writing to CSV files using dataframes:\n", 556 | "df[[\"Player\", \"GP\", \"G\", \"A\", \"P\"]].to_csv(\"ch18-playerstats-2013-2014-subset.csv\")" 557 | ] 558 | }, 559 | { 560 | "cell_type": "code", 561 | "execution_count": 21, 562 | "metadata": { 563 | "collapsed": false 564 | }, 565 | "outputs": [ 566 | { 567 | "name": "stdout", 568 | "output_type": "stream", 569 | "text": [ 570 | "Rank,Player,GP,G,A,P\n", 571 | "1,Sidney Crosby,80,36,68,104\n", 572 | "2,Ryan Getzlaf,77,31,56,87\n", 573 | "3,Claude Giroux,82,28,58,86\n", 574 | "4,Tyler Seguin,80,37,47,84\n" 575 | ] 576 | } 577 | ], 578 | "source": [ 579 | "!head -n 5 ch18-playerstats-2013-2014-subset.csv" 580 | ] 581 | }, 582 | { 583 | "cell_type": "markdown", 584 | "metadata": {}, 585 | "source": [ 586 | "### h5py\n", 587 | "* used for numerical data store\n", 588 | "* hierarchical format - orgs datasets within files: \"groups\" and \"datasets\"\n", 589 | "* groups & datasets can contain \"attributes\" (metadata)\n", 590 | "* Python libraries: h5py & PyTables" 591 | ] 592 | }, 593 | { 594 | "cell_type": "code", 595 | "execution_count": 22, 596 | "metadata": { 597 | "collapsed": true 598 | }, 599 | "outputs": [], 600 | "source": [ 601 | "import h5py" 602 | ] 603 | }, 604 | { 605 | "cell_type": "markdown", 606 | "metadata": {}, 607 | "source": [ 608 | "* file modes: \"w\" (create new file; truncate if exists), \"r\" (read-only; file must exist), \"w-\" (create new file; error if exists), \"r+\" (read-write; file must exist), \"a\" (read-write; create if needed)" 609 | ] 610 | }, 611 | { 612 | "cell_type": "code", 613 | "execution_count": 23, 614 | "metadata": { 615 | "collapsed": false 616 | }, 617 | "outputs": [ 618 | { 619 | "data": { 620 | "text/plain": [ 621 | "'r+'" 622 | ] 623 | }, 624 | "execution_count": 23, 625 | "metadata": {}, 626 | "output_type": "execute_result" 627 | } 628 | ], 629 | "source": [ 630 | "# create new read-write file\n", 631 | "f = h5py.File(\"ch18-data.h5\", \"w\")\n", 632 | "f.mode" 633 | ] 634 | }, 635 | { 636 | "cell_type": "code", 637 | "execution_count": 24, 638 | "metadata": { 639 | "collapsed": false 640 | }, 641 | "outputs": [], 642 | "source": [ 643 | "f.flush()\n", 644 | "f.close()" 645 | ] 646 | }, 647 | { 648 | "cell_type": "markdown", 649 | "metadata": {}, 650 | "source": [ 651 | "### Groups\n", 652 | "\n", 653 | "* File object creates both file handle and a \"root group\" object.\n", 654 | "* group name accessible via 'name'. root is '/'" 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": 25, 660 | "metadata": { 661 | "collapsed": false 662 | }, 663 | "outputs": [ 664 | { 665 | "data": { 666 | "text/plain": [ 667 | "'/'" 668 | ] 669 | }, 670 | "execution_count": 25, 671 | "metadata": {}, 672 | "output_type": "execute_result" 673 | } 674 | ], 675 | "source": [ 676 | "f = h5py.File(\"ch18-data.h5\", \"w\")\n", 677 | "f.name" 678 | ] 679 | }, 680 | { 681 | "cell_type": "code", 682 | "execution_count": 26, 683 | "metadata": { 684 | "collapsed": false 685 | }, 686 | "outputs": [ 687 | { 688 | "data": { 689 | "text/plain": [ 690 | "('/experiment1', '/experiment2/measurement', '/experiment2/simulation')" 691 | ] 692 | }, 693 | "execution_count": 26, 694 | "metadata": {}, 695 | "output_type": "execute_result" 696 | } 697 | ], 698 | "source": [ 699 | "# create hierarchical subgroups.\n", 700 | "grp1 = f.create_group(\"experiment1\")\n", 701 | "grp2_meas = f.create_group(\"experiment2/measurement\")\n", 702 | "grp2_sim = f.create_group(\"experiment2/simulation\")\n", 703 | "\n", 704 | "grp1.name, grp2_meas.name, grp2_sim.name" 705 | ] 706 | }, 707 | { 708 | "cell_type": "code", 709 | "execution_count": 27, 710 | "metadata": { 711 | "collapsed": false 712 | }, 713 | "outputs": [ 714 | { 715 | "data": { 716 | "text/plain": [ 717 | "" 718 | ] 719 | }, 720 | "execution_count": 27, 721 | "metadata": {}, 722 | "output_type": "execute_result" 723 | } 724 | ], 725 | "source": [ 726 | "# group access\n", 727 | "f[\"/experiment1\"]" 728 | ] 729 | }, 730 | { 731 | "cell_type": "code", 732 | "execution_count": 28, 733 | "metadata": { 734 | "collapsed": false 735 | }, 736 | "outputs": [ 737 | { 738 | "data": { 739 | "text/plain": [ 740 | "" 741 | ] 742 | }, 743 | "execution_count": 28, 744 | "metadata": {}, 745 | "output_type": "execute_result" 746 | } 747 | ], 748 | "source": [ 749 | "f[\"/experiment2/simulation\"]" 750 | ] 751 | }, 752 | { 753 | "cell_type": "code", 754 | "execution_count": 29, 755 | "metadata": { 756 | "collapsed": true 757 | }, 758 | "outputs": [], 759 | "source": [ 760 | "grp_expr2 = f[\"/experiment2\"]" 761 | ] 762 | }, 763 | { 764 | "cell_type": "code", 765 | "execution_count": 30, 766 | "metadata": { 767 | "collapsed": false 768 | }, 769 | "outputs": [ 770 | { 771 | "data": { 772 | "text/plain": [ 773 | "" 774 | ] 775 | }, 776 | "execution_count": 30, 777 | "metadata": {}, 778 | "output_type": "execute_result" 779 | } 780 | ], 781 | "source": [ 782 | "grp_expr2['simulation']" 783 | ] 784 | }, 785 | { 786 | "cell_type": "code", 787 | "execution_count": 31, 788 | "metadata": { 789 | "collapsed": false 790 | }, 791 | "outputs": [ 792 | { 793 | "data": { 794 | "text/plain": [ 795 | "['experiment1', 'experiment2']" 796 | ] 797 | }, 798 | "execution_count": 31, 799 | "metadata": {}, 800 | "output_type": "execute_result" 801 | } 802 | ], 803 | "source": [ 804 | "# keys = names of subgroups & datasets within a group\n", 805 | "list(f.keys())" 806 | ] 807 | }, 808 | { 809 | "cell_type": "code", 810 | "execution_count": 32, 811 | "metadata": { 812 | "collapsed": false 813 | }, 814 | "outputs": [ 815 | { 816 | "data": { 817 | "text/plain": [ 818 | "[('experiment1', ),\n", 819 | " ('experiment2', )]" 820 | ] 821 | }, 822 | "execution_count": 32, 823 | "metadata": {}, 824 | "output_type": "execute_result" 825 | } 826 | ], 827 | "source": [ 828 | "# items = tuples of (name, value) for each entity in each group\n", 829 | "list(f.items())" 830 | ] 831 | }, 832 | { 833 | "cell_type": "code", 834 | "execution_count": 33, 835 | "metadata": { 836 | "collapsed": false 837 | }, 838 | "outputs": [ 839 | { 840 | "name": "stdout", 841 | "output_type": "stream", 842 | "text": [ 843 | "experiment1\n", 844 | "experiment2\n", 845 | "experiment2/measurement\n", 846 | "experiment2/simulation\n" 847 | ] 848 | } 849 | ], 850 | "source": [ 851 | "# traverse group hierarchy\n", 852 | "f.visit(lambda x: print(x))" 853 | ] 854 | }, 855 | { 856 | "cell_type": "code", 857 | "execution_count": 34, 858 | "metadata": { 859 | "collapsed": false 860 | }, 861 | "outputs": [ 862 | { 863 | "name": "stdout", 864 | "output_type": "stream", 865 | "text": [ 866 | "experiment1 \n", 867 | "experiment2 \n", 868 | "experiment2/measurement \n", 869 | "experiment2/simulation \n" 870 | ] 871 | } 872 | ], 873 | "source": [ 874 | "# traverse group hierarchy with item & item name accessible in arg\n", 875 | "f.visititems(\n", 876 | " lambda name, \n", 877 | " value: print(name, value))" 878 | ] 879 | }, 880 | { 881 | "cell_type": "code", 882 | "execution_count": 35, 883 | "metadata": { 884 | "collapsed": false 885 | }, 886 | "outputs": [ 887 | { 888 | "data": { 889 | "text/plain": [ 890 | "True" 891 | ] 892 | }, 893 | "execution_count": 35, 894 | "metadata": {}, 895 | "output_type": "execute_result" 896 | } 897 | ], 898 | "source": [ 899 | "# membership testing\n", 900 | "\"experiment1\" in f" 901 | ] 902 | }, 903 | { 904 | "cell_type": "code", 905 | "execution_count": 36, 906 | "metadata": { 907 | "collapsed": false 908 | }, 909 | "outputs": [ 910 | { 911 | "data": { 912 | "text/plain": [ 913 | "True" 914 | ] 915 | }, 916 | "execution_count": 36, 917 | "metadata": {}, 918 | "output_type": "execute_result" 919 | } 920 | ], 921 | "source": [ 922 | "\"simulation\" in f[\"experiment2\"]" 923 | ] 924 | }, 925 | { 926 | "cell_type": "code", 927 | "execution_count": 37, 928 | "metadata": { 929 | "collapsed": false 930 | }, 931 | "outputs": [ 932 | { 933 | "data": { 934 | "text/plain": [ 935 | "False" 936 | ] 937 | }, 938 | "execution_count": 37, 939 | "metadata": {}, 940 | "output_type": "execute_result" 941 | } 942 | ], 943 | "source": [ 944 | "\"experiment3\" in f" 945 | ] 946 | }, 947 | { 948 | "cell_type": "code", 949 | "execution_count": 38, 950 | "metadata": { 951 | "collapsed": true 952 | }, 953 | "outputs": [], 954 | "source": [ 955 | "f.flush()" 956 | ] 957 | }, 958 | { 959 | "cell_type": "code", 960 | "execution_count": 39, 961 | "metadata": { 962 | "collapsed": false 963 | }, 964 | "outputs": [], 965 | "source": [ 966 | "# h5ls = command-line tool for viewing HDF5 contents\n", 967 | "# !h5ls ch18-data.h5" 968 | ] 969 | }, 970 | { 971 | "cell_type": "markdown", 972 | "metadata": {}, 973 | "source": [ 974 | "### HDF5 datasets\n", 975 | "* two main methods to create a dataset in an HDF5 file:\n", 976 | " - easiest: assign NumPy array to an item in an HDF5 group (dictionary index syntax)\n", 977 | " - use __create_dataset__ method." 978 | ] 979 | }, 980 | { 981 | "cell_type": "code", 982 | "execution_count": 40, 983 | "metadata": { 984 | "collapsed": true 985 | }, 986 | "outputs": [], 987 | "source": [ 988 | "data1 = np.arange(10)\n", 989 | "data2 = np.random.randn(100, 100)" 990 | ] 991 | }, 992 | { 993 | "cell_type": "code", 994 | "execution_count": 41, 995 | "metadata": { 996 | "collapsed": false 997 | }, 998 | "outputs": [], 999 | "source": [ 1000 | "f[\"array1\"] = data1\n", 1001 | "f[\"/experiment2/measurement/meas1\"] = data2" 1002 | ] 1003 | }, 1004 | { 1005 | "cell_type": "code", 1006 | "execution_count": 42, 1007 | "metadata": { 1008 | "collapsed": false 1009 | }, 1010 | "outputs": [ 1011 | { 1012 | "name": "stdout", 1013 | "output_type": "stream", 1014 | "text": [ 1015 | "array1 \n", 1016 | "experiment1 \n", 1017 | "experiment2 \n", 1018 | "experiment2/measurement \n", 1019 | "experiment2/measurement/meas1 \n", 1020 | "experiment2/simulation \n" 1021 | ] 1022 | } 1023 | ], 1024 | "source": [ 1025 | "# verify data was save correctly using visititems\n", 1026 | "\n", 1027 | "f.visititems(\n", 1028 | " lambda name, value: print(name, value))" 1029 | ] 1030 | }, 1031 | { 1032 | "cell_type": "code", 1033 | "execution_count": 43, 1034 | "metadata": { 1035 | "collapsed": false 1036 | }, 1037 | "outputs": [ 1038 | { 1039 | "data": { 1040 | "text/plain": [ 1041 | "" 1042 | ] 1043 | }, 1044 | "execution_count": 43, 1045 | "metadata": {}, 1046 | "output_type": "execute_result" 1047 | } 1048 | ], 1049 | "source": [ 1050 | "# to retrieve array1 dataset (in root group)\n", 1051 | "# array1 is a Dataset object, not a NumPy array\n", 1052 | "ds = f[\"array1\"]\n", 1053 | "ds " 1054 | ] 1055 | }, 1056 | { 1057 | "cell_type": "code", 1058 | "execution_count": 44, 1059 | "metadata": { 1060 | "collapsed": false 1061 | }, 1062 | "outputs": [ 1063 | { 1064 | "data": { 1065 | "text/plain": [ 1066 | "('/array1', dtype('int64'), (10,), 10)" 1067 | ] 1068 | }, 1069 | "execution_count": 44, 1070 | "metadata": {}, 1071 | "output_type": "execute_result" 1072 | } 1073 | ], 1074 | "source": [ 1075 | "ds.name, ds.dtype, ds.shape, ds.len()" 1076 | ] 1077 | }, 1078 | { 1079 | "cell_type": "code", 1080 | "execution_count": 45, 1081 | "metadata": { 1082 | "collapsed": false 1083 | }, 1084 | "outputs": [ 1085 | { 1086 | "data": { 1087 | "text/plain": [ 1088 | "array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])" 1089 | ] 1090 | }, 1091 | "execution_count": 45, 1092 | "metadata": {}, 1093 | "output_type": "execute_result" 1094 | } 1095 | ], 1096 | "source": [ 1097 | "ds.value" 1098 | ] 1099 | }, 1100 | { 1101 | "cell_type": "code", 1102 | "execution_count": 46, 1103 | "metadata": { 1104 | "collapsed": false 1105 | }, 1106 | "outputs": [ 1107 | { 1108 | "data": { 1109 | "text/plain": [ 1110 | "" 1111 | ] 1112 | }, 1113 | "execution_count": 46, 1114 | "metadata": {}, 1115 | "output_type": "execute_result" 1116 | } 1117 | ], 1118 | "source": [ 1119 | "# go deeper into hierarchy\n", 1120 | "ds = f[\"/experiment2/measurement/meas1\"]\n", 1121 | "ds" 1122 | ] 1123 | }, 1124 | { 1125 | "cell_type": "code", 1126 | "execution_count": 47, 1127 | "metadata": { 1128 | "collapsed": false 1129 | }, 1130 | "outputs": [ 1131 | { 1132 | "data": { 1133 | "text/plain": [ 1134 | "(dtype('float64'),\n", 1135 | " (100, 100),\n", 1136 | " >)" 1137 | ] 1138 | }, 1139 | "execution_count": 47, 1140 | "metadata": {}, 1141 | "output_type": "execute_result" 1142 | } 1143 | ], 1144 | "source": [ 1145 | "ds.dtype, ds.shape, ds.len" 1146 | ] 1147 | }, 1148 | { 1149 | "cell_type": "code", 1150 | "execution_count": 48, 1151 | "metadata": { 1152 | "collapsed": false 1153 | }, 1154 | "outputs": [ 1155 | { 1156 | "data": { 1157 | "text/plain": [ 1158 | "array([[-1.30652685, 1.65813068, -0.11816405, ..., 1.14110187,\n", 1159 | " 1.46657872, 0.85255194],\n", 1160 | " [-0.59865394, -1.11589699, 0.76666318, ..., -0.51423397,\n", 1161 | " -1.01804188, -0.07785476],\n", 1162 | " [ 0.38273243, -0.03424228, 1.09634685, ..., -0.21673147,\n", 1163 | " -0.9301565 , -0.17858909],\n", 1164 | " ...,\n", 1165 | " [-0.20211703, -0.833231 , 1.73360025, ..., 0.77025427,\n", 1166 | " -0.08612658, -0.85766795],\n", 1167 | " [ 0.6391736 , -0.24720034, 0.23337957, ..., 0.17974832,\n", 1168 | " 0.26792302, 0.7701867 ],\n", 1169 | " [ 1.31951239, -0.42585313, 0.09323029, ..., -0.51270866,\n", 1170 | " -0.44602375, 1.89001412]])" 1171 | ] 1172 | }, 1173 | "execution_count": 48, 1174 | "metadata": {}, 1175 | "output_type": "execute_result" 1176 | } 1177 | ], 1178 | "source": [ 1179 | "# alternative syntax using [...]\n", 1180 | "\n", 1181 | "data_full = ds[...]; data_full" 1182 | ] 1183 | }, 1184 | { 1185 | "cell_type": "code", 1186 | "execution_count": 49, 1187 | "metadata": { 1188 | "collapsed": false 1189 | }, 1190 | "outputs": [ 1191 | { 1192 | "data": { 1193 | "text/plain": [ 1194 | "(numpy.ndarray, (100, 100))" 1195 | ] 1196 | }, 1197 | "execution_count": 49, 1198 | "metadata": {}, 1199 | "output_type": "execute_result" 1200 | } 1201 | ], 1202 | "source": [ 1203 | "type(data_full), data_full.shape" 1204 | ] 1205 | }, 1206 | { 1207 | "cell_type": "code", 1208 | "execution_count": 50, 1209 | "metadata": { 1210 | "collapsed": false 1211 | }, 1212 | "outputs": [ 1213 | { 1214 | "data": { 1215 | "text/plain": [ 1216 | "(100,)" 1217 | ] 1218 | }, 1219 | "execution_count": 50, 1220 | "metadata": {}, 1221 | "output_type": "execute_result" 1222 | } 1223 | ], 1224 | "source": [ 1225 | "# retrieve only first column (a 100 element array)\n", 1226 | "data_col = ds[:, 0]\n", 1227 | "data_col.shape" 1228 | ] 1229 | }, 1230 | { 1231 | "cell_type": "code", 1232 | "execution_count": 51, 1233 | "metadata": { 1234 | "collapsed": false 1235 | }, 1236 | "outputs": [ 1237 | { 1238 | "data": { 1239 | "text/plain": [ 1240 | "array([[ 0.60270766, -0.34804638, -0.813596 , -1.29737966],\n", 1241 | " [ 0.91320192, -1.06343294, 0.22734595, 0.52759738],\n", 1242 | " [ 1.25774422, -0.32775492, 1.4849256 , 0.28005786],\n", 1243 | " [-0.84907287, -0.30000358, 1.79691852, -0.19871506]])" 1244 | ] 1245 | }, 1246 | "execution_count": 51, 1247 | "metadata": {}, 1248 | "output_type": "execute_result" 1249 | } 1250 | ], 1251 | "source": [ 1252 | "# Dataset objects support strided indexing:\n", 1253 | "ds[10:20:3, 10:20:3]" 1254 | ] 1255 | }, 1256 | { 1257 | "cell_type": "code", 1258 | "execution_count": 52, 1259 | "metadata": { 1260 | "collapsed": false 1261 | }, 1262 | "outputs": [ 1263 | { 1264 | "data": { 1265 | "text/plain": [ 1266 | "(3, 100)" 1267 | ] 1268 | }, 1269 | "execution_count": 52, 1270 | "metadata": {}, 1271 | "output_type": "execute_result" 1272 | } 1273 | ], 1274 | "source": [ 1275 | "# Dataset objects support \"fancy\" indexing:\n", 1276 | "ds[[1,2,3], :].shape" 1277 | ] 1278 | }, 1279 | { 1280 | "cell_type": "code", 1281 | "execution_count": 53, 1282 | "metadata": { 1283 | "collapsed": true 1284 | }, 1285 | "outputs": [], 1286 | "source": [ 1287 | "# Boolean masking support\n", 1288 | "mask = ds[:, 0] > 2.0" 1289 | ] 1290 | }, 1291 | { 1292 | "cell_type": "code", 1293 | "execution_count": 54, 1294 | "metadata": { 1295 | "collapsed": false 1296 | }, 1297 | "outputs": [ 1298 | { 1299 | "data": { 1300 | "text/plain": [ 1301 | "((100,), dtype('bool'))" 1302 | ] 1303 | }, 1304 | "execution_count": 54, 1305 | "metadata": {}, 1306 | "output_type": "execute_result" 1307 | } 1308 | ], 1309 | "source": [ 1310 | "mask.shape, mask.dtype" 1311 | ] 1312 | }, 1313 | { 1314 | "cell_type": "code", 1315 | "execution_count": 55, 1316 | "metadata": { 1317 | "collapsed": false 1318 | }, 1319 | "outputs": [ 1320 | { 1321 | "data": { 1322 | "text/plain": [ 1323 | "array([[ 2.04253623, -0.91946118, 0.11467003, -0.1374237 , 1.36552692],\n", 1324 | " [ 2.1041854 , 0.22725706, -1.1291663 , -0.28133197, -0.7394167 ],\n", 1325 | " [ 2.05689385, 0.18041971, -0.06670925, -0.02835398, 0.48480475]])" 1326 | ] 1327 | }, 1328 | "execution_count": 55, 1329 | "metadata": {}, 1330 | "output_type": "execute_result" 1331 | } 1332 | ], 1333 | "source": [ 1334 | "# Single out first 5 columns (index :5 on 2nd axis) for each row\n", 1335 | "# whose 1st column value is larger than 2.\n", 1336 | "ds[mask, :5]" 1337 | ] 1338 | }, 1339 | { 1340 | "cell_type": "markdown", 1341 | "metadata": {}, 1342 | "source": [ 1343 | "### Creating empty data sets, assign and update datasets" 1344 | ] 1345 | }, 1346 | { 1347 | "cell_type": "code", 1348 | "execution_count": 56, 1349 | "metadata": { 1350 | "collapsed": false 1351 | }, 1352 | "outputs": [ 1353 | { 1354 | "data": { 1355 | "text/plain": [ 1356 | "array([0, 2, 2, 4, 7, 3, 7, 2, 4, 1])" 1357 | ] 1358 | }, 1359 | "execution_count": 56, 1360 | "metadata": {}, 1361 | "output_type": "execute_result" 1362 | } 1363 | ], 1364 | "source": [ 1365 | "ds = f.create_dataset(\n", 1366 | " \"array2\", \n", 1367 | " data=np.random.randint(10, size=10))\n", 1368 | "ds.value" 1369 | ] 1370 | }, 1371 | { 1372 | "cell_type": "code", 1373 | "execution_count": 57, 1374 | "metadata": { 1375 | "collapsed": false 1376 | }, 1377 | "outputs": [ 1378 | { 1379 | "data": { 1380 | "text/plain": [ 1381 | "array([[-1., -1., -1., -1., -1.],\n", 1382 | " [-1., -1., -1., -1., -1.],\n", 1383 | " [-1., -1., -1., -1., -1.],\n", 1384 | " [-1., -1., -1., -1., -1.],\n", 1385 | " [-1., -1., -1., -1., -1.]], dtype=float32)" 1386 | ] 1387 | }, 1388 | "execution_count": 57, 1389 | "metadata": {}, 1390 | "output_type": "execute_result" 1391 | } 1392 | ], 1393 | "source": [ 1394 | "ds = f.create_dataset(\n", 1395 | " \"/experiment2/simulation/data1\", \n", 1396 | " shape=(5, 5), \n", 1397 | " fillvalue=-1)\n", 1398 | "ds.value" 1399 | ] 1400 | }, 1401 | { 1402 | "cell_type": "code", 1403 | "execution_count": 58, 1404 | "metadata": { 1405 | "collapsed": false 1406 | }, 1407 | "outputs": [ 1408 | { 1409 | "data": { 1410 | "text/plain": [ 1411 | "" 1412 | ] 1413 | }, 1414 | "execution_count": 58, 1415 | "metadata": {}, 1416 | "output_type": "execute_result" 1417 | } 1418 | ], 1419 | "source": [ 1420 | "ds = f.create_dataset(\n", 1421 | " \"/experiment1/simulation/data1\", \n", 1422 | " shape=(5000, 5000, 5000),\n", 1423 | " fillvalue=0, \n", 1424 | " compression='gzip') # HDF5 = smart compression\n", 1425 | "ds" 1426 | ] 1427 | }, 1428 | { 1429 | "cell_type": "code", 1430 | "execution_count": 59, 1431 | "metadata": { 1432 | "collapsed": false 1433 | }, 1434 | "outputs": [], 1435 | "source": [ 1436 | "ds[:, 0, 0] = np.random.rand(5000)\n", 1437 | "ds[1, :, 0] += np.random.rand(5000)" 1438 | ] 1439 | }, 1440 | { 1441 | "cell_type": "code", 1442 | "execution_count": 60, 1443 | "metadata": { 1444 | "collapsed": false 1445 | }, 1446 | "outputs": [ 1447 | { 1448 | "data": { 1449 | "text/plain": [ 1450 | "array([[0.6939344 , 0. , 0. , 0. , 0. ],\n", 1451 | " [1.4819994 , 0.01639538, 0.54387355, 0.11130908, 0.9928771 ]],\n", 1452 | " dtype=float32)" 1453 | ] 1454 | }, 1455 | "execution_count": 60, 1456 | "metadata": {}, 1457 | "output_type": "execute_result" 1458 | } 1459 | ], 1460 | "source": [ 1461 | "ds[:2, :5, 0]" 1462 | ] 1463 | }, 1464 | { 1465 | "cell_type": "code", 1466 | "execution_count": 61, 1467 | "metadata": { 1468 | "collapsed": false 1469 | }, 1470 | "outputs": [ 1471 | { 1472 | "data": { 1473 | "text/plain": [ 1474 | "0.0" 1475 | ] 1476 | }, 1477 | "execution_count": 61, 1478 | "metadata": {}, 1479 | "output_type": "execute_result" 1480 | } 1481 | ], 1482 | "source": [ 1483 | "# if you need a reminder of the default value of a dataset:\n", 1484 | "ds.fillvalue" 1485 | ] 1486 | }, 1487 | { 1488 | "cell_type": "code", 1489 | "execution_count": 62, 1490 | "metadata": { 1491 | "collapsed": false 1492 | }, 1493 | "outputs": [ 1494 | { 1495 | "name": "stdout", 1496 | "output_type": "stream", 1497 | "text": [ 1498 | "simulation \n", 1499 | "simulation/data1 \n" 1500 | ] 1501 | } 1502 | ], 1503 | "source": [ 1504 | "f[\"experiment1\"].visititems(\n", 1505 | " lambda name, \n", 1506 | " value: print(name, value))" 1507 | ] 1508 | }, 1509 | { 1510 | "cell_type": "code", 1511 | "execution_count": 63, 1512 | "metadata": { 1513 | "collapsed": false 1514 | }, 1515 | "outputs": [ 1516 | { 1517 | "data": { 1518 | "text/plain": [ 1519 | "'ch18-data.h5'" 1520 | ] 1521 | }, 1522 | "execution_count": 63, 1523 | "metadata": {}, 1524 | "output_type": "execute_result" 1525 | } 1526 | ], 1527 | "source": [ 1528 | "f.flush()\n", 1529 | "f.filename" 1530 | ] 1531 | }, 1532 | { 1533 | "cell_type": "code", 1534 | "execution_count": 64, 1535 | "metadata": { 1536 | "collapsed": false 1537 | }, 1538 | "outputs": [ 1539 | { 1540 | "name": "stdout", 1541 | "output_type": "stream", 1542 | "text": [ 1543 | "-rw-rw-r-- 1 bjpcjp bjpcjp 357K May 19 17:47 ch18-data.h5\n" 1544 | ] 1545 | } 1546 | ], 1547 | "source": [ 1548 | "# HDF5 = smart about file compression. Very larget dataset --> relatively small file size\n", 1549 | "!ls -lh ch18-data.h5" 1550 | ] 1551 | }, 1552 | { 1553 | "cell_type": "code", 1554 | "execution_count": 65, 1555 | "metadata": { 1556 | "collapsed": false 1557 | }, 1558 | "outputs": [], 1559 | "source": [ 1560 | "# Deleting items from HDF5 file:\n", 1561 | "del f[\"/experiment1/simulation/data1\"]" 1562 | ] 1563 | }, 1564 | { 1565 | "cell_type": "code", 1566 | "execution_count": 66, 1567 | "metadata": { 1568 | "collapsed": false 1569 | }, 1570 | "outputs": [ 1571 | { 1572 | "name": "stdout", 1573 | "output_type": "stream", 1574 | "text": [ 1575 | "simulation \n" 1576 | ] 1577 | } 1578 | ], 1579 | "source": [ 1580 | "# data1 should now be gone\n", 1581 | "f[\"experiment1\"].visititems(\n", 1582 | " lambda name, \n", 1583 | " value: print(name, value))" 1584 | ] 1585 | }, 1586 | { 1587 | "cell_type": "code", 1588 | "execution_count": 67, 1589 | "metadata": { 1590 | "collapsed": false 1591 | }, 1592 | "outputs": [], 1593 | "source": [ 1594 | "f.close()" 1595 | ] 1596 | }, 1597 | { 1598 | "cell_type": "markdown", 1599 | "metadata": {}, 1600 | "source": [ 1601 | "### HDF5 Atributes\n", 1602 | "\n", 1603 | "* Attributes make HDF5 great for annotating data & self-describing data (metadata)." 1604 | ] 1605 | }, 1606 | { 1607 | "cell_type": "code", 1608 | "execution_count": 68, 1609 | "metadata": { 1610 | "collapsed": false 1611 | }, 1612 | "outputs": [ 1613 | { 1614 | "data": { 1615 | "text/plain": [ 1616 | "" 1617 | ] 1618 | }, 1619 | "execution_count": 68, 1620 | "metadata": {}, 1621 | "output_type": "execute_result" 1622 | } 1623 | ], 1624 | "source": [ 1625 | "f = h5py.File(\"ch18-data.h5\")\n", 1626 | "f.attrs" 1627 | ] 1628 | }, 1629 | { 1630 | "cell_type": "code", 1631 | "execution_count": 69, 1632 | "metadata": { 1633 | "collapsed": true 1634 | }, 1635 | "outputs": [], 1636 | "source": [ 1637 | "# create an attribute\n", 1638 | "f.attrs[\"desc\"] = \"Result sets from experiments and simulations\"" 1639 | ] 1640 | }, 1641 | { 1642 | "cell_type": "code", 1643 | "execution_count": 70, 1644 | "metadata": { 1645 | "collapsed": true 1646 | }, 1647 | "outputs": [], 1648 | "source": [ 1649 | "f[\"experiment1\"].attrs[\"date\"] = \"2015-1-1\"\n", 1650 | "f[\"experiment2\"].attrs[\"date\"] = \"2015-1-2\"\n", 1651 | "\n", 1652 | "f[\"experiment2/simulation/data1\"].attrs[\"k\"] = 1.5\n", 1653 | "f[\"experiment2/simulation/data1\"].attrs[\"T\"] = 1000" 1654 | ] 1655 | }, 1656 | { 1657 | "cell_type": "code", 1658 | "execution_count": 71, 1659 | "metadata": { 1660 | "collapsed": false 1661 | }, 1662 | "outputs": [ 1663 | { 1664 | "data": { 1665 | "text/plain": [ 1666 | "['date']" 1667 | ] 1668 | }, 1669 | "execution_count": 71, 1670 | "metadata": {}, 1671 | "output_type": "execute_result" 1672 | } 1673 | ], 1674 | "source": [ 1675 | "list(f[\"experiment1\"].attrs.keys())" 1676 | ] 1677 | }, 1678 | { 1679 | "cell_type": "code", 1680 | "execution_count": 72, 1681 | "metadata": { 1682 | "collapsed": false 1683 | }, 1684 | "outputs": [ 1685 | { 1686 | "data": { 1687 | "text/plain": [ 1688 | "[('k', 1.5), ('T', 1000)]" 1689 | ] 1690 | }, 1691 | "execution_count": 72, 1692 | "metadata": {}, 1693 | "output_type": "execute_result" 1694 | } 1695 | ], 1696 | "source": [ 1697 | "list(f[\"experiment2/simulation/data1\"].attrs.items())" 1698 | ] 1699 | }, 1700 | { 1701 | "cell_type": "code", 1702 | "execution_count": 73, 1703 | "metadata": { 1704 | "collapsed": false 1705 | }, 1706 | "outputs": [ 1707 | { 1708 | "data": { 1709 | "text/plain": [ 1710 | "True" 1711 | ] 1712 | }, 1713 | "execution_count": 73, 1714 | "metadata": {}, 1715 | "output_type": "execute_result" 1716 | } 1717 | ], 1718 | "source": [ 1719 | "# Existence testing:\n", 1720 | "\"T\" in f[\"experiment2/simulation/data1\"].attrs" 1721 | ] 1722 | }, 1723 | { 1724 | "cell_type": "code", 1725 | "execution_count": 74, 1726 | "metadata": { 1727 | "collapsed": true 1728 | }, 1729 | "outputs": [], 1730 | "source": [ 1731 | "# Deleting existing attributes:\n", 1732 | "del f[\"experiment2/simulation/data1\"].attrs[\"T\"]" 1733 | ] 1734 | }, 1735 | { 1736 | "cell_type": "code", 1737 | "execution_count": 75, 1738 | "metadata": { 1739 | "collapsed": false 1740 | }, 1741 | "outputs": [ 1742 | { 1743 | "data": { 1744 | "text/plain": [ 1745 | "False" 1746 | ] 1747 | }, 1748 | "execution_count": 75, 1749 | "metadata": {}, 1750 | "output_type": "execute_result" 1751 | } 1752 | ], 1753 | "source": [ 1754 | "\"T\" in f[\"experiment2/simulation/data1\"].attrs" 1755 | ] 1756 | }, 1757 | { 1758 | "cell_type": "code", 1759 | "execution_count": 76, 1760 | "metadata": { 1761 | "collapsed": false 1762 | }, 1763 | "outputs": [], 1764 | "source": [ 1765 | "f[\"experiment2/simulation/data1\"].attrs[\"t\"] = np.array([1, 2, 3])" 1766 | ] 1767 | }, 1768 | { 1769 | "cell_type": "code", 1770 | "execution_count": 77, 1771 | "metadata": { 1772 | "collapsed": false 1773 | }, 1774 | "outputs": [ 1775 | { 1776 | "data": { 1777 | "text/plain": [ 1778 | "array([1, 2, 3])" 1779 | ] 1780 | }, 1781 | "execution_count": 77, 1782 | "metadata": {}, 1783 | "output_type": "execute_result" 1784 | } 1785 | ], 1786 | "source": [ 1787 | "f[\"experiment2/simulation/data1\"].attrs[\"t\"]" 1788 | ] 1789 | }, 1790 | { 1791 | "cell_type": "code", 1792 | "execution_count": 78, 1793 | "metadata": { 1794 | "collapsed": true 1795 | }, 1796 | "outputs": [], 1797 | "source": [ 1798 | "f.close()" 1799 | ] 1800 | }, 1801 | { 1802 | "cell_type": "markdown", 1803 | "metadata": {}, 1804 | "source": [ 1805 | "### pytables\n", 1806 | "* alternate HDF5 interface" 1807 | ] 1808 | }, 1809 | { 1810 | "cell_type": "code", 1811 | "execution_count": 79, 1812 | "metadata": { 1813 | "collapsed": false 1814 | }, 1815 | "outputs": [], 1816 | "source": [ 1817 | "df = pd.read_csv(\n", 1818 | " \"ch18-playerstats-2013-2014-top30.csv\", skiprows=1)\n", 1819 | "\n", 1820 | "df = df.set_index(\"Rank\")" 1821 | ] 1822 | }, 1823 | { 1824 | "cell_type": "code", 1825 | "execution_count": 80, 1826 | "metadata": { 1827 | "collapsed": false 1828 | }, 1829 | "outputs": [ 1830 | { 1831 | "data": { 1832 | "text/html": [ 1833 | "
\n", 1834 | "\n", 1847 | "\n", 1848 | " \n", 1849 | " \n", 1850 | " \n", 1851 | " \n", 1852 | " \n", 1853 | " \n", 1854 | " \n", 1855 | " \n", 1856 | " \n", 1857 | " \n", 1858 | " \n", 1859 | " \n", 1860 | " \n", 1861 | " \n", 1862 | " \n", 1863 | " \n", 1864 | " \n", 1865 | " \n", 1866 | " \n", 1867 | " \n", 1868 | " \n", 1869 | " \n", 1870 | " \n", 1871 | " \n", 1872 | " \n", 1873 | " \n", 1874 | " \n", 1875 | " \n", 1876 | " \n", 1877 | " \n", 1878 | " \n", 1879 | " \n", 1880 | " \n", 1881 | " \n", 1882 | " \n", 1883 | " \n", 1884 | " \n", 1885 | " \n", 1886 | " \n", 1887 | " \n", 1888 | " \n", 1889 | " \n", 1890 | " \n", 1891 | " \n", 1892 | " \n", 1893 | " \n", 1894 | " \n", 1895 | " \n", 1896 | " \n", 1897 | " \n", 1898 | " \n", 1899 | " \n", 1900 | " \n", 1901 | " \n", 1902 | " \n", 1903 | " \n", 1904 | " \n", 1905 | " \n", 1906 | " \n", 1907 | " \n", 1908 | " \n", 1909 | " \n", 1910 | " \n", 1911 | " \n", 1912 | " \n", 1913 | " \n", 1914 | " \n", 1915 | " \n", 1916 | " \n", 1917 | " \n", 1918 | " \n", 1919 | " \n", 1920 | " \n", 1921 | " \n", 1922 | " \n", 1923 | " \n", 1924 | " \n", 1925 | " \n", 1926 | " \n", 1927 | " \n", 1928 | " \n", 1929 | "
PlayerPosGPPGAS%Shift/GP
Rank
1Sidney CrosbyC80104366813.924.0
2Ryan GetzlafC7787315615.225.2
3Claude GirouxC8286285812.625.1
4Tyler SeguinC8084374712.623.4
5Corey PerryR8182433915.423.2
\n", 1930 | "
" 1931 | ], 1932 | "text/plain": [ 1933 | " Player Pos GP P G A S% Shift/GP\n", 1934 | "Rank \n", 1935 | "1 Sidney Crosby C 80 104 36 68 13.9 24.0\n", 1936 | "2 Ryan Getzlaf C 77 87 31 56 15.2 25.2\n", 1937 | "3 Claude Giroux C 82 86 28 58 12.6 25.1\n", 1938 | "4 Tyler Seguin C 80 84 37 47 12.6 23.4\n", 1939 | "5 Corey Perry R 81 82 43 39 15.4 23.2" 1940 | ] 1941 | }, 1942 | "execution_count": 80, 1943 | "metadata": {}, 1944 | "output_type": "execute_result" 1945 | } 1946 | ], 1947 | "source": [ 1948 | "df[[\"Player\", \"Pos\", \"GP\", \"P\", \"G\", \"A\", \"S%\", \"Shift/GP\"]].head(5)" 1949 | ] 1950 | }, 1951 | { 1952 | "cell_type": "code", 1953 | "execution_count": 81, 1954 | "metadata": { 1955 | "collapsed": false 1956 | }, 1957 | "outputs": [], 1958 | "source": [ 1959 | "# create new PyTables HDF5 file\n", 1960 | "f = tables.open_file(\n", 1961 | " \"ch18-playerstats-2013-2014.h5\", mode=\"w\")" 1962 | ] 1963 | }, 1964 | { 1965 | "cell_type": "code", 1966 | "execution_count": 82, 1967 | "metadata": { 1968 | "collapsed": false 1969 | }, 1970 | "outputs": [ 1971 | { 1972 | "data": { 1973 | "text/plain": [ 1974 | "/season_2013_2014 (Group) 'NHL player statistics for the 2013/2014 season'\n", 1975 | " children := []" 1976 | ] 1977 | }, 1978 | "execution_count": 82, 1979 | "metadata": {}, 1980 | "output_type": "execute_result" 1981 | } 1982 | ], 1983 | "source": [ 1984 | "# create HDF5 groups\n", 1985 | "grp = f.create_group(\n", 1986 | " \"/\", \n", 1987 | " \"season_2013_2014\", \n", 1988 | " title=\"NHL player statistics for the 2013/2014 season\")\n", 1989 | "grp" 1990 | ] 1991 | }, 1992 | { 1993 | "cell_type": "code", 1994 | "execution_count": 83, 1995 | "metadata": { 1996 | "collapsed": false 1997 | }, 1998 | "outputs": [ 1999 | { 2000 | "data": { 2001 | "text/plain": [ 2002 | "/ (RootGroup) ''\n", 2003 | " children := ['season_2013_2014' (Group)]" 2004 | ] 2005 | }, 2006 | "execution_count": 83, 2007 | "metadata": {}, 2008 | "output_type": "execute_result" 2009 | } 2010 | ], 2011 | "source": [ 2012 | "# Unlike h5py, PyTables file objects do not represent root groups in the HDF5 file.\n", 2013 | "# Use the root attribute to access it instead.\n", 2014 | "f.root" 2015 | ] 2016 | }, 2017 | { 2018 | "cell_type": "code", 2019 | "execution_count": 84, 2020 | "metadata": { 2021 | "collapsed": false 2022 | }, 2023 | "outputs": [], 2024 | "source": [ 2025 | "# PyTables makes it easy to create mixed column types.\n", 2026 | "\n", 2027 | "class PlayerStat(tables.IsDescription):\n", 2028 | " player = tables.StringCol(20, dflt=\"\")\n", 2029 | " position = tables.StringCol(1, dflt=\"C\")\n", 2030 | " games_played = tables.UInt8Col(dflt=0)\n", 2031 | " points = tables.UInt16Col(dflt=0)\n", 2032 | " goals = tables.UInt16Col(dflt=0)\n", 2033 | " assists = tables.UInt16Col(dflt=0)\n", 2034 | " shooting_percentage = tables.Float64Col(dflt=0.0)\n", 2035 | " shifts_per_game_played = tables.Float64Col(dflt=0.0) " 2036 | ] 2037 | }, 2038 | { 2039 | "cell_type": "code", 2040 | "execution_count": 85, 2041 | "metadata": { 2042 | "collapsed": false 2043 | }, 2044 | "outputs": [], 2045 | "source": [ 2046 | "top30_table = f.create_table(\n", 2047 | " grp, 'top30', PlayerStat, \"Top 30 point leaders\")" 2048 | ] 2049 | }, 2050 | { 2051 | "cell_type": "code", 2052 | "execution_count": 86, 2053 | "metadata": { 2054 | "collapsed": true 2055 | }, 2056 | "outputs": [ 2057 | { 2058 | "data": { 2059 | "text/plain": [ 2060 | "tables.tableextension.Row" 2061 | ] 2062 | }, 2063 | "execution_count": 86, 2064 | "metadata": {}, 2065 | "output_type": "execute_result" 2066 | } 2067 | ], 2068 | "source": [ 2069 | "playerstat = top30_table.row\n", 2070 | "type(playerstat)" 2071 | ] 2072 | }, 2073 | { 2074 | "cell_type": "code", 2075 | "execution_count": 87, 2076 | "metadata": { 2077 | "collapsed": false 2078 | }, 2079 | "outputs": [], 2080 | "source": [ 2081 | "# to insert data into table, use row attribute of table object\n", 2082 | "# when row object is initialized, use append to insert data.\n", 2083 | "\n", 2084 | "for index, row_series in df.iterrows():\n", 2085 | " playerstat[\"player\"] = row_series[\"Player\"] \n", 2086 | " playerstat[\"position\"] = row_series[\"Pos\"] \n", 2087 | " playerstat[\"games_played\"] = row_series[\"GP\"] \n", 2088 | " playerstat[\"points\"] = row_series[\"P\"] \n", 2089 | " playerstat[\"goals\"] = row_series[\"G\"]\n", 2090 | " playerstat[\"assists\"] = row_series[\"A\"] \n", 2091 | " playerstat[\"shooting_percentage\"] = row_series[\"S%\"]\n", 2092 | " playerstat[\"shifts_per_game_played\"] = row_series[\"Shift/GP\"]\n", 2093 | " playerstat.append()" 2094 | ] 2095 | }, 2096 | { 2097 | "cell_type": "code", 2098 | "execution_count": 88, 2099 | "metadata": { 2100 | "collapsed": false 2101 | }, 2102 | "outputs": [], 2103 | "source": [ 2104 | "# flush forces a file write\n", 2105 | "top30_table.flush()" 2106 | ] 2107 | }, 2108 | { 2109 | "cell_type": "code", 2110 | "execution_count": 89, 2111 | "metadata": { 2112 | "collapsed": false 2113 | }, 2114 | "outputs": [ 2115 | { 2116 | "data": { 2117 | "text/plain": [ 2118 | "array([b'Sidney Crosby', b'Ryan Getzlaf', b'Claude Giroux',\n", 2119 | " b'Tyler Seguin', b'Corey Perry'], dtype='|S20')" 2120 | ] 2121 | }, 2122 | "execution_count": 89, 2123 | "metadata": {}, 2124 | "output_type": "execute_result" 2125 | } 2126 | ], 2127 | "source": [ 2128 | "# access table data using cols attribute\n", 2129 | "top30_table.cols.player[:5]" 2130 | ] 2131 | }, 2132 | { 2133 | "cell_type": "code", 2134 | "execution_count": 90, 2135 | "metadata": { 2136 | "collapsed": false 2137 | }, 2138 | "outputs": [ 2139 | { 2140 | "data": { 2141 | "text/plain": [ 2142 | "array([104, 87, 86, 84, 82], dtype=uint16)" 2143 | ] 2144 | }, 2145 | "execution_count": 90, 2146 | "metadata": {}, 2147 | "output_type": "execute_result" 2148 | } 2149 | ], 2150 | "source": [ 2151 | "top30_table.cols.points[:5]" 2152 | ] 2153 | }, 2154 | { 2155 | "cell_type": "code", 2156 | "execution_count": 91, 2157 | "metadata": { 2158 | "collapsed": true 2159 | }, 2160 | "outputs": [], 2161 | "source": [ 2162 | "# Use iterrows to create an iterator for row-wise data access.\n", 2163 | "def print_playerstat(row):\n", 2164 | " print(\"%20s\\t%s\\t%s\\t%s\" %\n", 2165 | " (row[\"player\"].decode('UTF-8'), row[\"points\"], row[\"goals\"], row[\"assists\"]))" 2166 | ] 2167 | }, 2168 | { 2169 | "cell_type": "code", 2170 | "execution_count": 92, 2171 | "metadata": { 2172 | "collapsed": false 2173 | }, 2174 | "outputs": [ 2175 | { 2176 | "name": "stdout", 2177 | "output_type": "stream", 2178 | "text": [ 2179 | " Sidney Crosby\t104\t36\t68\n", 2180 | " Ryan Getzlaf\t87\t31\t56\n", 2181 | " Claude Giroux\t86\t28\t58\n", 2182 | " Tyler Seguin\t84\t37\t47\n", 2183 | " Corey Perry\t82\t43\t39\n", 2184 | " Phil Kessel\t80\t37\t43\n", 2185 | " Taylor Hall\t80\t27\t53\n", 2186 | " Alex Ovechkin\t79\t51\t28\n", 2187 | " Joe Pavelski\t79\t41\t38\n", 2188 | " Jamie Benn\t79\t34\t45\n", 2189 | " Nicklas Backstrom\t79\t18\t61\n", 2190 | " Patrick Sharp\t78\t34\t44\n", 2191 | " Joe Thornton\t76\t11\t65\n", 2192 | " Erik Karlsson\t74\t20\t54\n", 2193 | " Evgeni Malkin\t72\t23\t49\n", 2194 | " Patrick Marleau\t70\t33\t37\n", 2195 | " Anze Kopitar\t70\t29\t41\n", 2196 | " Matt Duchene\t70\t23\t47\n", 2197 | " Martin St. Louis\t69\t30\t39\n", 2198 | " Patrick Kane\t69\t29\t40\n", 2199 | " Blake Wheeler\t69\t28\t41\n", 2200 | " Kyle Okposo\t69\t27\t42\n", 2201 | " David Krejci\t69\t19\t50\n", 2202 | " Chris Kunitz\t68\t35\t33\n", 2203 | " Jonathan Toews\t68\t28\t40\n", 2204 | " Thomas Vanek\t68\t27\t41\n", 2205 | " Jaromir Jagr\t67\t24\t43\n", 2206 | " John Tavares\t66\t24\t42\n", 2207 | " Jason Spezza\t66\t23\t43\n", 2208 | " Jordan Eberle\t65\t28\t37\n" 2209 | ] 2210 | } 2211 | ], 2212 | "source": [ 2213 | "for row in top30_table.iterrows():\n", 2214 | " print_playerstat(row)" 2215 | ] 2216 | }, 2217 | { 2218 | "cell_type": "code", 2219 | "execution_count": 93, 2220 | "metadata": { 2221 | "collapsed": false 2222 | }, 2223 | "outputs": [ 2224 | { 2225 | "name": "stdout", 2226 | "output_type": "stream", 2227 | "text": [ 2228 | " Phil Kessel\t80\t37\t43\n", 2229 | " Taylor Hall\t80\t27\t53\n", 2230 | " Alex Ovechkin\t79\t51\t28\n", 2231 | " Joe Pavelski\t79\t41\t38\n", 2232 | " Jamie Benn\t79\t34\t45\n", 2233 | " Nicklas Backstrom\t79\t18\t61\n", 2234 | " Patrick Sharp\t78\t34\t44\n", 2235 | " Joe Thornton\t76\t11\t65\n" 2236 | ] 2237 | } 2238 | ], 2239 | "source": [ 2240 | "# PyTables support SQL-like queries.\n", 2241 | "for row in top30_table.where(\"(points > 75) & (points <= 80)\"):\n", 2242 | " print_playerstat(row)" 2243 | ] 2244 | }, 2245 | { 2246 | "cell_type": "code", 2247 | "execution_count": 94, 2248 | "metadata": { 2249 | "collapsed": false 2250 | }, 2251 | "outputs": [ 2252 | { 2253 | "name": "stdout", 2254 | "output_type": "stream", 2255 | "text": [ 2256 | " Alex Ovechkin\t79\t51\t28\n", 2257 | " Joe Pavelski\t79\t41\t38\n" 2258 | ] 2259 | } 2260 | ], 2261 | "source": [ 2262 | "# PyTables queries using multiple column conditions:\n", 2263 | "for row in top30_table.where(\"(goals > 40) & (points < 80)\"):\n", 2264 | " print_playerstat(row)" 2265 | ] 2266 | }, 2267 | { 2268 | "cell_type": "code", 2269 | "execution_count": 95, 2270 | "metadata": { 2271 | "collapsed": false 2272 | }, 2273 | "outputs": [ 2274 | { 2275 | "data": { 2276 | "text/plain": [ 2277 | "File(filename=ch18-playerstats-2013-2014.h5, title='', mode='w', root_uep='/', filters=Filters(complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None))\n", 2278 | "/ (RootGroup) ''\n", 2279 | "/season_2013_2014 (Group) 'NHL player statistics for the 2013/2014 season'\n", 2280 | "/season_2013_2014/top30 (Table(30,)) 'Top 30 point leaders'\n", 2281 | " description := {\n", 2282 | " \"assists\": UInt16Col(shape=(), dflt=0, pos=0),\n", 2283 | " \"games_played\": UInt8Col(shape=(), dflt=0, pos=1),\n", 2284 | " \"goals\": UInt16Col(shape=(), dflt=0, pos=2),\n", 2285 | " \"player\": StringCol(itemsize=20, shape=(), dflt=b'', pos=3),\n", 2286 | " \"points\": UInt16Col(shape=(), dflt=0, pos=4),\n", 2287 | " \"position\": StringCol(itemsize=1, shape=(), dflt=b'C', pos=5),\n", 2288 | " \"shifts_per_game_played\": Float64Col(shape=(), dflt=0.0, pos=6),\n", 2289 | " \"shooting_percentage\": Float64Col(shape=(), dflt=0.0, pos=7)}\n", 2290 | " byteorder := 'little'\n", 2291 | " chunkshape := (1489,)" 2292 | ] 2293 | }, 2294 | "execution_count": 95, 2295 | "metadata": {}, 2296 | "output_type": "execute_result" 2297 | } 2298 | ], 2299 | "source": [ 2300 | "# inspect HDF5 file structure\n", 2301 | "f" 2302 | ] 2303 | }, 2304 | { 2305 | "cell_type": "code", 2306 | "execution_count": 96, 2307 | "metadata": { 2308 | "collapsed": true 2309 | }, 2310 | "outputs": [], 2311 | "source": [ 2312 | "# done? let's flush the buffers, force a write & close the file.\n", 2313 | "f.flush()\n", 2314 | "f.close()" 2315 | ] 2316 | }, 2317 | { 2318 | "cell_type": "markdown", 2319 | "metadata": {}, 2320 | "source": [ 2321 | "### Pandas hdfstore\n", 2322 | "* 3rd method to use HDF5 files - using HDFStore object in Pandas\n", 2323 | "* HDFStore object can be used as a dictionary for Pandas dataframes." 2324 | ] 2325 | }, 2326 | { 2327 | "cell_type": "code", 2328 | "execution_count": 97, 2329 | "metadata": { 2330 | "collapsed": true 2331 | }, 2332 | "outputs": [], 2333 | "source": [ 2334 | "import pandas as pd" 2335 | ] 2336 | }, 2337 | { 2338 | "cell_type": "code", 2339 | "execution_count": 98, 2340 | "metadata": { 2341 | "collapsed": true 2342 | }, 2343 | "outputs": [], 2344 | "source": [ 2345 | "store = pd.HDFStore('store.h5')" 2346 | ] 2347 | }, 2348 | { 2349 | "cell_type": "code", 2350 | "execution_count": 99, 2351 | "metadata": { 2352 | "collapsed": true 2353 | }, 2354 | "outputs": [], 2355 | "source": [ 2356 | "df = pd.DataFrame(np.random.rand(5,5))\n", 2357 | "store[\"df1\"] = df" 2358 | ] 2359 | }, 2360 | { 2361 | "cell_type": "code", 2362 | "execution_count": 101, 2363 | "metadata": { 2364 | "collapsed": false 2365 | }, 2366 | "outputs": [], 2367 | "source": [ 2368 | "df = pd.read_csv(\"ch18-playerstats-2013-2014-top30.csv\", skiprows=1)\n", 2369 | "store[\"df2\"] = df" 2370 | ] 2371 | }, 2372 | { 2373 | "cell_type": "code", 2374 | "execution_count": 102, 2375 | "metadata": { 2376 | "collapsed": false 2377 | }, 2378 | "outputs": [ 2379 | { 2380 | "data": { 2381 | "text/plain": [ 2382 | "['/df1', '/df2']" 2383 | ] 2384 | }, 2385 | "execution_count": 102, 2386 | "metadata": {}, 2387 | "output_type": "execute_result" 2388 | } 2389 | ], 2390 | "source": [ 2391 | "# What's in the HDFstore object?\n", 2392 | "store.keys()" 2393 | ] 2394 | }, 2395 | { 2396 | "cell_type": "code", 2397 | "execution_count": 103, 2398 | "metadata": { 2399 | "collapsed": false 2400 | }, 2401 | "outputs": [ 2402 | { 2403 | "data": { 2404 | "text/plain": [ 2405 | "True" 2406 | ] 2407 | }, 2408 | "execution_count": 103, 2409 | "metadata": {}, 2410 | "output_type": "execute_result" 2411 | } 2412 | ], 2413 | "source": [ 2414 | "# test for existence\n", 2415 | "'df2' in store" 2416 | ] 2417 | }, 2418 | { 2419 | "cell_type": "code", 2420 | "execution_count": 104, 2421 | "metadata": { 2422 | "collapsed": false 2423 | }, 2424 | "outputs": [], 2425 | "source": [ 2426 | "# retrieve an object\n", 2427 | "df = store[\"df1\"]" 2428 | ] 2429 | }, 2430 | { 2431 | "cell_type": "code", 2432 | "execution_count": 105, 2433 | "metadata": { 2434 | "collapsed": false 2435 | }, 2436 | "outputs": [ 2437 | { 2438 | "data": { 2439 | "text/plain": [ 2440 | "/ (RootGroup) ''\n", 2441 | " children := ['df1' (Group), 'df2' (Group)]" 2442 | ] 2443 | }, 2444 | "execution_count": 105, 2445 | "metadata": {}, 2446 | "output_type": "execute_result" 2447 | } 2448 | ], 2449 | "source": [ 2450 | "# access underlying HDF5 handle\n", 2451 | "store.root" 2452 | ] 2453 | }, 2454 | { 2455 | "cell_type": "code", 2456 | "execution_count": 106, 2457 | "metadata": { 2458 | "collapsed": true 2459 | }, 2460 | "outputs": [], 2461 | "source": [ 2462 | "store.close()" 2463 | ] 2464 | }, 2465 | { 2466 | "cell_type": "code", 2467 | "execution_count": 107, 2468 | "metadata": { 2469 | "collapsed": true 2470 | }, 2471 | "outputs": [], 2472 | "source": [ 2473 | "# HDF5 is a std file format. We can open a file & see how data is arranged.\n", 2474 | "f = h5py.File(\"store.h5\")" 2475 | ] 2476 | }, 2477 | { 2478 | "cell_type": "code", 2479 | "execution_count": 108, 2480 | "metadata": { 2481 | "collapsed": false 2482 | }, 2483 | "outputs": [ 2484 | { 2485 | "name": "stdout", 2486 | "output_type": "stream", 2487 | "text": [ 2488 | "df1 \t\t\t \n", 2489 | "df1/axis0 \t\t \n", 2490 | "df1/axis1 \t\t \n", 2491 | "df1/block0_items \t \n", 2492 | "df1/block0_values \t \n", 2493 | "df2 \t\t\t \n", 2494 | "df2/axis0 \t\t \n", 2495 | "df2/axis1 \t\t \n", 2496 | "df2/block0_items \t \n", 2497 | "df2/block0_values \t \n", 2498 | "df2/block1_items \t \n", 2499 | "df2/block1_values \t \n", 2500 | "df2/block2_items \t \n", 2501 | "df2/block2_values \t \n" 2502 | ] 2503 | } 2504 | ], 2505 | "source": [ 2506 | "f.visititems(\n", 2507 | " lambda x, y: \n", 2508 | " print(x, \"\\t\" * int(3 - len(str(x))//8), y))" 2509 | ] 2510 | }, 2511 | { 2512 | "cell_type": "code", 2513 | "execution_count": 109, 2514 | "metadata": { 2515 | "collapsed": false 2516 | }, 2517 | "outputs": [ 2518 | { 2519 | "data": { 2520 | "text/plain": [ 2521 | "array([b'S%', b'Shift/GP', b'FO%'], dtype='|S8')" 2522 | ] 2523 | }, 2524 | "execution_count": 109, 2525 | "metadata": {}, 2526 | "output_type": "execute_result" 2527 | } 2528 | ], 2529 | "source": [ 2530 | "# HDF5Store objects store dataframes in distinct groups.\n", 2531 | "# Each dataframe is split into heterogeneous \"blocks\" with columns grouped by data type\n", 2532 | "# Column names & values are stored in separate HDF5 datasets.\n", 2533 | "\n", 2534 | "f[\"/df2/block0_items\"].value " 2535 | ] 2536 | }, 2537 | { 2538 | "cell_type": "code", 2539 | "execution_count": 110, 2540 | "metadata": { 2541 | "collapsed": false 2542 | }, 2543 | "outputs": [ 2544 | { 2545 | "data": { 2546 | "text/plain": [ 2547 | "array([[13.9, 24. , 52.5],\n", 2548 | " [15.2, 25.2, 49. ],\n", 2549 | " [12.6, 25.1, 52.9]])" 2550 | ] 2551 | }, 2552 | "execution_count": 110, 2553 | "metadata": {}, 2554 | "output_type": "execute_result" 2555 | } 2556 | ], 2557 | "source": [ 2558 | "f[\"/df2/block0_values\"][:3]" 2559 | ] 2560 | }, 2561 | { 2562 | "cell_type": "code", 2563 | "execution_count": 111, 2564 | "metadata": { 2565 | "collapsed": false 2566 | }, 2567 | "outputs": [ 2568 | { 2569 | "data": { 2570 | "text/plain": [ 2571 | "array([b'Rank', b'GP', b'G', b'A', b'P', b'+/-', b'PIM', b'PPG', b'PPP',\n", 2572 | " b'SHG', b'SHP', b'GW', b'OT', b'S'], dtype='|S4')" 2573 | ] 2574 | }, 2575 | "execution_count": 111, 2576 | "metadata": {}, 2577 | "output_type": "execute_result" 2578 | } 2579 | ], 2580 | "source": [ 2581 | "f[\"/df2/block1_items\"].value " 2582 | ] 2583 | }, 2584 | { 2585 | "cell_type": "code", 2586 | "execution_count": 112, 2587 | "metadata": { 2588 | "collapsed": false 2589 | }, 2590 | "outputs": [ 2591 | { 2592 | "data": { 2593 | "text/plain": [ 2594 | "array([[ 1, 80, 36, 68, 104],\n", 2595 | " [ 2, 77, 31, 56, 87],\n", 2596 | " [ 3, 82, 28, 58, 86]])" 2597 | ] 2598 | }, 2599 | "execution_count": 112, 2600 | "metadata": {}, 2601 | "output_type": "execute_result" 2602 | } 2603 | ], 2604 | "source": [ 2605 | "f[\"/df2/block1_values\"][:3, :5]" 2606 | ] 2607 | }, 2608 | { 2609 | "cell_type": "markdown", 2610 | "metadata": {}, 2611 | "source": [ 2612 | "### JSON\n", 2613 | "* Human-readable, lightweight plain-text format\n", 2614 | "* Ideal for storing lists & dictionaries - no tabular data restrictions" 2615 | ] 2616 | }, 2617 | { 2618 | "cell_type": "code", 2619 | "execution_count": 113, 2620 | "metadata": { 2621 | "collapsed": true 2622 | }, 2623 | "outputs": [ 2624 | { 2625 | "data": { 2626 | "text/plain": [ 2627 | "'[\"string\", 1.0, 2, null]'" 2628 | ] 2629 | }, 2630 | "execution_count": 113, 2631 | "metadata": {}, 2632 | "output_type": "execute_result" 2633 | } 2634 | ], 2635 | "source": [ 2636 | "# storing Python lists as JSON strings\n", 2637 | "data = [\"string\", 1.0, 2, None]\n", 2638 | "data_json = json.dumps(data)\n", 2639 | "data_json" 2640 | ] 2641 | }, 2642 | { 2643 | "cell_type": "code", 2644 | "execution_count": 114, 2645 | "metadata": { 2646 | "collapsed": true 2647 | }, 2648 | "outputs": [ 2649 | { 2650 | "data": { 2651 | "text/plain": [ 2652 | "['string', 1.0, 2, None]" 2653 | ] 2654 | }, 2655 | "execution_count": 114, 2656 | "metadata": {}, 2657 | "output_type": "execute_result" 2658 | } 2659 | ], 2660 | "source": [ 2661 | "# parsing JSON string to a Python object\n", 2662 | "data2 = json.loads(data_json)\n", 2663 | "data2" 2664 | ] 2665 | }, 2666 | { 2667 | "cell_type": "code", 2668 | "execution_count": 115, 2669 | "metadata": { 2670 | "collapsed": true 2671 | }, 2672 | "outputs": [ 2673 | { 2674 | "name": "stdout", 2675 | "output_type": "stream", 2676 | "text": [ 2677 | "{\"one\": 1, \"two\": 2.0, \"three\": \"three\"}\n" 2678 | ] 2679 | } 2680 | ], 2681 | "source": [ 2682 | "# Storing Python dictionaries as JSON strings\n", 2683 | "data = {\"one\": 1, \"two\": 2.0, \"three\": \"three\"}\n", 2684 | "data_json = json.dumps(data)\n", 2685 | "print(data_json)" 2686 | ] 2687 | }, 2688 | { 2689 | "cell_type": "code", 2690 | "execution_count": 116, 2691 | "metadata": { 2692 | "collapsed": true 2693 | }, 2694 | "outputs": [ 2695 | { 2696 | "data": { 2697 | "text/plain": [ 2698 | "(2.0, 'three')" 2699 | ] 2700 | }, 2701 | "execution_count": 116, 2702 | "metadata": {}, 2703 | "output_type": "execute_result" 2704 | } 2705 | ], 2706 | "source": [ 2707 | "# Parsing JSON string back to Python object\n", 2708 | "data = json.loads(data_json)\n", 2709 | "data[\"two\"], data[\"three\"]" 2710 | ] 2711 | }, 2712 | { 2713 | "cell_type": "code", 2714 | "execution_count": 117, 2715 | "metadata": { 2716 | "collapsed": true 2717 | }, 2718 | "outputs": [], 2719 | "source": [ 2720 | "# JSON can handle variable-size elements\n", 2721 | "data = {\"one\": [1], \n", 2722 | " \"two\": [1, 2], \n", 2723 | " \"three\": [1, 2, 3]}" 2724 | ] 2725 | }, 2726 | { 2727 | "cell_type": "code", 2728 | "execution_count": 118, 2729 | "metadata": { 2730 | "collapsed": true 2731 | }, 2732 | "outputs": [ 2733 | { 2734 | "name": "stdout", 2735 | "output_type": "stream", 2736 | "text": [ 2737 | "{\n", 2738 | " \"one\": [\n", 2739 | " 1\n", 2740 | " ],\n", 2741 | " \"two\": [\n", 2742 | " 1,\n", 2743 | " 2\n", 2744 | " ],\n", 2745 | " \"three\": [\n", 2746 | " 1,\n", 2747 | " 2,\n", 2748 | " 3\n", 2749 | " ]\n", 2750 | "}\n" 2751 | ] 2752 | } 2753 | ], 2754 | "source": [ 2755 | "# indent=True obtains indented JSON code = easier to read.\n", 2756 | "data_json = json.dumps(data, indent=True)\n", 2757 | "print(data_json)" 2758 | ] 2759 | }, 2760 | { 2761 | "cell_type": "code", 2762 | "execution_count": 119, 2763 | "metadata": { 2764 | "collapsed": true 2765 | }, 2766 | "outputs": [], 2767 | "source": [ 2768 | "# another complex data structure, saved to a JSON file\n", 2769 | "data = {\"one\": [1], \n", 2770 | " \"two\": {\"one\": 1, \"two\": 2}, \n", 2771 | " \"three\": [(1,), (1, 2), (1, 2, 3)],\n", 2772 | " \"four\": \"a text string\"}" 2773 | ] 2774 | }, 2775 | { 2776 | "cell_type": "code", 2777 | "execution_count": 120, 2778 | "metadata": { 2779 | "collapsed": true 2780 | }, 2781 | "outputs": [], 2782 | "source": [ 2783 | "with open(\"data.json\", \"w\") as f:\n", 2784 | " json.dump(data, f)" 2785 | ] 2786 | }, 2787 | { 2788 | "cell_type": "code", 2789 | "execution_count": 121, 2790 | "metadata": { 2791 | "collapsed": false 2792 | }, 2793 | "outputs": [ 2794 | { 2795 | "name": "stdout", 2796 | "output_type": "stream", 2797 | "text": [ 2798 | "{\"one\": [1], \"two\": {\"one\": 1, \"two\": 2}, \"three\": [[1], [1, 2], [1, 2, 3]], \"four\": \"a text string\"}" 2799 | ] 2800 | } 2801 | ], 2802 | "source": [ 2803 | "!cat data.json" 2804 | ] 2805 | }, 2806 | { 2807 | "cell_type": "code", 2808 | "execution_count": 122, 2809 | "metadata": { 2810 | "collapsed": false 2811 | }, 2812 | "outputs": [], 2813 | "source": [ 2814 | "# read back into a Python object\n", 2815 | "with open(\"data.json\", \"r\") as f:\n", 2816 | " data_from_file = json.load(f)" 2817 | ] 2818 | }, 2819 | { 2820 | "cell_type": "code", 2821 | "execution_count": 123, 2822 | "metadata": { 2823 | "collapsed": false 2824 | }, 2825 | "outputs": [ 2826 | { 2827 | "data": { 2828 | "text/plain": [ 2829 | "{'one': 1, 'two': 2}" 2830 | ] 2831 | }, 2832 | "execution_count": 123, 2833 | "metadata": {}, 2834 | "output_type": "execute_result" 2835 | } 2836 | ], 2837 | "source": [ 2838 | "data_from_file[\"two\"]" 2839 | ] 2840 | }, 2841 | { 2842 | "cell_type": "code", 2843 | "execution_count": 124, 2844 | "metadata": { 2845 | "collapsed": false 2846 | }, 2847 | "outputs": [ 2848 | { 2849 | "data": { 2850 | "text/plain": [ 2851 | "[[1], [1, 2], [1, 2, 3]]" 2852 | ] 2853 | }, 2854 | "execution_count": 124, 2855 | "metadata": {}, 2856 | "output_type": "execute_result" 2857 | } 2858 | ], 2859 | "source": [ 2860 | "data_from_file[\"three\"]" 2861 | ] 2862 | }, 2863 | { 2864 | "cell_type": "code", 2865 | "execution_count": 125, 2866 | "metadata": { 2867 | "collapsed": false 2868 | }, 2869 | "outputs": [ 2870 | { 2871 | "name": "stdout", 2872 | "output_type": "stream", 2873 | "text": [ 2874 | "{\n", 2875 | " \"C\": {\n", 2876 | " \"color\": \"#149848\", \n", 2877 | " \"transfers\": [\n", 2878 | " [\n", 2879 | " \"C3\", \n", 2880 | " \"F15\"\n", 2881 | " ], \n", 2882 | " [\n", 2883 | " \"C4\", \n", 2884 | " \"Z2\"\n", 2885 | " ], \n", 2886 | " [\n", 2887 | " \"C4\", \n", 2888 | " \"G2\"\n", 2889 | " ], \n", 2890 | " [\n", 2891 | " \"C7\", \n", 2892 | " \"M14\"\n", 2893 | " ], \n" 2894 | ] 2895 | } 2896 | ], 2897 | "source": [ 2898 | "# revisit Tokyo metro dataset (JSON format):\n", 2899 | "!head -n 20 tokyo-metro.json" 2900 | ] 2901 | }, 2902 | { 2903 | "cell_type": "code", 2904 | "execution_count": 128, 2905 | "metadata": { 2906 | "collapsed": true 2907 | }, 2908 | "outputs": [], 2909 | "source": [ 2910 | "with open(\"tokyo-metro.json\", \"r\") as f:\n", 2911 | " data = json.load(f)" 2912 | ] 2913 | }, 2914 | { 2915 | "cell_type": "code", 2916 | "execution_count": 129, 2917 | "metadata": { 2918 | "collapsed": false 2919 | }, 2920 | "outputs": [ 2921 | { 2922 | "data": { 2923 | "text/plain": [ 2924 | "dict_keys(['C', 'G', 'F', 'H', 'M', 'N', 'T', 'Y', 'Z'])" 2925 | ] 2926 | }, 2927 | "execution_count": 129, 2928 | "metadata": {}, 2929 | "output_type": "execute_result" 2930 | } 2931 | ], 2932 | "source": [ 2933 | "# dictionary with a key for each metro line\n", 2934 | "data.keys()" 2935 | ] 2936 | }, 2937 | { 2938 | "cell_type": "code", 2939 | "execution_count": 130, 2940 | "metadata": { 2941 | "collapsed": false 2942 | }, 2943 | "outputs": [ 2944 | { 2945 | "data": { 2946 | "text/plain": [ 2947 | "dict_keys(['color', 'transfers', 'travel_times'])" 2948 | ] 2949 | }, 2950 | "execution_count": 130, 2951 | "metadata": {}, 2952 | "output_type": "execute_result" 2953 | } 2954 | ], 2955 | "source": [ 2956 | "data[\"C\"].keys()" 2957 | ] 2958 | }, 2959 | { 2960 | "cell_type": "code", 2961 | "execution_count": 131, 2962 | "metadata": { 2963 | "collapsed": false 2964 | }, 2965 | "outputs": [ 2966 | { 2967 | "data": { 2968 | "text/plain": [ 2969 | "'#149848'" 2970 | ] 2971 | }, 2972 | "execution_count": 131, 2973 | "metadata": {}, 2974 | "output_type": "execute_result" 2975 | } 2976 | ], 2977 | "source": [ 2978 | "data[\"C\"][\"color\"]" 2979 | ] 2980 | }, 2981 | { 2982 | "cell_type": "code", 2983 | "execution_count": 132, 2984 | "metadata": { 2985 | "collapsed": false 2986 | }, 2987 | "outputs": [ 2988 | { 2989 | "data": { 2990 | "text/plain": [ 2991 | "[['C3', 'F15'],\n", 2992 | " ['C4', 'Z2'],\n", 2993 | " ['C4', 'G2'],\n", 2994 | " ['C7', 'M14'],\n", 2995 | " ['C7', 'N6'],\n", 2996 | " ['C7', 'G6'],\n", 2997 | " ['C8', 'M15'],\n", 2998 | " ['C8', 'H6'],\n", 2999 | " ['C9', 'H7'],\n", 3000 | " ['C9', 'Y18'],\n", 3001 | " ['C11', 'T9'],\n", 3002 | " ['C11', 'M18'],\n", 3003 | " ['C11', 'Z8'],\n", 3004 | " ['C12', 'M19'],\n", 3005 | " ['C18', 'H21']]" 3006 | ] 3007 | }, 3008 | "execution_count": 132, 3009 | "metadata": {}, 3010 | "output_type": "execute_result" 3011 | } 3012 | ], 3013 | "source": [ 3014 | "data[\"C\"][\"transfers\"]" 3015 | ] 3016 | }, 3017 | { 3018 | "cell_type": "code", 3019 | "execution_count": 133, 3020 | "metadata": { 3021 | "collapsed": false 3022 | }, 3023 | "outputs": [ 3024 | { 3025 | "data": { 3026 | "text/plain": [ 3027 | "[('C3', 'C4', 1), ('C7', 'C8', 1), ('C9', 'C10', 1)]" 3028 | ] 3029 | }, 3030 | "execution_count": 133, 3031 | "metadata": {}, 3032 | "output_type": "execute_result" 3033 | } 3034 | ], 3035 | "source": [ 3036 | "# now we can iterate & filter items with Python list syntax\n", 3037 | "# below: select connected nodes in graph, on C line, with travel time = 1 minute.\n", 3038 | "\n", 3039 | "[(s, e, tt) for s, e, tt in data[\"C\"][\"travel_times\"] if tt == 1]" 3040 | ] 3041 | }, 3042 | { 3043 | "cell_type": "markdown", 3044 | "metadata": {}, 3045 | "source": [ 3046 | "### Serialization\n", 3047 | "\n", 3048 | "* JSON files aren't space efficient, and can represent only limited set of datatypes.\n", 3049 | "* Two alternatives: __msgpack__ library, and Python's __pickle__ module." 3050 | ] 3051 | }, 3052 | { 3053 | "cell_type": "code", 3054 | "execution_count": 135, 3055 | "metadata": { 3056 | "collapsed": false 3057 | }, 3058 | "outputs": [ 3059 | { 3060 | "name": "stdout", 3061 | "output_type": "stream", 3062 | "text": [ 3063 | "-rw-rw-r-- 1 bjpcjp bjpcjp 27K May 19 16:47 tokyo-metro.json\n" 3064 | ] 3065 | } 3066 | ], 3067 | "source": [ 3068 | "!ls -lh tokyo-metro.json" 3069 | ] 3070 | }, 3071 | { 3072 | "cell_type": "code", 3073 | "execution_count": 138, 3074 | "metadata": { 3075 | "collapsed": false 3076 | }, 3077 | "outputs": [ 3078 | { 3079 | "data": { 3080 | "text/plain": [ 3081 | "(bytes, 3021)" 3082 | ] 3083 | }, 3084 | "execution_count": 138, 3085 | "metadata": {}, 3086 | "output_type": "execute_result" 3087 | } 3088 | ], 3089 | "source": [ 3090 | "# packing the JSON \"data\" file (above) --> considerably smaller file\n", 3091 | "data_pack = msgpack.packb(data)\n", 3092 | "type(data_pack), len(data_pack)" 3093 | ] 3094 | }, 3095 | { 3096 | "cell_type": "code", 3097 | "execution_count": 139, 3098 | "metadata": { 3099 | "collapsed": false 3100 | }, 3101 | "outputs": [], 3102 | "source": [ 3103 | "with open(\"tokyo-metro.msgpack\", \"wb\") as f:\n", 3104 | " f.write(data_pack)" 3105 | ] 3106 | }, 3107 | { 3108 | "cell_type": "code", 3109 | "execution_count": 140, 3110 | "metadata": { 3111 | "collapsed": false 3112 | }, 3113 | "outputs": [ 3114 | { 3115 | "name": "stdout", 3116 | "output_type": "stream", 3117 | "text": [ 3118 | "-rw-rw-r-- 1 bjpcjp bjpcjp 3.0K May 19 18:46 tokyo-metro.msgpack\n" 3119 | ] 3120 | } 3121 | ], 3122 | "source": [ 3123 | "!ls -lh tokyo-metro.msgpack" 3124 | ] 3125 | }, 3126 | { 3127 | "cell_type": "code", 3128 | "execution_count": 141, 3129 | "metadata": { 3130 | "collapsed": false 3131 | }, 3132 | "outputs": [], 3133 | "source": [ 3134 | "# unpack the msgpack file, back into JSON:\n", 3135 | "with open(\"tokyo-metro.msgpack\", \"rb\") as f:\n", 3136 | " data_msgpack = f.read()\n", 3137 | " data = msgpack.unpackb(data_msgpack)" 3138 | ] 3139 | }, 3140 | { 3141 | "cell_type": "code", 3142 | "execution_count": 142, 3143 | "metadata": { 3144 | "collapsed": false 3145 | }, 3146 | "outputs": [ 3147 | { 3148 | "data": { 3149 | "text/plain": [ 3150 | "[b'C', b'G', b'F', b'H', b'M', b'N', b'T', b'Y', b'Z']" 3151 | ] 3152 | }, 3153 | "execution_count": 142, 3154 | "metadata": {}, 3155 | "output_type": "execute_result" 3156 | } 3157 | ], 3158 | "source": [ 3159 | "list(data.keys())" 3160 | ] 3161 | }, 3162 | { 3163 | "cell_type": "markdown", 3164 | "metadata": {}, 3165 | "source": [ 3166 | "### Serialization - pickle\n", 3167 | "* Advantage: almost any type of Python object can be serialized.\n", 3168 | "* Disadvantage: Pickles can't be read by any non-Python code. Also, not recommended for long-term storage due to version thrashing." 3169 | ] 3170 | }, 3171 | { 3172 | "cell_type": "code", 3173 | "execution_count": 143, 3174 | "metadata": { 3175 | "collapsed": false 3176 | }, 3177 | "outputs": [], 3178 | "source": [ 3179 | "with open(\"tokyo-metro.pickle\", \"wb\") as f:\n", 3180 | " cPickle.dump(data, f)" 3181 | ] 3182 | }, 3183 | { 3184 | "cell_type": "code", 3185 | "execution_count": 144, 3186 | "metadata": { 3187 | "collapsed": false 3188 | }, 3189 | "outputs": [ 3190 | { 3191 | "name": "stdout", 3192 | "output_type": "stream", 3193 | "text": [ 3194 | "-rw-rw-r-- 1 bjpcjp bjpcjp 8.6K May 19 18:53 tokyo-metro.pickle\n" 3195 | ] 3196 | } 3197 | ], 3198 | "source": [ 3199 | "!ls -lh tokyo-metro.pickle" 3200 | ] 3201 | }, 3202 | { 3203 | "cell_type": "code", 3204 | "execution_count": 145, 3205 | "metadata": { 3206 | "collapsed": false 3207 | }, 3208 | "outputs": [], 3209 | "source": [ 3210 | "with open(\"tokyo-metro.pickle\", \"rb\") as f:\n", 3211 | " data = pickle.load(f)" 3212 | ] 3213 | }, 3214 | { 3215 | "cell_type": "code", 3216 | "execution_count": 146, 3217 | "metadata": { 3218 | "collapsed": false 3219 | }, 3220 | "outputs": [ 3221 | { 3222 | "data": { 3223 | "text/plain": [ 3224 | "dict_keys([b'C', b'G', b'F', b'H', b'M', b'N', b'T', b'Y', b'Z'])" 3225 | ] 3226 | }, 3227 | "execution_count": 146, 3228 | "metadata": {}, 3229 | "output_type": "execute_result" 3230 | } 3231 | ], 3232 | "source": [ 3233 | "data.keys()" 3234 | ] 3235 | } 3236 | ], 3237 | "metadata": { 3238 | "kernelspec": { 3239 | "display_name": "Python 3", 3240 | "language": "python", 3241 | "name": "python3" 3242 | }, 3243 | "language_info": { 3244 | "codemirror_mode": { 3245 | "name": "ipython", 3246 | "version": 3 3247 | }, 3248 | "file_extension": ".py", 3249 | "mimetype": "text/x-python", 3250 | "name": "python", 3251 | "nbconvert_exporter": "python", 3252 | "pygments_lexer": "ipython3", 3253 | "version": "3.6.5" 3254 | } 3255 | }, 3256 | "nbformat": 4, 3257 | "nbformat_minor": 2 3258 | } 3259 | -------------------------------------------------------------------------------- /ch18-data.csv: -------------------------------------------------------------------------------- 1 | # Random x, y, z coordinates 2 | x, y, z 3 | 1.764052345967664026e+00,4.001572083672232938e-01,9.787379841057392005e-01 4 | 2.240893199201457797e+00,1.867557990149967484e+00,-9.772778798764110153e-01 5 | 9.500884175255893682e-01,-1.513572082976978872e-01,-1.032188517935578448e-01 6 | 4.105985019383723289e-01,1.440435711608779867e-01,1.454273506962975082e+00 7 | 7.610377251469934157e-01,1.216750164928284139e-01,4.438632327454256621e-01 8 | 3.336743273742668325e-01,1.494079073157606130e+00,-2.051582637658008745e-01 9 | 3.130677016509013644e-01,-8.540957393017247767e-01,-2.552989815834078691e+00 10 | 6.536185954403606058e-01,8.644361988595057333e-01,-7.421650204064419443e-01 11 | 2.269754623987607633e+00,-1.454365674598764757e+00,4.575851730144606805e-02 12 | -1.871838500258335980e-01,1.532779214358457542e+00,1.469358769900285022e+00 13 | 1.549474256969163022e-01,3.781625196021735635e-01,-8.877857476301127537e-01 14 | -1.980796468223926965e+00,-3.479121493261526088e-01,1.563489691039800511e-01 15 | 1.230290680727720742e+00,1.202379848784411287e+00,-3.873268174079522730e-01 16 | -3.023027505753355659e-01,-1.048552965067092613e+00,-1.420017937178975176e+00 17 | -1.706270190625012617e+00,1.950775395231789666e+00,-5.096521817516534991e-01 18 | -4.380743016111863786e-01,-1.252795360049926199e+00,7.774903558319100627e-01 19 | -1.613897847557951515e+00,-2.127402802139687077e-01,-8.954665611936756253e-01 20 | 3.869024978592620068e-01,-5.108051375688730245e-01,-1.180632184122412109e+00 21 | -2.818222833865486818e-02,4.283318705304176577e-01,6.651722238316788716e-02 22 | 3.024718977397813924e-01,-6.343220936809635946e-01,-3.627411659871381255e-01 23 | -6.724604477759510424e-01,-3.595531615405412884e-01,-8.131462820444540496e-01 24 | -1.726282602331676852e+00,1.774261422537528332e-01,-4.017809362082618851e-01 25 | -1.630198346966044598e+00,4.627822555257741777e-01,-9.072983643832421841e-01 26 | 5.194539579613895175e-02,7.290905621775368717e-01,1.289829107574106681e-01 27 | 1.139400684543300679e+00,-1.234825820353652626e+00,4.023416411775490031e-01 28 | -6.848100909403131986e-01,-8.707971491818817800e-01,-5.788496647644154613e-01 29 | -3.115525321273726611e-01,5.616534222974543800e-02,-1.165149840783356483e+00 30 | 9.008264869541871001e-01,4.656624397304598428e-01,-1.536243686277223741e+00 31 | 1.488252193795599698e+00,1.895889176030583156e+00,1.178779571159650708e+00 32 | -1.799248358123509139e-01,-1.070752621510542513e+00,1.054451726931136646e+00 33 | -4.031769469731796285e-01,1.222445070382427446e+00,2.082749780768602954e-01 34 | 9.766390364837127525e-01,3.563663971744018832e-01,7.065731681919481533e-01 35 | 1.050002072082047849e-02,1.785870493905835188e+00,1.269120927036199165e-01 36 | 4.019893634447016528e-01,1.883150697056254375e+00,-1.347759061142446368e+00 37 | -1.270484998485733596e+00,9.693967081580111556e-01,-1.173123405114159912e+00 38 | 1.943621185649292649e+00,-4.136189807597473456e-01,-7.474548114407577604e-01 39 | 1.922942026480384703e+00,1.480514791434424327e+00,1.867558960426569881e+00 40 | 9.060446582753852729e-01,-8.612256850547025389e-01,1.910064953099033680e+00 41 | -2.680033709513803819e-01,8.024563957963951610e-01,9.472519677737479826e-01 42 | -1.550100930908341890e-01,6.140793703460802888e-01,9.222066715665268033e-01 43 | 3.764255311556294337e-01,-1.099400790584194487e+00,2.982381742060559726e-01 44 | 1.326385896687030330e+00,-6.945678597313654778e-01,-1.496345403276707620e-01 45 | -4.351535517216374438e-01,1.849263728479341840e+00,6.722947570124354622e-01 46 | 4.074618362411104311e-01,-7.699160744453164007e-01,5.392491912918172536e-01 47 | -6.743326606573760706e-01,3.183055827435118251e-02,-6.358460783788809634e-01 48 | 6.764332949464997302e-01,5.765908166149409375e-01,-2.082987555779948763e-01 49 | 3.960067126616452771e-01,-1.093061508730505782e+00,-1.491257592705605539e+00 50 | 4.393917012645369158e-01,1.666734953725290402e-01,6.350314368921063934e-01 51 | 2.383144774863942050e+00,9.444794869904138412e-01,-9.128222254441585859e-01 52 | 1.117016288095852961e+00,-1.315907410511521158e+00,-4.615846048147089764e-01 53 | -6.824160532463123541e-02,1.713342721649366629e+00,-7.447548220484399017e-01 54 | -8.264385386590143989e-01,-9.845252442543230009e-02,-6.634782863621073723e-01 55 | 1.126635922106506982e+00,-1.079931508363423331e+00,-1.147468652411102408e+00 56 | -4.378200447444340337e-01,-4.980324506923048955e-01,1.929532053816985782e+00 57 | 9.494208069257608074e-01,8.755124138519089494e-02,-1.225435518830167991e+00 58 | 8.443629764015471162e-01,-1.000215347389564746e+00,-1.544771096777611596e+00 59 | 1.188029792352301772e+00,3.169426119248496265e-01,9.208588237808189580e-01 60 | 3.187276529430211891e-01,8.568306119026911638e-01,-6.510255933001468653e-01 61 | -1.034242841784464684e+00,6.815945182816269821e-01,-8.034096641738410760e-01 62 | -6.895497777502005432e-01,-4.555325035173431458e-01,1.747915902505672858e-02 63 | -3.539939112534839505e-01,-1.374951293418018805e+00,-6.436184028328905127e-01 64 | -2.223403152224426638e+00,6.252314510271874681e-01,-1.602057655606747621e+00 65 | -1.104383339428450572e+00,5.216507926097440490e-02,-7.395629963913132876e-01 66 | 1.543014595406735801e+00,-1.292856909723448577e+00,2.670508693491829288e-01 67 | -3.928281822749560281e-02,-1.168093497741197417e+00,5.232766605317537012e-01 68 | -1.715463312222481052e-01,7.717905512136673529e-01,8.235041539637314445e-01 69 | 2.163235949280689852e+00,1.336527949436391971e+00,-3.691818379424435803e-01 70 | -2.393791775759263885e-01,1.099659595887113150e+00,6.552637307225978036e-01 71 | 6.401315260975920518e-01,-1.616956044310834395e+00,-2.432612439893563558e-02 72 | -7.380309092056887010e-01,2.799245990432382425e-01,-9.815038964295794144e-02 73 | 9.101789080925919428e-01,3.172182151913020554e-01,7.863279621089761529e-01 74 | -4.664190967359430617e-01,-9.444462559182503769e-01,-4.100496932025484709e-01 75 | -1.702041386144059407e-02,3.791517355550818036e-01,2.259308950690852136e+00 76 | -4.225715166064269307e-02,-9.559450004927769751e-01,-3.459817756993864291e-01 77 | -4.635959746460941999e-01,4.814814737734621763e-01,-1.540797014444624802e+00 78 | 6.326199420033171150e-02,1.565065379653755862e-01,2.321810362002757755e-01 79 | -5.973160689653627120e-01,-2.379217297360070038e-01,-1.424060908982531570e+00 80 | -4.933198833621940671e-01,-5.428614760167177478e-01,4.160500462614254991e-01 81 | -1.156182431821912715e+00,7.811981017099933755e-01,1.494484544491368805e+00 82 | -2.069985025013532542e+00,4.262587307781009471e-01,6.769080350302455473e-01 83 | -6.374370255522290307e-01,-3.972718143287976567e-01,-1.328805775869556227e-01 84 | -2.977908794017283256e-01,-3.090129690471222212e-01,-1.676003806329976697e+00 85 | 1.152331564783120044e+00,1.079618592036821090e+00,-8.133642592042028552e-01 86 | -1.466424327802513972e+00,5.210648764527585630e-01,-5.757879698130661250e-01 87 | 1.419531633207796739e-01,-3.193284171450951892e-01,6.915387510701865859e-01 88 | 6.947491436560059297e-01,-7.255973784635842971e-01,-1.383363955395055411e+00 89 | -1.582938397335081904e+00,6.103793791072051889e-01,-1.188859257784028900e+00 90 | -5.068163542986875436e-01,-5.963140384505081215e-01,-5.256729626954628753e-02 91 | -1.936279805846506941e+00,1.887785967938285514e-01,5.238910238342056047e-01 92 | 8.842208704466140989e-02,-3.108861716984717138e-01,9.740016626878340855e-02 93 | 3.990463456401301956e-01,-2.772592756426650151e+00,1.955912308250694176e+00 94 | 3.900933226879264581e-01,-6.524085823870200418e-01,-3.909533751876010932e-01 95 | 4.937417773491884487e-01,-1.161039390343665267e-01,-2.030684467781494362e+00 96 | 2.064492861359319420e+00,-1.105406572324726111e-01,1.020172711715799707e+00 97 | -6.920498477843911589e-01,1.536377054245797735e+00,2.863436888922795687e-01 98 | 6.088438344754507625e-01,-1.045253366146954743e+00,1.211145289682700854e+00 99 | 6.898181645347883917e-01,1.301846229564998403e+00,-6.280875596415789186e-01 100 | -4.810271184607877171e-01,2.303916697683941806e+00,-1.060015822721547263e+00 101 | -1.359497006783208228e-01,1.136891362602695299e+00,9.772496771485560119e-02 102 | 5.829536797532935966e-01,-3.994490292628751771e-01,3.700558878475187496e-01 103 | -------------------------------------------------------------------------------- /ch18-data.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/ch18-data.h5 -------------------------------------------------------------------------------- /ch18-playerstats-2013-2014-subset.csv: -------------------------------------------------------------------------------- 1 | Rank,Player,GP,G,A,P 2 | 1,Sidney Crosby,80,36,68,104 3 | 2,Ryan Getzlaf,77,31,56,87 4 | 3,Claude Giroux,82,28,58,86 5 | 4,Tyler Seguin,80,37,47,84 6 | 5,Corey Perry,81,43,39,82 7 | -------------------------------------------------------------------------------- /ch18-playerstats-2013-2014-top30.csv: -------------------------------------------------------------------------------- 1 | # 2013-2014 / Regular Season / All Skaters / Summary / Points 2 | Rank,Player,Team,Pos,GP,G,A,P,+/-,PIM,PPG,PPP,SHG,SHP,GW,OT,S,S%,TOI/GP,Shift/GP,FO% 3 | 1,Sidney Crosby,PIT,C,80,36,68,104,+18,46,11,38,0,0,5,1,259,13.9,21:58,24.0,52.5 4 | 2,Ryan Getzlaf,ANA,C,77,31,56,87,+28,31,5,23,0,0,7,1,204,15.2,21:17,25.2,49.0 5 | 3,Claude Giroux,PHI,C,82,28,58,86,+7,46,7,37,0,0,7,1,223,12.6,20:26,25.1,52.9 6 | 4,Tyler Seguin,DAL,C,80,37,47,84,+16,18,11,25,0,0,8,0,294,12.6,19:20,23.4,41.5 7 | 5,Corey Perry,ANA,R,81,43,39,82,+32,65,8,18,0,0,9,1,280,15.4,19:28,23.2,36.0 8 | 6,Phil Kessel,TOR,R,82,37,43,80,-5,27,8,20,0,0,6,0,305,12.1,20:39,24.5,14.3 9 | 7,Taylor Hall,EDM,L,75,27,53,80,-15,44,7,17,0,1,1,1,250,10.8,20:00,25.4,45.7 10 | 8,Alex Ovechkin,WSH,L,78,51,28,79,-35,48,24,39,0,1,10,3,386,13.2,20:32,21.8,66.7 11 | 9,Joe Pavelski,SJS,C,82,41,38,79,+23,32,16,31,1,2,3,0,225,18.2,19:51,27.1,56.0 12 | 10,Jamie Benn,DAL,L,81,34,45,79,+21,64,5,19,1,3,3,1,279,12.2,19:09,25.0,52.8 13 | 11,Nicklas Backstrom,WSH,C,82,18,61,79,-20,54,6,44,1,1,1,0,196,9.2,19:48,23.3,50.4 14 | 12,Patrick Sharp,CHI,L,82,34,44,78,+13,40,10,25,0,0,3,1,313,10.9,18:53,22.7,54.6 15 | 13,Joe Thornton,SJS,C,82,11,65,76,+20,32,2,19,0,1,3,1,122,9.0,18:55,26.3,56.1 16 | 14,Erik Karlsson,OTT,D,82,20,54,74,-15,36,5,31,0,0,1,0,257,7.8,27:04,28.6,0.0 17 | 15,Evgeni Malkin,PIT,C,60,23,49,72,+10,62,7,30,0,0,3,0,191,12.0,20:03,21.4,48.8 18 | 16,Patrick Marleau,SJS,L,82,33,37,70,+0,18,11,23,2,2,4,0,285,11.6,20:31,27.3,52.9 19 | 17,Anze Kopitar,LAK,C,82,29,41,70,+34,24,10,23,0,0,9,2,200,14.5,20:53,25.4,53.3 20 | 18,Matt Duchene,COL,C,71,23,47,70,+8,19,5,17,0,0,6,1,217,10.6,18:29,22.0,50.3 21 | 19,Martin St. Louis,"TBL, NYR",R,81,30,39,69,+13,10,9,21,1,2,5,1,204,14.7,20:56,25.7,40.7 22 | 20,Patrick Kane,CHI,R,69,29,40,69,+7,22,10,25,0,0,6,0,227,12.8,19:36,22.9,50.0 23 | 21,Blake Wheeler,WPG,R,82,28,41,69,+4,63,8,19,0,0,4,2,225,12.4,18:41,24.0,37.5 24 | 22,Kyle Okposo,NYI,R,71,27,42,69,-9,51,5,15,0,0,4,1,195,13.8,20:26,22.2,47.5 25 | 23,David Krejci,BOS,C,80,19,50,69,+39,28,3,19,0,0,6,1,169,11.2,19:07,21.3,51.2 26 | 24,Chris Kunitz,PIT,L,78,35,33,68,+25,66,13,22,0,0,8,0,218,16.1,19:09,22.2,75.0 27 | 25,Jonathan Toews,CHI,C,76,28,40,68,+26,34,5,15,3,5,5,0,193,14.5,20:28,25.9,57.2 28 | 26,Thomas Vanek,"BUF, NYI, MTL",L,78,27,41,68,+7,46,8,18,0,0,4,0,248,10.9,19:21,21.6,43.5 29 | 27,Jaromir Jagr,NJD,R,82,24,43,67,+16,46,5,17,0,0,6,1,231,10.4,19:09,22.8,0.0 30 | 28,John Tavares,NYI,C,59,24,42,66,-6,40,8,25,0,0,4,0,188,12.8,21:14,22.3,49.1 31 | 29,Jason Spezza,OTT,C,75,23,43,66,-26,46,9,22,0,0,5,0,223,10.3,18:12,23.8,54.0 32 | 30,Jordan Eberle,EDM,R,80,28,37,65,-11,18,7,20,1,1,4,1,200,14.0,19:32,25.4,38.1 -------------------------------------------------------------------------------- /ch18-playerstats-2013-2014.csv: -------------------------------------------------------------------------------- 1 | # 2013-2014 / Regular Season / All Skaters / Summary / Points 2 | Rank,Player,Team,Pos,GP,G,A,P,+/-,PIM,PPG,PPP,SHG,SHP,GW,OT,S,S%,TOI/GP,Shift/GP,FO% 3 | 1,Sidney Crosby,PIT,C,80,36,68,104,+18,46,11,38,0,0,5,1,259,13.9,21:58,24.0,52.5 4 | 2,Ryan Getzlaf,ANA,C,77,31,56,87,+28,31,5,23,0,0,7,1,204,15.2,21:17,25.2,49.0 5 | 3,Claude Giroux,PHI,C,82,28,58,86,+7,46,7,37,0,0,7,1,223,12.6,20:26,25.1,52.9 6 | 4,Tyler Seguin,DAL,C,80,37,47,84,+16,18,11,25,0,0,8,0,294,12.6,19:20,23.4,41.5 7 | 5,Corey Perry,ANA,R,81,43,39,82,+32,65,8,18,0,0,9,1,280,15.4,19:28,23.2,36.0 -------------------------------------------------------------------------------- /ch18-playerstats-2013-2014.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/ch18-playerstats-2013-2014.h5 -------------------------------------------------------------------------------- /cy_cumsum.pyx: -------------------------------------------------------------------------------- 1 | # cy_cumsum.pyx 2 | cimport numpy 3 | import numpy 4 | 5 | def cy_cumsum(data): 6 | out = numpy.zeros_like(data) 7 | s = 0 8 | for n in range(len(data)): 9 | s += data[n] 10 | out[n] =s 11 | return out -------------------------------------------------------------------------------- /cy_sum.cpython-36m-x86_64-linux-gnu.so: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/cy_sum.cpython-36m-x86_64-linux-gnu.so -------------------------------------------------------------------------------- /cy_sum.pyx: -------------------------------------------------------------------------------- 1 | # cy_sum.pyx 2 | 3 | def cy_sum(data): 4 | s = 0.0 5 | for d in data: 6 | s += d 7 | return s -------------------------------------------------------------------------------- /data.csv: -------------------------------------------------------------------------------- 1 | random x,y,z coords 2 | x,y,z 3 | 1.764052345967664026e+00,4.001572083672232938e-01,9.787379841057392005e-01 4 | 2.240893199201457797e+00,1.867557990149967484e+00,-9.772778798764110153e-01 5 | 9.500884175255893682e-01,-1.513572082976978872e-01,-1.032188517935578448e-01 6 | 4.105985019383723289e-01,1.440435711608779867e-01,1.454273506962975082e+00 7 | 7.610377251469934157e-01,1.216750164928284139e-01,4.438632327454256621e-01 8 | 3.336743273742668325e-01,1.494079073157606130e+00,-2.051582637658008745e-01 9 | 3.130677016509013644e-01,-8.540957393017247767e-01,-2.552989815834078691e+00 10 | 6.536185954403606058e-01,8.644361988595057333e-01,-7.421650204064419443e-01 11 | 2.269754623987607633e+00,-1.454365674598764757e+00,4.575851730144606805e-02 12 | -1.871838500258335980e-01,1.532779214358457542e+00,1.469358769900285022e+00 13 | 1.549474256969163022e-01,3.781625196021735635e-01,-8.877857476301127537e-01 14 | -1.980796468223926965e+00,-3.479121493261526088e-01,1.563489691039800511e-01 15 | 1.230290680727720742e+00,1.202379848784411287e+00,-3.873268174079522730e-01 16 | -3.023027505753355659e-01,-1.048552965067092613e+00,-1.420017937178975176e+00 17 | -1.706270190625012617e+00,1.950775395231789666e+00,-5.096521817516534991e-01 18 | -4.380743016111863786e-01,-1.252795360049926199e+00,7.774903558319100627e-01 19 | -1.613897847557951515e+00,-2.127402802139687077e-01,-8.954665611936756253e-01 20 | 3.869024978592620068e-01,-5.108051375688730245e-01,-1.180632184122412109e+00 21 | -2.818222833865486818e-02,4.283318705304176577e-01,6.651722238316788716e-02 22 | 3.024718977397813924e-01,-6.343220936809635946e-01,-3.627411659871381255e-01 23 | -6.724604477759510424e-01,-3.595531615405412884e-01,-8.131462820444540496e-01 24 | -1.726282602331676852e+00,1.774261422537528332e-01,-4.017809362082618851e-01 25 | -1.630198346966044598e+00,4.627822555257741777e-01,-9.072983643832421841e-01 26 | 5.194539579613895175e-02,7.290905621775368717e-01,1.289829107574106681e-01 27 | 1.139400684543300679e+00,-1.234825820353652626e+00,4.023416411775490031e-01 28 | -6.848100909403131986e-01,-8.707971491818817800e-01,-5.788496647644154613e-01 29 | -3.115525321273726611e-01,5.616534222974543800e-02,-1.165149840783356483e+00 30 | 9.008264869541871001e-01,4.656624397304598428e-01,-1.536243686277223741e+00 31 | 1.488252193795599698e+00,1.895889176030583156e+00,1.178779571159650708e+00 32 | -1.799248358123509139e-01,-1.070752621510542513e+00,1.054451726931136646e+00 33 | -4.031769469731796285e-01,1.222445070382427446e+00,2.082749780768602954e-01 34 | 9.766390364837127525e-01,3.563663971744018832e-01,7.065731681919481533e-01 35 | 1.050002072082047849e-02,1.785870493905835188e+00,1.269120927036199165e-01 36 | 4.019893634447016528e-01,1.883150697056254375e+00,-1.347759061142446368e+00 37 | -1.270484998485733596e+00,9.693967081580111556e-01,-1.173123405114159912e+00 38 | 1.943621185649292649e+00,-4.136189807597473456e-01,-7.474548114407577604e-01 39 | 1.922942026480384703e+00,1.480514791434424327e+00,1.867558960426569881e+00 40 | 9.060446582753852729e-01,-8.612256850547025389e-01,1.910064953099033680e+00 41 | -2.680033709513803819e-01,8.024563957963951610e-01,9.472519677737479826e-01 42 | -1.550100930908341890e-01,6.140793703460802888e-01,9.222066715665268033e-01 43 | 3.764255311556294337e-01,-1.099400790584194487e+00,2.982381742060559726e-01 44 | 1.326385896687030330e+00,-6.945678597313654778e-01,-1.496345403276707620e-01 45 | -4.351535517216374438e-01,1.849263728479341840e+00,6.722947570124354622e-01 46 | 4.074618362411104311e-01,-7.699160744453164007e-01,5.392491912918172536e-01 47 | -6.743326606573760706e-01,3.183055827435118251e-02,-6.358460783788809634e-01 48 | 6.764332949464997302e-01,5.765908166149409375e-01,-2.082987555779948763e-01 49 | 3.960067126616452771e-01,-1.093061508730505782e+00,-1.491257592705605539e+00 50 | 4.393917012645369158e-01,1.666734953725290402e-01,6.350314368921063934e-01 51 | 2.383144774863942050e+00,9.444794869904138412e-01,-9.128222254441585859e-01 52 | 1.117016288095852961e+00,-1.315907410511521158e+00,-4.615846048147089764e-01 53 | -6.824160532463123541e-02,1.713342721649366629e+00,-7.447548220484399017e-01 54 | -8.264385386590143989e-01,-9.845252442543230009e-02,-6.634782863621073723e-01 55 | 1.126635922106506982e+00,-1.079931508363423331e+00,-1.147468652411102408e+00 56 | -4.378200447444340337e-01,-4.980324506923048955e-01,1.929532053816985782e+00 57 | 9.494208069257608074e-01,8.755124138519089494e-02,-1.225435518830167991e+00 58 | 8.443629764015471162e-01,-1.000215347389564746e+00,-1.544771096777611596e+00 59 | 1.188029792352301772e+00,3.169426119248496265e-01,9.208588237808189580e-01 60 | 3.187276529430211891e-01,8.568306119026911638e-01,-6.510255933001468653e-01 61 | -1.034242841784464684e+00,6.815945182816269821e-01,-8.034096641738410760e-01 62 | -6.895497777502005432e-01,-4.555325035173431458e-01,1.747915902505672858e-02 63 | -3.539939112534839505e-01,-1.374951293418018805e+00,-6.436184028328905127e-01 64 | -2.223403152224426638e+00,6.252314510271874681e-01,-1.602057655606747621e+00 65 | -1.104383339428450572e+00,5.216507926097440490e-02,-7.395629963913132876e-01 66 | 1.543014595406735801e+00,-1.292856909723448577e+00,2.670508693491829288e-01 67 | -3.928281822749560281e-02,-1.168093497741197417e+00,5.232766605317537012e-01 68 | -1.715463312222481052e-01,7.717905512136673529e-01,8.235041539637314445e-01 69 | 2.163235949280689852e+00,1.336527949436391971e+00,-3.691818379424435803e-01 70 | -2.393791775759263885e-01,1.099659595887113150e+00,6.552637307225978036e-01 71 | 6.401315260975920518e-01,-1.616956044310834395e+00,-2.432612439893563558e-02 72 | -7.380309092056887010e-01,2.799245990432382425e-01,-9.815038964295794144e-02 73 | 9.101789080925919428e-01,3.172182151913020554e-01,7.863279621089761529e-01 74 | -4.664190967359430617e-01,-9.444462559182503769e-01,-4.100496932025484709e-01 75 | -1.702041386144059407e-02,3.791517355550818036e-01,2.259308950690852136e+00 76 | -4.225715166064269307e-02,-9.559450004927769751e-01,-3.459817756993864291e-01 77 | -4.635959746460941999e-01,4.814814737734621763e-01,-1.540797014444624802e+00 78 | 6.326199420033171150e-02,1.565065379653755862e-01,2.321810362002757755e-01 79 | -5.973160689653627120e-01,-2.379217297360070038e-01,-1.424060908982531570e+00 80 | -4.933198833621940671e-01,-5.428614760167177478e-01,4.160500462614254991e-01 81 | -1.156182431821912715e+00,7.811981017099933755e-01,1.494484544491368805e+00 82 | -2.069985025013532542e+00,4.262587307781009471e-01,6.769080350302455473e-01 83 | -6.374370255522290307e-01,-3.972718143287976567e-01,-1.328805775869556227e-01 84 | -2.977908794017283256e-01,-3.090129690471222212e-01,-1.676003806329976697e+00 85 | 1.152331564783120044e+00,1.079618592036821090e+00,-8.133642592042028552e-01 86 | -1.466424327802513972e+00,5.210648764527585630e-01,-5.757879698130661250e-01 87 | 1.419531633207796739e-01,-3.193284171450951892e-01,6.915387510701865859e-01 88 | 6.947491436560059297e-01,-7.255973784635842971e-01,-1.383363955395055411e+00 89 | -1.582938397335081904e+00,6.103793791072051889e-01,-1.188859257784028900e+00 90 | -5.068163542986875436e-01,-5.963140384505081215e-01,-5.256729626954628753e-02 91 | -1.936279805846506941e+00,1.887785967938285514e-01,5.238910238342056047e-01 92 | 8.842208704466140989e-02,-3.108861716984717138e-01,9.740016626878340855e-02 93 | 3.990463456401301956e-01,-2.772592756426650151e+00,1.955912308250694176e+00 94 | 3.900933226879264581e-01,-6.524085823870200418e-01,-3.909533751876010932e-01 95 | 4.937417773491884487e-01,-1.161039390343665267e-01,-2.030684467781494362e+00 96 | 2.064492861359319420e+00,-1.105406572324726111e-01,1.020172711715799707e+00 97 | -6.920498477843911589e-01,1.536377054245797735e+00,2.863436888922795687e-01 98 | 6.088438344754507625e-01,-1.045253366146954743e+00,1.211145289682700854e+00 99 | 6.898181645347883917e-01,1.301846229564998403e+00,-6.280875596415789186e-01 100 | -4.810271184607877171e-01,2.303916697683941806e+00,-1.060015822721547263e+00 101 | -1.359497006783208228e-01,1.136891362602695299e+00,9.772496771485560119e-02 102 | 5.829536797532935966e-01,-3.994490292628751771e-01,3.700558878475187496e-01 103 | -------------------------------------------------------------------------------- /data.json: -------------------------------------------------------------------------------- 1 | {"one": [1], "two": {"one": 1, "two": 2}, "three": [[1], [1, 2], [1, 2, 3]], "four": "a text string"} -------------------------------------------------------------------------------- /european_cities.csv: -------------------------------------------------------------------------------- 1 | Rank,City,State,Population,Date of census/estimate 2 | 1,London[2], United Kingdom,"8,615,246",1 June 2014 3 | 2,Berlin, Germany,"3,437,916",31 May 2014 4 | 3,Madrid, Spain,"3,165,235",1 January 2014 5 | 4,Rome, Italy,"2,872,086",30 September 2014 6 | 5,Paris, France,"2,273,305",1 January 2013 7 | 6,Bucharest, Romania,"1,883,425",20 October 2011 8 | 7,Vienna, Austria,"1,794,770",1 January 2015 9 | 8,Hamburg[10], Germany,"1,746,342",30 December 2013 10 | 9,Budapest, Hungary,"1,744,665",1 January 2014 11 | 10,Warsaw, Poland,"1,729,119",31 March 2014 12 | 11,Barcelona, Spain,"1,602,386",1 January 2014 13 | 12,Munich, Germany,"1,407,836",31 December 2013 14 | 13,Milan, Italy,"1,332,516",30 September 2014 15 | 14,Sofia, Bulgaria,"1,291,895",14 December 2014 16 | 15,Prague, Czech Republic,"1,246,780",1 January 2013 17 | 16,Brussels[17], Belgium,"1,175,831",1 January 2014 18 | 17,Birmingham, United Kingdom,"1,092,330",30 June 2013 19 | 18,Cologne, Germany,"1,034,175",31 December 2013 20 | 19,Naples, Italy,"989,845",30 September 2014 21 | 20,Stockholm, Sweden,"909,976",31 January 2014 22 | 21,Turin, Italy,"898,095",30 September 2014 23 | 22,Marseille, France,"852,516",31 December 2012 24 | 23,Amsterdam, Netherlands,"813,562",31 May 2014 25 | 24,Zagreb, Croatia,"790,017",31 March 2011 26 | 25,Valencia, Spain,"786,424",1 January 2014 27 | 26,Kraków, Poland,"760,700",31 March 2014 28 | 27,Leeds, United Kingdom,"757,655",30 June 2012 29 | 28,Łódź, Poland,"709,757",31 March 2014 30 | 29,Frankfurt, Germany,"701,350",31 December 2013 31 | 30,Riga, Latvia,"701,185",1 January 2014 32 | 31,Seville, Spain,"696,676",1 January 2014 33 | 32,Palermo, Italy,"677,015",31 July 2014 34 | 33,Zaragoza, Spain,"666,058",1 January 2014 35 | 34,Athens, Greece,"664,046",24 May 2011 36 | 35,Wrocław, Poland,"632,432",31 March 2014 37 | 37,Rotterdam, Netherlands,"616,528",31 January 2013 38 | 38,Helsinki, Finland,"605,523",28 February 2013 39 | 39,Stuttgart, Germany,"604,297",31 December 2013 40 | 40,Glasgow, United Kingdom,"596,550",31 December 2013 41 | 41,Genoa, Italy,"594,774",31 July 2014 42 | 42,Düsseldorf, Germany,"593,682",31 December 2012 43 | 42,Dortmund, Germany,"575,944",31 December 2013 44 | 43,Essen, Germany,"569,884",31 December 2013 45 | 44,Málaga, Spain,"566,913",1 January 2014 46 | 45,Copenhagen, Denmark,"559,440",1 January 2013 47 | 46,Sheffield, United Kingdom,"557,382",30 June 2012 48 | 47,Lisbon, Portugal,"547,631",1 January 2011 49 | 48,Poznań, Poland,"547,161",31 March 2014 50 | 49,Bremen, Germany,"546,451",31 December 2012 51 | 50,Vilnius, Lithuania,"537,152",1 January 2013 52 | 51,Leipzig, Germany,"531,562",31 December 2013 53 | 52,Dresden, Germany,"530,754",31 December 2013 54 | 53,Gothenburg, Sweden,"528,014",31 March 2013 55 | 54,Dublin, Ireland,"527,612",10 April 2011 56 | 55,Bradford, United Kingdom,"524,619",30 June 2012 57 | 56,Hanover, Germany,"514,137",31 December 2012 58 | 57,The Hague, Netherlands,"510,909",1 May 2014 59 | 58,Manchester, United Kingdom,"510,772",30 June 2012 60 | 59,Antwerp, Belgium,"510,610",1 January 2014 61 | 60,Edinburgh, United Kingdom,"495,360",30 June 2011 62 | 61,Nuremberg, Germany,"495,121",31 December 2012 63 | 62,Duisburg, Germany,"486,816",31 December 2012 64 | 63,Lyon, France,"484,344",31 December 2010 65 | 64,Liverpool, United Kingdom,"469,690",30 June 2012 66 | 65,Gdańsk, Poland,"460,354",30 June 2012 67 | 66,Toulouse, France,"441,802",31 December 2010 68 | 67,Murcia, Spain,"441,354",1 January 2012 69 | 68,Tallinn, Estonia,"434,810",1 February 2015 70 | 69,Bristol, United Kingdom,"432,451",30 June 2012 71 | 70,Bratislava,Slovakia Slovak Republic,"417,389",31 December 2013 72 | 71,Szczecin, Poland,"409,211",30 June 2012 73 | 72,Palma de Mallorca, Spain,"407,648",1 January 2012 74 | 73,Bologna, Italy,"384,202",31 December 2013 75 | 74,Las Palmas, Spain,"382,296",1 January 2012 76 | 75,Florence, Italy,"377,207",31 December 2013 77 | 76,Brno, Czech Republic,"378,327",1 January 2013 78 | 77,Bydgoszcz, Poland,"362,286",30 June 2012 79 | 78,Bochum, Germany,"362,213",31 December 2012 80 | 79,Bilbao, Spain,"351,629",1 January 2012 81 | 80,Cardiff, United Kingdom,"348,493",30 June 2012 82 | 81,Lublin, Poland,"348,120",30 June 2012 83 | 82,Nice, France,"343,304",31 December 2010 84 | 83,Wuppertal, Germany,"342,885",31 December 2012 85 | 84,Plovdiv, Bulgaria,"341,041",31 December 2013 86 | 85,Varna, Bulgaria,"335,819",31 December 2013 87 | 86,Alicante, Spain,"334,678",1 January 2012 88 | 87,Leicester, United Kingdom,"331,606",30 June 2012 89 | 88,Utrecht, Netherlands,"330,772",1 May 2014 90 | 89,Córdoba, Spain,"328,841",1 January 2012 91 | 90,Bielefeld, Germany,"328,314",31 December 2012 92 | 91,Wakefield, United Kingdom,"327,627",30 June 2012 93 | 92,Aarhus, Denmark,"326,676",1 October 2014 94 | 93,Cluj-Napoca, Romania,"324,576",20 October 2011 95 | 94,Coventry, United Kingdom,"323,132",30 June 2012 96 | 95,Bari, Italy,"322,751",31 December 2013 97 | 96,Thessaloniki, Greece,"322,240",24 May 2011 98 | 97,Wirral, United Kingdom,"320,229",30 June 2012 99 | 98,Timișoara, Romania,"319,279",20 October 2011 100 | 99,Catania, Italy,"315,576",31 December 2013 101 | 100,Valladolid, Spain,"311,501",1 January 2012 102 | 101,Bonn, Germany,"309,869",31 December 2012 103 | 102,Malmö, Sweden,"309,105",31 March 2013 104 | 103,Nottingham, United Kingdom,"308,735",30 June 2012 105 | 104,Katowice, Poland,"308,269",30 June 2012 106 | 105,Kaunas, Lithuania,"306,888",1 January 2013 107 | -------------------------------------------------------------------------------- /fib.py: -------------------------------------------------------------------------------- 1 | 2 | def fib(N): 3 | """ 4 | Return a list of the first N Fibonacci numbers. 5 | """ 6 | f0, f1 = 0, 1 7 | f = [1] * N 8 | for n in range(1, N): 9 | f[n] = f0 + f1 10 | f0, f1 = f1, f[n] 11 | 12 | return f 13 | 14 | print(fib(10)) 15 | -------------------------------------------------------------------------------- /file1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/file1.py -------------------------------------------------------------------------------- /file2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/file2.py -------------------------------------------------------------------------------- /file3.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/file3.py -------------------------------------------------------------------------------- /guitar-echo.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/guitar-echo.wav -------------------------------------------------------------------------------- /hello.py: -------------------------------------------------------------------------------- 1 | print("Hello from Python!") 2 | -------------------------------------------------------------------------------- /pics/aggregate-funcs-illustrated.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/aggregate-funcs-illustrated.png -------------------------------------------------------------------------------- /pics/aggregate-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/aggregate-funcs.png -------------------------------------------------------------------------------- /pics/arithmetic-ops.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/arithmetic-ops.png -------------------------------------------------------------------------------- /pics/array-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/array-funcs.png -------------------------------------------------------------------------------- /pics/array-gen-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/array-gen-funcs.png -------------------------------------------------------------------------------- /pics/array-gen-funcs2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/array-gen-funcs2.png -------------------------------------------------------------------------------- /pics/array-slice-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/array-slice-funcs.png -------------------------------------------------------------------------------- /pics/autocompletion.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/autocompletion.png -------------------------------------------------------------------------------- /pics/bookcover-2nd-ed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/bookcover-2nd-ed.png -------------------------------------------------------------------------------- /pics/bookcover-reduced.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/bookcover-reduced.png -------------------------------------------------------------------------------- /pics/bookcover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/bookcover.png -------------------------------------------------------------------------------- /pics/broadcasting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/broadcasting.png -------------------------------------------------------------------------------- /pics/ch08-numerical-integration-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/ch08-numerical-integration-example.png -------------------------------------------------------------------------------- /pics/ch4-figure-1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/ch4-figure-1.pdf -------------------------------------------------------------------------------- /pics/ch4-figure-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/ch4-figure-2.pdf -------------------------------------------------------------------------------- /pics/conditional-logical-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/conditional-logical-funcs.png -------------------------------------------------------------------------------- /pics/constants-specials.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/constants-specials.png -------------------------------------------------------------------------------- /pics/convex-vs-nonconvex.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/convex-vs-nonconvex.png -------------------------------------------------------------------------------- /pics/coupled-damped-oscilators.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/coupled-damped-oscilators.png -------------------------------------------------------------------------------- /pics/dense-vs-sparse.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/dense-vs-sparse.png -------------------------------------------------------------------------------- /pics/direction-field-numint.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/direction-field-numint.png -------------------------------------------------------------------------------- /pics/direction-fields.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/direction-fields.png -------------------------------------------------------------------------------- /pics/direction-graphs-2nd-ex.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/direction-graphs-2nd-ex.png -------------------------------------------------------------------------------- /pics/dolfin-api-p1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/dolfin-api-p1.png -------------------------------------------------------------------------------- /pics/dolfin-api-p2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/dolfin-api-p2.png -------------------------------------------------------------------------------- /pics/dolfin-mshr-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/dolfin-mshr-funcs.png -------------------------------------------------------------------------------- /pics/element-wise-math-functs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/element-wise-math-functs.png -------------------------------------------------------------------------------- /pics/element-wise-math.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/element-wise-math.png -------------------------------------------------------------------------------- /pics/expression-tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/expression-tree.png -------------------------------------------------------------------------------- /pics/figures-axes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/figures-axes.png -------------------------------------------------------------------------------- /pics/font-properties.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/font-properties.png -------------------------------------------------------------------------------- /pics/graph.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/graph.pdf -------------------------------------------------------------------------------- /pics/graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/graph.png -------------------------------------------------------------------------------- /pics/graphtypes-2d.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/graphtypes-2d.png -------------------------------------------------------------------------------- /pics/hypothesis-test-cases.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/hypothesis-test-cases.png -------------------------------------------------------------------------------- /pics/indexing-viz.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/indexing-viz.png -------------------------------------------------------------------------------- /pics/lineprops1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/lineprops1.png -------------------------------------------------------------------------------- /pics/lineprops2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/lineprops2.png -------------------------------------------------------------------------------- /pics/lorentz-odes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/lorentz-odes.png -------------------------------------------------------------------------------- /pics/matrix-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/matrix-funcs.png -------------------------------------------------------------------------------- /pics/networkx-ops.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/networkx-ops.png -------------------------------------------------------------------------------- /pics/numerical-error-and-condition-num-vs-p.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/numerical-error-and-condition-num-vs-p.png -------------------------------------------------------------------------------- /pics/numpy-array-index-methods.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/numpy-array-index-methods.png -------------------------------------------------------------------------------- /pics/patsy-syntax.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/patsy-syntax.png -------------------------------------------------------------------------------- /pics/plot-types-1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-1.pdf -------------------------------------------------------------------------------- /pics/plot-types-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-2.pdf -------------------------------------------------------------------------------- /pics/plot-types-3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-3.pdf -------------------------------------------------------------------------------- /pics/plot-types-4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-4.pdf -------------------------------------------------------------------------------- /pics/plot-types-5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-5.pdf -------------------------------------------------------------------------------- /pics/plot-types-6.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-6.pdf -------------------------------------------------------------------------------- /pics/plot-types-7.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-7.pdf -------------------------------------------------------------------------------- /pics/plot-types-8.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-8.pdf -------------------------------------------------------------------------------- /pics/plot-types-9.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/plot-types-9.pdf -------------------------------------------------------------------------------- /pics/predator-prey-populations.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/predator-prey-populations.png -------------------------------------------------------------------------------- /pics/random-var-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/random-var-funcs.png -------------------------------------------------------------------------------- /pics/rcm-ordering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/rcm-ordering.png -------------------------------------------------------------------------------- /pics/rectangular-linear-equation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/rectangular-linear-equation.png -------------------------------------------------------------------------------- /pics/reshape-ops.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/reshape-ops.png -------------------------------------------------------------------------------- /pics/set-funcs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/set-funcs.png -------------------------------------------------------------------------------- /pics/simplifications.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/simplifications.png -------------------------------------------------------------------------------- /pics/software-stack-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/software-stack-overview.png -------------------------------------------------------------------------------- /pics/sparse-a-b-tensorprod.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/sparse-a-b-tensorprod.png -------------------------------------------------------------------------------- /pics/sparse-matrix-representations.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/sparse-matrix-representations.png -------------------------------------------------------------------------------- /pics/sparse-nz2diags.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/sparse-nz2diags.png -------------------------------------------------------------------------------- /pics/sum-by-axis.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/sum-by-axis.png -------------------------------------------------------------------------------- /pics/symbol-arguments.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/symbol-arguments.png -------------------------------------------------------------------------------- /pics/symbol-attributes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/symbol-attributes.png -------------------------------------------------------------------------------- /pics/sympy-matrix-ops.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/sympy-matrix-ops.png -------------------------------------------------------------------------------- /pics/tokyo-metro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/pics/tokyo-metro.png -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # usage: $python setup.py build_ext --inplace 2 | # instructs distutils to build extension module in same directory as source. 3 | 4 | from distutils.core import setup 5 | from Cython.Build import cythonize 6 | import numpy as np 7 | 8 | setup(ext_modules=cythonize('cy_sum.pyx'), 9 | include_dirs=[np.get_include()], 10 | requires=['Cython','numpy']) -------------------------------------------------------------------------------- /store.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/store.h5 -------------------------------------------------------------------------------- /tokyo-metro.json: -------------------------------------------------------------------------------- 1 | { 2 | "C": { 3 | "color": "#149848", 4 | "transfers": [ 5 | [ 6 | "C3", 7 | "F15" 8 | ], 9 | [ 10 | "C4", 11 | "Z2" 12 | ], 13 | [ 14 | "C4", 15 | "G2" 16 | ], 17 | [ 18 | "C7", 19 | "M14" 20 | ], 21 | [ 22 | "C7", 23 | "N6" 24 | ], 25 | [ 26 | "C7", 27 | "G6" 28 | ], 29 | [ 30 | "C8", 31 | "M15" 32 | ], 33 | [ 34 | "C8", 35 | "H6" 36 | ], 37 | [ 38 | "C9", 39 | "H7" 40 | ], 41 | [ 42 | "C9", 43 | "Y18" 44 | ], 45 | [ 46 | "C11", 47 | "T9" 48 | ], 49 | [ 50 | "C11", 51 | "M18" 52 | ], 53 | [ 54 | "C11", 55 | "Z8" 56 | ], 57 | [ 58 | "C12", 59 | "M19" 60 | ], 61 | [ 62 | "C18", 63 | "H21" 64 | ] 65 | ], 66 | "travel_times": [ 67 | [ 68 | "C1", 69 | "C2", 70 | 2 71 | ], 72 | [ 73 | "C2", 74 | "C3", 75 | 2 76 | ], 77 | [ 78 | "C3", 79 | "C4", 80 | 1 81 | ], 82 | [ 83 | "C4", 84 | "C5", 85 | 2 86 | ], 87 | [ 88 | "C5", 89 | "C6", 90 | 2 91 | ], 92 | [ 93 | "C6", 94 | "C7", 95 | 2 96 | ], 97 | [ 98 | "C7", 99 | "C8", 100 | 1 101 | ], 102 | [ 103 | "C8", 104 | "C9", 105 | 3 106 | ], 107 | [ 108 | "C9", 109 | "C10", 110 | 1 111 | ], 112 | [ 113 | "C10", 114 | "C11", 115 | 2 116 | ], 117 | [ 118 | "C11", 119 | "C12", 120 | 2 121 | ], 122 | [ 123 | "C12", 124 | "C13", 125 | 2 126 | ], 127 | [ 128 | "C13", 129 | "C14", 130 | 2 131 | ], 132 | [ 133 | "C14", 134 | "C15", 135 | 2 136 | ], 137 | [ 138 | "C15", 139 | "C16", 140 | 2 141 | ], 142 | [ 143 | "C16", 144 | "C17", 145 | 3 146 | ], 147 | [ 148 | "C17", 149 | "C18", 150 | 3 151 | ], 152 | [ 153 | "C18", 154 | "C19", 155 | 3 156 | ] 157 | ] 158 | }, 159 | "G": { 160 | "color": "#f59230", 161 | "transfers": [ 162 | [ 163 | "G1", 164 | "Z1" 165 | ], 166 | [ 167 | "G1", 168 | "F16" 169 | ], 170 | [ 171 | "G2", 172 | "Z2" 173 | ], 174 | [ 175 | "G2", 176 | "C4" 177 | ], 178 | [ 179 | "G4", 180 | "Z3" 181 | ], 182 | [ 183 | "G5", 184 | "M13" 185 | ], 186 | [ 187 | "G5", 188 | "Y16" 189 | ], 190 | [ 191 | "G5", 192 | "Z4" 193 | ], 194 | [ 195 | "G5", 196 | "N7" 197 | ], 198 | [ 199 | "G6", 200 | "N6" 201 | ], 202 | [ 203 | "G6", 204 | "M14" 205 | ], 206 | [ 207 | "G6", 208 | "C7" 209 | ], 210 | [ 211 | "G9", 212 | "M16" 213 | ], 214 | [ 215 | "G9", 216 | "H8" 217 | ], 218 | [ 219 | "G11", 220 | "T10" 221 | ], 222 | [ 223 | "G12", 224 | "Z9" 225 | ], 226 | [ 227 | "G15", 228 | "H16" 229 | ], 230 | [ 231 | "G16", 232 | "H17" 233 | ] 234 | ], 235 | "travel_times": [ 236 | [ 237 | "G1", 238 | "G2", 239 | 2 240 | ], 241 | [ 242 | "G2", 243 | "G3", 244 | 1 245 | ], 246 | [ 247 | "G3", 248 | "G4", 249 | 2 250 | ], 251 | [ 252 | "G4", 253 | "G5", 254 | 2 255 | ], 256 | [ 257 | "G5", 258 | "G6", 259 | 2 260 | ], 261 | [ 262 | "G6", 263 | "G7", 264 | 2 265 | ], 266 | [ 267 | "G7", 268 | "G8", 269 | 2 270 | ], 271 | [ 272 | "G8", 273 | "G9", 274 | 2 275 | ], 276 | [ 277 | "G9", 278 | "G10", 279 | 1 280 | ], 281 | [ 282 | "G10", 283 | "G11", 284 | 2 285 | ], 286 | [ 287 | "G11", 288 | "G12", 289 | 2 290 | ], 291 | [ 292 | "G12", 293 | "G13", 294 | 1 295 | ], 296 | [ 297 | "G13", 298 | "G14", 299 | 2 300 | ], 301 | [ 302 | "G14", 303 | "G15", 304 | 2 305 | ], 306 | [ 307 | "G15", 308 | "G16", 309 | 1 310 | ], 311 | [ 312 | "G16", 313 | "G17", 314 | 2 315 | ], 316 | [ 317 | "G17", 318 | "G18", 319 | 1 320 | ], 321 | [ 322 | "G18", 323 | "G19", 324 | 2 325 | ] 326 | ] 327 | }, 328 | "F": { 329 | "color": "#b96528", 330 | "transfers": [ 331 | [ 332 | "F1", 333 | "Y1" 334 | ], 335 | [ 336 | "F2", 337 | "Y2" 338 | ], 339 | [ 340 | "F3", 341 | "Y3" 342 | ], 343 | [ 344 | "F4", 345 | "Y4" 346 | ], 347 | [ 348 | "F5", 349 | "Y5" 350 | ], 351 | [ 352 | "F6", 353 | "Y6" 354 | ], 355 | [ 356 | "F7", 357 | "Y7" 358 | ], 359 | [ 360 | "F8", 361 | "Y8" 362 | ], 363 | [ 364 | "F9", 365 | "Y9" 366 | ], 367 | [ 368 | "F9", 369 | "M25" 370 | ], 371 | [ 372 | "F13", 373 | "M9" 374 | ], 375 | [ 376 | "F15", 377 | "C3" 378 | ], 379 | [ 380 | "F16", 381 | "Z1" 382 | ], 383 | [ 384 | "F16", 385 | "G1" 386 | ] 387 | ], 388 | "travel_times": [ 389 | [ 390 | "F1", 391 | "F2", 392 | 3 393 | ], 394 | [ 395 | "F2", 396 | "F3", 397 | 2 398 | ], 399 | [ 400 | "F3", 401 | "F4", 402 | 3 403 | ], 404 | [ 405 | "F4", 406 | "F5", 407 | 2 408 | ], 409 | [ 410 | "F5", 411 | "F6", 412 | 2 413 | ], 414 | [ 415 | "F6", 416 | "F7", 417 | 2 418 | ], 419 | [ 420 | "F7", 421 | "F8", 422 | 2 423 | ], 424 | [ 425 | "F8", 426 | "F9", 427 | 2 428 | ], 429 | [ 430 | "F9", 431 | "F10", 432 | 3 433 | ], 434 | [ 435 | "F10", 436 | "F11", 437 | 2 438 | ], 439 | [ 440 | "F11", 441 | "F12", 442 | 2 443 | ], 444 | [ 445 | "F12", 446 | "F13", 447 | 2 448 | ], 449 | [ 450 | "F13", 451 | "F14", 452 | 3 453 | ], 454 | [ 455 | "F14", 456 | "F15", 457 | 2 458 | ], 459 | [ 460 | "F15", 461 | "F16", 462 | 2 463 | ] 464 | ] 465 | }, 466 | "H": { 467 | "color": "#9cacb5", 468 | "transfers": [ 469 | [ 470 | "H6", 471 | "M15" 472 | ], 473 | [ 474 | "H6", 475 | "C8" 476 | ], 477 | [ 478 | "H7", 479 | "Y18" 480 | ], 481 | [ 482 | "H7", 483 | "C9" 484 | ], 485 | [ 486 | "H8", 487 | "M16" 488 | ], 489 | [ 490 | "H8", 491 | "G9" 492 | ], 493 | [ 494 | "H12", 495 | "T11" 496 | ], 497 | [ 498 | "H16", 499 | "G15" 500 | ], 501 | [ 502 | "H17", 503 | "G16" 504 | ], 505 | [ 506 | "H21", 507 | "C18" 508 | ] 509 | ], 510 | "travel_times": [ 511 | [ 512 | "H1", 513 | "H2", 514 | 3 515 | ], 516 | [ 517 | "H2", 518 | "H3", 519 | 3 520 | ], 521 | [ 522 | "H3", 523 | "H4", 524 | 3 525 | ], 526 | [ 527 | "H4", 528 | "H5", 529 | 3 530 | ], 531 | [ 532 | "H5", 533 | "H6", 534 | 2 535 | ], 536 | [ 537 | "H6", 538 | "H7", 539 | 3 540 | ], 541 | [ 542 | "H7", 543 | "H8", 544 | 1 545 | ], 546 | [ 547 | "H8", 548 | "H9", 549 | 2 550 | ], 551 | [ 552 | "H9", 553 | "H10", 554 | 2 555 | ], 556 | [ 557 | "H10", 558 | "H11", 559 | 2 560 | ], 561 | [ 562 | "H11", 563 | "H12", 564 | 1 565 | ], 566 | [ 567 | "H12", 568 | "H13", 569 | 3 570 | ], 571 | [ 572 | "H13", 573 | "H14", 574 | 1 575 | ], 576 | [ 577 | "H14", 578 | "H15", 579 | 2 580 | ], 581 | [ 582 | "H15", 583 | "H16", 584 | 2 585 | ], 586 | [ 587 | "H16", 588 | "H17", 589 | 1 590 | ], 591 | [ 592 | "H17", 593 | "H18", 594 | 2 595 | ], 596 | [ 597 | "H18", 598 | "H19", 599 | 2 600 | ], 601 | [ 602 | "H19", 603 | "H20", 604 | 2 605 | ], 606 | [ 607 | "H20", 608 | "H21", 609 | 3 610 | ] 611 | ] 612 | }, 613 | "M": { 614 | "color": "#ff0000", 615 | "transfers": [ 616 | [ 617 | "M9", 618 | "F13" 619 | ], 620 | [ 621 | "M12", 622 | "N8" 623 | ], 624 | [ 625 | "M13", 626 | "G5" 627 | ], 628 | [ 629 | "M13", 630 | "Y16" 631 | ], 632 | [ 633 | "M13", 634 | "Z4" 635 | ], 636 | [ 637 | "M13", 638 | "N7" 639 | ], 640 | [ 641 | "M14", 642 | "C7" 643 | ], 644 | [ 645 | "M14", 646 | "G6" 647 | ], 648 | [ 649 | "M14", 650 | "N6" 651 | ], 652 | [ 653 | "M15", 654 | "H6" 655 | ], 656 | [ 657 | "M15", 658 | "C8" 659 | ], 660 | [ 661 | "M16", 662 | "G9" 663 | ], 664 | [ 665 | "M16", 666 | "H8" 667 | ], 668 | [ 669 | "M18", 670 | "T9" 671 | ], 672 | [ 673 | "M18", 674 | "C11" 675 | ], 676 | [ 677 | "M18", 678 | "Z8" 679 | ], 680 | [ 681 | "M19", 682 | "C12" 683 | ], 684 | [ 685 | "M22", 686 | "N11" 687 | ], 688 | [ 689 | "M25", 690 | "Y9" 691 | ], 692 | [ 693 | "M25", 694 | "F9" 695 | ] 696 | ], 697 | "travel_times": [ 698 | [ 699 | "M1", 700 | "M2", 701 | 2 702 | ], 703 | [ 704 | "M2", 705 | "M3", 706 | 2 707 | ], 708 | [ 709 | "M3", 710 | "M4", 711 | 2 712 | ], 713 | [ 714 | "M4", 715 | "M5", 716 | 2 717 | ], 718 | [ 719 | "M5", 720 | "M6", 721 | 2 722 | ], 723 | [ 724 | "M6", 725 | "M7", 726 | 2 727 | ], 728 | [ 729 | "M7", 730 | "M8", 731 | 2 732 | ], 733 | [ 734 | "M8", 735 | "M9", 736 | 2 737 | ], 738 | [ 739 | "M9", 740 | "M10", 741 | 1 742 | ], 743 | [ 744 | "M10", 745 | "M11", 746 | 2 747 | ], 748 | [ 749 | "M11", 750 | "M12", 751 | 2 752 | ], 753 | [ 754 | "M12", 755 | "M13", 756 | 3 757 | ], 758 | [ 759 | "M13", 760 | "M14", 761 | 2 762 | ], 763 | [ 764 | "M14", 765 | "M15", 766 | 1 767 | ], 768 | [ 769 | "M15", 770 | "M16", 771 | 3 772 | ], 773 | [ 774 | "M16", 775 | "M17", 776 | 2 777 | ], 778 | [ 779 | "M17", 780 | "M18", 781 | 2 782 | ], 783 | [ 784 | "M18", 785 | "M19", 786 | 2 787 | ], 788 | [ 789 | "M19", 790 | "M20", 791 | 1 792 | ], 793 | [ 794 | "M20", 795 | "M21", 796 | 2 797 | ], 798 | [ 799 | "M21", 800 | "M22", 801 | 2 802 | ], 803 | [ 804 | "M22", 805 | "M23", 806 | 3 807 | ], 808 | [ 809 | "M23", 810 | "M24", 811 | 2 812 | ], 813 | [ 814 | "M24", 815 | "M25", 816 | 3 817 | ], 818 | [ 819 | "m3", 820 | "m4", 821 | 2 822 | ], 823 | [ 824 | "m4", 825 | "m5", 826 | 2 827 | ], 828 | [ 829 | "m5", 830 | "M6", 831 | 2 832 | ] 833 | ] 834 | }, 835 | "N": { 836 | "color": "#1aaca9", 837 | "transfers": [ 838 | [ 839 | "N1", 840 | "T1" 841 | ], 842 | [ 843 | "N2", 844 | "T2" 845 | ], 846 | [ 847 | "N3", 848 | "T3" 849 | ], 850 | [ 851 | "N6", 852 | "G6" 853 | ], 854 | [ 855 | "N6", 856 | "M14" 857 | ], 858 | [ 859 | "N6", 860 | "C7" 861 | ], 862 | [ 863 | "N7", 864 | "Y16" 865 | ], 866 | [ 867 | "N7", 868 | "Z4" 869 | ], 870 | [ 871 | "N7", 872 | "G5" 873 | ], 874 | [ 875 | "N7", 876 | "M13" 877 | ], 878 | [ 879 | "N8", 880 | "M12" 881 | ], 882 | [ 883 | "N9", 884 | "Y14" 885 | ], 886 | [ 887 | "N10", 888 | "Y13" 889 | ], 890 | [ 891 | "N10", 892 | "T6" 893 | ], 894 | [ 895 | "N11", 896 | "M22" 897 | ] 898 | ], 899 | "travel_times": [ 900 | [ 901 | "N1", 902 | "N2", 903 | 2 904 | ], 905 | [ 906 | "N2", 907 | "N3", 908 | 2 909 | ], 910 | [ 911 | "N3", 912 | "N4", 913 | 2 914 | ], 915 | [ 916 | "N4", 917 | "N5", 918 | 2 919 | ], 920 | [ 921 | "N5", 922 | "N6", 923 | 2 924 | ], 925 | [ 926 | "N6", 927 | "N7", 928 | 2 929 | ], 930 | [ 931 | "N7", 932 | "N8", 933 | 2 934 | ], 935 | [ 936 | "N8", 937 | "N9", 938 | 2 939 | ], 940 | [ 941 | "N9", 942 | "N10", 943 | 2 944 | ], 945 | [ 946 | "N10", 947 | "N11", 948 | 2 949 | ], 950 | [ 951 | "N11", 952 | "N12", 953 | 3 954 | ], 955 | [ 956 | "N12", 957 | "N13", 958 | 2 959 | ], 960 | [ 961 | "N13", 962 | "N14", 963 | 2 964 | ], 965 | [ 966 | "N14", 967 | "N15", 968 | 3 969 | ], 970 | [ 971 | "N15", 972 | "N16", 973 | 1 974 | ], 975 | [ 976 | "N16", 977 | "N17", 978 | 3 979 | ], 980 | [ 981 | "N17", 982 | "N18", 983 | 2 984 | ], 985 | [ 986 | "N18", 987 | "N19", 988 | 2 989 | ] 990 | ] 991 | }, 992 | "T": { 993 | "color": "#1aa7d8", 994 | "transfers": [ 995 | [ 996 | "T6", 997 | "N10" 998 | ], 999 | [ 1000 | "T6", 1001 | "Y13" 1002 | ], 1003 | [ 1004 | "T7", 1005 | "Z6" 1006 | ], 1007 | [ 1008 | "T9", 1009 | "M18" 1010 | ], 1011 | [ 1012 | "T9", 1013 | "C11" 1014 | ], 1015 | [ 1016 | "T9", 1017 | "Z8" 1018 | ], 1019 | [ 1020 | "T10", 1021 | "G11" 1022 | ], 1023 | [ 1024 | "T11", 1025 | "H12" 1026 | ] 1027 | ], 1028 | "travel_times": [ 1029 | [ 1030 | "T1", 1031 | "T2", 1032 | 0 1033 | ], 1034 | [ 1035 | "T2", 1036 | "T3", 1037 | 3 1038 | ], 1039 | [ 1040 | "T3", 1041 | "T4", 1042 | 6 1043 | ], 1044 | [ 1045 | "T4", 1046 | "T5", 1047 | 9 1048 | ], 1049 | [ 1050 | "T5", 1051 | "T6", 1052 | 11 1053 | ], 1054 | [ 1055 | "T6", 1056 | "T7", 1057 | 13 1058 | ], 1059 | [ 1060 | "T7", 1061 | "T8", 1062 | 14 1063 | ], 1064 | [ 1065 | "T8", 1066 | "T9", 1067 | 16 1068 | ], 1069 | [ 1070 | "T9", 1071 | "T10", 1072 | 18 1073 | ], 1074 | [ 1075 | "T10", 1076 | "T11", 1077 | 20 1078 | ], 1079 | [ 1080 | "T11", 1081 | "T12", 1082 | 21 1083 | ], 1084 | [ 1085 | "T12", 1086 | "T13", 1087 | 24 1088 | ], 1089 | [ 1090 | "T13", 1091 | "T14", 1092 | 26 1093 | ], 1094 | [ 1095 | "T14", 1096 | "T15", 1097 | 27 1098 | ], 1099 | [ 1100 | "T15", 1101 | "T16", 1102 | 30 1103 | ], 1104 | [ 1105 | "T16", 1106 | "T17", 1107 | 33 1108 | ], 1109 | [ 1110 | "T17", 1111 | "T18", 1112 | 35 1113 | ], 1114 | [ 1115 | "T18", 1116 | "T19", 1117 | 37 1118 | ], 1119 | [ 1120 | "T19", 1121 | "T20", 1122 | 39 1123 | ], 1124 | [ 1125 | "T20", 1126 | "T21", 1127 | 41 1128 | ], 1129 | [ 1130 | "T21", 1131 | "T22", 1132 | 43 1133 | ], 1134 | [ 1135 | "T22", 1136 | "T23", 1137 | 46 1138 | ], 1139 | [ 1140 | "T23", 1141 | "T24", 1142 | 49 1143 | ] 1144 | ] 1145 | }, 1146 | "Y": { 1147 | "color": "#ede7c3", 1148 | "transfers": [ 1149 | [ 1150 | "Y1", 1151 | "F1" 1152 | ], 1153 | [ 1154 | "Y2", 1155 | "F2" 1156 | ], 1157 | [ 1158 | "Y3", 1159 | "F3" 1160 | ], 1161 | [ 1162 | "Y4", 1163 | "F4" 1164 | ], 1165 | [ 1166 | "Y5", 1167 | "F5" 1168 | ], 1169 | [ 1170 | "Y6", 1171 | "F6" 1172 | ], 1173 | [ 1174 | "Y7", 1175 | "F7" 1176 | ], 1177 | [ 1178 | "Y8", 1179 | "F8" 1180 | ], 1181 | [ 1182 | "Y9", 1183 | "F9" 1184 | ], 1185 | [ 1186 | "Y9", 1187 | "M25" 1188 | ], 1189 | [ 1190 | "Y13", 1191 | "T6" 1192 | ], 1193 | [ 1194 | "Y13", 1195 | "N10" 1196 | ], 1197 | [ 1198 | "Y14", 1199 | "N9" 1200 | ], 1201 | [ 1202 | "Y16", 1203 | "Z4" 1204 | ], 1205 | [ 1206 | "Y16", 1207 | "N7" 1208 | ], 1209 | [ 1210 | "Y16", 1211 | "G5" 1212 | ], 1213 | [ 1214 | "Y16", 1215 | "M13" 1216 | ], 1217 | [ 1218 | "Y18", 1219 | "H7" 1220 | ], 1221 | [ 1222 | "Y18", 1223 | "C9" 1224 | ] 1225 | ], 1226 | "travel_times": [ 1227 | [ 1228 | "Y1", 1229 | "Y2", 1230 | 4 1231 | ], 1232 | [ 1233 | "Y2", 1234 | "Y3", 1235 | 2 1236 | ], 1237 | [ 1238 | "Y3", 1239 | "Y4", 1240 | 3 1241 | ], 1242 | [ 1243 | "Y4", 1244 | "Y5", 1245 | 2 1246 | ], 1247 | [ 1248 | "Y5", 1249 | "Y6", 1250 | 2 1251 | ], 1252 | [ 1253 | "Y6", 1254 | "Y7", 1255 | 2 1256 | ], 1257 | [ 1258 | "Y7", 1259 | "Y8", 1260 | 2 1261 | ], 1262 | [ 1263 | "Y8", 1264 | "Y9", 1265 | 3 1266 | ], 1267 | [ 1268 | "Y9", 1269 | "Y10", 1270 | 2 1271 | ], 1272 | [ 1273 | "Y10", 1274 | "Y11", 1275 | 2 1276 | ], 1277 | [ 1278 | "Y11", 1279 | "Y12", 1280 | 2 1281 | ], 1282 | [ 1283 | "Y12", 1284 | "Y13", 1285 | 3 1286 | ], 1287 | [ 1288 | "Y13", 1289 | "Y14", 1290 | 2 1291 | ], 1292 | [ 1293 | "Y14", 1294 | "Y15", 1295 | 2 1296 | ], 1297 | [ 1298 | "Y15", 1299 | "Y16", 1300 | 1 1301 | ], 1302 | [ 1303 | "Y16", 1304 | "Y17", 1305 | 2 1306 | ], 1307 | [ 1308 | "Y17", 1309 | "Y18", 1310 | 2 1311 | ], 1312 | [ 1313 | "Y18", 1314 | "Y19", 1315 | 2 1316 | ], 1317 | [ 1318 | "Y19", 1319 | "Y20", 1320 | 2 1321 | ], 1322 | [ 1323 | "Y20", 1324 | "Y21", 1325 | 2 1326 | ], 1327 | [ 1328 | "Y21", 1329 | "Y22", 1330 | 2 1331 | ], 1332 | [ 1333 | "Y22", 1334 | "Y23", 1335 | 3 1336 | ], 1337 | [ 1338 | "Y23", 1339 | "Y24", 1340 | 2 1341 | ] 1342 | ] 1343 | }, 1344 | "Z": { 1345 | "color": "#a384bf", 1346 | "transfers": [ 1347 | [ 1348 | "Z1", 1349 | "F16" 1350 | ], 1351 | [ 1352 | "Z1", 1353 | "G1" 1354 | ], 1355 | [ 1356 | "Z2", 1357 | "C4" 1358 | ], 1359 | [ 1360 | "Z2", 1361 | "G2" 1362 | ], 1363 | [ 1364 | "Z3", 1365 | "G4" 1366 | ], 1367 | [ 1368 | "Z4", 1369 | "Y16" 1370 | ], 1371 | [ 1372 | "Z4", 1373 | "N7" 1374 | ], 1375 | [ 1376 | "Z4", 1377 | "M13" 1378 | ], 1379 | [ 1380 | "Z4", 1381 | "G5" 1382 | ], 1383 | [ 1384 | "Z6", 1385 | "T7" 1386 | ], 1387 | [ 1388 | "Z8", 1389 | "M18" 1390 | ], 1391 | [ 1392 | "Z8", 1393 | "C11" 1394 | ], 1395 | [ 1396 | "Z8", 1397 | "T9" 1398 | ], 1399 | [ 1400 | "Z9", 1401 | "G12" 1402 | ] 1403 | ], 1404 | "travel_times": [ 1405 | [ 1406 | "Z1", 1407 | "Z2", 1408 | 3 1409 | ], 1410 | [ 1411 | "Z2", 1412 | "Z3", 1413 | 2 1414 | ], 1415 | [ 1416 | "Z3", 1417 | "Z4", 1418 | 2 1419 | ], 1420 | [ 1421 | "Z4", 1422 | "Z5", 1423 | 2 1424 | ], 1425 | [ 1426 | "Z5", 1427 | "Z6", 1428 | 2 1429 | ], 1430 | [ 1431 | "Z6", 1432 | "Z7", 1433 | 2 1434 | ], 1435 | [ 1436 | "Z7", 1437 | "Z8", 1438 | 2 1439 | ], 1440 | [ 1441 | "Z8", 1442 | "Z9", 1443 | 2 1444 | ], 1445 | [ 1446 | "Z9", 1447 | "Z10", 1448 | 3 1449 | ], 1450 | [ 1451 | "Z10", 1452 | "Z11", 1453 | 3 1454 | ], 1455 | [ 1456 | "Z11", 1457 | "Z12", 1458 | 3 1459 | ], 1460 | [ 1461 | "Z12", 1462 | "Z13", 1463 | 2 1464 | ], 1465 | [ 1466 | "Z13", 1467 | "Z14", 1468 | 2 1469 | ] 1470 | ] 1471 | } 1472 | } -------------------------------------------------------------------------------- /tokyo-metro.msgpack: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/tokyo-metro.msgpack -------------------------------------------------------------------------------- /tokyo-metro.pickle: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpcjp/Numeric-Python/528b9ca61c1e8f229add2c0409dbb90912d565d8/tokyo-metro.pickle --------------------------------------------------------------------------------