├── .gitignore ├── LICENSE ├── README.md ├── app-master ├── app.py ├── rf.gz ├── templates │ ├── base.html │ ├── form.html │ ├── plot.html │ ├── simple_page.html │ └── upload.html └── utils.py ├── app ├── app.py └── utils.py ├── dash ├── CoreGamma.csv ├── PorePermDensity.csv └── plug_explore.py ├── environment.yml ├── nb-master ├── _Hitting_our_web_API.ipynb └── _Hitting_some_web_APIs.ipynb └── notebooks ├── Building_a_flask_app.ipynb ├── Data Exploration-Clustering.ipynb ├── Fossil_classifier_minimal.ipynb ├── Hitting_our_web_API.ipynb ├── Hitting_some_web_APIs.ipynb ├── examples.txt └── images ├── 2226ff34-b4d5-43dd-a294-9f818fe49856-sunfish.jpg ├── 32971485793_13647ecbb2_b.jpg ├── 42485-116.jpg ├── 91228bcba9db1b55ab7cbe7f7f280e7c.png ├── Elphidium.jpg └── fish-fossil-green-river.jpg /.gitignore: -------------------------------------------------------------------------------- 1 | passwords.py 2 | 3 | # Byte-compiled / optimized / DLL files 4 | __pycache__/ 5 | *.py[cod] 6 | *$py.class 7 | 8 | # C extensions 9 | *.so 10 | 11 | # Distribution / packaging 12 | .Python 13 | build/ 14 | develop-eggs/ 15 | dist/ 16 | downloads/ 17 | eggs/ 18 | .eggs/ 19 | lib/ 20 | lib64/ 21 | parts/ 22 | sdist/ 23 | var/ 24 | wheels/ 25 | pip-wheel-metadata/ 26 | share/python-wheels/ 27 | *.egg-info/ 28 | .installed.cfg 29 | *.egg 30 | MANIFEST 31 | 32 | # PyInstaller 33 | # Usually these files are written by a python script from a template 34 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 35 | *.manifest 36 | *.spec 37 | 38 | # Installer logs 39 | pip-log.txt 40 | pip-delete-this-directory.txt 41 | 42 | # Unit test / coverage reports 43 | htmlcov/ 44 | .tox/ 45 | .nox/ 46 | .coverage 47 | .coverage.* 48 | .cache 49 | nosetests.xml 50 | coverage.xml 51 | *.cover 52 | *.py,cover 53 | .hypothesis/ 54 | .pytest_cache/ 55 | 56 | # Translations 57 | *.mo 58 | *.pot 59 | 60 | # Django stuff: 61 | *.log 62 | local_settings.py 63 | db.sqlite3 64 | db.sqlite3-journal 65 | 66 | # Flask stuff: 67 | instance/ 68 | .webassets-cache 69 | 70 | # Scrapy stuff: 71 | .scrapy 72 | 73 | # Sphinx documentation 74 | docs/_build/ 75 | 76 | # PyBuilder 77 | target/ 78 | 79 | # Jupyter Notebook 80 | .ipynb_checkpoints 81 | 82 | # IPython 83 | profile_default/ 84 | ipython_config.py 85 | 86 | # pyenv 87 | .python-version 88 | 89 | # pipenv 90 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 91 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 92 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 93 | # install all needed dependencies. 94 | #Pipfile.lock 95 | 96 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 97 | __pypackages__/ 98 | 99 | # Celery stuff 100 | celerybeat-schedule 101 | celerybeat.pid 102 | 103 | # SageMath parsed files 104 | *.sage.py 105 | 106 | # Environments 107 | .env 108 | .venv 109 | env/ 110 | venv/ 111 | ENV/ 112 | env.bak/ 113 | venv.bak/ 114 | 115 | # Spyder project settings 116 | .spyderproject 117 | .spyproject 118 | 119 | # Rope project settings 120 | .ropeproject 121 | 122 | # mkdocs documentation 123 | /site 124 | 125 | # mypy 126 | .mypy_cache/ 127 | .dmypy.json 128 | dmypy.json 129 | 130 | # Pyre type checker 131 | .pyre/ 132 | 133 | .DS_Store 134 | .vscode/ 135 | *.gz -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # transform-2020-tutorial 2 | 3 | **_Idea to MVP_ tutorial at TRANSFORM 2020** 4 | 5 | ---- 6 | 7 | This repo contains everything you need to follow along with the tutorial. You can clone it with `git` or use the green button to download a ZIP file. Put it where you would normally keep your project files. 8 | 9 | I'll assume you have the `conda` package and environment manager. (If you don't have it, you can install it [from here](https://docs.conda.io/en/latest/miniconda.html).) 10 | 11 | 12 | ## Create an environment 13 | 14 | conda env create -f environment.yml 15 | conda activate t20-fri-mvp 16 | python -m ipykernel install --user --name t20-fri-mvp 17 | 18 | Now you're ready to use the notebooks and the app in this repo. 19 | 20 | 21 | ## Agenda 22 | 23 | - Introduction to this problem 24 | - `notebooks/Data Exploration-Clustering.ipynb` 25 | - `notebooks/Fossil_classifier_minimal.ipynb` 26 | - `notebooks/Hitting_some_web_APIs.ipynb` 27 | - `notebooks/Building_a_flask_app.ipynb` (note that this is not a 'normal' notebook) 28 | - `notebooks/Hitting_our_web_API.ipynb` 29 | 30 | We will build everything else from scratch. However, there are 'complete' notebooks in `nb-master` if you prefer to read rather than type. And there is a complete app in `app-master` if you want the complete app. 31 | -------------------------------------------------------------------------------- /app-master/app.py: -------------------------------------------------------------------------------- 1 | from io import BytesIO 2 | import base64 3 | import joblib 4 | 5 | from flask import Flask, request, render_template, jsonify 6 | from PIL import Image 7 | 8 | import utils 9 | 10 | 11 | app = Flask(__name__) 12 | 13 | # Before running with `flask run` we must put ourselves in development mode: 14 | # Mac/Linux: export FLASK_ENV=development 15 | # Windows: set FLASK_ENV=development 16 | 17 | CLF = joblib.load('rf.gz') 18 | 19 | @app.route('/') 20 | def root(): 21 | """ 22 | (0, 1, 2) The most basic web app in the world. 23 | """ 24 | # print(request.headers) 25 | 26 | # name = request.args.get('name') 27 | 28 | # vp = float(request.args.get('vp') or 0) 29 | # rho = float(request.args.get('rho') or 0) 30 | # return "Impedance: {}".format(vp * rho) 31 | return "Hello world" 32 | 33 | @app.route('/impedance') 34 | def impedance(): 35 | """ 36 | (2) A ridiculously simple calculator. 37 | """ 38 | vp = float(request.args.get('vp') or 0) 39 | rho = float(request.args.get('rho') or 0) 40 | imp = vp * rho 41 | return "Impedance: {}".format(imp) 42 | 43 | 44 | @app.route('/hello/') 45 | def hello(name): 46 | """ 47 | (aside) 48 | Getting resources from the path. This is good for querying databases: 49 | path parameters represent entities (tables in your DB, more or less). 50 | """ 51 | return "Hello {}".format(name) 52 | 53 | 54 | @app.route('/predict') 55 | def predict(): 56 | """ 57 | (3) Make a prediction from a URL given via GET request. 58 | 59 | Using a URL means we can still just accept a string as an arg. 60 | 61 | There's still no human interface. 62 | """ 63 | url = request.args.get('url') 64 | img = utils.fetch_image(url) 65 | result = utils.predict_from_image(CLF, img) 66 | 67 | # Deal with not getting a URL. 68 | # if url: 69 | # img = utils.fetch_image(url) 70 | # result = utils.predict_from_image(CLF, img) 71 | # else: 72 | # result = 'Please provide a URL' 73 | 74 | return jsonify(result) 75 | 76 | ## We can also do that GET from a Python script! 77 | 78 | @app.route('/simple', methods=['GET']) 79 | def simple(): 80 | """ 81 | (4a) Render a template. 82 | """ 83 | return render_template('simple_page.html') 84 | 85 | ## You could add an About page as an exercise. 86 | 87 | ## A real website should also have a Terms page, especially 88 | ## if users are uploading anything or creating user instances. 89 | ## Check out https://www.termsfeed.com/ 90 | 91 | @app.route('/form', methods=['GET']) 92 | def form(): 93 | """ 94 | (4b) Make a prediction from a URL given via a GET form. 95 | """ 96 | url = request.args.get('url') 97 | if url: 98 | img = utils.fetch_image(url) 99 | result = utils.predict_from_image(CLF, img) 100 | result['url'] = url # If we add this back, we can display it. 101 | else: 102 | result = {} 103 | 104 | return render_template('form.html', result=result) 105 | 106 | 107 | @app.route('/upload', methods=['GET', 'POST']) 108 | def upload(): 109 | """ 110 | (5) Make a prediction from an image uploaded via a form. 111 | 112 | Bonus: on a mobile device, there will automatically be an option to 113 | capture via the camera. 114 | """ 115 | if request.method == 'POST': 116 | data = request.files['image'].read() 117 | img = Image.open(BytesIO(data)) 118 | result = utils.predict_from_image(CLF, img) 119 | result['image'] = base64.b64encode(data).decode('utf-8') 120 | else: 121 | result = {} 122 | 123 | return render_template('upload.html', result=result) 124 | 125 | ## Can you combine both forms into one, letting the user upload an image 126 | ## or provide a URL? 127 | 128 | @app.route('/plot', methods=["GET", "POST"]) 129 | def plot(): 130 | """ 131 | (6) This time we'll also send back a plot of the probabilities. 132 | 133 | We'll use the exact same code as (3), except we'll add the plot. 134 | """ 135 | if request.method == 'POST': 136 | data = request.files.get('image').read() 137 | img = Image.open(BytesIO(data)) 138 | result = utils.predict_from_image(CLF, img) 139 | result['image'] = base64.b64encode(data).decode('utf-8') 140 | 141 | # This is the only new line. 142 | result['plot'] = utils.plot(result['probs'], CLF.classes_) 143 | else: 144 | result = {} 145 | 146 | return render_template('plot.html', result=result) 147 | 148 | 149 | @app.route('/post', methods=['POST']) 150 | def post(): 151 | """ 152 | (7) Make a prediction from a URL provided via POST request. 153 | """ 154 | url = request.json.get('url') 155 | img = utils.fetch_image(url) 156 | result = utils.predict_from_image(CLF, img) 157 | return jsonify(result) 158 | 159 | 160 | @app.route('/api/v0.1', methods=['POST']) 161 | def api(): 162 | """ 163 | (8) Make a prediction from a base64-encoded image via POST request. 164 | 165 | If accessing the web API from code, you may not have a URL to pass to 166 | the service, and there is no form for doing a file upload. So we need 167 | a way to pass the image as data. There are lots of ways to do this; one 168 | way is to encode as base64. 169 | """ 170 | data = request.json.get('image') 171 | if data.startswith('http'): 172 | img = utils.fetch_image(data) 173 | else: 174 | img = Image.open(BytesIO(base64.b64decode(data))) 175 | result = utils.predict_from_image(CLF, img) 176 | return jsonify(result) 177 | -------------------------------------------------------------------------------- /app-master/rf.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/app-master/rf.gz -------------------------------------------------------------------------------- /app-master/templates/base.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | {% block title %}{% endblock %} - fossilnet 9 | 10 | 11 | 12 | 13 | 14 |
15 |

fossilnet

16 | 20 |
21 | 22 | {% block body %}{% endblock %} 23 | 24 | 28 | 29 | 30 | 31 | -------------------------------------------------------------------------------- /app-master/templates/form.html: -------------------------------------------------------------------------------- 1 | {% extends "base.html" %} 2 | {% block title %}Form{% endblock %} 3 | {% block body %} 4 | 5 | 8 | 9 |
10 | 11 | 12 |
13 | 14 | 20 | 21 | {% if result %} 22 | 23 |

{{ result['class'] }}: {{ "%.3f"|format(result['prob']) }}

24 | 25 | 32 | 33 | {% endif %} 34 | 35 | {% endblock %} -------------------------------------------------------------------------------- /app-master/templates/plot.html: -------------------------------------------------------------------------------- 1 | {% extends "base.html" %} 2 | {% block title %}Plot{% endblock %} 3 | {% block body %} 4 | 5 |
6 | 7 | 8 |
9 | 10 | 23 | 24 | {% if result %} 25 | 26 |

27 | {{ result['class'] }} 28 | 29 | p = {{ "%.3f"|format(result['prob']) }} 30 |

31 | 32 | 33 | 34 | 35 | {% endif %} 36 | 37 | {% endblock %} -------------------------------------------------------------------------------- /app-master/templates/simple_page.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Simple page - fossilnet 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 |
16 |

fossilnet

17 |
18 | 19 | 20 |

Hello world!

21 | 22 | 26 | 27 | 28 | 29 | -------------------------------------------------------------------------------- /app-master/templates/upload.html: -------------------------------------------------------------------------------- 1 | {% extends "base.html" %} 2 | {% block title %}Index{% endblock %} 3 | {% block body %} 4 | 5 |
6 | 7 | 8 |
9 | 10 | {% if result %} 11 | 12 |

13 | {{ result['class'] }}   14 | p = {{ "%.3f"|format(result['prob']) }} 15 |

16 | 17 | {% if result['image'] %} 18 | 19 | {% endif %} 20 | 21 | {% endif %} 22 | 23 | {% endblock %} -------------------------------------------------------------------------------- /app-master/utils.py: -------------------------------------------------------------------------------- 1 | import base64 2 | from io import BytesIO 3 | 4 | import numpy as np 5 | import requests 6 | from PIL import Image 7 | import matplotlib.pyplot as plt 8 | 9 | 10 | def plot(probs, class_names): 11 | """ 12 | Make a plot and return a base64 encoded string. 13 | """ 14 | y = list(range(len(probs))) 15 | y_min, y_max = y[0]-0.75, y[-1]+0.75 16 | 17 | fig, ax = plt.subplots(figsize=(8, 4)) 18 | bars = ax.barh(y, probs, color='orange', align='center', edgecolor='none') 19 | bars[np.argmax(probs)].set_color('red') 20 | ax.set_yticks(y) 21 | ax.set_yticklabels(class_names, size=12) 22 | ax.set_xscale('log') 23 | ax.set_ylim(y_max, y_min) # Label top-down. 24 | ax.grid(c='black', alpha=0.15, which='both') 25 | ax.patch.set_facecolor("white") 26 | fig.patch.set_facecolor("none") 27 | 28 | for i, p in enumerate(probs): 29 | ax.text(min(probs), i, "{:0.2e}".format(p), va='center') 30 | 31 | plt.tight_layout() 32 | 33 | # Put in memory. 34 | handle = BytesIO() 35 | plt.savefig(handle, format='png', facecolor=fig.get_facecolor()) 36 | plt.close() 37 | 38 | # Encode. 39 | handle.seek(0) 40 | figdata_png = base64.b64encode(handle.getvalue()).decode('utf8') 41 | 42 | return figdata_png 43 | 44 | def img_to_arr(img): 45 | """ 46 | Apply the same processing we used in training: greyscale and resize. 47 | """ 48 | img = img.convert(mode='L').resize((32, 32)) 49 | return np.asarray(img).ravel() / 255 50 | 51 | 52 | def fetch_image(url): 53 | """ 54 | Download an image from the web and pass to the image processing function. 55 | """ 56 | r = requests.get(url) 57 | f = BytesIO(r.content) 58 | return Image.open(f) 59 | 60 | 61 | def predict_from_image(clf, img): 62 | """ 63 | Classify an image. 64 | """ 65 | arr = img_to_arr(img) 66 | X = np.atleast_2d(arr) 67 | probs = clf.predict_proba(X) 68 | result = { 69 | 'class': clf.classes_[np.argmax(probs)], 70 | 'prob': probs.max(), 71 | 'classes': clf.classes_.tolist(), 72 | 'probs': np.squeeze(probs).tolist(), # Must be serializable. 73 | } 74 | return result 75 | -------------------------------------------------------------------------------- /app/app.py: -------------------------------------------------------------------------------- 1 | from flask import Flask 2 | 3 | app = Flask(__name__) 4 | 5 | # Before running with `flask run` we must put ourselves in development mode: 6 | # Mac/Linux: export FLASK_ENV=development 7 | # Windows: set FLASK_ENV=development 8 | 9 | @app.route('/') 10 | def root(): 11 | """ 12 | (0) The most basic web app in the world. 13 | """ 14 | return "Hello world!" 15 | -------------------------------------------------------------------------------- /app/utils.py: -------------------------------------------------------------------------------- 1 | import base64 2 | from io import BytesIO 3 | 4 | import numpy as np 5 | import requests 6 | from PIL import Image 7 | import matplotlib.pyplot as plt 8 | 9 | 10 | def plot(probs, class_names): 11 | """ 12 | Make a plot and return a base64 encoded string. 13 | """ 14 | y = list(range(len(probs))) 15 | y_min, y_max = y[0]-0.75, y[-1]+0.75 16 | 17 | fig, ax = plt.subplots(figsize=(8, 4)) 18 | bars = ax.barh(y, probs, color='orange', align='center', edgecolor='none') 19 | bars[np.argmax(probs)].set_color('red') 20 | ax.set_yticks(y) 21 | ax.set_yticklabels(class_names, size=12) 22 | ax.set_xscale('log') 23 | ax.set_ylim(y_max, y_min) # Label top-down. 24 | ax.grid(c='black', alpha=0.15, which='both') 25 | ax.patch.set_facecolor("white") 26 | fig.patch.set_facecolor("none") 27 | 28 | for i, p in enumerate(probs): 29 | ax.text(min(probs), i, "{:0.2e}".format(p), va='center') 30 | 31 | plt.tight_layout() 32 | 33 | # Put in memory. 34 | handle = BytesIO() 35 | plt.savefig(handle, format='png', facecolor=fig.get_facecolor()) 36 | plt.close() 37 | 38 | # Encode. 39 | handle.seek(0) 40 | figdata_png = base64.b64encode(handle.getvalue()).decode('utf8') 41 | 42 | return figdata_png 43 | 44 | # We need the following: 45 | # - im_to_arr 46 | # - fetch_image 47 | # - predict_from_image 48 | -------------------------------------------------------------------------------- /dash/CoreGamma.csv: -------------------------------------------------------------------------------- 1 | Gamma,Depth 2 | 105.3,4963.01 3 | 106.3,4963.06 4 | 105.8,4963.11 5 | 105.3,4963.16 6 | 103.0,4963.21 7 | 100.9,4963.26 8 | 97.8,4963.31 9 | 98.0,4963.36 10 | 97.2,4963.41 11 | 93.8,4963.46 12 | 91.0,4963.51 13 | 85.1,4963.56 14 | 81.4,4963.61 15 | 80.2,4963.66 16 | 80.7,4963.71 17 | 82.0,4963.76 18 | 80.9,4963.81 19 | 82.1,4963.86 20 | 83.7,4963.91 21 | 87.5,4963.96 22 | 90.3,4964.01 23 | 91.6,4964.06 24 | 94.1,4964.11 25 | 96.8,4964.16 26 | 99.3,4964.21 27 | 100.6,4964.26 28 | 100.2,4964.31 29 | 101.2,4964.36 30 | 103.9,4964.41 31 | 108.9,4964.46 32 | 113.4,4964.51 33 | 118.2,4964.56 34 | 122.7,4964.61 35 | 126.7,4964.66 36 | 131.7,4964.71 37 | 136.9,4964.76 38 | 142.0,4964.81 39 | 148.9,4964.86 40 | 155.9,4964.91 41 | 164.5,4964.96 42 | 164.5,4964.96 43 | 169.8,4965.01 44 | 172.5,4965.06 45 | 173.9,4965.11 46 | 175.7,4965.16 47 | 177.1,4965.21 48 | 178.3,4965.26 49 | 180.3,4965.31 50 | 182.6,4965.36 51 | 184.0,4965.41 52 | 184.3,4965.46 53 | 182.7,4965.51 54 | 181.2,4965.56 55 | 176.6,4965.61 56 | 173.0,4965.66 57 | 169.5,4965.71 58 | 169.7,4965.76 59 | 168.3,4965.81 60 | 165.8,4965.86 61 | 165.0,4965.91 62 | 166.2,4965.96 63 | 164.0,4966.01 64 | 162.0,4966.06 65 | 162.4,4966.11 66 | 161.7,4966.16 67 | 161.0,4966.21 68 | 162.2,4966.26 69 | 164.5,4966.31 70 | 165.8,4966.36 71 | 164.1,4966.41 72 | 163.7,4966.46 73 | 164.7,4966.51 74 | 163.8,4966.56 75 | 161.7,4966.61 76 | 160.8,4966.66 77 | 160.5,4966.71 78 | 160.2,4966.76 79 | 160.3,4966.81 80 | 158.6,4966.86 81 | 158.1,4966.91 82 | 155.4,4966.96 83 | 154.7,4967.01 84 | 155.1,4967.06 85 | 156.1,4967.11 86 | 156.9,4967.16 87 | 155.8,4967.21 88 | 155.2,4967.26 89 | 156.2,4967.31 90 | 156.0,4967.36 91 | 153.5,4967.41 92 | 152.9,4967.46 93 | 152.7,4967.51 94 | 151.0,4967.56 95 | 149.3,4967.61 96 | 146.7,4967.66 97 | 145.7,4967.71 98 | 142.4,4967.76 99 | 140.3,4967.81 100 | 140.5,4967.86 101 | 140.8,4967.91 102 | 140.6,4967.96 103 | 140.7,4968.01 104 | 140.4,4968.06 105 | 139.0,4968.11 106 | 138.7,4968.16 107 | 139.1,4968.21 108 | 139.6,4968.26 109 | 142.1,4968.31 110 | 142.9,4968.36 111 | 144.6,4968.41 112 | 144.5,4968.46 113 | 143.0,4968.51 114 | 140.3,4968.56 115 | 140.1,4968.61 116 | 139.5,4968.66 117 | 139.3,4968.71 118 | 139.8,4968.76 119 | 140.4,4968.81 120 | 139.4,4968.86 121 | 140.9,4968.91 122 | 141.0,4968.96 123 | 140.6,4969.01 124 | 140.5,4969.06 125 | 141.5,4969.11 126 | 142.8,4969.16 127 | 144.5,4969.21 128 | 143.5,4969.26 129 | 144.9,4969.31 130 | 147.1,4969.36 131 | 148.3,4969.41 132 | 149.4,4969.46 133 | 150.8,4969.51 134 | 150.6,4969.56 135 | 150.2,4969.61 136 | 149.1,4969.66 137 | 147.9,4969.71 138 | 147.4,4969.76 139 | 147.5,4969.81 140 | 148.4,4969.86 141 | 148.5,4969.91 142 | 146.9,4969.96 143 | 146.2,4970.01 144 | 145.2,4970.06 145 | 145.8,4970.11 146 | 144.8,4970.16 147 | 143.7,4970.21 148 | 143.0,4970.26 149 | 142.1,4970.31 150 | 142.3,4970.36 151 | 139.2,4970.41 152 | 134.3,4970.46 153 | 127.9,4970.51 154 | 122.1,4970.56 155 | 116.0,4970.61 156 | 109.7,4970.66 157 | 103.0,4970.71 158 | 95.2,4970.76 159 | 89.7,4970.81 160 | 84.5,4970.86 161 | 77.6,4970.91 162 | 72.1,4970.96 163 | 65.7,4971.01 164 | 62.0,4971.06 165 | 59.7,4971.11 166 | 58.5,4971.16 167 | 57.4,4971.21 168 | 58.3,4971.26 169 | 57.7,4971.31 170 | 57.4,4971.36 171 | 57.3,4971.41 172 | 56.9,4971.46 173 | 56.8,4971.51 174 | 58.1,4971.56 175 | 58.9,4971.61 176 | 60.6,4971.66 177 | 61.1,4971.71 178 | 59.7,4971.76 179 | 62.0,4971.81 180 | 61.9,4971.86 181 | 60.0,4971.91 182 | 58.5,4971.96 183 | 58.5,4972.01 184 | 58.1,4972.06 185 | 57.2,4972.11 186 | 56.1,4972.16 187 | 54.8,4972.21 188 | 54.0,4972.26 189 | 52.0,4972.31 190 | 52.4,4972.36 191 | 53.7,4972.41 192 | 51.5,4972.46 193 | 52.7,4972.51 194 | 54.2,4972.56 195 | 55.7,4972.61 196 | 57.7,4972.66 197 | 59.2,4972.71 198 | 60.8,4972.76 199 | 61.7,4972.81 200 | 64.11,4972.86 201 | 65.3,4972.91 202 | 67.2,4972.96 203 | 66.4,4973.01 204 | 67.3,4973.06 205 | 67.4,4973.11 206 | 67.2,4973.16 207 | 67.4,4973.21 208 | 68.8,4973.26 209 | 68.6,4973.31 210 | 68.2,4973.36 211 | 68.6,4973.41 212 | 69.0,4973.46 213 | 66.7,4973.51 214 | 66.6,4973.56 215 | 64.1,4973.61 216 | 63.9,4973.66 217 | 62.3,4973.71 218 | 62.1,4973.76 219 | 61.1,4973.81 220 | 59.8,4973.86 221 | 58.8,4973.91 222 | 57.1,4973.96 223 | 56.3,4974.01 224 | 55.7,4974.06 225 | 56.0,4974.11 226 | 57.0,4974.16 227 | 56.6,4974.21 228 | 58.7,4974.26 229 | 58.2,4974.31 230 | 57.5,4974.36 231 | 57.9,4974.41 232 | 56.9,4974.46 233 | 56.5,4974.51 234 | 55.6,4974.56 235 | 56.6,4974.61 236 | 57.1,4974.66 237 | 57.5,4974.71 238 | 57.3,4974.76 239 | 57.3,4974.81 240 | 58.0,4974.86 241 | 58.0,4974.91 242 | 59.8,4974.96 243 | 63.2,4975.01 244 | 64.5,4975.06 245 | 68.8,4975.11 246 | 70.4,4975.16 247 | 71.1,4975.21 248 | 70.6,4975.26 249 | 70.1,4975.31 250 | 70.2,4975.36 251 | 69.7,4975.41 252 | 70.2,4975.46 253 | 69.8,4975.51 254 | 70.0,4975.56 255 | 71.1,4975.61 256 | 68.9,4975.66 257 | 69.3,4975.71 258 | 67.1,4975.76 259 | 67.1,4975.81 260 | 67.7,4975.86 261 | 68.8,4975.91 262 | 72.1,4975.96 263 | 74.9,4976.01 264 | 77.6,4976.06 265 | 81.8,4976.11 266 | 86.7,4976.16 267 | 93.4,4976.21 268 | 98.8,4976.26 269 | 107.2,4976.31 270 | 114.3,4976.36 271 | 123.0,4976.41 272 | 129.1,4976.46 273 | 137.4,4976.51 274 | 146.1,4976.56 275 | 151.9,4976.61 276 | 157.6,4976.66 277 | 165.1,4976.71 278 | 172.0,4976.76 279 | 179.2,4976.81 280 | 182.2,4976.86 281 | 184.8,4976.91 282 | 186.9,4976.96 283 | 188.5,4977.01 284 | 189.2,4977.06 285 | 191.6,4977.11 286 | 191.5,4977.16 287 | 190.7,4977.21 288 | 190.1,4977.26 289 | 189.8,4977.31 290 | 189.3,4977.36 291 | 188.9,4977.41 292 | 188.9,4977.46 293 | 190.5,4977.51 294 | 193.1,4977.56 295 | 195.5,4977.61 296 | 197.1,4977.66 297 | 197.7,4977.71 298 | 194.1,4977.76 299 | 186.0,4977.81 300 | 175.8,4977.86 301 | 165.4,4977.91 302 | 155.5,4977.96 303 | 142.8,4978.01 304 | 129.1,4978.06 305 | 115.5,4978.11 306 | 101.5,4978.16 307 | 86.4,4978.21 308 | 71.4,4978.26 309 | 57.3,4978.31 310 | 44.1,4978.36 311 | 36.1,4978.41 312 | 33.3,4978.46 313 | 32.3,4978.51 314 | 30.5,4978.56 315 | 29.4,4978.61 316 | 30.1,4978.66 317 | 30.1,4978.71 318 | 29.5,4978.76 319 | 28.4,4978.81 320 | 28.2,4978.86 321 | 27.9,4978.91 322 | 27.1,4978.96 323 | 27.2,4979.01 324 | 28.1,4979.06 325 | 29.2,4979.11 326 | 28.9,4979.16 327 | 31.2,4979.21 328 | 32.2,4979.26 329 | 33.7,4979.31 330 | 36.4,4979.36 331 | 41.7,4979.41 332 | 47.4,4979.46 333 | 51.8,4979.51 334 | 52.4,4979.56 335 | 52.0,4979.61 336 | 51.3,4979.66 337 | 49.1,4979.71 338 | 45.8,4979.76 339 | 44.9,4979.81 340 | 42.7,4979.86 341 | 39.9,4979.91 342 | 36.3,4979.96 343 | 32.0,4980.01 344 | 24.4,4980.06 345 | 17.4,4980.11 346 | 10.2,4980.16 347 | 8.5,4980.21 348 | 8.0,4980.26 349 | 7.2,4980.31 350 | 6.4,4980.36 351 | 6.6,4980.41 352 | 6.0,4980.46 353 | 5.5,4980.51 354 | 5.1,4980.56 355 | 4.9,4980.61 356 | 4.1,4980.66 357 | 4.2,4980.71 358 | 4.9,4980.76 359 | 5.8,4980.81 360 | 5.1,4980.86 361 | 5.4,4980.91 362 | 5.5,4980.96 363 | 5.4,4981.01 364 | 5.9,4981.06 365 | 6.6,4981.11 366 | 6.3,4981.16 367 | 7.6,4981.21 368 | 7.6,4981.26 369 | 8.0,4981.31 370 | 7.9,4981.36 371 | 7.9,4981.41 372 | 7.8,4981.46 373 | 7.9,4981.51 374 | 7.7,4981.56 375 | 6.6,4981.61 376 | 7.7,4981.66 377 | 7.7,4981.71 378 | 6.8,4981.76 379 | 7.3,4981.81 380 | 5.8,4981.86 381 | 5.7,4981.91 382 | 6.2,4981.96 383 | 7.7,4982.01 384 | 7.8,4982.06 385 | 8.8,4982.11 386 | 7.9,4982.16 387 | 6.7,4982.21 388 | 8.5,4982.26 389 | 8.8,4982.31 390 | 8.2,4982.36 391 | 9.4,4982.41 392 | 8.9,4982.46 393 | 9.1,4982.51 394 | 9.6,4982.56 395 | 9.0,4982.61 396 | 8.8,4982.66 397 | 8.5,4982.71 398 | 8.8,4982.76 399 | 8.9,4982.81 400 | 9.5,4982.86 401 | 8.1,4982.91 402 | 7.4,4982.96 403 | 7.0,4983.01 404 | 6.34,4983.06 405 | 6.5,4983.11 406 | 5.6,4983.16 407 | 5.2,4983.21 408 | 6.3,4983.26 409 | 6.2,4983.31 410 | 7.4,4983.36 411 | 7.6,4983.41 412 | 8.0,4983.46 413 | 8.7,4983.51 414 | 8.8,4983.56 415 | 9.0,4983.61 416 | 9.2,4983.66 417 | 9.6,4983.71 418 | 9.3,4983.76 419 | 11.0,4983.81 420 | 11.7,4983.86 421 | 11.7,4983.91 422 | 10.3,4983.96 423 | 8.4,4984.01 424 | 6.3,4984.06 425 | 6.3,4984.11 426 | 5.5,4984.16 427 | 5.1,4984.21 428 | 3.8,4984.26 429 | 3.8,4984.31 430 | 3.4,4984.36 431 | 3.0,4984.41 432 | 2.0,4984.46 433 | 1.0,4984.51 434 | 0.7,4984.56 435 | 0.0,4984.61 436 | 0.9,4984.66 437 | 2.4,4984.71 438 | 2.4,4984.76 439 | 2.8,4984.81 440 | 4.5,4984.86 441 | 4.9,4984.91 442 | 5.6,4984.96 443 | 5.3,4985.01 444 | 6.5,4985.06 445 | 7.2,4985.11 446 | 8.0,4985.16 447 | 6.9,4985.21 448 | 8.3,4985.26 449 | 8.0,4985.31 450 | 8.0,4985.36 451 | 8.2,4985.41 452 | 8.7,4985.46 453 | 8.6,4985.51 454 | 9.2,4985.56 455 | 9.8,4985.61 456 | 11.9,4985.66 457 | 12.5,4985.71 458 | 11.8,4985.76 459 | 10.6,4985.81 460 | 11.1,4985.86 461 | 10.9,4985.91 462 | 11.7,4985.96 463 | 11.6,4986.01 464 | 11.6,4986.06 465 | 11.3,4986.11 466 | 9.7,4986.16 467 | 9.8,4986.21 468 | 8.2,4986.26 469 | 6.6,4986.31 470 | 5.6,4986.36 471 | 5.4,4986.41 472 | 7.4,4986.46 473 | 7.7,4986.51 474 | 7.7,4986.56 475 | 7.4,4986.61 476 | 6.7,4986.66 477 | 7.6,4986.71 478 | 8.5,4986.76 479 | 9.8,4986.81 480 | 9.2,4986.86 481 | 10.1,4986.91 482 | 10.3,4986.96 483 | 9.2,4987.01 484 | 8.5,4987.06 485 | 7.1,4987.11 486 | 6.2,4987.16 487 | 7.2,4987.21 488 | 6.8,4987.26 489 | 7.3,4987.31 490 | 6.2,4987.36 491 | 4.7,4987.41 492 | 3.3,4987.46 493 | 4.7,4987.51 494 | 4.4,4987.56 495 | 3.3,4987.61 496 | 3.6,4987.66 497 | 5.1,4987.71 498 | 5.4,4987.76 499 | 6.1,4987.81 500 | 4.7,4987.86 501 | 4.4,4987.91 502 | 4.2,4987.96 503 | 5.3,4988.01 504 | 6.6,4988.06 505 | 6.7,4988.11 506 | 5.9,4988.16 507 | 5.7,4988.21 508 | 6.8,4988.26 509 | 6.7,4988.31 510 | 7.2,4988.36 511 | 7.0,4988.41 512 | 7.8,4988.46 513 | 7.9,4988.51 514 | 8.2,4988.56 515 | 8.7,4988.61 516 | 6.7,4988.66 517 | 6.2,4988.71 518 | 5.8,4988.76 519 | 5.4,4988.81 520 | 5.1,4988.86 521 | 5.2,4988.91 522 | 6.2,4988.96 523 | 6.0,4989.01 524 | 5.5,4989.06 525 | 5.4,4989.11 526 | 6.1,4989.16 527 | 6.2,4989.21 528 | 5.7,4989.26 529 | 7.1,4989.31 530 | 7.0,4989.36 531 | 6.5,4989.41 532 | 6.2,4989.46 533 | 5.8,4989.51 534 | 5.3,4989.56 535 | 6.4,4989.61 536 | 5.6,4989.66 537 | 6.3,4989.71 538 | 5.5,4989.76 539 | 4.7,4989.81 540 | 4.3,4989.86 541 | 3.9,4989.91 542 | 13.0,4993.06 543 | 13.3,4993.11 544 | 14.3,4993.16 545 | 14.4,4993.21 546 | 15.2,4993.26 547 | 14.2,4993.31 548 | 13.0,4993.36 549 | 14.8,4993.41 550 | 14.8,4993.46 551 | 15.4,4993.51 552 | 15.4,4993.56 553 | 14.6,4993.61 554 | 14.4,4993.66 555 | 14.3,4993.71 556 | 14.0,4993.76 557 | 14.6,4993.81 558 | 14.8,4993.86 559 | 15.8,4993.91 560 | 16.6,4993.96 561 | 18.0,4994.01 562 | 17.2,4994.06 563 | 16.0,4994.11 564 | 17.3,4994.16 565 | 20.8,4994.21 566 | 23.8,4994.26 567 | 28.0,4994.31 568 | 30.4,4994.36 569 | 32.5,4994.41 570 | 31.0,4994.46 571 | 30.5,4994.51 572 | 27.7,4994.56 573 | 27.8,4994.61 574 | 26.7,4994.66 575 | 27.5,4994.71 576 | 28.2,4994.76 577 | 26.8,4994.81 578 | 22.8,4994.86 579 | 19.1,4994.91 580 | 13.3,4994.96 581 | 13.3,4994.96 582 | 10.4,4995.01 583 | 7.7,4995.06 584 | 7.6,4995.11 585 | 7.8,4995.16 586 | 8.9,4995.21 587 | 9.0,4995.26 588 | 9.3,4995.31 589 | 8.2,4995.36 590 | 7.5,4995.41 591 | 7.5,4995.46 592 | 6.6,4995.51 593 | 6.2,4995.56 594 | 8.2,4995.61 595 | 8.7,4995.66 596 | 8.7,4995.71 597 | 7.7,4995.76 598 | 7.2,4995.81 599 | 7.1,4995.86 600 | 6.8,4995.91 601 | 6.6,4995.96 602 | 6.5,4996.01 603 | 7.0,4996.06 604 | 6.9,4996.11 605 | 7.5,4996.16 606 | 9.2,4996.21 607 | 8.7,4996.26 608 | 8.9,4996.31 609 | 8.5,4996.36 610 | 10.0,4996.41 611 | 9.1,4996.46 612 | 8.8,4996.51 613 | 8.6,4996.56 614 | 8.2,4996.61 615 | 9.6,4996.66 616 | 8.9,4996.71 617 | 9.0,4996.76 618 | 10.8,4996.81 619 | 10.7,4996.86 620 | 10.2,4996.91 621 | 10.3,4996.96 622 | 10.3,4997.01 623 | 10.0,4997.06 624 | 11.5,4997.11 625 | 11.5,4997.16 626 | 11.1,4997.21 627 | 11.8,4997.26 628 | 10.6,4997.31 629 | 11.4,4997.36 630 | 12.6,4997.41 631 | 11.9,4997.46 632 | 10.5,4997.51 633 | 11.0,4997.56 634 | 10.6,4997.61 635 | 12.6,4997.66 636 | 12.6,4997.71 637 | 13.3,4997.76 638 | 14.0,4997.81 639 | 15.3,4997.86 640 | 15.3,4997.91 641 | 16.8,4997.96 642 | 16.2,4998.01 643 | 14.2,4998.06 644 | 13.2,4998.11 645 | 13.7,4998.16 646 | 13.3,4998.21 647 | 13.2,4998.26 648 | 13.3,4998.31 649 | 12.8,4998.36 650 | 11.7,4998.41 651 | 12.6,4998.46 652 | 12.6,4998.51 653 | 14.3,4998.56 654 | 14.6,4998.61 655 | 16.1,4998.66 656 | 17.1,4998.71 657 | 18.2,4998.76 658 | 19.9,4998.81 659 | 22.0,4998.86 660 | 24.0,4998.91 661 | 22.7,4998.96 662 | 22.6,4999.01 663 | 23.6,4999.06 664 | 22.6,4999.11 665 | 21.6,4999.16 666 | 20.8,4999.21 667 | 19.6,4999.26 668 | 18.8,4999.31 669 | 18.5,4999.36 670 | 18.5,4999.41 671 | 17.5,4999.46 672 | 16.2,4999.51 673 | 15.2,4999.56 674 | 16.9,4999.61 675 | 19.5,4999.66 676 | 20.5,4999.71 677 | 19.2,4999.76 678 | 20.3,4999.81 679 | 20.5,4999.86 680 | 21.4,4999.91 681 | 22.2,4999.96 682 | 23.5,5000.01 683 | 22.8,5000.06 684 | 23.4,5000.11 685 | 22.4,5000.16 686 | 23.7,5000.21 687 | 22.8,5000.26 688 | 20.9,5000.31 689 | 19.0,5000.36 690 | 19.7,5000.41 691 | 19.2,5000.46 692 | 18.1,5000.51 693 | 16.5,5000.56 694 | 16.1,5000.61 695 | 14.5,5000.66 696 | 14.7,5000.71 697 | 14.3,5000.76 698 | 15.4,5000.81 699 | 13.7,5000.86 700 | 12.8,5000.91 701 | 13.7,5000.96 702 | 13.6,5001.01 703 | 13.2,5001.06 704 | 13.3,5001.11 705 | 13.6,5001.16 706 | 13.7,5001.21 707 | 13.0,5001.26 708 | 12.8,5001.31 709 | 13.4,5001.36 710 | 12.4,5001.41 711 | 12.3,5001.46 712 | 12.3,5001.51 713 | 12.3,5001.56 714 | 10.2,5001.61 715 | 9.8,5001.66 716 | 10.5,5001.71 717 | 10.5,5001.76 718 | 10.2,5001.81 719 | 10.3,5001.86 720 | 10.8,5001.91 721 | 12.0,5001.96 722 | 12.9,5002.01 723 | 14.8,5002.06 724 | 14.1,5002.11 725 | 13.8,5002.16 726 | 13.7,5002.21 727 | 15.0,5002.26 728 | 14.6,5002.31 729 | 13.8,5002.36 730 | 12.8,5002.41 731 | 13.3,5002.46 732 | 14.0,5002.51 733 | 12.2,5002.56 734 | 10.9,5002.61 735 | 8.3,5002.66 736 | 6.2,5002.71 737 | 5.2,5002.76 738 | 5.9,5002.81 739 | 6.2,5002.86 740 | 4.8,5002.91 741 | 5.7,5002.96 742 | 7.2,5003.01 743 | 7.1,5003.06 744 | 6.7,5003.11 745 | 5.7,5003.16 746 | 7.0,5003.21 747 | 8.2,5003.26 748 | 9.5,5003.31 749 | 10.2,5003.36 750 | 12.1,5003.41 751 | 10.9,5003.46 752 | 10.6,5003.51 753 | 11.3,5003.56 754 | 10.9,5003.61 755 | 10.3,5003.66 756 | 11.7,5003.71 757 | 11.7,5003.76 758 | 11.3,5003.81 759 | 12.2,5003.86 760 | 11.3,5003.91 761 | 9.9,5003.96 762 | 9.3,5004.01 763 | 8.3,5004.06 764 | 8.1,5004.11 765 | 8.8,5004.16 766 | 10.0,5004.21 767 | 10.6,5004.26 768 | 9.6,5004.31 769 | 9.7,5004.36 770 | 11.4,5004.41 771 | 14.1,5004.46 772 | 15.4,5004.51 773 | 19.7,5004.56 774 | 25.7,5004.61 775 | 31.5,5004.66 776 | 37.8,5004.71 777 | 47.5,5004.76 778 | 53.8,5004.81 779 | 60.5,5004.86 780 | 67.7,5004.91 781 | 75.5,5004.96 782 | 82.7,5005.01 783 | 87.7,5005.06 784 | 92.2,5005.11 785 | 96.5,5005.16 786 | 98.8,5005.21 787 | 99.6,5005.26 788 | 101.1,5005.31 789 | 102.6,5005.36 790 | 102.0,5005.41 791 | 102.6,5005.46 792 | 102.5,5005.51 793 | 103.7,5005.56 794 | 105.3,5005.61 795 | 105.8,5005.66 796 | 107.2,5005.71 797 | 107.6,5005.76 798 | 108.7,5005.81 799 | 112.0,5005.86 800 | 114.8,5005.91 801 | 116.3,5005.96 802 | 116.7,5006.01 803 | 116.6,5006.06 804 | 116.2,5006.11 805 | 116.3,5006.16 806 | 114.6,5006.21 807 | 110.8,5006.26 808 | 106.3,5006.31 809 | 102.4,5006.36 810 | 100.2,5006.41 811 | 97.5,5006.46 812 | 91.8,5006.51 813 | 86.3,5006.56 814 | 79.3,5006.61 815 | 72.2,5006.66 816 | 64.5,5006.71 817 | 58.2,5006.76 818 | 51.1,5006.81 819 | 44.3,5006.86 820 | 39.7,5006.91 821 | 34.6,5006.96 822 | 30.2,5007.01 823 | 26.4,5007.06 824 | 21.6,5007.11 825 | 18.3,5007.16 826 | 16.8,5007.21 827 | 16.0,5007.26 828 | 15.0,5007.31 829 | 14.4,5007.36 830 | 13.7,5007.41 831 | 14.1,5007.46 832 | 16.2,5007.51 833 | 16.6,5007.56 834 | 18.2,5007.61 835 | 18.3,5007.66 836 | 18.0,5007.71 837 | 18.2,5007.76 838 | 19.5,5007.81 839 | 19.7,5007.86 840 | 20.8,5007.91 841 | 24.8,5007.96 842 | 29.9,5008.01 843 | 35.0,5008.06 844 | 36.5,5008.11 845 | 36.8,5008.16 846 | 37.3,5008.21 847 | 37.8,5008.26 848 | 40.2,5008.31 849 | 42.2,5008.36 850 | 42.5,5008.41 851 | 43.9,5008.46 852 | 44.1,5008.51 853 | 43.5,5008.56 854 | 40.8,5008.61 855 | 36.8,5008.66 856 | 33.5,5008.71 857 | 33.8,5008.76 858 | 35.3,5008.81 859 | 37.0,5008.86 860 | 40.1,5008.91 861 | 40.0,5008.96 862 | 39.8,5009.01 863 | 41.1,5009.06 864 | 40.1,5009.11 865 | 39.1,5009.16 866 | 37.9,5009.21 867 | 38.4,5009.26 868 | 39.7,5009.31 869 | 39.2,5009.36 870 | 39.2,5009.41 871 | 37.5,5009.46 872 | 36.4,5009.51 873 | 33.3,5009.56 874 | 32.4,5009.61 875 | 31.9,5009.66 876 | 30.8,5009.71 877 | 29.8,5009.76 878 | 30.1,5009.81 879 | 31.5,5009.86 880 | 30.6,5009.91 881 | 29.4,5009.96 882 | 30.8,5010.01 883 | 31.3,5010.06 884 | 33.4,5010.11 885 | 33.9,5010.16 886 | 33.2,5010.21 887 | 33.1,5010.26 888 | 32.1,5010.31 889 | 32.8,5010.36 890 | 33.2,5010.41 891 | 34.9,5010.46 892 | 36.4,5010.51 893 | 37.1,5010.56 894 | 36.7,5010.61 895 | 35.0,5010.66 896 | 32.7,5010.71 897 | 29.1,5010.76 898 | 27.7,5010.81 899 | 29.1,5010.86 900 | 28.7,5010.91 901 | 29.1,5010.96 902 | 28.4,5011.01 903 | 28.0,5011.06 904 | 25.1,5011.11 905 | 22.9,5011.16 906 | 21.4,5011.21 907 | 21.7,5011.26 908 | 21.5,5011.31 909 | 22.3,5011.36 910 | 21.8,5011.41 911 | 21.8,5011.46 912 | 21.4,5011.51 913 | 21.4,5011.56 914 | 21.2,5011.61 915 | 20.6,5011.66 916 | 20.4,5011.71 917 | 21.3,5011.76 918 | 21.2,5011.81 919 | 21.3,5011.86 920 | 20.3,5011.91 921 | 19.3,5011.96 922 | 18.6,5012.01 923 | 18.6,5012.06 924 | 18.8,5012.11 925 | 18.0,5012.16 926 | 17.4,5012.21 927 | 16.8,5012.26 928 | 15.8,5012.31 929 | 14.3,5012.36 930 | 13.1,5012.41 931 | 12.6,5012.46 932 | 11.5,5012.51 933 | 11.3,5012.56 934 | 11.8,5012.61 935 | 9.4,5012.66 936 | 8.4,5012.71 937 | 7.5,5012.76 938 | 7.4,5012.81 939 | 7.6,5012.86 940 | 6.3,5012.91 941 | 7.9,5012.96 942 | 8.0,5013.01 943 | 8.8,5013.06 944 | 8.7,5013.11 945 | 9.4,5013.16 946 | 9.1,5013.21 947 | 9.6,5013.26 948 | 11.1,5013.31 949 | 11.9,5013.36 950 | 12.9,5013.41 951 | 12.2,5013.46 952 | 11.2,5013.51 953 | 12.1,5013.56 954 | 11.1,5013.61 955 | 12.4,5013.66 956 | 11.3,5013.71 957 | 11.1,5013.76 958 | 11.0,5013.81 959 | 10.7,5013.86 960 | 9.8,5013.91 961 | 8.9,5013.96 962 | 9.1,5014.01 963 | 9.0,5014.06 964 | 9.4,5014.11 965 | 9.7,5014.16 966 | 10.8,5014.21 967 | 11.4,5014.26 968 | 11.1,5014.31 969 | 11.8,5014.36 970 | 11.1,5014.41 971 | 12.7,5014.46 972 | 14.0,5014.51 973 | 15.5,5014.56 974 | 16.4,5014.61 975 | 16.0,5014.66 976 | 15.8,5014.71 977 | 16.2,5014.76 978 | 16.7,5014.81 979 | 15.9,5014.86 980 | 14.6,5014.91 981 | 13.4,5014.96 982 | 12.9,5015.01 983 | 12.9,5015.06 984 | 11.1,5015.11 985 | 10.8,5015.16 986 | 9.2,5015.21 987 | 9.0,5015.26 988 | 8.7,5015.31 989 | 7.9,5015.36 990 | 7.5,5015.41 991 | 7.7,5015.46 992 | 7.1,5015.51 993 | 7.6,5015.56 994 | 8.9,5015.61 995 | 11.1,5015.66 996 | 12.6,5015.71 997 | 13.5,5015.76 998 | 14.2,5015.81 999 | 14.9,5015.86 1000 | 15.2,5015.91 1001 | 16.2,5015.96 1002 | 17.0,5016.01 1003 | 16.5,5016.06 1004 | 16.2,5016.11 1005 | 15.8,5016.16 1006 | 16.4,5016.21 1007 | 16.5,5016.26 1008 | 14.7,5016.31 1009 | 14.3,5016.36 1010 | 14.4,5016.41 1011 | 11.9,5016.48 1012 | 13.5,5016.53 1013 | 13.4,5016.58 1014 | 14.1,5016.63 1015 | 12.8,5016.68 1016 | 12.4,5016.73 1017 | 11.5,5016.78 1018 | 9.8,5016.83 1019 | 10.9,5016.88 1020 | 10.6,5016.93 1021 | 12.9,5016.98 1022 | 13.9,5017.03 1023 | 14.3,5017.08 1024 | 14.3,5017.13 1025 | 13.4,5017.18 1026 | 13.2,5017.23 1027 | 12.6,5017.28 1028 | 13.4,5017.33 1029 | 13.6,5017.38 1030 | 14.6,5017.43 1031 | 15.4,5017.48 1032 | 16.3,5017.53 1033 | 15.9,5017.58 1034 | 14.2,5017.63 1035 | 13.8,5017.68 1036 | 13.2,5017.73 1037 | 14.7,5017.78 1038 | 14.8,5017.83 1039 | 15.1,5017.88 1040 | 14.7,5017.93 1041 | 15.6,5017.98 1042 | 15.9,5018.03 1043 | 16.6,5018.08 1044 | 15.6,5018.13 1045 | 14.1,5018.18 1046 | 15.2,5018.23 1047 | 14.1,5018.28 1048 | 13.0,5018.33 1049 | 13.3,5018.38 1050 | 12.9,5018.43 1051 | 12.9,5018.43 1052 | 13.2,5018.48 1053 | 13.5,5018.53 1054 | 13.4,5018.58 1055 | 11.9,5018.63 1056 | 12.8,5018.68 1057 | 11.8,5018.73 1058 | 12.8,5018.78 1059 | 12.6,5018.83 1060 | 12.1,5018.88 1061 | 13.5,5018.93 1062 | 13.8,5018.98 1063 | 14.0,5019.03 1064 | 12.8,5019.08 1065 | 14.8,5019.13 1066 | 14.7,5019.18 1067 | 17.2,5019.23 1068 | 18.3,5019.28 1069 | 17.7,5019.33 1070 | 18.8,5019.38 1071 | 18.7,5019.43 1072 | 19.7,5019.48 1073 | 19.3,5019.53 1074 | 19.3,5019.58 1075 | 19.9,5019.63 1076 | 21.0,5019.68 1077 | 22.7,5019.73 1078 | 20.3,5019.78 1079 | 20.5,5019.83 1080 | 18.5,5019.88 1081 | 17.6,5019.93 1082 | 17.4,5019.98 1083 | 17.6,5020.03 1084 | 18.4,5020.08 1085 | 18.4,5020.13 1086 | 20.0,5020.18 1087 | 19.4,5020.23 1088 | 19.5,5020.28 1089 | 18.0,5020.33 1090 | 16.9,5020.38 1091 | 16.5,5020.43 1092 | 16.2,5020.48 1093 | 16.1,5020.53 1094 | 16.8,5020.58 1095 | 15.5,5020.63 1096 | 15.3,5020.68 1097 | 14.1,5020.73 1098 | 13.7,5020.78 1099 | 12.5,5020.83 1100 | 12.1,5020.88 1101 | 11.8,5020.93 1102 | 12.1,5020.98 1103 | 12.4,5021.03 1104 | 12.2,5021.08 1105 | 13.1,5021.13 1106 | 12.6,5021.18 1107 | 12.4,5021.23 1108 | 12.2,5021.28 1109 | 11.1,5021.33 1110 | 12.5,5021.38 1111 | 12.5,5021.43 1112 | 11.3,5021.48 1113 | 12.7,5021.53 1114 | 13.4,5021.58 1115 | 13.2,5021.63 1116 | 13.4,5021.68 1117 | 12.9,5021.73 1118 | 12.3,5021.78 1119 | 13.1,5021.83 1120 | 13.5,5021.88 1121 | 14.1,5021.93 1122 | 14.2,5021.98 1123 | 12.2,5022.03 1124 | 10.9,5022.08 1125 | 12.9,5022.13 1126 | 14.4,5022.18 1127 | 13.7,5022.23 1128 | 14.1,5022.28 1129 | 13.7,5022.33 1130 | 13.6,5022.38 1131 | 13.6,5022.43 1132 | 14.2,5022.48 1133 | 14.6,5022.53 1134 | 15.3,5022.58 1135 | 15.0,5022.63 1136 | 14.6,5022.68 1137 | 16.4,5022.73 1138 | 16.2,5022.78 1139 | 14.2,5022.83 1140 | 14.2,5022.88 1141 | 15.2,5022.93 1142 | 15.8,5022.98 1143 | 19.0,5023.03 1144 | 19.9,5023.08 1145 | 19.7,5023.13 1146 | 20.6,5023.18 1147 | 21.1,5023.23 1148 | 23.4,5023.28 1149 | 27.2,5023.33 1150 | 30.2,5023.38 1151 | 32.5,5023.43 1152 | 34.9,5023.48 1153 | 36.5,5023.53 1154 | 37.7,5023.58 1155 | 40.6,5023.63 1156 | 42.2,5023.68 1157 | 45.2,5023.73 1158 | 49.4,5023.78 1159 | 54.9,5023.83 1160 | 63.0,5023.88 1161 | 71.3,5023.93 1162 | 76.5,5023.98 1163 | 77.7,5024.03 1164 | 78.5,5024.08 1165 | 81.0,5024.13 1166 | 84.6,5024.18 1167 | 87.4,5024.23 1168 | 88.9,5024.28 1169 | 89.5,5024.33 1170 | 89.8,5024.38 1171 | 89.4,5024.43 1172 | 86.1,5024.48 1173 | 80.4,5024.53 1174 | 73.1,5024.58 1175 | 67.1,5024.63 1176 | 63.4,5024.68 1177 | 61.0,5024.73 1178 | 57.5,5024.78 1179 | 53.3,5024.83 1180 | 49.3,5024.88 1181 | 46.4,5024.93 1182 | 44.3,5024.98 1183 | 43.4,5025.03 1184 | 42.6,5025.08 1185 | 41.0,5025.13 1186 | 39.4,5025.18 1187 | 38.5,5025.23 1188 | 38.7,5025.28 1189 | 39.7,5025.33 1190 | 40.3,5025.38 1191 | 41.5,5025.43 1192 | 90.3,5025.46 1193 | 92.0,5025.51 1194 | 92.9,5025.56 1195 | 92.4,5025.61 1196 | 91.6,5025.66 1197 | 90.8,5025.71 1198 | 87.9,5025.76 1199 | 89.0,5025.81 1200 | 87.4,5025.86 1201 | 83.9,5025.91 1202 | 81.3,5025.96 1203 | 80.5,5026.01 1204 | 78.8,5026.06 1205 | 79.1,5026.11 1206 | 78.1,5026.16 1207 | 79.4,5026.21 1208 | 81.4,5026.26 1209 | 86.6,5026.31 1210 | 91.2,5026.36 1211 | 96.3,5026.41 1212 | 99.6,5026.46 1213 | 102.0,5026.51 1214 | 102.4,5026.56 1215 | 102.2,5026.61 1216 | 99.9,5026.66 1217 | 98.1,5026.71 1218 | 95.6,5026.76 1219 | 93.9,5026.81 1220 | 89.7,5026.86 1221 | 85.3,5026.91 1222 | 77.6,5026.96 1223 | 71.3,5027.01 1224 | 66.8,5027.06 1225 | 64.9,5027.11 1226 | 64.6,5027.16 1227 | 64.4,5027.21 1228 | 64.4,5027.26 1229 | 67.7,5027.31 1230 | 70.5,5027.36 1231 | 73.3,5027.41 1232 | 73.3,5027.41 1233 | 76.3,5027.46 1234 | 79.5,5027.51 1235 | 81.9,5027.56 1236 | 84.6,5027.61 1237 | 85.6,5027.66 1238 | 84.4,5027.71 1239 | 82.3,5027.76 1240 | 76.8,5027.81 1241 | 73.0,5027.86 1242 | 68.7,5027.91 1243 | 61.2,5027.96 1244 | 53.9,5028.01 1245 | 46.3,5028.06 1246 | 39.0,5028.11 1247 | 31.8,5028.16 1248 | 27.3,5028.21 1249 | 21.7,5028.26 1250 | 16.3,5028.31 1251 | 14.7,5028.36 1252 | 11.0,5028.41 1253 | 12.0,5028.46 1254 | 11.9,5028.51 1255 | 12.0,5028.56 1256 | 13.1,5028.61 1257 | 13.8,5028.66 1258 | 14.3,5028.71 1259 | 14.8,5028.76 1260 | 15.4,5028.81 1261 | 14.1,5028.86 1262 | 13.5,5028.91 1263 | 14.2,5028.96 1264 | 13.0,5029.01 1265 | 15.0,5029.06 1266 | 15.3,5029.11 1267 | 15.8,5029.16 1268 | 15.4,5029.21 1269 | 16.1,5029.26 1270 | 15.9,5029.31 1271 | 15.9,5029.36 1272 | 15.6,5029.41 1273 | 15.5,5029.46 1274 | 15.4,5029.51 1275 | 14.8,5029.56 1276 | 14.5,5029.61 1277 | 13.6,5029.66 1278 | 13.1,5029.71 1279 | 13.2,5029.76 1280 | 13.5,5029.81 1281 | 12.8,5029.86 1282 | 11.3,5029.91 1283 | 10.8,5029.96 1284 | 10.4,5030.01 1285 | 10.1,5030.06 1286 | 10.8,5030.11 1287 | 11.0,5030.16 1288 | 12.5,5030.21 1289 | 13.5,5030.26 1290 | 14.7,5030.31 1291 | 15.7,5030.36 1292 | 15.7,5030.41 1293 | 16.3,5030.46 1294 | 17.0,5030.51 1295 | 16.9,5030.56 1296 | 17.3,5030.61 1297 | 17.9,5030.66 1298 | 17.9,5030.71 1299 | 17.5,5030.76 1300 | 17.9,5030.81 1301 | 17.2,5030.86 1302 | 16.2,5030.91 1303 | 16.7,5030.96 1304 | 16.3,5031.01 1305 | 16.4,5031.06 1306 | 15.0,5031.11 1307 | 16.4,5031.16 1308 | 16.8,5031.21 1309 | 16.8,5031.26 1310 | 17.2,5031.31 1311 | 17.2,5031.36 1312 | 17.3,5031.41 1313 | 16.7,5031.46 1314 | 18.2,5031.51 1315 | 19.1,5031.56 1316 | 19.2,5031.61 1317 | 20.8,5031.66 1318 | 21.8,5031.71 1319 | 24.1,5031.76 1320 | 25.2,5031.81 1321 | 25.9,5031.86 1322 | 25.7,5031.91 1323 | 25.2,5031.96 1324 | 25.2,5032.01 1325 | 24.9,5032.06 1326 | 25.5,5032.11 1327 | 24.8,5032.16 1328 | 25.5,5032.21 1329 | 25.1,5032.26 1330 | 24.3,5032.31 1331 | 23.5,5032.36 1332 | 21.8,5032.41 1333 | 19.6,5032.46 1334 | 18.7,5032.51 1335 | 18.3,5032.56 1336 | 19.3,5032.61 1337 | 21.9,5032.66 1338 | 22.1,5032.71 1339 | 21.7,5032.76 1340 | 22.6,5032.81 1341 | 22.0,5032.86 1342 | 21.8,5032.91 1343 | 20.7,5032.96 1344 | 20.6,5033.01 1345 | 20.3,5033.06 1346 | 20.6,5033.11 1347 | 20.9,5033.16 1348 | 21.7,5033.21 1349 | 21.2,5033.26 1350 | 19.8,5033.31 1351 | 19.6,5033.36 1352 | 18.8,5033.41 1353 | 18.0,5033.46 1354 | 18.0,5033.51 1355 | 18.5,5033.56 1356 | 19.5,5033.61 1357 | 19.9,5033.66 1358 | 19.9,5033.71 1359 | 20.4,5033.76 1360 | 20.8,5033.81 1361 | 20.6,5033.86 1362 | 20.3,5033.91 1363 | 19.8,5033.96 1364 | 19.4,5034.01 1365 | 19.9,5034.06 1366 | 19.1,5034.11 1367 | 18.9,5034.16 1368 | 19.5,5034.21 1369 | 19.0,5034.26 1370 | 18.3,5034.31 1371 | 18.3,5034.36 1372 | 17.5,5034.41 1373 | 17.6,5034.46 1374 | 17.3,5034.51 1375 | 16.3,5034.56 1376 | 15.5,5034.61 1377 | 15.6,5034.66 1378 | 15.5,5034.71 1379 | 16.5,5034.76 1380 | 16.8,5034.81 1381 | 15.4,5034.86 1382 | 16.2,5034.91 1383 | 16.4,5034.96 1384 | 16.2,5035.01 1385 | 16.9,5035.06 1386 | 15.7,5035.11 1387 | 15.8,5035.16 1388 | 16.6,5035.21 1389 | 17.0,5035.26 1390 | 17.0,5035.31 1391 | 17.5,5035.36 1392 | 16.1,5035.41 1393 | 16.0,5035.46 1394 | 16.4,5035.51 1395 | 15.6,5035.56 1396 | 15.3,5035.61 1397 | 15.4,5035.66 1398 | 16.8,5035.71 1399 | 19.1,5035.76 1400 | 23.6,5035.81 1401 | 31.3,5035.86 1402 | 38.4,5035.91 1403 | 45.8,5035.96 1404 | 53.3,5036.01 1405 | 62.0,5036.06 1406 | 69.9,5036.11 1407 | 78.5,5036.16 1408 | 88.9,5036.21 1409 | 96.7,5036.26 1410 | 102.1,5036.31 1411 | 103.7,5036.36 1412 | 103.9,5036.41 1413 | 100.2,5036.46 1414 | 92.4,5036.51 1415 | 84.4,5036.56 1416 | 77.0,5036.61 1417 | 69.4,5036.66 1418 | 61.2,5036.71 1419 | 53.3,5036.76 1420 | 44.1,5036.81 1421 | 33.4,5036.86 1422 | 24.8,5036.91 1423 | 19.6,5036.96 1424 | 15.6,5037.01 1425 | 13.4,5037.06 1426 | 12.6,5037.11 1427 | 13.1,5037.16 1428 | 14.2,5037.21 1429 | 14.1,5037.26 1430 | 13.6,5037.31 1431 | 12.8,5037.36 1432 | 12.6,5037.41 1433 | 14.7,5037.46 1434 | 17.0,5037.51 1435 | 21.4,5037.56 1436 | 30.1,5037.61 1437 | 40.3,5037.66 1438 | 49.6,5037.71 1439 | 53.3,5037.76 1440 | 52.9,5037.81 1441 | 52.8,5037.86 1442 | 54.7,5037.91 1443 | 55.7,5037.96 1444 | 58.5,5038.01 1445 | 58.2,5038.06 1446 | 55.8,5038.11 1447 | 53.4,5038.16 1448 | 49.0,5038.21 1449 | 39.9,5038.26 1450 | 31.1,5038.31 1451 | 21.9,5038.36 1452 | 17.0,5038.41 1453 | 16.0,5038.46 1454 | 14.8,5038.51 1455 | 12.4,5038.56 1456 | 11.4,5038.61 1457 | 7.9,5038.66 1458 | 7.5,5038.71 1459 | 6.6,5038.76 1460 | 7.4,5038.81 1461 | 8.1,5038.86 1462 | 6.5,5038.91 1463 | 5.9,5038.96 1464 | 6.3,5039.01 1465 | 7.1,5039.06 1466 | 8.3,5039.11 1467 | 10.0,5039.16 1468 | 11.3,5039.21 1469 | 12.6,5039.26 1470 | 14.5,5039.31 1471 | 14.0,5039.36 1472 | 16.9,5039.41 1473 | 16.3,5039.46 1474 | 14.4,5039.51 1475 | 16.4,5039.56 1476 | 15.5,5039.61 1477 | 14.7,5039.66 1478 | 14.7,5039.71 1479 | 13.5,5039.76 1480 | 13.4,5039.81 1481 | 12.1,5039.86 1482 | 11.4,5039.91 1483 | 11.0,5039.96 1484 | 11.4,5040.01 1485 | 8.8,5040.06 1486 | 7.7,5040.11 1487 | 8.0,5040.16 1488 | 7.6,5040.21 1489 | 8.3,5040.26 1490 | 8.6,5040.31 1491 | 9.1,5040.36 1492 | 9.4,5040.41 1493 | 8.6,5040.46 1494 | 8.1,5040.51 1495 | 8.5,5040.56 1496 | 8.8,5040.61 1497 | 8.3,5040.66 1498 | 8.0,5040.71 1499 | 8.5,5040.76 1500 | 9.5,5040.81 1501 | 9.4,5040.86 1502 | 8.4,5040.91 1503 | 8.4,5040.96 1504 | 8.7,5041.01 1505 | 9.3,5041.06 1506 | 11.4,5041.11 1507 | 12.6,5041.16 1508 | 13.0,5041.21 1509 | 13.2,5041.26 1510 | 14.2,5041.31 1511 | 14.5,5041.36 1512 | 13.2,5041.41 1513 | 12.7,5041.46 1514 | 12.5,5041.51 1515 | 13.1,5041.56 1516 | 13.6,5041.61 1517 | 12.4,5041.66 1518 | 10.9,5041.71 1519 | 8.9,5041.76 1520 | 7.7,5041.81 1521 | 7.1,5041.86 1522 | 6.8,5041.91 1523 | 6.4,5041.96 1524 | 7.4,5042.01 1525 | 9.3,5042.06 1526 | 10.2,5042.11 1527 | 10.6,5042.16 1528 | 10.8,5042.21 1529 | 10.4,5042.26 1530 | 9.9,5042.31 1531 | 10.4,5042.36 1532 | 10.0,5042.41 1533 | 10.6,5042.46 1534 | 10.2,5042.51 1535 | 10.3,5042.56 1536 | 11.0,5042.61 1537 | 10.6,5042.66 1538 | 10.5,5042.71 1539 | 11.2,5042.76 1540 | 10.9,5042.81 1541 | 11.1,5042.86 1542 | 10.9,5042.91 1543 | 12.6,5042.96 1544 | 13.1,5043.01 1545 | 13.4,5043.06 1546 | 13.3,5043.11 1547 | 14.0,5043.16 1548 | 14.1,5043.21 1549 | 12.4,5043.26 1550 | 12.5,5043.31 1551 | 11.4,5043.36 1552 | 9.8,5043.41 1553 | 10.5,5043.46 1554 | 10.5,5043.51 1555 | 10.5,5043.56 1556 | 9.2,5043.61 1557 | 9.1,5043.66 1558 | 9.7,5043.71 1559 | 10.3,5043.76 1560 | 9.0,5043.81 1561 | 8.8,5043.86 1562 | 9.7,5043.91 1563 | 8.7,5043.96 1564 | 8.5,5044.01 1565 | 9.6,5044.06 1566 | 9.1,5044.11 1567 | 9.3,5044.16 1568 | 9.3,5044.21 1569 | 9.8,5044.26 1570 | 9.4,5044.31 1571 | 8.8,5044.36 1572 | 8.0,5044.41 1573 | 8.1,5044.46 -------------------------------------------------------------------------------- /dash/PorePermDensity.csv: -------------------------------------------------------------------------------- 1 | Depth,Kinf,Kair,Porosity,Gdensity 2 | 4970.86,0.078,0.154,3.3,2.732 3 | 4971.24,0.003,0.011,2.9,2.721 4 | 4971.55,0.318,0.399,3.5,2.723 5 | 4971.85,0.036,0.082,2.5,2.713 6 | 4972.33,0.003,0.010,4.1,2.721 7 | 4972.67,0.037,0.085,2.6,2.742 8 | 4973.20,0.383,0.418,3.6,2.743 9 | 4973.55,0.009,0.027,2.9,2.761 10 | 4973.92,2.63,2.86,3.0,2.705 11 | 4974.25,0.013,0.036,2.9,2.705 12 | 4974.57, -, -, -, - 13 | 4974.86,0.053,0.091,3.0,2.716 14 | 4975.20, -, -, -,2.762 15 | 4975.50,0.261,0.281,3.0,2.807 16 | 4975.78,0.078,0.107,2.4,2.754 17 | 4978.16,0.062,0.102,5.9,2.752 18 | 4978.46,0.155,0.242,7.7,2.661 19 | 4978.83,0.146,0.218,6.8,2.670 20 | 4979.15,0.149,0.222,7.4,2.658 21 | 4979.43,0.169,0.230,6.5,2.680 22 | 4979.63,0.166,0.236,6.0,2.678 23 | 4980.02,682,747,13.3,2.654 24 | 4980.30,1190,1210,13.2,2.650 25 | 4980.60,1290,1310,14.0,2.651 26 | 4980.87,1360,1380,14.2,2.640 27 | 4981.30,2550,2580,15.0,2.656 28 | 4981.60,1460,1480,13.7,2.652 29 | 4981.90,2160,2180,15.0,2.654 30 | 4982.30,1560,1580,14.5,2.643 31 | 4982.60,1320,1340,13.0,2.639 32 | 4982.82,750,751,12.0,2.689 33 | 4983.15,1740,1760,13.8,2.643 34 | 4983.50,912,925,12.0,2.653 35 | 4983.86,94.7,97.1,8.3,2.650 36 | 4984.14,1320,1330,13.6,2.647 37 | 4984.49,1260,1270,13.0,2.640 38 | 4984.84,1240,1260,13.2,2.641 39 | 4985.14,393,399,10.4,2.654 40 | 4985.50,864,877,12.1,2.663 41 | 4985.84,28.0,28.9,9.2,2.639 42 | 4986.14,195,201,12.7,2.664 43 | 4986.50,195,201,13.3,2.660 44 | 4986.83,301,303,13.4,2.649 45 | 4987.17,91.4,97.6,10.5,2.727 46 | 4987.24,213,214,11.2,2.639 47 | 4987.48,54.9,58.9,10.7,2.638 48 | 4987.84,52.4,53.8,9.8,2.635 49 | 4987.89,122,131,11.5,2.635 50 | 4988.12,245,257,12.8,2.669 51 | 4988.50,128,143,12.4,2.639 52 | 4988.79,57.6,61.1,11.5,2.638 53 | 4989.16,244,257,10.7,2.638 54 | 4989.30,142,145,10.3,2.649 55 | 4993.16,52.6,57.4,10.3,2.688 56 | 4993.50,27.4,32.6,9.1,2.655 57 | 4993.85,17.2,19.1,9.3,2.639 58 | 4994.16,27.0,29.9,8.5,2.637 59 | 4994.53,0.278,0.379,2.4,2.633 60 | 4994.84,82.6,86.1,12.1,2.639 61 | 4995.14,1610,1630,14.3,2.641 62 | 4995.50,1830,1850,14.2,2.640 63 | 4995.85,1910,1950,14.8,2.641 64 | 4996.16,1530,1550,15.3,2.641 65 | 4996.45,1900,1910,15.8,2.641 66 | 4996.69,1580,1600,15.2,2.640 67 | 4996.94,803,841,13.0,2.644 68 | 4997.24,257,276,11.1,2.642 69 | 4997.50,670,677,12.8,2.638 70 | 4997.84,264,271,12.7,2.640 71 | 4998.28,333,342,11.8,2.643 72 | 4998.59,185,199,10.8,2.644 73 | 4998.76, -, -, -, - 74 | 4998.90,345,364,11.2,2.668 75 | 4999.23, -, -, -,2.644 76 | 4999.50,6.05,6.87,6.2,2.641 77 | 4999.85, -, -, -, - 78 | 5000.13, -, -, -, - 79 | 5000.18,44.2,54.6,9.7,2.641 80 | 5000.56,22.2,23.4,10.0,2.653 81 | 5000.80,97.8,102,12.1,2.655 82 | 5001.15,204,206,13.5,2.653 83 | 5001.50,137,138,12.1,2.650 84 | 5001.84,120,125,12.3,2.645 85 | 5002.17,21.2,22.6,9.8,2.640 86 | 5002.51,151,154,11.3,2.640 87 | 5002.81,185,186,11.7,2.640 88 | 5003.20,252,253,12.8,2.641 89 | 5003.53,587,598,13.2,2.640 90 | 5003.87,667,678,13.3,2.642 91 | 5004.15,460,461,12.6,2.644 92 | 5004.50,150,154,11.5,2.640 93 | 5004.82,0.258,0.335,5.6,2.655 94 | 5006.58,0.052,0.088,4.0,2.639 95 | 5006.84,0.154,0.196,3.9,2.639 96 | 5007.13,4.96,6.23,7.3,2.677 97 | 5007.31, -, -, -,2.644 98 | 5007.51,14.0,19.4,5.7,2.641 99 | 5007.81,9.41,14.7,6.3,2.699 100 | 5008.25, -, -, -,2.506 101 | 5008.44,77.4,109,7.7,2.663 102 | 5009.17,0.748,1.08,6.7,2.664 103 | 5012.56,4.47,5.04,9.1,2.639 104 | 5012.87,9.01,9.71,8.6,2.640 105 | 5013.15,24.6,25.4,8.3,2.640 106 | 5013.37,1.58,1.90,6.9,2.638 107 | 5013.91,5.82,6.44,7.7,2.639 108 | 5014.16,6.97,7.52,7.6,2.640 109 | 5014.37,0.267,0.399,5.4,2.656 110 | 5015.19,23.9,25.0,9.7,2.636 111 | 5015.54,24.9,25.8,8.8,2.640 112 | 5015.96,17.1,18.2,9.1,2.637 113 | 5016.14,2.82,3.25,7.7,2.648 114 | 5016.38,147,152,10.7,2.637 115 | 5016.95,271,275,12.4,2.638 116 | 5017.13,276,281,12.3,2.638 117 | 5017.54,141,142,11.0,2.640 118 | 5017.85,238,245,11.7,2.640 119 | 5018.14,143,144,11.5,2.638 120 | 5018.47,100,103,11.2,2.641 121 | 5018.89,38.9,40.0,9.5,2.639 122 | 5019.18,30.2,31.5,8.8,2.640 123 | 5019.44,0.873,1.09,5.0,2.649 124 | 5019.82,34.6,39.9,6.8,2.639 125 | 5020.11,98.0,131,5.7,2.638 126 | 5020.44,0.136,0.210,6.2,2.637 127 | 5020.70,5.27,5.82,7.0,2.639 128 | 5020.96,4.53,5.11,4.9,2.638 129 | 5021.29,76.9,81.1,9.2,2.637 130 | 5021.59,464,471,10.9,2.636 131 | 5021.89,642,685,11.4,2.637 132 | 5022.19,470,517,11.8,2.634 133 | 5022.49,430,469,10.9,2.641 134 | 5022.79,153,172,9.6,2.637 135 | 5023.10,2.81,3.43,6.8,2.640 136 | 5023.41,8.73,9.69,6.3,2.645 137 | 5023.70,0.439,0.602,6.7,2.648 138 | 5023.90,0.346,0.526,7.0,2.642 139 | 5024.30,0.095,0.150,5.5,2.656 140 | 5024.60,0.107,0.175,6.9,2.641 141 | 5024.91,0.439,0.604,7.4,2.654 142 | 5025.20,0.412,0.575,7.1,2.653 143 | 5025.44,0.073,0.124,6.0,2.740 144 | 5025.73,3.69,3.85,3.3,2.741 145 | 5025.98,0.093,0.142,3.3,2.700 146 | 5026.21,0.058,0.092,3.1,2.681 147 | 5026.51,0.022,0.044,3.4,2.730 148 | 5026.83,0.030,0.055,5.2,2.660 149 | 5027.11,0.037,0.069,5.2,2.660 150 | 5027.43,0.157,0.208,4.8,2.752 151 | 5027.67,0.042,0.078,3.9,2.668 152 | 5028.10,24.9,31.3,7.9,2.675 153 | 5028.31,7.49,8.52,10.3,2.645 154 | 5028.59,5.95,6.91,9.8,2.647 155 | 5028.89,0.539,0.735,7.9,2.637 156 | 5029.20,8.62,9.52,10.4,2.642 157 | 5029.51,12.9,15.3,10.2,2.649 158 | 5030.11,65.5,68.5,12.2,2.641 159 | 5030.41,2.03,2.47,8.9,2.659 160 | 5030.66,6.18,6.84,9.6,2.646 161 | 5030.97,39.0,40.9,11.8,2.640 162 | 5031.30,49.7,52.0,12.6,2.874 163 | 5031.59,25.8,27.5,12.1,2.641 164 | 5031.89,10.1,11.1,10.6,2.741 165 | 5032.20,3.73,4.38,10.3,2.643 166 | 5032.50,3.38,4.02,10.3,2.650 167 | 5032.80,4.41,5.08,9.3,2.638 168 | 5033.10,1.25,1.58,9.3,2.642 169 | 5033.41,2.64,3.13,9.0,2.641 170 | 5033.71,1.35,1.76,9.1,2.640 171 | 5034.07, -, -, -,2.706 172 | 5034.30,37.5,59.4,7.7,2.646 173 | 5034.59,1.42,1.66,5.5,2.641 174 | 5034.89,19.4,21.7,5.8,2.640 175 | 5035.21,2.08,2.54,9.0,2.641 176 | 5035.52,0.539,0.756,7.5,2.650 177 | 5035.82,2.24,2.71,9.4,2.654 178 | 5036.10,0.313,0.440,7.7,2.640 179 | 5036.47,0.208,0.312,1.8,2.672 180 | 5036.72,4.57,5.45,7.8,2.638 181 | 5036.97,5.73,6.79,5.9,2.640 182 | 5037.25,3.45,4.13,7.2,2.640 183 | 5037.60,16.9,18.8,8.6,2.636 184 | 5037.91,0.245,0.274,2.7,3.295 185 | 5038.20,1.03,1.27,5.1,2.649 186 | 5038.50,15.9,17.2,7.2,2.643 187 | 5038.81,98.9,105,8.8,2.637 188 | 5039.10,3.19,3.68,6.0,2.665 189 | 5039.37,5.91,6.59,4.2,2.639 190 | 5039.70,5.46,6.22,5.3,2.639 191 | 5039.89,0.729,0.961,5.2,2.643 192 | 5040.30,0.041,0.071,4.1,2.637 193 | 5040.60,3.72,4.13,5.9,2.643 194 | 5040.90,3.71,4.22,6.4,2.639 195 | 5041.20,2.22,2.65,6.4,2.640 196 | 5041.50,23.8,24.60,5.2,2.632 197 | 5041.81,4.05,4.52,6.2,2.661 198 | 5042.11,1.12,1.40,4.3,2.677 199 | 5042.41,0.125,0.192,4.0,2.717 200 | 5042.71,25.7,27.0,7.6,2.639 201 | 5042.97,0.162,0.244,4.5,2.673 202 | 5043.32,2.11,2.20,3.6,2.674 -------------------------------------------------------------------------------- /dash/plug_explore.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | import dash 4 | 5 | import dash_core_components as dcc 6 | import dash_html_components as html 7 | from dash.dependencies import Input, Output 8 | import dash_table 9 | 10 | from plotly import tools 11 | import plotly.graph_objs as go 12 | import plotly.figure_factory as ff 13 | 14 | import pandas as pd 15 | import numpy as np 16 | 17 | import re 18 | 19 | external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css'] 20 | 21 | app = dash.Dash(__name__, external_stylesheets=external_stylesheets) 22 | 23 | styles = { 24 | 'pre': { 25 | 'border': 'thin lightgrey solid', 26 | 'overflowX': 'scroll' 27 | } 28 | } 29 | 30 | df = pd.read_csv('PorePermDensity.csv') 31 | df = df.reset_index() 32 | df = df.rename(columns={'index': 'Sample Number'}) 33 | 34 | gamma_df = pd.read_csv('CoreGamma.csv') 35 | 36 | available_indicators = ['Kinf','Porosity','Gdensity','Depth'] 37 | 38 | app.layout = html.Div([ 39 | html.Div([ 40 | html.Div([ 41 | html.H1( 42 | 'Plug Data Explorer', 43 | className='eight columns' 44 | ), 45 | html.Img( 46 | src="https://images.squarespace-cdn.com/content/v1/58a4b31dbebafb6777c575b4/1552228857222-IT0RG669ZUOLLLR597EU/ke17ZwdGBToddI8pDm48kLPswmMOqQZ9-Q6KHLjvbpZ7gQa3H78H3Y0txjaiv_0fDoOvxcdMmMKkDsyUqMSsMWxHk725yiiHCCLfrh8O1z5QPOohDIaIeljMHgDF5CVlOqpeNLcJ80NK65_fV7S1UcaEU9usEQgaPMYSSHCLDdjcUDfwtSR5qjoqJbWx-aCIZDqXZYzu2fuaodM4POSZ4w/swung_round_no_text.png?format=200w", 47 | className='one columns', 48 | style={ 49 | 'height': '50', 50 | 'float': 'right', 51 | 'position': 'relative', 52 | }, 53 | ), 54 | ],className='row'), 55 | html.Div([ 56 | html.Label('X Axis'), 57 | dcc.Dropdown( 58 | id='xaxis-column', 59 | options=[{'label': i, 'value': i} for i in available_indicators], 60 | value=available_indicators[1] 61 | ), 62 | dcc.RadioItems( 63 | id='xaxis-type', 64 | options=[{'label': i, 'value': i} for i in ['Linear', 'Log']], 65 | value='Linear', 66 | labelStyle={'display': 'inline-block'} 67 | ) 68 | ], 69 | style={'width': '30%', 'display': 'inline-block'}), 70 | 71 | html.Div([ 72 | html.Label('Y Axis'), 73 | dcc.Dropdown( 74 | id='yaxis-column', 75 | options=[{'label': i, 'value': i} for i in available_indicators], 76 | value=available_indicators[0] 77 | ), 78 | dcc.RadioItems( 79 | id='yaxis-type', 80 | options=[{'label': i, 'value': i} for i in ['Linear', 'Log']], 81 | value='Log', 82 | labelStyle={'display': 'inline-block'} 83 | ) 84 | ],style={'width': '30%','display': 'inline-block'}), 85 | ]), 86 | 87 | html.Div([ 88 | html.Label('Select Cores to Include'), 89 | dcc.Graph(id='cross-plot'), 90 | ], style={'width': '60%', 'display': 'inline-block', 'padding': '0 20'}), 91 | 92 | 93 | html.Div([ 94 | html.Label('Core Gamma'), 95 | dcc.Graph(id='well-log-plot'), 96 | ], style={'display': 'inline-block', 'width': '40%','float': 'right' }), 97 | ]) 98 | 99 | @app.callback( 100 | Output('cross-plot', 'figure'), 101 | [Input('xaxis-column', 'value'), 102 | Input('yaxis-column', 'value'), 103 | Input('xaxis-type', 'value'), 104 | Input('yaxis-type', 'value'), 105 | ]) 106 | def update_graph(xaxis_column_name, yaxis_column_name, 107 | xaxis_type, yaxis_type): 108 | 109 | dff = df 110 | 111 | traces = [] 112 | 113 | df_by_color = dff 114 | traces.append(dict( 115 | x=df_by_color[xaxis_column_name], 116 | y=df_by_color[yaxis_column_name], 117 | customdata=df_by_color['Sample Number'], 118 | text=['Sample Number: {}
Kinf: {}
Porosity: {}
Grain Density: {}
Depth: {}'.format(int(n), lf, d, c, ltrf) for n, lf, d, c, ltrf in zip(df_by_color['Sample Number'],df_by_color['Kinf'], df_by_color['Porosity'], df_by_color['Gdensity'],df_by_color['Depth'] ) ], 119 | hovertemplate = '%{text}', # the extra bit is for the area beside the main tooltip. make it blank so name doesn't appear 120 | mode='markers', 121 | marker={ 122 | 'size': 15, 123 | 'opacity': 0.5, 124 | }, 125 | hoverinfo='skip', 126 | )) 127 | return { 128 | 'data': traces, 129 | 'layout': dict( 130 | xaxis={ 131 | 'title': xaxis_column_name, 132 | 'type': 'linear' if xaxis_type == 'Linear' else 'log' 133 | }, 134 | yaxis={ 135 | 'title': yaxis_column_name, 136 | 'type': 'linear' if yaxis_type == 'Linear' else 'log' 137 | }, 138 | margin={'l': 40, 'b': 40, 't': 10, 'r': 0}, 139 | hovermode='closest' 140 | ) 141 | } 142 | 143 | @app.callback( 144 | Output('well-log-plot', 'figure'), 145 | [Input('cross-plot', 'selectedData')]) 146 | def generate_log_curves(selectedData): 147 | 148 | height=500 149 | width=300 150 | bg_color='white' 151 | font_size=10 152 | tick_font_size=8 153 | line_width=1.0 154 | 155 | selectedpoints = df['Sample Number'] 156 | ind1 = selectedpoints.index.values 157 | 158 | if selectedData is not None: 159 | 160 | for selected_data in [selectedData]: 161 | 162 | if selected_data and selected_data['points']: 163 | selectedpoints, ind1, ind2 = np.intersect1d(selectedpoints, 164 | [p['customdata'] for p in selected_data['points']], return_indices=True) 165 | 166 | fig = tools.make_subplots(rows=1, cols=1, 167 | shared_yaxes=True, 168 | horizontal_spacing=0) 169 | 170 | fig.append_trace(go.Scatter( 171 | x=gamma_df['Gamma'], 172 | y=gamma_df['Depth'], 173 | xaxis='x', 174 | name='CoreGamma', 175 | line={'width': line_width, 176 | 'dash': 'dashdot'}, 177 | showlegend = False, 178 | hoverinfo='skip', 179 | 180 | ), row=1, col=1) 181 | fig.append_trace(go.Scatter( 182 | x=df['Kinf'], 183 | y=df['Depth'], 184 | xaxis='x2', 185 | selectedpoints= ind1, #selected_indices, 186 | name='PERM-plug', 187 | mode='markers', 188 | hovertemplate= 189 | "Depth: %{y:.2f}m
" + 190 | "PERM: %{x:.3e}md
" + 191 | "", 192 | marker={ 193 | 'size': 8, 194 | 'line': {'width': 0.5, 'color': 'white'} 195 | }, 196 | unselected={ 197 | 'marker': { 'opacity': 0.2 }, 198 | # make text transparent when not selected 199 | 'textfont': { 'color': 'rgba(0, 0, 0, 0)' } 200 | }, 201 | showlegend = False, 202 | ),row=1, col=1) 203 | 204 | fig['layout']['xaxis'].update( 205 | title='Gamma', 206 | type='log', 207 | side='bottom' 208 | ) 209 | fig['layout'].update( 210 | xaxis2={"title": "Kinf", "side": "top"}, 211 | ) 212 | # y axis title 213 | fig['layout']['yaxis'].update( 214 | title='Depth
[m]', 215 | autorange='reversed' 216 | ) 217 | 218 | fig['layout'].update( 219 | 220 | plot_bgcolor=bg_color, 221 | paper_bgcolor=bg_color, 222 | hovermode='closest', 223 | legend={ 224 | 'font': { 225 | 'size': tick_font_size 226 | } 227 | }, 228 | margin=go.layout.Margin( 229 | l=20,r=20, t=10, b= 40 230 | ) 231 | ) 232 | 233 | return fig 234 | 235 | if __name__ == '__main__': 236 | app.run_server(debug=True) -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: t20-fri-mvp 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - flask=1.1.2 7 | - jupyter=1.0.0 8 | - matplotlib=3.2.1 9 | - notebook=6.0.3 10 | - numpy=1.18.5 11 | - pillow=7.1.2 12 | - pip=20.1.1 13 | - python=3.7.6 14 | - requests=2.23.0 15 | - scikit-learn=0.21.3 16 | - scipy=1.4.1 17 | - pip: 18 | - brotli==1.0.7 19 | - dash==1.12.0 20 | - dash-core-components==1.10.0 21 | - dash-html-components==1.0.3 22 | - dash-renderer==1.4.1 23 | - dash-table==4.7.0 24 | - flask-compress==1.5.0 25 | - future==0.18.2 26 | - jupyter-dash==0.2.1.post1 27 | - plotly==4.8.1 28 | - pyqt5-sip==4.19.18 29 | - pyqtchart==5.12 30 | - pyqtwebengine==5.12.1 31 | - retrying==1.3.3 32 | 33 | -------------------------------------------------------------------------------- /notebooks/Building_a_flask_app.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Building a web app" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "
\n", 15 | " \n", 16 | "

NOTE

\n", 17 | "\n", 18 | "

Much of the code in this notebook doesn't work in a Notebook. It needs to reside in a `flask` application. Usually this means putting it in a file called `app.py`, which can be run by typing `flask run` on the command line in the same directory.

\n", 19 | " \n", 20 | "
" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "## 0. Minimal\n", 28 | "\n", 29 | "This is the smallest possible flask application. It serves a single string on the root path." 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": null, 35 | "metadata": {}, 36 | "outputs": [], 37 | "source": [ 38 | "from flask import Flask\n", 39 | "\n", 40 | "app = Flask(__name__)\n", 41 | "\n", 42 | "@app.route('/')\n", 43 | "def root():\n", 44 | " \"\"\"\n", 45 | " Simplest possible. Return a string.\n", 46 | " \"\"\"\n", 47 | " return \"Hello world!\"" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | "❗ **IMPORTANT**\n", 55 | "\n", 56 | "To run, we must put ourselves in `development` mode:\n", 57 | "\n", 58 | " export FLASK_ENV=development\n", 59 | " flask run\n", 60 | "\n", 61 | "Or, in Windows CMD:\n", 62 | "\n", 63 | " set FLASK_ENV=development\n", 64 | " flask run" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "## 1. Getting an arg\n", 72 | "\n", 73 | "We have access to an object called `request`, which contains the data passed to the server when the request was made. For example, the headers passed by the browser (try printing `request.headers`), and query parameters (if any). " 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "from flask import Flask, request\n", 83 | "\n", 84 | "app = Flask(__name__)\n", 85 | "\n", 86 | "@app.route('/')\n", 87 | "def root():\n", 88 | " \"\"\"\n", 89 | " Get one argument as a string.\n", 90 | " \"\"\"\n", 91 | " name = request.args.get('name', 'world')\n", 92 | " return \"Hello {}!\".format(name)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "metadata": {}, 98 | "source": [ 99 | "## 2. Calculator" 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "execution_count": null, 105 | "metadata": {}, 106 | "outputs": [], 107 | "source": [ 108 | "@app.route('/')\n", 109 | "def root():\n", 110 | " \"\"\"\n", 111 | " (2) A simple calculator with GET requests.\n", 112 | " \"\"\"\n", 113 | " vp = float(request.args.get('vp') or 0)\n", 114 | " rho = float(request.args.get('rho') or 0)\n", 115 | " return \"Impedance: {}\".format(vp * rho)" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "
\n", 123 | " \n", 124 | "

EXERCISE

\n", 125 | "\n", 126 | "

Add another endpoint to the site to compute Gardner's equation, $\\rho = 310 V_\\mathrm{P}^{0.25}$ (or your favourite equation!).\n", 127 | " \n", 128 | "You will need to add a function to handle GET requests on a new path.

\n", 129 | " \n", 130 | "
" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "### Path parameters\n", 138 | "\n", 139 | "As well as passing a query string like `?vp=2400&rho=2500`, which turns into a dictionary `{'vp': '2400', 'rho': '2500'}` (called `request.args`), we can also pass variables in the path itself.\n", 140 | "\n", 141 | " This is good for querying databases: path parameters represent entities (tables in your DB, more or less).\n", 142 | " \n", 143 | " Here's an example:" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": null, 149 | "metadata": {}, 150 | "outputs": [], 151 | "source": [ 152 | "@app.route('/hello/')\n", 153 | "def hello(name):\n", 154 | " return \"Hello {}\".format(name)" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "Here's [one explanation](https://stackoverflow.com/a/31261026/3381305) of why you might do this:\n", 162 | "\n", 163 | "> Best practice for RESTful API design is that path params are used to identify a specific resource or resources, while query parameters are used to sort/filter those resources.\n", 164 | "\n", 165 | "> Here's an example. Suppose you are implementing RESTful API endpoints for an entity called Car. You would structure your endpoints like this:\n", 166 | "\n", 167 | "> GET /cars\n", 168 | "> GET /cars/:id\n", 169 | "> POST /cars\n", 170 | "> PUT /cars/:id\n", 171 | "> DELETE /cars/:id\n", 172 | "\n", 173 | "> This way you are only using path parameters when you are specifying which resource to fetch, but this does not sort/filter the resources in any way." 174 | ] 175 | }, 176 | { 177 | "cell_type": "markdown", 178 | "metadata": {}, 179 | "source": [ 180 | "## 3. Classify image via URL given as query\n", 181 | "\n", 182 | "To do this, we need some new functions:\n", 183 | "\n", 184 | "- one to process the image as in training.\n", 185 | "- one to fetch an image from a URL.\n", 186 | "- one to make a prediction from an image.\n", 187 | "\n", 188 | "First, here's the image processing function:" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": null, 194 | "metadata": {}, 195 | "outputs": [], 196 | "source": [ 197 | "def img_to_arr(img):\n", 198 | " \"\"\"\n", 199 | " Apply the same processing we used in training: greyscale and resize.\n", 200 | " \"\"\"\n", 201 | " img = img.convert(mode='L').resize((32, 32))\n", 202 | " return np.asarray(img).ravel() / 255" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "Here's the `fetch_image()` and `predict_from_image()` functions:" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "def fetch_image(url):\n", 219 | " \"\"\"\n", 220 | " Download an image from the web and pass to the image processing function.\n", 221 | " \"\"\"\n", 222 | " r = requests.get(url)\n", 223 | " f = BytesIO(r.content)\n", 224 | " return Image.open(f) \n", 225 | "\n", 226 | "\n", 227 | "def predict_from_image(clf, img):\n", 228 | " \"\"\"\n", 229 | " Classify an image.\n", 230 | " \"\"\"\n", 231 | " arr = img_to_arr(img)\n", 232 | " X = np.atleast_2d(arr)\n", 233 | " probs = clf.predict_proba(X)\n", 234 | " result = {\n", 235 | " 'class': clf.classes_[np.argmax(probs)],\n", 236 | " 'prob': probs.max(),\n", 237 | " 'classes': clf.classes_.tolist(),\n", 238 | " 'probs': np.squeeze(probs).tolist(), # Must be serializable.\n", 239 | " }\n", 240 | " return result" 241 | ] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "metadata": {}, 246 | "source": [ 247 | "Now we can write the prediction function for the app:" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": null, 253 | "metadata": {}, 254 | "outputs": [], 255 | "source": [ 256 | "import joblib\n", 257 | "from flask import Flask, request, jsonify\n", 258 | "\n", 259 | "app = Flask(__name__)\n", 260 | "\n", 261 | "CLF = joblib.load('../app/data/rf.gz')\n", 262 | "\n", 263 | "@app.route('/predict')\n", 264 | "def predict():\n", 265 | " \"\"\"\n", 266 | " (3) Make a prediction from a URL given via GET request.\n", 267 | " \n", 268 | " Using a URL means we can still just accept a string as an arg.\n", 269 | "\n", 270 | " There's still no human interface.\n", 271 | " \"\"\"\n", 272 | " url = request.args.get('url')\n", 273 | " img = utils.fetch_image(url)\n", 274 | " result = utils.predict_from_image(CLF, img)\n", 275 | "\n", 276 | " # Deal with not getting a URL.\n", 277 | " # if url:\n", 278 | " # img = utils.fetch_image(url)\n", 279 | " # result = utils.predict_from_image(CLF, img)\n", 280 | " # else:\n", 281 | " # result = 'Please provide a URL'\n", 282 | "\n", 283 | " return jsonify(result)" 284 | ] 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": {}, 289 | "source": [ 290 | "## 4. Provide URL via a GET form" 291 | ] 292 | }, 293 | { 294 | "cell_type": "markdown", 295 | "metadata": {}, 296 | "source": [ 297 | "First we'll just show a 'Hello world' page. The `simple_page.html` file can contain any HTML:" 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": null, 303 | "metadata": {}, 304 | "outputs": [], 305 | "source": [ 306 | "from flask import render_template\n", 307 | "\n", 308 | "@app.route('/simple')\n", 309 | "def simple():\n", 310 | " \"\"\"\n", 311 | " (4a) Render a template.\n", 312 | " \"\"\"\n", 313 | " return render_template('simple_page.html')" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "It's tempting to defer all design etc until later, but I strongly recommend using some styling from the start so you can evolve it alongside the content. There are a couple of 'off the shelf' styles I use:\n", 321 | "\n", 322 | "- [Bootstrap](https://getbootstrap.com/), the gold standard, originally from Twitter. Downside: looks like every other website in the universe.\n", 323 | "- [new.css](https://newcss.net/), arguably the simplest possible HTML framework and the one I'm using here.\n", 324 | "\n", 325 | "There are loads of others, eg [Skeleton](http://getskeleton.com/) and [Foundation](https://get.foundation/). Find more by Googling something like _responsive HTML5 frameworks_." 326 | ] 327 | }, 328 | { 329 | "cell_type": "markdown", 330 | "metadata": {}, 331 | "source": [ 332 | "Now we can add a form:" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": null, 338 | "metadata": {}, 339 | "outputs": [], 340 | "source": [ 341 | "@app.route('/form', methods=['GET'])\n", 342 | "def form():\n", 343 | " \"\"\"\n", 344 | " (4b) Make a prediction from a URL given via a GET form.\n", 345 | " \"\"\"\n", 346 | " url = request.args.get('url')\n", 347 | " if url:\n", 348 | " img = utils.fetch_image(url)\n", 349 | " result = utils.predict_from_image(CLF, img)\n", 350 | " result['url'] = url # If we add this back, we can display it.\n", 351 | " else:\n", 352 | " result = {}\n", 353 | "\n", 354 | " return render_template('form.html', result=result)" 355 | ] 356 | }, 357 | { 358 | "cell_type": "markdown", 359 | "metadata": {}, 360 | "source": [ 361 | "The key part of the form HTML document, `form.html`, looks like this:\n", 362 | "\n", 363 | "
\n", 364 | " \n", 365 | " \n", 366 | "
\n", 367 | " \n", 368 | "Then we have a bit of Jinja2 'code' in the page to show items from a `result` dictionary, if that dictionary contains any items:\n", 369 | "\n", 370 | " {% if result %}\n", 371 | "\n", 372 | "

{{ result['class'] }}: {{ \"%.3f\"|format(result['prob']) }}

\n", 373 | "\n", 374 | " {% endif %}\n", 375 | " \n", 376 | "That's it!" 377 | ] 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "metadata": {}, 382 | "source": [ 383 | "
\n", 384 | " \n", 385 | "

EXERCISE

\n", 386 | "\n", 387 | "

Add an About page to the site. You will need to add a template (an HTML file), and a function to handle GET requests on the /about path.

\n", 388 | " \n", 389 | "
" 390 | ] 391 | }, 392 | { 393 | "cell_type": "markdown", 394 | "metadata": {}, 395 | "source": [ 396 | "## GET vs POST\n", 397 | "\n", 398 | "In a nutshell, `GET` is awesome because the data is passed in the URL. This means we can share the result of a query (say) just by sharing the URL with someone. But the URL is not secure, e.g. it's recorded in the server's logs, and it limits both the size and type of data we can pass to the server.\n", 399 | "\n", 400 | "If we want to pass a lot of data, or structured data (e.g. a nested dictionary), or a bunch of non-text bytes, or we want to pass secure data, we'll need to use the `POST` method instead.\n", 401 | "\n", 402 | "Our handlers may stil want to handle `GET` requests, since that's how a browser is going to ask for a resource." 403 | ] 404 | }, 405 | { 406 | "cell_type": "markdown", 407 | "metadata": {}, 408 | "source": [ 409 | "We could convert the GET form to a post form simply by changing the `method` (and handling that on the back-end):\n", 410 | "\n", 411 | "
\n", 412 | " \n", 413 | " \n", 414 | "
" 415 | ] 416 | }, 417 | { 418 | "cell_type": "markdown", 419 | "metadata": {}, 420 | "source": [ 421 | "## 5. Upload image via a POST form" 422 | ] 423 | }, 424 | { 425 | "cell_type": "code", 426 | "execution_count": null, 427 | "metadata": {}, 428 | "outputs": [], 429 | "source": [ 430 | "@app.route('/upload', methods=['GET', 'POST'])\n", 431 | "def upload():\n", 432 | " \"\"\"\n", 433 | " (5) Make a prediction from an image uploaded via a POST form.\n", 434 | "\n", 435 | " Bonus: on a mobile device, there will automatically be an option to\n", 436 | " capture via the camera.\n", 437 | " \"\"\"\n", 438 | " if request.method == 'POST':\n", 439 | " data = request.files['image'].read()\n", 440 | " img = Image.open(BytesIO(data))\n", 441 | " result = utils.predict_from_image(CLF, img)\n", 442 | " # result['image'] = base64.b64encode(data).decode('utf-8')\n", 443 | " else:\n", 444 | " result = {}\n", 445 | "\n", 446 | " return render_template('upload.html', result=result)" 447 | ] 448 | }, 449 | { 450 | "cell_type": "markdown", 451 | "metadata": {}, 452 | "source": [ 453 | "The HTML for the form is like this:\n", 454 | "\n", 455 | "
\n", 456 | " \n", 457 | " \n", 458 | "
\n", 459 | "\n", 460 | "The `enctype` parameter is needed for sending more complex data types such as binary files. The default is `enctype=\"multipart/form-data\"` ([more about this](https://www.w3schools.com/tags/att_form_enctype.asp))." 461 | ] 462 | }, 463 | { 464 | "cell_type": "markdown", 465 | "metadata": {}, 466 | "source": [ 467 | "### Displaying the image\n", 468 | "\n", 469 | "We can send the image back to the front-end by adding a base64-encoded string to the `result` dictionary (uncomment the line in the handler function above). Then we can display the image with some HTML:\n", 470 | "\n", 471 | " \n", 472 | " \n", 473 | "`base64` is an encoding similar to binary, octal and hexadecimal, but with 64 symbols instead of 2, 8 and 16 respectively. Each symbol therefore represents 6 bits of data (26 = 64), so you can pack 3 bytes into 4 symbols. [Read more about base64 encoding.](https://en.wikipedia.org/wiki/Base64)" 474 | ] 475 | }, 476 | { 477 | "cell_type": "markdown", 478 | "metadata": {}, 479 | "source": [ 480 | "## 6. Sending a plot back to the user\n", 481 | "\n", 482 | "In a simple app, I usually try to reduce complexity as much as possible. So it's useful to know how to store a `matplotlib` plot in memory, by tricking it into thinking we're saving a file, then sending those bytes to the front-end.\n", 483 | "\n", 484 | "Here's the code to save the `matplotlib` plot:" 485 | ] 486 | }, 487 | { 488 | "cell_type": "code", 489 | "execution_count": null, 490 | "metadata": {}, 491 | "outputs": [], 492 | "source": [ 493 | "import matplotlib.pyplot as plt\n", 494 | "from io import BytesIO\n", 495 | "import base64\n", 496 | "\n", 497 | "fig, ax = plt.subplots()\n", 498 | "ax.plot([0, 1, 2, 3, 4, 5])\n", 499 | "\n", 500 | "# Put in memory.\n", 501 | "handle = BytesIO()\n", 502 | "plt.savefig(handle, format='png', facecolor=fig.get_facecolor())\n", 503 | "plt.close()\n", 504 | "\n", 505 | "# Encode.\n", 506 | "handle.seek(0)\n", 507 | "image_string = base64.b64encode(handle.getvalue()).decode('utf8')" 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "Now we add one line to the handler:" 515 | ] 516 | }, 517 | { 518 | "cell_type": "code", 519 | "execution_count": null, 520 | "metadata": {}, 521 | "outputs": [], 522 | "source": [ 523 | "result['plot'] = utils.plot(result['probs'], CLF.classes_)" 524 | ] 525 | }, 526 | { 527 | "cell_type": "markdown", 528 | "metadata": {}, 529 | "source": [ 530 | "## 7. A web API for POST requests\n", 531 | "\n", 532 | "Instead of passing data via a form, programmers would often like to pass data to a server programmatically. To handle requests like this, we can write a special endpoint.\n", 533 | "\n", 534 | "We'll always want to return JSON from an endpoint like this, so we don't need templates or any HTML:" 535 | ] 536 | }, 537 | { 538 | "cell_type": "code", 539 | "execution_count": null, 540 | "metadata": {}, 541 | "outputs": [], 542 | "source": [ 543 | "@app.route('/post', methods=['POST'])\n", 544 | "def post():\n", 545 | " \"\"\"\n", 546 | " (7) Make a prediction from a URL provided via POST request.\n", 547 | " \"\"\"\n", 548 | " url = request.json.get('url')\n", 549 | " img = utils.fetch_image(url)\n", 550 | " result = utils.predict_from_image(CLF, img)\n", 551 | " return jsonify(result)" 552 | ] 553 | }, 554 | { 555 | "cell_type": "markdown", 556 | "metadata": {}, 557 | "source": [ 558 | "## 8. A web API handling images or URLs\n", 559 | "\n", 560 | "If accessing the web API from code, you may not have a URL to pass to the service, and there is no form for doing a file upload. So we need a way to pass the image as data. There are lots of ways to do this; one way is to encode as base64." 561 | ] 562 | }, 563 | { 564 | "cell_type": "code", 565 | "execution_count": null, 566 | "metadata": {}, 567 | "outputs": [], 568 | "source": [ 569 | "@app.route('/api/v0.1', methods=['POST'])\n", 570 | "def api():\n", 571 | " \"\"\"\n", 572 | " (8) Extend example (7) to make a prediction from a URL or a\n", 573 | " base64-encoded image via POST request. \n", 574 | " \"\"\"\n", 575 | " data = request.json.get('image')\n", 576 | " if data.startswith('http'):\n", 577 | " img = utils.fetch_image(url)\n", 578 | " else:\n", 579 | " img = Image.open(BytesIO(base64.b64decode(data)))\n", 580 | " result = utils.predict_from_image(CLF, img)\n", 581 | " return jsonify(result)" 582 | ] 583 | }, 584 | { 585 | "cell_type": "markdown", 586 | "metadata": {}, 587 | "source": [ 588 | "Check out [Hitting_our_web_API.ipynb](./Hitting_our_web_API.ipynb) to see how to use this API." 589 | ] 590 | }, 591 | { 592 | "cell_type": "code", 593 | "execution_count": null, 594 | "metadata": {}, 595 | "outputs": [], 596 | "source": [] 597 | } 598 | ], 599 | "metadata": { 600 | "kernelspec": { 601 | "display_name": "transform", 602 | "language": "python", 603 | "name": "transform" 604 | }, 605 | "language_info": { 606 | "codemirror_mode": { 607 | "name": "ipython", 608 | "version": 3 609 | }, 610 | "file_extension": ".py", 611 | "mimetype": "text/x-python", 612 | "name": "python", 613 | "nbconvert_exporter": "python", 614 | "pygments_lexer": "ipython3", 615 | "version": "3.7.6" 616 | } 617 | }, 618 | "nbformat": 4, 619 | "nbformat_minor": 2 620 | } -------------------------------------------------------------------------------- /notebooks/Fossil_classifier_minimal.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Build a fossil classifier" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Download the data from [Swung's public Amazon S3 bucket](https://swung-data.s3.amazonaws.com/fossilnet/fossilnet-png-224px.zip)." 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "data_url = \"https://swung-data.s3.amazonaws.com/fossilnet/fossilnet-png-224px.zip\"" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": null, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "import requests \n", 33 | "import zipfile\n", 34 | "from io import BytesIO\n", 35 | "\n", 36 | "r = requests.get(data_url, stream=True)\n", 37 | "z = zipfile.ZipFile(BytesIO(r.content))\n", 38 | "z.extractall(path='..')" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "There should be 3000 PNG files there." 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "ls -l ../fossilnet-png-224px/*/*/*.png | wc | awk '{print $1 \" PNG files\"}'" 55 | ] 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "metadata": {}, 60 | "source": [ 61 | "Please be aware of the licensing of these images. Note that it relies on the [_Fair use doctrine_](https://en.wikipedia.org/wiki/Fair_use) (also called _Fair dealing_) and that individual images are not open; only the collection is." 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "from IPython.display import Markdown\n", 71 | "\n", 72 | "Markdown('../fossilnet-png-224px/fossilnet-copyright-info.md')" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "## Make train and test" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | "We're not trying to make a fantastic model here, and this is a hard dataset.\n", 87 | "\n", 88 | "So I'm only going to use `train` and `val`, and I'm only going to use 4 classes.\n", 89 | "\n", 90 | "Let's read the files, do a bit of processing on them (make greyscale and resize), and I'll also save a flipped version, so I'll have 2 versions of each image." 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": {}, 97 | "outputs": [], 98 | "source": [ 99 | "import numpy as np\n", 100 | "\n", 101 | "def img_to_arr(img):\n", 102 | " \"\"\"\n", 103 | " Apply the same processing we used in training: greyscale and resize.\n", 104 | " \"\"\"\n", 105 | " img = img.convert(mode='L').resize((32, 32))\n", 106 | " return np.asarray(img).ravel() / 255" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "metadata": {}, 113 | "outputs": [], 114 | "source": [ 115 | "import os\n", 116 | "from glob import glob\n", 117 | "\n", 118 | "from PIL import Image\n", 119 | "from collections import defaultdict\n", 120 | "\n", 121 | "\n", 122 | "sets = ['train', 'val']\n", 123 | "classes = ['trilobites', 'fishes', 'forams', 'dinosaurs']\n", 124 | "\n", 125 | "data = defaultdict(list)\n", 126 | "labels = defaultdict(list)\n", 127 | "\n", 128 | "for set_ in sets:\n", 129 | " for class_ in classes:\n", 130 | " for fname in glob(f'../fossilnet/{set_}/{class_}/*.png'):\n", 131 | "\n", 132 | " img = Image.open(fname)\n", 133 | " arr = img_to_arr(img)\n", 134 | " data[set_].append(arr.ravel())\n", 135 | " data[set_].append(np.fliplr(arr.reshape(32, 32)).ravel())\n", 136 | " labels[set_] += 2 * [class_]\n", 137 | "\n", 138 | "X_train = np.array(data['train'])\n", 139 | "X_val = np.array(data['val'])\n", 140 | "\n", 141 | "y_train = np.array(labels['train'])\n", 142 | "y_val = np.array(labels['val'])" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": null, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [ 151 | "X_train.shape" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "%matplotlib inline\n", 161 | "import matplotlib.pyplot as plt\n", 162 | "\n", 163 | "plt.imshow(data['train'][503].reshape(32, 32))\n", 164 | "plt.axis('off')\n", 165 | "plt.show()" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "## Select and evaluate a model\n", 173 | "\n", 174 | "A very simple model, with only 2 hyperparameters to ensure the trees can't overfit." 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": {}, 181 | "outputs": [], 182 | "source": [ 183 | "from sklearn.ensemble import RandomForestClassifier\n", 184 | "\n", 185 | "clf = RandomForestClassifier(random_state=11,\n", 186 | " n_estimators=100,\n", 187 | " max_depth=5,\n", 188 | " min_samples_leaf=5)\n", 189 | "\n", 190 | "clf.fit(X_train, y_train)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": {}, 197 | "outputs": [], 198 | "source": [ 199 | "from sklearn.metrics import f1_score\n", 200 | "\n", 201 | "y_pred = clf.predict(X_train)\n", 202 | "\n", 203 | "f1_score(y_train, y_pred, average='weighted')" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": null, 209 | "metadata": {}, 210 | "outputs": [], 211 | "source": [ 212 | "from sklearn.metrics import f1_score\n", 213 | "\n", 214 | "y_pred = clf.predict(X_val)\n", 215 | "\n", 216 | "f1_score(y_val, y_pred, average='weighted')" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": {}, 222 | "source": [ 223 | "It's a bit overtrained... let's worry about it later. Or maybe it's SEP." 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": null, 229 | "metadata": {}, 230 | "outputs": [], 231 | "source": [ 232 | "from sklearn.metrics import confusion_matrix\n", 233 | "\n", 234 | "confusion_matrix(y_val, y_pred)" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": null, 240 | "metadata": {}, 241 | "outputs": [], 242 | "source": [ 243 | "clf.classes_" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "It's okay at forams, not great at dinos." 251 | ] 252 | }, 253 | { 254 | "cell_type": "markdown", 255 | "metadata": {}, 256 | "source": [ 257 | "## Train the model\n", 258 | "\n", 259 | "Now we can train on all the data so I'll concatenate the arrays together. (We have more data too, as we didn't use the `test` directory, but oh well." 260 | ] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "execution_count": null, 265 | "metadata": {}, 266 | "outputs": [], 267 | "source": [ 268 | "X = np.vstack([X_train, X_val])\n", 269 | "y = np.hstack([y_train, y_val])" 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": null, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [ 278 | "clf = RandomForestClassifier(random_state=11,\n", 279 | " n_estimators=100,\n", 280 | " max_depth=5,\n", 281 | " min_samples_leaf=5)\n", 282 | "\n", 283 | "clf.fit(X, y)" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": {}, 290 | "outputs": [], 291 | "source": [ 292 | "import joblib\n", 293 | "\n", 294 | "joblib.dump(clf, '../app/rf.gz')" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "----\n", 302 | "\n", 303 | "## Make predictions from new images - using URL\n", 304 | "\n", 305 | "We'd like to use this trained model to make predictions from random images people throw at us. So we need a function to process those images to be exactly like the ones we trained on.\n", 306 | "\n", 307 | "Let's stick to using the same tool we used before: `skimage`." 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "execution_count": null, 313 | "metadata": {}, 314 | "outputs": [], 315 | "source": [ 316 | "import requests\n", 317 | "from io import BytesIO\n", 318 | "import numpy as np\n", 319 | "import joblib\n", 320 | "from PIL import Image\n", 321 | "\n", 322 | "clf = joblib.load('../app/rf.gz')\n", 323 | "\n", 324 | "\n", 325 | "def fetch_image(url):\n", 326 | " \"\"\"\n", 327 | " Download an image from the web and pass to the image processing function.\n", 328 | " \"\"\"\n", 329 | " r = requests.get(url)\n", 330 | " f = BytesIO(r.content)\n", 331 | " return Image.open(f) \n", 332 | "\n", 333 | "def predict_from_image(clf, img):\n", 334 | " \"\"\"\n", 335 | " Classify an image.\n", 336 | " \"\"\"\n", 337 | " arr = img_to_arr(img)\n", 338 | " X = np.atleast_2d(arr)\n", 339 | " probs = clf.predict_proba(X)\n", 340 | " result = {\n", 341 | " 'class': clf.classes_[np.argmax(probs)],\n", 342 | " 'prob': probs.max(),\n", 343 | " 'classes': clf.classes_.tolist(),\n", 344 | " 'probs': np.squeeze(probs).tolist(), # Must be serializable.\n", 345 | " }\n", 346 | " return result" 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "execution_count": null, 352 | "metadata": {}, 353 | "outputs": [], 354 | "source": [ 355 | "with open(\"examples.txt\") as f:\n", 356 | " urls = [l.strip() for l in f.readlines()]" 357 | ] 358 | }, 359 | { 360 | "cell_type": "code", 361 | "execution_count": null, 362 | "metadata": {}, 363 | "outputs": [], 364 | "source": [ 365 | "import IPython.display\n", 366 | "\n", 367 | "IPython.display.Image(urls[2])" 368 | ] 369 | }, 370 | { 371 | "cell_type": "code", 372 | "execution_count": null, 373 | "metadata": {}, 374 | "outputs": [], 375 | "source": [ 376 | "predict_from_image(clf, fetch_image(urls[2]))" 377 | ] 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": null, 382 | "metadata": {}, 383 | "outputs": [], 384 | "source": [] 385 | } 386 | ], 387 | "metadata": { 388 | "kernelspec": { 389 | "display_name": "t20-fri-mvp", 390 | "language": "python", 391 | "name": "t20-fri-mvp" 392 | }, 393 | "language_info": { 394 | "codemirror_mode": { 395 | "name": "ipython", 396 | "version": 3 397 | }, 398 | "file_extension": ".py", 399 | "mimetype": "text/x-python", 400 | "name": "python", 401 | "nbconvert_exporter": "python", 402 | "pygments_lexer": "ipython3", 403 | "version": "3.7.6" 404 | } 405 | }, 406 | "nbformat": 4, 407 | "nbformat_minor": 2 408 | } 409 | -------------------------------------------------------------------------------- /notebooks/Hitting_some_web_APIs.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Hitting some web APIs" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "You can read websites with Python too! The easiest way is with `requests`:" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "## Geology\n", 36 | "\n", 37 | "Let's go look at [Macrostrat](https://macrostrat.org), a fantastic project at University of Wisconsin-Madison." 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "## Language analysis\n", 59 | "\n", 60 | "This one uses a rpoprietary web API, so you need a secret key to access it. You can sign up and get a key at [MeaningCloud](https://www.meaningcloud.com/). Just substitute yours in as the `meaningcloud` variable here." 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "I'm using an article from [Canadian Mining Journal](http://www.canadianminingjournal.com/news/gold-victoria-continues-to-ramp-up-eagle/)." 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "text = \"\"\"YUKON – Victoria Gold’s Eagle gold mine has produced 25,068 oz. of gold from the start of this year until the end of May, and is expected to achieve commercial production at the end of the second quarter or early in the third quarter.\n", 77 | "\n", 78 | "In a release, the company’s management noted that productivity in May was impacted by COVID-19. “May production numbers were below our expectations as we felt the impact of COVID measures on productivity,” John McConnell, the company’s president and CEO, said in a release.\n", 79 | "\n", 80 | "Between January and the end of May, Victoria mined and stacked 2.4 million tonnes of ore grading 0.84 g/t gold and produced 25,068 oz. as a result.\n", 81 | "\n", 82 | "The company has also made a US$7.1-million principal repayment on its secured credit facility at the end of May. The next principal and interest payment is due at the end of August; the company has the option to repay this debt early.\n", 83 | "\n", 84 | "Victoria wholly owns the 555-sq.-km Dublin Gulch property in central Yukon, which hosts the Eagle and Olive gold deposits.\n", 85 | "\n", 86 | "Proven and probable reserves across Eagle and Olive total 155 million tonnes grading 0.65 g/t gold for a total of 3.3 million oz. Cut-off grades for the reserves vary with deposit and material type and range from 0.15 g/t gold to 0.58 g/t gold.\n", 87 | "\n", 88 | "Based on a technical report released last December, the Eagle mine is expected to produce an average of over 210,000 oz. of gold a year at all-in sustaining costs of US$774 per oz. over a mine life in excess of 10 years.\"\"\"" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": { 95 | "scrolled": true 96 | }, 97 | "outputs": [], 98 | "source": [ 99 | "from passwords import meaningcloud" 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "execution_count": null, 105 | "metadata": {}, 106 | "outputs": [], 107 | "source": [ 108 | "def analyse_text(t, txtf='plain'):\n", 109 | " \n", 110 | " url = \"http://api.meaningcloud.com/topics-2.0\"\n", 111 | "\n", 112 | " data = {\n", 113 | " 'key': meaningcloud,\n", 114 | " 'lang': 'en',\n", 115 | " 'txt': t,\n", 116 | " 'txtf': txtf,\n", 117 | " 'tt': 'a'\n", 118 | " }\n", 119 | "\n", 120 | " headers = {'content-type': 'application/x-www-form-urlencoded'}\n", 121 | "\n", 122 | " r = requests.request(\"POST\", url, data=data, headers=headers)\n", 123 | "\n", 124 | " return r.json()" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": null, 130 | "metadata": {}, 131 | "outputs": [], 132 | "source": [ 133 | "j = analyse_text(text)" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "j.keys()" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": null, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [ 151 | "j['money_expression_list']" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "j['quotation_list']" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": null, 166 | "metadata": {}, 167 | "outputs": [], 168 | "source": [ 169 | "j['entity_list']" 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": null, 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "def get_places(j):\n", 179 | " return [x['form'] for x in j['entity_list'] if 'Location' in x['sementity']['type']]\n", 180 | "\n", 181 | "get_places(j)" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "## Geofignet\n", 189 | "\n", 190 | "Let's [take a look](https://geofignet.geosci.ai/api).\n", 191 | "\n", 192 | "We can try it on one of Conor O'Sullivan's tweets, like [this one](https://twitter.com/cmnosullivan/status/1264176075791360001)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": null, 198 | "metadata": {}, 199 | "outputs": [], 200 | "source": [] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [] 208 | } 209 | ], 210 | "metadata": { 211 | "kernelspec": { 212 | "display_name": "t20-fri-mvp", 213 | "language": "python", 214 | "name": "t20-fri-mvp" 215 | }, 216 | "language_info": { 217 | "codemirror_mode": { 218 | "name": "ipython", 219 | "version": 3 220 | }, 221 | "file_extension": ".py", 222 | "mimetype": "text/x-python", 223 | "name": "python", 224 | "nbconvert_exporter": "python", 225 | "pygments_lexer": "ipython3", 226 | "version": "3.8.3" 227 | } 228 | }, 229 | "nbformat": 4, 230 | "nbformat_minor": 4 231 | } 232 | -------------------------------------------------------------------------------- /notebooks/examples.txt: -------------------------------------------------------------------------------- 1 | https://images.fineartamerica.com/images-medium-large-5/23-trilobite-fossil-sinclair-stammersscience-photo-library.jpg 2 | https://assets3.fossilera.com/sp2/150226/diplomystus/620x400/diplomystus-dentatus.jpg 3 | https://i.pinimg.com/originals/44/ae/11/44ae11cfe407ffea220a34c9a9a3112f.jpg 4 | https://www.bgs.ac.uk/discoveringGeology/time/Fossilfocus/images/foraminifera/Elphidium.jpg 5 | https://earthsky.org/upl/2016/03/fish-fossil-green-river.jpg -------------------------------------------------------------------------------- /notebooks/images/2226ff34-b4d5-43dd-a294-9f818fe49856-sunfish.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/2226ff34-b4d5-43dd-a294-9f818fe49856-sunfish.jpg -------------------------------------------------------------------------------- /notebooks/images/32971485793_13647ecbb2_b.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/32971485793_13647ecbb2_b.jpg -------------------------------------------------------------------------------- /notebooks/images/42485-116.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/42485-116.jpg -------------------------------------------------------------------------------- /notebooks/images/91228bcba9db1b55ab7cbe7f7f280e7c.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/91228bcba9db1b55ab7cbe7f7f280e7c.png -------------------------------------------------------------------------------- /notebooks/images/Elphidium.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/Elphidium.jpg -------------------------------------------------------------------------------- /notebooks/images/fish-fossil-green-river.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/agilescientific/transform-2020-tutorial/833ddefffe60647e42817c713bf9d267ade44d2f/notebooks/images/fish-fossil-green-river.jpg --------------------------------------------------------------------------------