├── .gitignore ├── LICENSE ├── README.md ├── homeworks ├── catalysis.csv ├── hw2_p1B_data.txt ├── hw2_p2_data.txt ├── hw3_data1.txt ├── hw3_data1_test.txt ├── hw_01.ipynb ├── hw_02.ipynb ├── hw_03.ipynb ├── hw_04.ipynb ├── hw_05.ipynb ├── mauna_loa_co2.txt ├── me597_project_final.pdf ├── motor.dat ├── spe10_permx.dat └── stress_strain.txt └── lectures ├── catalysis.csv ├── challenger_data.csv ├── coal_mining_disasters.csv ├── coin_flipping.png ├── coin_toss_bayes_1 ├── coin_toss_bayes_2 ├── coin_toss_bayes_3 ├── coin_toss_bayes_plate ├── coin_toss_g ├── demos.zip ├── demos ├── __init__.py ├── _cache.py ├── _cached_function.py ├── _model.py ├── _numpy_array_cache.py ├── _utils.py ├── advection │ ├── __init__.py │ ├── _forward_advection.py │ ├── covarMatrix50.npy │ ├── data_concentrations_advection.npy │ ├── generate_data.py │ └── transport_model.py ├── catalysis │ ├── __init__.py │ ├── _forward_model.py │ ├── model_1.py │ ├── model_2.py │ └── system.py └── diffusion │ ├── __init__.py │ ├── _forward_diffusion.py │ ├── _forward_diffusion.pyo │ ├── _forward_diffusion_centers.py │ ├── _forward_diffusion_left.py │ ├── _forward_diffusion_upperleft.py │ ├── generate_data.py │ ├── transport_model.py │ ├── transport_model.pyo │ ├── transport_model_centers.py │ ├── transport_model_left_corner.py │ └── transport_model_upperleft.py ├── infer_normal_mu ├── lecture_01.ipynb ├── lecture_02.ipynb ├── lecture_03.ipynb ├── lecture_04.ipynb ├── lecture_05.ipynb ├── lecture_06.ipynb ├── lecture_07.ipynb ├── lecture_08.ipynb ├── lecture_09.ipynb ├── lecture_10.ipynb ├── lecture_11.ipynb ├── lecture_12.ipynb ├── lecture_13.ipynb ├── lecture_14.ipynb ├── lecture_15.ipynb ├── lecture_17.ipynb ├── lecture_18.ipynb ├── lecture_19.ipynb ├── lecture_20.ipynb ├── lecture_21.ipynb ├── lecture_22.ipynb ├── lecture_23.ipynb ├── lecture_24.ipynb ├── lecture_25.ipynb ├── lecture_26.ipynb ├── motor.dat ├── omega_X ├── sample_smc.py ├── trailer.png ├── trailer_g ├── trailer_m_g ├── urn.png ├── urn1_graph ├── urn2_graph ├── urn3_graph ├── urn4_graph ├── venn.png └── venn_sum_rule.png /.gitignore: -------------------------------------------------------------------------------- 1 | *.ipynb_checkpoints* 2 | *.bbl 3 | *.blg 4 | *.spl 5 | *.swo 6 | *.docx\# 7 | *.aux 8 | *.log 9 | *.pdf 10 | *.~ 11 | *.dot 12 | *.pyc 13 | *.html 14 | *.pickle 15 | *.inv 16 | *.js 17 | *.out 18 | *doctree* 19 | *.so 20 | *.sh 21 | *.swp 22 | *.gz 23 | *.png 24 | *.eps 25 | *.gif 26 | *.h5 27 | *.pcl 28 | *.o??????? 29 | *.glo 30 | *.gls 31 | *.ilg 32 | *.DS_Store 33 | src/CMakeFiles/ 34 | src/Makefile 35 | src/cmake_install.cmake 36 | build 37 | backup 38 | *.fls 39 | *.pdfsync 40 | *.fdb_latexmk 41 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Predictive Science Laboratory 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to Uncertainty Quantification (old UQ course) 2 | 3 | This course has been taught three times by Prof. [Ilias Bilionis](https://www.predictivesciencelab.org/authors/ebilionis/). You can find the old versions at the following tags: 4 | 5 | + [Spring 2018](https://github.com/PredictiveScienceLab/uq-course/tree/SP2018) 6 | + [Spring 2020](https://github.com/PredictiveScienceLab/uq-course/tree/SP2020) 7 | 8 | The Spring 2018 version was recorded and the videos can be found [here](https://nanohub.org/resources/27789). 9 | 10 | The course evolved to the following two courses: 11 | + The "ME 539 Introduction to Scientific Machine Learning" which is hosted in the [data-analytics-se](https://github.com/PredictiveScienceLab/data-analytics-se) GitHub repository. This class is a standard Purdue course and it is also offered through [EdX](https://courses.edx.org/courses/course-v1:PurdueX+ME597x+2T2020/course/). It is currently offered only to Purdue online students, but it will soon be available to everyone; and 12 | + The "ME 297 Introduction to Data Science for Mechanical Engineers" which is hosted in the [me-297-intro-to-data-science](https://github.com/PurdueMechanicalEngineering/me-297-intro-to-data-science) GitHub repository. 13 | 14 | So, note that the Jupyter notebooks in this repository are provided without any support. Visit the above-mentioned GitHub repositories if you want to see to what they have evolved. 15 | -------------------------------------------------------------------------------- /homeworks/catalysis.csv: -------------------------------------------------------------------------------- 1 | Time,NO3,NO2,N2,NH3,N2O 2 | 0,500.00,0.00,0.00,0.00,0.00 3 | 30,250.95,107.32,18.51,3.33,4.98 4 | 60,123.66,132.33,74.85,7.34,20.14 5 | 90,84.47,98.81,166.19,13.14,42.10 6 | 120,30.24,38.74,249.78,19.54,55.98 7 | 150,27.94,10.42,292.32,24.07,60.65 8 | 180,13.54,6.11,309.50,27.26,62.54 -------------------------------------------------------------------------------- /homeworks/hw2_p1B_data.txt: -------------------------------------------------------------------------------- 1 | 3.613446899587822395e-03 2 | 5.185520515790512791e-02 3 | 1.117226669837600474e-02 4 | 2.065714957751426176e-02 5 | 5.992068831807514928e-02 6 | 9.794197745635039101e-03 7 | 4.592085725129269047e-02 8 | 9.214392689039574424e-02 9 | 1.793975092838155302e-02 10 | 9.095692247556988463e-02 11 | 7.571491612794943593e-02 12 | 2.026745954467959465e-02 13 | 2.940031852943273960e-01 14 | 1.469655077111428898e-01 15 | 2.562375379225268618e-02 16 | 9.745122339342082371e-03 17 | 4.314429788054167936e-02 18 | 3.867714779485811882e-02 19 | 1.597989970461724529e-01 20 | 8.530804193222889242e-02 21 | 3.541815881322513127e-02 22 | 1.985268825690557262e-03 23 | 1.287927002932563458e-01 24 | 1.756170144316487880e-02 25 | 4.545388413035546527e-01 26 | 3.567298515885820170e-02 27 | 2.333875338456117277e-01 28 | 1.979428766903147119e-01 29 | 8.985898033290849740e-02 30 | 3.200078120924058390e-03 31 | 1.367449724887005169e-01 32 | 3.794513593640644333e-02 33 | 9.511202750660775751e-02 34 | 1.803824585506891842e-01 35 | 8.857832509220447958e-02 36 | 6.642573667262068049e-02 37 | 2.382109608127428065e-03 38 | 3.262295775800173186e-02 39 | 1.147180507412354084e-01 40 | 2.070662524760396583e-01 41 | 3.698786912357122925e-02 42 | 2.839388239353032861e-01 43 | 4.694204979033497066e-02 44 | 1.832146010382563681e-02 45 | 3.599230825557304253e-02 46 | 2.752872780181541898e-02 47 | 1.365862258405238228e-01 48 | 2.322285956676898785e-02 49 | 1.334064627637751299e-01 50 | 5.196570728300027586e-02 51 | 2.289796916735925558e-01 52 | 5.952811095112994777e-02 53 | 5.234070519618443801e-02 54 | 7.199930816744821993e-02 55 | 8.114139083645936779e-02 56 | 2.345307559474401615e-01 57 | 1.267575413303221876e-01 58 | 9.880474433834818965e-03 59 | 6.254001027217565012e-01 60 | 2.459007281226357772e-03 61 | 1.153391592307758534e-02 62 | 5.191322886508508913e-03 63 | 1.797611210118462560e-02 64 | 3.471378034510737981e-02 65 | 8.349422044451310088e-02 66 | 5.083810666055920552e-02 67 | 2.722208502150566534e-02 68 | 1.510179306339861083e-01 69 | 3.524088903250663374e-02 70 | 1.091096272088546426e-01 71 | 2.929960614659426685e-02 72 | 6.036536974365219743e-02 73 | 3.556579376041076740e-02 74 | 1.499602942024291265e-02 75 | 5.910551918926740950e-02 76 | 2.312033213006577936e-01 77 | 6.261154436286224534e-02 78 | 9.935369703771693151e-02 79 | 6.683806900493070857e-02 80 | 6.193114119619710922e-02 81 | 1.953100699914694371e-01 82 | 1.190265653102081933e-01 83 | 7.806495756519728213e-02 84 | 6.300878930970531255e-02 85 | 1.936762103322600126e-01 86 | 2.783952858179282175e-01 87 | 7.726849027501105603e-03 88 | 4.334312716897890244e-02 89 | 3.605561267994766145e-02 90 | 1.116952418410752101e-02 91 | 2.231676818679511481e-02 92 | 5.833870916186609246e-02 93 | 6.582536664089627676e-02 94 | 2.416898180843103094e-01 95 | 4.434073496703647838e-02 96 | 1.636095618295507412e-02 97 | 7.876741383784356329e-03 98 | 2.050502460357003093e-02 99 | 1.601212038967751369e-02 100 | 1.220427702427507449e-01 101 | 3.088397110010858426e-02 102 | 8.029146665609386210e-03 103 | 3.494494641283041325e-01 104 | 1.619053326312969721e-01 105 | 2.817883648678856656e-02 106 | 3.374107264497683972e-02 107 | 1.010588200597577480e-01 108 | 4.159494939191650358e-03 109 | 6.696642159889047707e-02 110 | 2.359040120833960294e-02 111 | 2.044632556149152125e-01 112 | 5.155427619341330578e-03 113 | 8.301039622327351097e-02 114 | 6.703374022717725061e-03 115 | 9.493896446443973736e-02 116 | 1.296839462251769548e-01 117 | 3.982559120849093331e-02 118 | 1.939673859877830142e-02 119 | 4.086151456671961724e-02 120 | 7.409828474082118255e-02 121 | 5.032999226485566219e-02 122 | 1.132631159925042152e-01 123 | 6.954244556200399330e-02 124 | 2.405141257621937895e-02 125 | 2.308313380228078915e-01 126 | 5.116767774392778911e-02 127 | 5.666326926069139083e-03 128 | 2.882763557617566419e-01 129 | 2.600381543175835173e-03 130 | 2.516561894251839182e-01 131 | 2.563559697807120674e-01 132 | 1.262214097002885560e-01 133 | 1.092016268331056916e-01 134 | 1.824909838208108037e-01 135 | 3.445135548123667152e-02 136 | 1.306899045677859666e-02 137 | 4.607339288757433989e-03 138 | 2.076504224741852755e-01 139 | 4.467173700436588568e-02 140 | 3.190927643350637694e-02 141 | 1.595627690503892970e-01 142 | 2.409412013635575112e-01 143 | 2.851052930985398248e-02 144 | 6.857721327721759408e-02 145 | 1.022816952752888114e-01 146 | 2.231089117360508800e-02 147 | 3.433138849584132374e-02 148 | 1.469153125583430412e-02 149 | 4.313467343197079779e-03 150 | 1.902778650112225620e-02 151 | 4.593692983239673899e-01 152 | 5.648292634317347849e-03 153 | 3.043046719087498797e-01 154 | 1.457563024613351410e-01 155 | 6.247516619878948352e-03 156 | 2.155170968839456302e-01 157 | 8.859134309949612263e-02 158 | 1.149809938045475051e-02 159 | 1.679555368856386899e-01 160 | 2.754464838172598351e-01 161 | 1.763096233038972602e-02 162 | 1.236558194460589122e-01 163 | 1.020962350021854120e-01 164 | 1.380160352623693298e-01 165 | 3.996437539621902624e-02 166 | 1.201342008730012581e-02 167 | 4.066987506406039010e-02 168 | 4.018549672010113816e-01 169 | 5.796787978293729826e-02 170 | 8.772092010953928820e-02 171 | 4.196005689719740570e-03 172 | 1.056548918190987891e-05 173 | 2.947269492451496525e-02 174 | 1.284073875160285649e-01 175 | 5.295727984862033666e-02 176 | 7.835057604539982790e-02 177 | 4.353122604333740930e-02 178 | 5.226826457831054706e-02 179 | 3.371178770909515759e-02 180 | 3.885664979414524267e-02 181 | 2.547841394237639057e-02 182 | 4.972091255011910904e-02 183 | 1.135721687883581715e-01 184 | 1.133055745899301664e-01 185 | 4.970438327410412784e-03 186 | 1.951147428203384854e-02 187 | 2.577241783880103443e-02 188 | 8.087945381896377572e-02 189 | 4.081247996926071209e-02 190 | 5.999877723252740325e-02 191 | 6.263040323017532440e-02 192 | 1.298247686331860939e-01 193 | 2.223177740723034090e-02 194 | 6.630632489651788752e-03 195 | 4.932670213094409589e-02 196 | 6.049055069059964262e-01 197 | 1.118041440208440290e-01 198 | 5.755972052233482988e-03 199 | 2.448795250724965511e-01 200 | 5.749991226937406497e-02 201 | 1.943439158385378562e-01 202 | 1.095711825554012003e-01 203 | 8.060803820543151199e-02 204 | 4.631505567024649106e-02 205 | 1.163941680974778931e-01 206 | 7.053042139095283547e-02 207 | 1.285937538456497289e-03 208 | 1.823444648924863951e-02 209 | 1.412960404491558353e-01 210 | 3.785150858574274496e-02 211 | 5.541578382574398332e-02 212 | 7.275434789865665997e-02 213 | 3.825974754974835468e-02 214 | 6.792045487536538251e-02 215 | 3.252399938696606818e-02 216 | 1.919075824010452180e-02 217 | 4.519767961370724924e-02 218 | 8.904750999397853528e-02 219 | 1.327409733143578019e-02 220 | 2.662914672461663135e-01 221 | 1.083016606350253180e-01 222 | 2.056165920256477764e-01 223 | 3.433887120017662525e-02 224 | 1.433602379639929919e-03 225 | 6.239910132637364187e-02 226 | 2.868266156796928784e-02 227 | 1.892756405637013306e-01 228 | 1.357782745526603385e-01 229 | 1.111435211357046371e-01 230 | 1.171808444211073497e-01 231 | 1.439851278534080015e-01 232 | 6.388435890737735341e-04 233 | 3.315387814671762468e-01 234 | 1.305977796456936693e-02 235 | 3.971389833216118492e-02 236 | 1.362641639187557206e-03 237 | 3.130170618599767973e-01 238 | 2.414548344848546635e-01 239 | 1.396782363118343750e-01 240 | 1.219447105883767479e-01 241 | 6.418309487098265709e-02 242 | 2.771346613558527475e-02 243 | 4.902490863232854962e-02 244 | 3.511229368106687926e-02 245 | 5.420128754979279366e-03 246 | 6.891739548786596226e-02 247 | 2.665287301157791104e-03 248 | 4.204507296675293743e-02 249 | 7.347264717320529936e-02 250 | 1.384627306371929469e-02 251 | 2.816041210620533197e-02 252 | 2.331247098831781828e-01 253 | 1.123413623070316017e-02 254 | 5.735751765480151532e-02 255 | 2.613104667496744110e-02 256 | 2.995477337312359434e-01 257 | 2.545763922380329166e-03 258 | 1.116112357989608450e-01 259 | 1.019770481755719188e-01 260 | 5.362106914900214433e-02 261 | 1.783561168580977130e-01 262 | 1.267049867105373984e-01 263 | 1.693698417628347608e-02 264 | 6.880128229986792743e-02 265 | 1.417550717848322095e-01 266 | 5.780884335029046195e-02 267 | 8.317487393263267581e-02 268 | 2.649605008322773458e-01 269 | 2.765768631676308043e-01 270 | 1.280555781408286853e-01 271 | 2.310549009235791651e-01 272 | 1.219611145841074662e-01 273 | 1.138376487887992755e-01 274 | 2.242200287134243075e-01 275 | 9.718524526074241872e-05 276 | 7.822404942594481103e-02 277 | 9.887739789165880810e-02 278 | 1.371362224022334153e-01 279 | 5.511378440418948643e-03 280 | 6.124552192805832185e-03 281 | 6.349662181324010335e-02 282 | 3.475110115181221371e-01 283 | 7.462885876474521785e-02 284 | 1.771013819415539689e-02 285 | 1.585133469003908654e-02 286 | 2.033505857243376180e-01 287 | 9.834275781764773017e-02 288 | 4.002370103404754437e-02 289 | 1.677566651766154993e-01 290 | 4.096157082015550871e-02 291 | 9.154105331896970899e-02 292 | 1.835556494275766282e-01 293 | 5.562186534160978857e-02 294 | 2.790111111238055726e-03 295 | 9.025010662559803873e-03 296 | 2.015227644821583342e-01 297 | 5.495639598521732255e-02 298 | 9.107138670793633178e-03 299 | 1.336567560810212330e-01 300 | 4.244039728923829080e-02 301 | 1.090777697469297591e-01 302 | 3.106538963168770096e-01 303 | 2.233109482728147521e-02 304 | 7.101972915138677310e-03 305 | 1.731319381936600316e-02 306 | 9.376799397561026705e-02 307 | 3.345555610341877095e-02 308 | 2.650427049090000112e-02 309 | 2.326181954452608558e-01 310 | 9.562902779133294201e-03 311 | 9.818076342951767566e-02 312 | 1.016657074685107381e-01 313 | 1.619170665497340345e-01 314 | 2.765075908450215919e-02 315 | 2.367351199883924728e-01 316 | 5.431231015765958237e-02 317 | 3.555243819476503475e-01 318 | 4.836732026692239228e-02 319 | 7.318839586242173456e-03 320 | 2.574659413819332943e-02 321 | 1.712098741398714874e-01 322 | 8.390972966949314035e-03 323 | 7.828692691955260741e-02 324 | 5.595978347562117691e-03 325 | 2.364869911669901029e-02 326 | 4.670212090446560632e-02 327 | 1.712427534016599806e-01 328 | 1.186768799555694143e-01 329 | 1.345694866635300559e-02 330 | 2.962009057476297502e-02 331 | 2.396473874217163597e-02 332 | 1.868959206968631159e-01 333 | 1.321551569268926141e-01 334 | 1.357507577335515837e-02 335 | 8.315456002254451029e-02 336 | 2.019354797707006988e-02 337 | 3.193689653096528497e-02 338 | 3.213302610118125291e-02 339 | 5.894044293250803174e-02 340 | 2.268814836114984856e-01 341 | 3.626713822845062940e-01 342 | 1.377985879573736250e-01 343 | 4.558376797155125337e-02 344 | 3.659632098473681802e-04 345 | 3.763960135060080425e-01 346 | 1.285080101808303878e-02 347 | 1.598696628618808194e-02 348 | 8.068658144722080927e-02 349 | 6.575515639073905433e-03 350 | 3.925661792620854101e-02 351 | 1.816796642766854708e-01 352 | 7.389055942688120548e-02 353 | 3.394825995197875423e-02 354 | 7.291976185167096947e-02 355 | 1.100152891521404969e-01 356 | 1.252013728077192578e-01 357 | 1.498697162060170784e-01 358 | 9.053680123453328871e-03 359 | 1.841398686802659182e-03 360 | 1.559375619581276629e-01 361 | 4.788679318793213624e-01 362 | 1.162780166922100095e-01 363 | 1.751740632255240071e-02 364 | 3.154756177999350902e-02 365 | 2.044844156733604396e-02 366 | 4.411638162834933749e-03 367 | 6.138655040755634090e-02 368 | 3.267982048771378434e-01 369 | 1.488103720208246716e-01 370 | 8.262425816762469788e-02 371 | 1.874354378345029909e-02 372 | 2.280234969708597181e-01 373 | 1.172218492677853247e-02 374 | 5.984638568737457587e-02 375 | 1.575047300932398420e-01 376 | 8.029802979362943988e-03 377 | 4.789020294706198416e-02 378 | 4.094363236899122710e-02 379 | 9.146355840201248963e-03 380 | 1.654295125914938963e-02 381 | 3.946838032916575451e-02 382 | 7.926548080654952089e-02 383 | 2.594729277362184394e-01 384 | 6.060219227507177703e-02 385 | 4.416763794262649340e-02 386 | 2.211366394077021735e-01 387 | 1.102547162067693542e-01 388 | 9.610804190079001719e-02 389 | 1.544883326753071773e-01 390 | 2.711386570285329589e-01 391 | 1.328694264654761426e-01 392 | 1.176665149334336358e-01 393 | 1.414644957131322156e-01 394 | 2.861526282656784437e-01 395 | 4.273268628613829390e-01 396 | 1.034037783054481130e-01 397 | 1.703503267815668562e-02 398 | 1.458590686042006146e-01 399 | 1.437479097978667519e-01 400 | 1.539533040749668791e-01 401 | 5.799183410367197977e-02 402 | 6.212746541357298119e-02 403 | 1.863775599659414361e-01 404 | 1.937194625033899793e-03 405 | 4.365007674413305325e-03 406 | 4.814547346258210569e-02 407 | 1.877689478785805194e-02 408 | 2.658667768047282554e-01 409 | 8.329630667543497236e-02 410 | 8.651308688820437409e-02 411 | 3.698930695672865698e-01 412 | 2.278592707257175443e-01 413 | 6.279952426069307370e-02 414 | 1.982884926131426445e-02 415 | 5.552147900131153974e-02 416 | 2.812387248792392463e-02 417 | 3.523779991545245016e-02 418 | 4.575435063665900248e-02 419 | 1.095030388841590901e-01 420 | 1.163254466810550725e-01 421 | 9.946774444283162930e-02 422 | 8.853023049359588781e-02 423 | 1.402522367272392200e-03 424 | 4.806076444266423747e-02 425 | 8.626089066520804469e-03 426 | 1.495533243732963258e-01 427 | 1.698081147251363776e-01 428 | 8.249513667097611858e-02 429 | 1.267196619713571448e-01 430 | 5.242934500693561400e-02 431 | 5.083102596720386057e-02 432 | 2.618755104402893941e-01 433 | 9.275426676323102193e-02 434 | 4.179940575733434582e-02 435 | 4.038913073302865392e-02 436 | 6.315573037214335106e-02 437 | 3.459643711580797504e-02 438 | 1.533337126240769122e-01 439 | 7.799364352402188630e-03 440 | 1.565900768133494658e-01 441 | 2.741441182612699978e-02 442 | 4.682110768887206252e-03 443 | 9.212564873998119186e-02 444 | 4.604826879717845156e-02 445 | 1.060115813912006037e-02 446 | 1.970060653082535487e-02 447 | 1.524063927136574959e-01 448 | 3.169344219267105078e-01 449 | 2.235707179978253523e-01 450 | 5.156116227199128736e-04 451 | 3.633259898431256407e-01 452 | 1.110196621167583997e-01 453 | 6.092911097257690950e-02 454 | 6.471427325660560059e-02 455 | 4.097817271962222313e-02 456 | 1.100750919627855179e-01 457 | 8.133999215599124755e-02 458 | 1.165420811382065214e-01 459 | 1.017516501854704852e-01 460 | 1.126379995628632030e-01 461 | 1.498425443993003092e-01 462 | 5.601670236427912924e-02 463 | 9.038824056368613413e-02 464 | 3.122774567656402475e-02 465 | 6.746423586758422408e-02 466 | 1.064344733758626760e-01 467 | 4.899993423064871734e-02 468 | 4.453937900965817404e-02 469 | 1.433865586011535409e-01 470 | 3.535541735215301518e-03 471 | 9.705157014545684468e-02 472 | 5.208319108789854351e-02 473 | 5.317603403891249525e-02 474 | 3.419360132853230816e-01 475 | 6.981679903292748075e-03 476 | 1.219320656430354088e-01 477 | 6.378219707957383244e-02 478 | 1.408185405844566696e-01 479 | 2.627205959531886403e-02 480 | 3.180712708140100847e-02 481 | 7.198517773896136174e-02 482 | 9.190788348764855620e-02 483 | 5.682607877766863297e-02 484 | 3.780714100137114869e-02 485 | 3.496926428007437332e-02 486 | 6.007857373209048862e-03 487 | 4.087430653237175138e-01 488 | 6.117657647979002355e-02 489 | 1.208440220644038665e-02 490 | 4.119555324612005909e-01 491 | 5.961738051501890706e-02 492 | 1.251403917027311408e-01 493 | 3.403072500806796769e-02 494 | 3.946721897594752254e-03 495 | 9.359144822734301317e-02 496 | 8.491419693192321649e-02 497 | 1.356575553238578424e-01 498 | 9.318329314082959489e-03 499 | 8.634302658185931401e-02 500 | 1.294345337655695982e-02 501 | 1.072365693977554157e-01 502 | 1.750869586224932176e-02 503 | 5.422187323948719773e-02 504 | 1.808699012994285232e-02 505 | 2.284808598007764281e-04 506 | 6.109740294128146454e-02 507 | 1.884319694275149448e-01 508 | 2.363016151713072388e-01 509 | 8.536522047475775254e-02 510 | 1.034989278559805270e-01 511 | 1.285487439704882835e-03 512 | 1.724544073590246196e-01 513 | 7.822687730867904088e-02 514 | 1.189695573218086389e-01 515 | 5.809862476789406927e-02 516 | 1.160942035290099678e-01 517 | 1.062956858928156695e-02 518 | 2.297144995283555147e-01 519 | 6.382295221684673725e-02 520 | 4.952967470788941645e-02 521 | 5.103562794506935792e-03 522 | 5.893654319829255001e-02 523 | 2.194430418426476098e-02 524 | 4.939143330413319627e-02 525 | 4.597991793658161991e-02 526 | 5.597626677728059774e-02 527 | 2.799765547194555393e-01 528 | 1.005078385551660125e-01 529 | 1.455842188290540906e-01 530 | 4.098101460177780986e-01 531 | 1.291482762293642839e-01 532 | 9.696173108024496237e-03 533 | 1.249545536960523506e-01 534 | 4.063263700544691992e-02 535 | 9.186886188264900208e-03 536 | 1.048067801414661604e-01 537 | 4.170474866771084477e-02 538 | 2.396830273673984310e-02 539 | 7.464801290215579421e-02 540 | 4.684371487405016715e-02 541 | 2.228995804338729236e-02 542 | 1.797967028193239031e-02 543 | 1.856654042786128889e-01 544 | 7.590219155768432313e-02 545 | 3.509552125526515381e-02 546 | 7.281675488938459118e-02 547 | 8.435718586743949643e-02 548 | 6.745232611060923700e-02 549 | 1.006633366587102590e-01 550 | 5.189315458389137659e-02 551 | 8.790470779773269794e-02 552 | 5.373091374459579295e-02 553 | 1.643735265606559998e-02 554 | 1.010569075863245597e-01 555 | 1.809539372654850597e-01 556 | 5.601349048549541382e-02 557 | 4.585521822413521353e-02 558 | 9.685332999130397136e-02 559 | 2.214344663054154116e-01 560 | 2.595647555255607419e-01 561 | 1.037230322784899600e-01 562 | 2.367943230751460779e-02 563 | 1.580506915606542107e-01 564 | 1.693622431373074111e-01 565 | 2.200637760919370300e-02 566 | 9.829831182443364568e-02 567 | 5.915481273345002400e-02 568 | 2.692498422219691268e-01 569 | 1.330546866009908735e-01 570 | 3.892166923322832543e-03 571 | 4.538867512211591793e-02 572 | 6.382354127533471436e-03 573 | 6.993079150623502382e-01 574 | 1.253814748029301318e-01 575 | 1.415267076821507541e-01 576 | 1.929559907340749544e-02 577 | 5.000501835800452016e-02 578 | 1.995997425483509932e-02 579 | 2.455383949639501628e-02 580 | 1.970399725757722084e-02 581 | 4.162082666843591738e-02 582 | 7.319566607089555976e-02 583 | 1.938258829945235240e-01 584 | 8.971542945651575507e-02 585 | 6.695579438270655992e-03 586 | 7.693632250291822239e-02 587 | 3.391853202779016585e-01 588 | 7.556849443880692130e-02 589 | 1.433903043061392490e-01 590 | 2.431522029679132613e-03 591 | 3.633830967365677539e-02 592 | 6.741036307148762230e-02 593 | 1.797410757808991233e-02 594 | 8.878845067645639577e-02 595 | 1.368627460797341155e-01 596 | 1.163847593243031442e-01 597 | 3.168568154410642684e-02 598 | 3.432170949135425475e-02 599 | 4.987007234061972827e-03 600 | 3.558491538314225711e-01 601 | 1.717263090590678276e-01 602 | 1.862966286095645574e-01 603 | 1.556476948032610268e-02 604 | 1.457326349457069847e-01 605 | 3.210503696080475172e-02 606 | 2.210134827949790659e-01 607 | 7.936929220989258696e-03 608 | 8.319106305302596815e-02 609 | 6.517836870110410574e-02 610 | 1.793932522916328043e-01 611 | 1.367219261882931247e-01 612 | 1.051300645601795924e-01 613 | 2.157006789955440706e-01 614 | 3.823648676613787545e-02 615 | 8.089687623119840021e-02 616 | 3.241994052066753899e-03 617 | 6.497312515541120437e-02 618 | 4.423532967527344512e-02 619 | 9.678273295507570928e-02 620 | 9.256105893465121914e-02 621 | 1.184839726319725783e-01 622 | 1.467421333335249545e-01 623 | 3.692344533150579422e-01 624 | 4.772885812587861210e-02 625 | 7.172699674124494162e-02 626 | 2.121923475722812535e-01 627 | 9.696026117812056588e-02 628 | 2.372748983578662485e-02 629 | 4.356602314073141410e-04 630 | 2.622892239191367339e-02 631 | 5.450546878831383224e-02 632 | 2.657173884967202471e-02 633 | 6.114396184061894612e-02 634 | 2.764262672287896572e-01 635 | 9.655907276944192263e-02 636 | 2.407138619345776387e-02 637 | 3.426509728608109068e-02 638 | 3.147884934052622846e-02 639 | 2.539716245376073649e-03 640 | 2.408370969448188625e-01 641 | 2.773206287944848802e-02 642 | 5.234646858845975331e-02 643 | 4.161015572526027828e-01 644 | 7.325769558775074852e-02 645 | 1.841570687121598937e-01 646 | 3.071986967487434772e-02 647 | 8.004408249339953207e-02 648 | 1.031658928163095047e-02 649 | 3.674616660904051618e-02 650 | 4.939559804721593839e-02 651 | 1.394965560052219278e-02 652 | 1.374165790788724195e-01 653 | 1.160706793002017484e-01 654 | 9.340252119702346922e-02 655 | 9.502094153872406301e-02 656 | 1.672861608119327256e-02 657 | 6.343264182394407757e-02 658 | 3.671045755271218347e-01 659 | 1.563251048202958793e-01 660 | 1.026701045791196705e-02 661 | 1.968658969562116501e-01 662 | 2.611767920189613304e-02 663 | 1.415180348603844729e-01 664 | 1.913779297348790209e-01 665 | 1.290114968174335165e-02 666 | 4.174494800612420509e-01 667 | 4.887049848819193243e-01 668 | 1.532151906043760503e-01 669 | 1.387805858296012884e-01 670 | 1.267661531781784923e-01 671 | 1.501482169409353584e-01 672 | 1.810436693296669508e-01 673 | 4.391492041261513268e-02 674 | 3.973206098123407276e-02 675 | 9.030658560498716625e-02 676 | 1.119002055291435510e-01 677 | 6.268495509860431969e-02 678 | 2.161203216727724044e-02 679 | 1.578751587006666235e-01 680 | 4.063527373479092208e-02 681 | 6.036363748536652252e-02 682 | 1.251584864726471569e-01 683 | 4.301426403159098327e-02 684 | 1.178472770325963981e-01 685 | 1.296454896242232591e-01 686 | 5.148116217424028584e-02 687 | 1.678073551818483611e-01 688 | 2.109058088461929492e-01 689 | 4.820682809375806399e-02 690 | 4.966227674148432532e-03 691 | 4.270422162016024314e-02 692 | 1.483317706492033594e-01 693 | 8.087622573644344393e-02 694 | 1.242311201685630286e-03 695 | 1.344067423603586275e-01 696 | 2.648291169950036869e-02 697 | 1.697933933238355286e-01 698 | 6.519223615336744437e-02 699 | 9.590984208672316547e-02 700 | 1.385055279227936642e-01 701 | 1.285534467942979486e-01 702 | 4.447740648902941824e-02 703 | 5.416362095501071547e-03 704 | 5.948929326015955227e-02 705 | 2.556346812163598492e-01 706 | 2.584032717674132895e-01 707 | 3.923463512141170048e-03 708 | 6.692413015874240645e-02 709 | 8.880179667558781020e-02 710 | 3.509811381687404364e-01 711 | 1.681148368515599234e-01 712 | 2.495285379715552962e-02 713 | 3.124003900943017262e-02 714 | 6.031400800553800290e-02 715 | 9.834646246640356415e-03 716 | 1.700842754767561138e-01 717 | 1.399394198806196976e-01 718 | 2.039012568517953872e-01 719 | 4.237517418366497368e-02 720 | 3.327785775759157377e-02 721 | 1.943207724937920031e-02 722 | 1.098386977186631697e-01 723 | 1.341914012358020880e-01 724 | 7.080822748846522796e-02 725 | 3.404338917501571055e-02 726 | 4.499779639581590052e-02 727 | 1.103340569748936101e-01 728 | 2.505309622972791694e-01 729 | 2.964283483317524115e-01 730 | 1.086240590140281626e-01 731 | 2.534449387191140685e-02 732 | 6.784252628868381318e-02 733 | 2.697608395030750006e-01 734 | 1.566739401788672381e-01 735 | 2.619303862746153455e-01 736 | 1.697109988676758219e-01 737 | 9.437112795437206658e-02 738 | 5.010286039595168717e-02 739 | 8.678490263831844034e-03 740 | 3.903332433392419851e-03 741 | 1.712236176436742197e-01 742 | 7.000357229902831602e-02 743 | 2.822661667588443057e-01 744 | 5.594318158632710300e-03 745 | 3.623255626312518046e-02 746 | 3.502112025294818604e-02 747 | 1.887841513557680651e-01 748 | 2.206929693717104013e-01 749 | 1.379705161897519516e-02 750 | 1.976478700739582248e-02 751 | 1.237483375347114278e-01 752 | 2.004395845939206666e-02 753 | 2.428240048100818171e-01 754 | 3.626403450771673032e-01 755 | 2.466462864982744280e-02 756 | 2.743840204924474224e-01 757 | 8.538399721882776461e-02 758 | 4.538237770650943026e-01 759 | 1.102075859052428254e-03 760 | 8.649519319157827901e-02 761 | 1.088030882742000924e-02 762 | 2.130692227495222524e-01 763 | 7.142496476133417538e-01 764 | 1.307269683288405648e-01 765 | 1.053172009401359688e-01 766 | 5.340994771933370994e-02 767 | 5.634273741080857850e-02 768 | 1.109140103054982868e-01 769 | 1.071960692378296359e-01 770 | 6.231641016647011810e-02 771 | 2.102287219516289080e-02 772 | 2.539614500194815050e-01 773 | 2.724589551030023524e-02 774 | 1.346817298441914390e-01 775 | 2.102062465230246369e-02 776 | 1.885091138726252558e-02 777 | 3.771615204907811658e-01 778 | 8.019404950543856242e-03 779 | 1.086146045029910134e-01 780 | 6.900941020849730989e-02 781 | 1.484097192498741169e-01 782 | 4.952337540702787810e-02 783 | 4.840187492277803760e-01 784 | 8.835271735926028894e-03 785 | 3.983422144224453842e-01 786 | 8.897385893123274658e-02 787 | 3.235778539480729105e-02 788 | 1.273535633756348318e-01 789 | 1.563412288601999205e-03 790 | 7.929523810012679974e-02 791 | 6.504579862980178206e-02 792 | 1.722681434800251987e-01 793 | 2.021023015516939993e-03 794 | 2.956736374058144962e-02 795 | 7.931293510480828812e-02 796 | 1.472551142741797825e-01 797 | 1.465885798874704340e-02 798 | 1.047444847113342264e-01 799 | 2.119878171548426060e-01 800 | 3.889772996295356999e-01 801 | 2.243517537014295682e-01 802 | 2.677686558643221568e-02 803 | 2.042438523833155217e-02 804 | 1.080226735867167477e-02 805 | 1.448233722354009545e-01 806 | 2.402140441509809099e-01 807 | 2.076453011975049404e-02 808 | 3.196309960367336306e-02 809 | 1.751184707625012479e-02 810 | 2.371173536588341568e-01 811 | 3.305663696953039382e-02 812 | 8.792613253946204854e-02 813 | 1.880316183647633144e-01 814 | 1.060273634930242875e-02 815 | 2.762485906972838093e-02 816 | 2.626601119407929834e-03 817 | 3.406046126487256576e-01 818 | 4.069904741528242165e-02 819 | 1.128993765286150919e-01 820 | 1.024482211295940220e-01 821 | 6.471660022537735479e-02 822 | 6.243861588478014438e-04 823 | 5.864618384189154709e-02 824 | 1.770201866272523572e-01 825 | 7.662370479272097301e-02 826 | 9.499812435419474621e-02 827 | 5.485600980104043956e-02 828 | 8.256611377828908249e-02 829 | 1.619104502908434717e-01 830 | 5.716887403229869297e-02 831 | 1.234200079195162669e-01 832 | 2.392322133009192331e-01 833 | 8.096317418211862460e-02 834 | 6.521759237481634941e-02 835 | 4.047261010773038625e-02 836 | 2.431128415847854107e-01 837 | 1.519545868636000274e-01 838 | 2.142285301216158289e-02 839 | 8.079250266241964906e-02 840 | 3.093994071374127288e-04 841 | 2.080808841874821555e-02 842 | 7.504301670575752592e-02 843 | 2.328816621002184183e-03 844 | 2.294339993465231431e-01 845 | 1.711969872603979776e-03 846 | 2.158423772834174026e-01 847 | 2.665167577850154068e-02 848 | 1.271871596967929230e-01 849 | 1.087330310154054624e-01 850 | 1.756680941316215994e-02 851 | 2.441442733059441192e-02 852 | 5.422734359641143143e-02 853 | 1.930241406814429961e-01 854 | 2.056453768973764973e-01 855 | 8.273957589384683606e-03 856 | 5.173757237487164318e-02 857 | 2.043318612120293906e-02 858 | 6.077505979646333212e-03 859 | 1.414656763394341876e-01 860 | 1.463926196919058953e-01 861 | 1.246520172410794068e-02 862 | 1.251073712210243050e-01 863 | 1.056296021214014474e-01 864 | 1.796312705323224179e-01 865 | 1.705589198481980984e-01 866 | 1.367215043788907736e-01 867 | 6.946097420519961770e-03 868 | 1.130880016715288816e-01 869 | 2.383991618442972416e-01 870 | 1.095636699372826151e-01 871 | 9.787248623436032528e-04 872 | 1.676152732845375901e-02 873 | 1.553198112450689117e-01 874 | 1.908257827689140151e-02 875 | 6.394104980527771931e-02 876 | 2.143718528928573522e-02 877 | 2.309172761749918590e-01 878 | 8.459528295151147459e-02 879 | 6.348079008609042173e-02 880 | 5.127250832147342563e-02 881 | 6.687701487546741930e-02 882 | 1.055417423159092094e-01 883 | 1.160358611932647693e-01 884 | 8.130066510495904952e-02 885 | 2.013301450072741139e-01 886 | 6.934076076427381219e-02 887 | 3.139681266572923179e-02 888 | 4.334045331381912169e-01 889 | 2.502804983611873341e-02 890 | 2.069560972563361245e-01 891 | 1.953079398719045207e-02 892 | 2.354625365381382562e-02 893 | 9.958521157985922301e-02 894 | 2.895839852092644412e-02 895 | 5.905198172119825728e-02 896 | 1.851277777415811787e-01 897 | 4.939556084560683957e-02 898 | 1.701804085824388515e-03 899 | 2.019326165948862398e-02 900 | 7.952038589853162309e-02 901 | 4.081557419002175557e-02 902 | 4.428558105427514782e-03 903 | 8.780534597241976269e-02 904 | 6.705843955062112949e-02 905 | 6.177520279094749628e-02 906 | 6.368770543647495064e-02 907 | 5.244419213788206746e-02 908 | 3.678883316954170007e-02 909 | 7.936578954943315778e-02 910 | 2.364744685599934337e-01 911 | 5.061939884845203727e-03 912 | 4.339478020144698321e-03 913 | 3.712257897614524849e-01 914 | 5.232242737293371637e-02 915 | 6.036811942211973980e-02 916 | 2.129725523944199506e-02 917 | 1.012201063869846984e-01 918 | 1.153744669823571489e-01 919 | 2.452346200308743590e-02 920 | 3.058352293101433694e-02 921 | 3.057476036263339844e-02 922 | 4.235084278566567800e-02 923 | 2.744209028629349123e-01 924 | 5.883594461725099797e-03 925 | 5.891727480535421946e-02 926 | 1.204660290150041391e-01 927 | 1.764805949741660857e-02 928 | 5.008632910675823308e-02 929 | 2.148006420661381644e-02 930 | 2.186697454180973887e-01 931 | 5.822646368632533598e-02 932 | 4.072339865625513450e-02 933 | 7.577238096178204318e-02 934 | 2.335852248726099367e-01 935 | 5.266788441890029454e-02 936 | 9.269118169675199040e-03 937 | 1.676537888513746077e-01 938 | 2.611180905281855202e-01 939 | 8.924397791980021033e-02 940 | 6.520601452180408986e-01 941 | 2.855651636570635810e-01 942 | 4.287557226283834205e-02 943 | 1.432217651808068426e-01 944 | 2.467664318416816147e-02 945 | 2.093937733373801291e-01 946 | 4.416510312337950869e-03 947 | 2.799388232668438281e-01 948 | 1.245646189973076967e-01 949 | 7.084852775223153543e-02 950 | 1.772814458131053914e-02 951 | 6.710580321148566785e-02 952 | 5.279209888880044554e-02 953 | 5.010955878819821699e-03 954 | 6.049327547874493866e-02 955 | 8.041326555613492397e-03 956 | 1.399309261285211770e-01 957 | 2.765869690358863653e-01 958 | 4.883296739701989408e-02 959 | 2.889452159808904644e-02 960 | 2.702475994461012859e-03 961 | 1.896683180134527136e-02 962 | 9.719694830738455360e-02 963 | 5.673573642395132732e-02 964 | 8.059952180391826912e-03 965 | 1.693842267057079631e-01 966 | 5.788922838062676018e-02 967 | 8.766989839975199150e-02 968 | 4.886505782127715386e-02 969 | 1.914344055876725584e-02 970 | 1.009439476052084173e-01 971 | 3.119246490913448952e-02 972 | 2.331387338689382138e-02 973 | 4.530835594943508071e-02 974 | 5.601874935020971849e-03 975 | 1.204431636269792244e-01 976 | 5.777131847517086305e-02 977 | 2.881823398622846560e-02 978 | 1.808111069560324746e-01 979 | 4.064190710399853690e-02 980 | 2.873476354970297408e-02 981 | 3.567041192916944881e-02 982 | 5.169685099362086173e-02 983 | 4.485212698060198067e-02 984 | 1.689156872500014450e-01 985 | 8.162393294629005647e-02 986 | 1.892034700443660095e-02 987 | 2.004245763081365928e-01 988 | 1.329153001484313368e-01 989 | 9.306596353191477766e-02 990 | 4.593544838843450183e-03 991 | 1.186864300235874192e-01 992 | 5.542870181548253256e-02 993 | 1.684925895364262557e-02 994 | 1.310929392261728221e-01 995 | 1.665256415325823414e-01 996 | 3.590168733255882927e-02 997 | 5.115607974136494873e-02 998 | 7.645473484790775431e-02 999 | 1.421722106277761577e-01 1000 | 8.934135745206987431e-03 1001 | -------------------------------------------------------------------------------- /homeworks/hw3_data1.txt: -------------------------------------------------------------------------------- 1 | 8.664103018084330365e-01 8.577490627486041852e-01 -5.461432144562790025e-01 4.872441060530816603e-01 3.893718322867183446e-01 -1.281823948437434968e-01 8.778513610189389027e-01 -9.219787601917812658e-01 -3.265248622613110552e-01 7.860439750719174778e-01 -8.938332017914198957e-01 -6.287047970416197451e-01 3.908607275911981649e-01 -8.450318705714161549e-01 4.473497542857962639e-01 1.063636697572074574e-01 -3.038096452951637438e-01 -6.828681127565117137e-01 -9.851515911539454606e-01 -4.474127873727058446e-01 2 | 1.303241491455559564e-01 -1.747301650388057848e-01 -6.081613310159188490e-01 9.681369646846307853e-02 -4.454273924201688217e-01 -6.474382821489710338e-02 1.401735668201074536e-01 -7.987487233885595117e-01 -3.440230872530442019e-01 2.636760650900552561e-01 -7.109467979688894879e-01 -6.726569851726263982e-01 8.790178671058726523e-02 -6.603706226057112216e-01 1.821065298465392168e-02 -2.970711449376042257e-01 -3.749429918950374363e-01 -4.172961414239574340e-01 -9.917942918527465901e-01 -5.983405130270973427e-01 3 | -------------------------------------------------------------------------------- /homeworks/hw3_data1_test.txt: -------------------------------------------------------------------------------- 1 | 8.102501747408332733e-01 -6.995307983349041692e-01 3.470997181199300119e-01 9.083720207856154083e-01 -6.175341732476615775e-01 6.213179657925915755e-01 -1.944658552965337073e-01 -8.824504273015021383e-02 -6.194024532326025678e-02 -3.490585313079743734e-01 7.573256153870855378e-01 8.800460570771904134e-01 -4.674247541954721363e-01 -9.082529528170280031e-01 -5.973084850213743469e-01 -9.139742143842215505e-01 5.943949922910689576e-01 -4.875143136541830735e-01 2.638542639837038184e-01 -1.982449246402919751e-01 2 | 2.040147373577534951e-01 -6.098796579329396339e-01 -2.470520208733037459e-01 4.154823273989486365e-01 -8.169510623745737110e-01 2.596219925837400244e-01 -5.222743387683155269e-01 -4.428399466936260298e-01 -4.378149486674891011e-01 -2.548214942590145693e-01 1.385240753486588483e-02 1.365147353872745439e-01 -5.992892909999170126e-01 -8.739441220422214940e-01 -5.623338988545931727e-01 -6.388258570916068857e-01 -6.872690960944397021e-02 -3.445444833362516501e-01 -1.383493401416524549e-03 -2.977657342071415858e-01 3 | -------------------------------------------------------------------------------- /homeworks/hw_02.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Homework 2\n", 8 | "\n", 9 | "**Due: 02/13/2020**\n", 10 | "\n", 11 | "\n", 12 | "## References\n", 13 | "\n", 14 | "+ Lectures 7-10 (inclusive).\n", 15 | "\n", 16 | "## Instructions\n", 17 | "\n", 18 | "+ Type your name and email in the \"Student details\" section below.\n", 19 | "+ Develop the code and generate the figures you need to solve the problems using this notebook.\n", 20 | "+ For the answers that require a mathematical proof or derivation you can either:\n", 21 | " \n", 22 | " - Type the answer using the built-in latex capabilities. In this case, simply export the notebook as a pdf and upload it on gradescope; or\n", 23 | " - You can print the notebook (after you are done with all the code), write your answers by hand, scan, turn your response to a single pdf, and upload on gradescope.\n", 24 | "\n", 25 | "\n", 26 | "**Note**: Please match all the pages corresponding to each of the questions when you submit on gradescope. " 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "## Student details\n", 34 | "\n", 35 | "+ **First Name:**\n", 36 | "+ **Last Name:**\n", 37 | "+ **Email:**" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "# Here are some modules that you may need - please run this block of code:\n", 47 | "import matplotlib.pyplot as plt\n", 48 | "%matplotlib inline\n", 49 | "import seaborn as sns\n", 50 | "sns.set_context('talk')\n", 51 | "import numpy as np\n", 52 | "import scipy\n", 53 | "import scipy.stats as st" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "## Problem 1\n", 61 | "### Part A\n", 62 | "Let $X$ be a continuous random variable with CDF:\n", 63 | "$$\n", 64 | "F(x) = p(X \\le x).\n", 65 | "$$\n", 66 | "Show that the random variable\n", 67 | "$$\n", 68 | "Z = F(X)\n", 69 | "$$\n", 70 | " is distributed uniformly in $[0,1]$.\n", 71 | " Hint: Show that:\n", 72 | "$$\n", 73 | " F_Z(z) := p(Z \\le z) = z.\n", 74 | "$$\n", 75 | "**Proof:**" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "*Type your proof here. Delete that ``
`` line (it just makes some white space).*\n", 83 | "









" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "### Part B" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "The theorem you have just proved is very useful when you want to test if a set of observations $x_1,x_2,\\dots,x_N$ has been indeed independent realizations of a random variable $X$ with CDF $F(x)$.\n", 98 | "Part A proves that, if this hypothesis is valid, then the transformed dataset $z_1,z_2,\\dots,z_N$, where\n", 99 | "$$\n", 100 | "z_i = F(x_i),\n", 101 | "$$\n", 102 | "should be distributed uniformely in $[0,1]$.\n", 103 | "In other words, the empirical histrogram of $z_1,z_2,\\dots,z_N$ should be a flat line.\n", 104 | "Use this observation to find the distribution from which this dataset was sampled:" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "metadata": {}, 111 | "outputs": [], 112 | "source": [ 113 | "data = np.loadtxt('hw2_p1b_data.txt')\n", 114 | "fig, ax = plt.subplots()\n", 115 | "ax.hist(data, density=True, alpha=0.5)\n", 116 | "ax.set_xlabel('$x$')\n", 117 | "ax.set_ylabel('Empirical PDF')\n", 118 | "ax.set_title('Histogram of data')" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "The correct distribution is one of the following:\n", 126 | "\n", 127 | "1. Standard normal, $\\mathcal{N}(0,1)$;\n", 128 | "\n", 129 | "2. Normal with mean 2 and variance 2, $\\mathcal{N}(2,2)$; or\n", 130 | "\n", 131 | "3. Exponential with rate parameter $1$, $\\mathcal{E}(1)$; or\n", 132 | "\n", 133 | "4. Exponential with rate parameter $2$, $\\mathcal{E}(2)$; or\n", 134 | "\n", 135 | "5. Exponential with rate parameter $10$, $\\mathcal{E}(10)$; or\n", 136 | "\n", 137 | "6. Gamma distribution with parameters $\\alpha=2.$ and $\\beta=3.$.\n", 138 | "\n", 139 | "Systematically go over these distributions and try to determine which one generated the data.\n", 140 | "All the required CDF's and inverse CDF's are implemented in [scipy.stats](https://docs.scipy.org/doc/scipy/reference/stats.html).\n", 141 | "Check also [scipy.stats.rv_continuous](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rv_continuous.html#scipy.stats.rv_continuous).)\n", 142 | "Please pay special attention to the defintiion of the probability distributions of the various random variables and how you can control their parameters.\n", 143 | "As a hint, here is how you can test for $\\mathcal{N}(0,1)$:" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": null, 149 | "metadata": {}, 150 | "outputs": [], 151 | "source": [ 152 | "# Testing for N(0,1):\n", 153 | "transformed_data = st.norm(loc=0, scale=1).cdf(data) # cdf() gives the CDF of a random variable\n", 154 | "# If the data came from N(0,1), the histogram of the transformed_data should match that of a uniform:\n", 155 | "fig, ax = plt.subplots()\n", 156 | "ax.hist(transformed_data, density=True, alpha=0.5)\n", 157 | "ax.plot(np.linspace(0, 1,50), np.ones(50), lw=2, color='r') # This is the line that you should try to much.\n", 158 | "ax.set_xlabel('$z = F(x)$')\n", 159 | "ax.set_ylabel('Empirical PDF')\n", 160 | "ax.set_title('Testing the $\\mathcal{N}(0,1)$');" 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "## Problem 2\n", 168 | "\n", 169 | "Consider the following data set $x_1,\\dots,x_N$:" 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": null, 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "data = np.loadtxt('hw2_p2_data.txt')\n", 179 | "data = np.array(data)\n", 180 | "fig, ax = plt.subplots()\n", 181 | "ax.hist(data, alpha=0.5)\n", 182 | "ax.set_xlabel('$x$')\n", 183 | "ax.set_ylabel('Empirical PDF')" 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "metadata": {}, 189 | "source": [ 190 | "Your goal is to generate a procedure that samples from the same distribution as the observed data. \n", 191 | "This is a variation of the standard *density estimation* problem.\n", 192 | "In general, this is a very difficult problem and we will see various ways to solve it later on.\n", 193 | "In this problem, you will develop a simple method that relies on the empirical CDF of the observed data.\n", 194 | "Needless to say, this method works only for one dimensional cases in which you have a lot of observations.\n", 195 | "\n", 196 | "The [empirical CDF](https://en.wikipedia.org/wiki/Empirical_distribution_function) of our data set, $x_1,\\dots,x_N$ is defined to be:\n", 197 | "$$\n", 198 | "\\hat{F}_N(x) = \\frac{\\text{Number of observations}\\;\\le x}{N} = \\frac{1}{N}\\sum_{i=1}^N 1_{[x_i,+\\infty]}(x),\n", 199 | "$$\n", 200 | "where $1_A(x)$ is the [indicator function](https://en.wikipedia.org/wiki/Indicator_function) of the set $A$.\n", 201 | "Using the, so called, [strong law of large numbers](https://en.wikipedia.org/wiki/Law_of_large_numbers#Strong_law), we can show that $\\hat{F}_N(x)$ converges to the true CDF of the data as $N\\rightarrow+\\infty$." 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": {}, 207 | "source": [ 208 | "### Part A\n", 209 | "\n", 210 | "Complete the code that calculates the empirical CDF:" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": 1, 216 | "metadata": {}, 217 | "outputs": [], 218 | "source": [ 219 | "def myECDF_base(x):\n", 220 | " \"\"\"\n", 221 | " Make this code work if ``x`` is a simple scalar.\n", 222 | " \n", 223 | " :param x: The point at which you want to observe the PDF.\n", 224 | " :returns: The value of the empirical CDF at ``x``.\n", 225 | " \"\"\"\n", 226 | " N = data.shape[0]\n", 227 | " # Write your code here (delete the next line and return the right value)\n", 228 | " raise NotImplementedError('Implement me!')\n", 229 | "\n", 230 | "# Vectorize your function (i.e., make it work with 1D numpy arrays).\n", 231 | "# See this: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.vectorize.html\n", 232 | "myECDF = np.vectorize(myECDF_base)" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": null, 238 | "metadata": {}, 239 | "outputs": [], 240 | "source": [ 241 | "# You can test your results by comparing the empirical CDF you can get with of scipy.stats\n", 242 | "# The two should match almost exactly\n", 243 | "hist_rv = st.rv_histogram(np.histogram(data, bins=1000))\n", 244 | "fig, ax = plt.subplots()\n", 245 | "# The range in which the x's takes values:\n", 246 | "x_min = data.min()\n", 247 | "x_max = data.max()\n", 248 | "xx = np.linspace(x_min, x_max, 100)\n", 249 | "ax.plot(xx, myECDF(xx), label='My empirical CDF')\n", 250 | "ax.plot(xx, hist_rv.cdf(xx), '--', label='Empirical CDF from scipy.stats')\n", 251 | "ax.set_xlabel('$x$')\n", 252 | "ax.set_ylabel('Empirical CDF')\n", 253 | "plt.legend(loc='best')" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": {}, 259 | "source": [ 260 | "### Part B\n", 261 | "\n", 262 | "Now complete the code that computes the inverse of the empirical CDF $\\hat{F}^{-1}$.\n", 263 | "There are may ways of doing this.\n", 264 | "Let's do it in a way that will teach us something about the root finding toolbox of numpy (see [this]()).\n", 265 | "Mathematically, we wish to find a function $F^{-1}$ such that\n", 266 | "$$\n", 267 | "F(F^{-1}(u))) = u,\n", 268 | "$$\n", 269 | "for any $u\\in[0,1]$ (the domain in which $F(x)$ takes values).\n", 270 | "It is obvious that $F^{-1}(u)$ is the solution to the *root finding* problem:\n", 271 | "$$\n", 272 | "F(x^*) = u.\n", 273 | "$$\n", 274 | "Since we know that $F$ is increasing, this problem must have a unique solution for any $u\\in[0,1]$.\n", 275 | "To find this solution, we can use [Brent's method](https://en.wikipedia.org/wiki/Brent%27s_method).\n", 276 | "Please note, that the problem that this code solves is of the form:\n", 277 | "$$\n", 278 | "g(x^*) = 0.\n", 279 | "$$\n", 280 | "So, you will have to reformulate the original problem as:\n", 281 | "$$\n", 282 | "F(x^*) - u = 0.\n", 283 | "$$\n", 284 | "Study the [numpy implementation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brentq.html#scipy.optimize.brentq) of Brent's method and complete the following code:" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": 2, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "from scipy import optimize # Gives you access to optimize.brentq\n", 294 | "\n", 295 | "def myiECDF_base(u):\n", 296 | " \"\"\"\n", 297 | " Evaluates the inverse of the empirical CDF.\n", 298 | " \n", 299 | " :param u: A scalar at which to evaluate the function.\n", 300 | " :returns: The value of the inverse of the empirical CDF at ``u``.\n", 301 | " \"\"\"\n", 302 | " # Write your code here\n", 303 | " # You will have to define a functioin to pass to optimize.brentq\n", 304 | " # You can define this function in here\n", 305 | " # (delete the next line and return the right value)\n", 306 | " raise NotImplementedError('Implement me!')\n", 307 | "\n", 308 | "# Vectorize your function (i.e., make it work with 1D numpy arrays).\n", 309 | "# See this: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.vectorize.html\n", 310 | "myiECDF = np.vectorize(myiECDF_base)" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": { 317 | "scrolled": false 318 | }, 319 | "outputs": [], 320 | "source": [ 321 | "# You can test your results by comparing the inverse of the empirical CDF you can get with scipy.stats\n", 322 | "# The two should match almost exactly\n", 323 | "hist_rv = st.rv_histogram(np.histogram(data, bins=1000))\n", 324 | "fig, ax = plt.subplots()\n", 325 | "uu = np.linspace(1e-3, 1, 100) # For convergence issues we cannot start at 0\n", 326 | "ax.plot(uu, myiECDF(uu), label='Inverse of my empirical CDF')\n", 327 | "ax.plot(uu, hist_rv.ppf(uu), '--', label='Inverse of empirical CDF from scipy.stats')\n", 328 | "ax.set_xlabel('$x$')\n", 329 | "ax.set_ylabel('Inverse of empirical CDF')\n", 330 | "plt.legend(loc='best')" 331 | ] 332 | }, 333 | { 334 | "cell_type": "markdown", 335 | "metadata": {}, 336 | "source": [ 337 | "### Part C\n", 338 | "\n", 339 | "Now use the *inverse transform sampling* method to generate samples from same distribution as the original data.\n", 340 | "That is, you can now generate uniform samples:\n", 341 | "$$\n", 342 | "u_i \\sim U([0,1]),\n", 343 | "$$\n", 344 | "and transform them as:\n", 345 | "$$\n", 346 | "\\hat{x}_i = {\\hat{F}}^{-1}(u_i).\n", 347 | "$$\n", 348 | "The $\\hat{x}_i$'s generated in this way should have the same distribution of the data you started with.\n", 349 | "Verify this by comparing the histrogram of $1,000$ $\\hat{x}_i$ samples with the original data of this problem." 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "# Write your code here.\n", 359 | "# Feel free to copy paste code from above." 360 | ] 361 | }, 362 | { 363 | "cell_type": "markdown", 364 | "metadata": {}, 365 | "source": [ 366 | "## Problem 3\n", 367 | "\n", 368 | "This is a classic uncertainty propagation problem that you will have to solve using Monte Carlo sampling.\n", 369 | "Consider the following stochastic harmonic oscillator:\n", 370 | "$$\n", 371 | "\\begin{array}{ccc}\n", 372 | "\\ddot{y} + 2 \\zeta \\omega(X) \\dot{y} + \\omega^2(X)y &=& 0,\\\\\n", 373 | "y(0) &=& y_0(X),\\\\\n", 374 | "\\dot{y}(0) &=& v_0(X),\n", 375 | "\\end{array}\n", 376 | "$$\n", 377 | "where:\n", 378 | "+ $X = (X_1, X_2, X_3)$,\n", 379 | "+ $X_i \\sim N(0, 1)$,\n", 380 | "+ $\\omega(X) = 2\\pi + X_1$, \n", 381 | "+ $\\zeta = 0.01$,\n", 382 | "+ $y_0(X) = 1+ 0.1 X_2$, and\n", 383 | "+ $v_0 = 0.1 X_3$.\n", 384 | "\n", 385 | "In words, this stochastic harmonic oscillator has an uncertain natural frequency and uncertain initial conditions.\n", 386 | "\n", 387 | "Our goal is to propagate uncertainty through this dynamical system, i.e., estimate the mean and variance of its solution.\n", 388 | "A solver for this dynamical system is given below:" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": null, 394 | "metadata": {}, 395 | "outputs": [], 396 | "source": [ 397 | "class Solver(object):\n", 398 | " def __init__(self, nt=100, T=5):\n", 399 | " \"\"\"\n", 400 | " This is the initializer of the class.\n", 401 | " \n", 402 | " Arguments:\n", 403 | " nt - The number of timesteps.\n", 404 | " T - The final time.\n", 405 | " \"\"\"\n", 406 | " self.nt = nt\n", 407 | " self.T = T\n", 408 | " self.t = np.linspace(0, T, nt) # The timesteps on which we will get the solution\n", 409 | " # The following are not essential, but they are convenient\n", 410 | " self.num_input = 3 # The number of inputs the class accepts\n", 411 | " self.num_output = nt # The number of outputs the class returns\n", 412 | " \n", 413 | " def __call__(self, x):\n", 414 | " \"\"\"\n", 415 | " This special class method emulates a function call.\n", 416 | " \n", 417 | " Arguments:\n", 418 | " x - A 1D numpy array with 3 elements. This represents the stochastic input x = (x1, x2, x3).\n", 419 | " \"\"\"\n", 420 | " ##uncertain quantities \n", 421 | " x1 = x[0]\n", 422 | " x2 = x[1]\n", 423 | " x3 = x[2]\n", 424 | " \n", 425 | " #ODE parameters \n", 426 | " omega = 2*np.pi + x1 \n", 427 | " y10 = 1 + 0.1*x2\n", 428 | " y20 = 0.1*x3\n", 429 | " y0 = np.array([y10, y20]) #initial conditions \n", 430 | " \n", 431 | " #coefficient matrix \n", 432 | " zeta = 0.01\n", 433 | " k = omega**2 ##spring constant\n", 434 | " c = 2*zeta*omega ##damping coeff. \n", 435 | " C = np.array([[0, 1],[-k, -c]])\n", 436 | " \n", 437 | " #RHS of the ODE system\n", 438 | " def rhs(y, t):\n", 439 | " return np.dot(C, y)\n", 440 | " \n", 441 | " y = scipy.integrate.odeint(rhs, y0, self.t)\n", 442 | " \n", 443 | " return y" 444 | ] 445 | }, 446 | { 447 | "cell_type": "markdown", 448 | "metadata": {}, 449 | "source": [ 450 | "Let's plot a few samples of the forward model to demonstrate how the solver works." 451 | ] 452 | }, 453 | { 454 | "cell_type": "code", 455 | "execution_count": null, 456 | "metadata": {}, 457 | "outputs": [], 458 | "source": [ 459 | "# 1. Create the solver object\n", 460 | "solver = Solver()\n", 461 | "fig1, ax1 = plt.subplots()\n", 462 | "ax1.set_xlabel('$t$ (Time)')\n", 463 | "ax1.set_ylabel('$y(t)$ (Position)')\n", 464 | "fig2, ax2 = plt.subplots()\n", 465 | "ax2.set_xlabel('$t$ (Time)')\n", 466 | "ax2.set_ylabel('$\\dot{y}(t)$ (Velocity)')\n", 467 | "for i in range(2):\n", 468 | " # Sample the random inputs (they are just standard normal)\n", 469 | " x = np.random.randn(solver.num_input) # solver.num_input is just 3\n", 470 | " # Evaluate the solver response\n", 471 | " y = solver(x) # This returns an (num timesteps) x (num states) array (100 x 2 here)\n", 472 | " # Plot the sample\n", 473 | " ax1.plot(solver.t, y[:, 0])\n", 474 | " ax2.plot(solver.t, y[:, 1], label='Sample {0:d}'.format(i+1))\n", 475 | "plt.legend(loc=True)" 476 | ] 477 | }, 478 | { 479 | "cell_type": "markdown", 480 | "metadata": {}, 481 | "source": [ 482 | "For your convenience, here is code that takes many samples of the solver at once:" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "def take_samples_from_solver(num_samples):\n", 492 | " \"\"\"\n", 493 | " Takes ``num_samples`` from the ODE solver.\n", 494 | " \n", 495 | " Returns them in an array of the form: ``num_samples x 100 x 2`` (100 timesteps, 2 states (position, velocity))\n", 496 | " \"\"\"\n", 497 | " samples = np.ndarray((num_samples, 100, 2))\n", 498 | " for i in range(num_samples):\n", 499 | " samples[i, :, :] = solver(np.random.randn(solver.num_input))\n", 500 | " return samples" 501 | ] 502 | }, 503 | { 504 | "cell_type": "markdown", 505 | "metadata": {}, 506 | "source": [ 507 | "It works like this:" 508 | ] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "execution_count": null, 513 | "metadata": {}, 514 | "outputs": [], 515 | "source": [ 516 | "samples = take_samples_from_solver(50)\n", 517 | "print(samples.shape)\n", 518 | "print(samples)" 519 | ] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": {}, 524 | "source": [ 525 | "### Part A\n", 526 | "Take 100 samples of the solver output and plot the estimated mean position and velocity as a function of time along with a 95\\% epistemic uncertainty interval around it. \n", 527 | "This interval captures how sure you are about the mean response when using only 100 Monte Carlo samples.\n", 528 | "You need to use the central limit theorem to find it (see the lecture notes)." 529 | ] 530 | }, 531 | { 532 | "cell_type": "code", 533 | "execution_count": null, 534 | "metadata": {}, 535 | "outputs": [], 536 | "source": [ 537 | "samples = take_samples_from_solver(100)\n", 538 | "# Sampled positions are: samples[:, :, 0]\n", 539 | "# Sampled velocities are: samples[:, :, 1]\n", 540 | "# Sampled position at the 10th timestep is: samples[:, 9, 0]\n", 541 | "# etc.\n", 542 | "\n", 543 | "# Your code here" 544 | ] 545 | }, 546 | { 547 | "cell_type": "markdown", 548 | "metadata": {}, 549 | "source": [ 550 | "### Part B\n", 551 | "\n", 552 | "Plot the epistemic uncertainty about the mean response at $t=5$s as a function of the number of samples. \n", 553 | "\n", 554 | "**Solution**:" 555 | ] 556 | }, 557 | { 558 | "cell_type": "code", 559 | "execution_count": null, 560 | "metadata": {}, 561 | "outputs": [], 562 | "source": [ 563 | "# Your code here" 564 | ] 565 | }, 566 | { 567 | "cell_type": "markdown", 568 | "metadata": {}, 569 | "source": [ 570 | "### Part C\n", 571 | "Repeat part A and B for the squared response.\n", 572 | "That is, do exactly the same thing as above, but consider $y^2(t)$ and $\\dot{y}^2(t)$ instead of $y(t)$ and $\\dot{y}(t)$.\n", 573 | "How many samples do you need to estimate the mean squared response at $t=5$s with negligible epistemic uncertainty?\n", 574 | "\n", 575 | "**Solution**:" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": null, 581 | "metadata": {}, 582 | "outputs": [], 583 | "source": [ 584 | "# Your code here" 585 | ] 586 | }, 587 | { 588 | "cell_type": "markdown", 589 | "metadata": {}, 590 | "source": [ 591 | "### Part D\n", 592 | "\n", 593 | "Now that you know how many samples you need to estimate the mean of the response and the square response, use the formula:\n", 594 | "$$\n", 595 | "\\mathbb{V}[y(t)] = \\mathbb{E}[y^2(t)] - \\left(\\mathbb{E}[y(t)]\\right)^2,\n", 596 | "$$\n", 597 | "and similarly for $\\dot{y}(t)$, to estimate the variance of the position and the velocity with negligible epistemic uncertainty.\n", 598 | "Plot both quantities as a function of time.\n", 599 | "\n", 600 | "**Solution**:" 601 | ] 602 | }, 603 | { 604 | "cell_type": "code", 605 | "execution_count": null, 606 | "metadata": {}, 607 | "outputs": [], 608 | "source": [ 609 | "# Your code here" 610 | ] 611 | }, 612 | { 613 | "cell_type": "markdown", 614 | "metadata": {}, 615 | "source": [ 616 | "### Part E\n", 617 | "\n", 618 | "Put together the estimated mean and variance to plot a 95\\% predictive interval for the position and the velocity as functions of time.\n", 619 | "\n", 620 | "**Solution**:" 621 | ] 622 | }, 623 | { 624 | "cell_type": "code", 625 | "execution_count": null, 626 | "metadata": {}, 627 | "outputs": [], 628 | "source": [ 629 | "# Your code here" 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "metadata": {}, 635 | "source": [ 636 | "-End-" 637 | ] 638 | } 639 | ], 640 | "metadata": { 641 | "anaconda-cloud": {}, 642 | "kernelspec": { 643 | "display_name": "Python 3", 644 | "language": "python", 645 | "name": "python3" 646 | }, 647 | "language_info": { 648 | "codemirror_mode": { 649 | "name": "ipython", 650 | "version": 3 651 | }, 652 | "file_extension": ".py", 653 | "mimetype": "text/x-python", 654 | "name": "python", 655 | "nbconvert_exporter": "python", 656 | "pygments_lexer": "ipython3", 657 | "version": "3.7.6" 658 | }, 659 | "latex_envs": { 660 | "bibliofile": "biblio.bib", 661 | "cite_by": "apalike", 662 | "current_citInitial": 1, 663 | "eqLabelWithNumbers": true, 664 | "eqNumInitial": 0 665 | } 666 | }, 667 | "nbformat": 4, 668 | "nbformat_minor": 1 669 | } 670 | -------------------------------------------------------------------------------- /homeworks/me597_project_final.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/homeworks/me597_project_final.pdf -------------------------------------------------------------------------------- /homeworks/motor.dat: -------------------------------------------------------------------------------- 1 | ### Motorcycle Data 2 | ### Data from a Simulated Motorcycle Accident 3 | ### 4 | ### SUMMARY: 5 | ### The motor data frame has 94 rows and 4 columns. The rows 6 | ### are obtained by removing replicate values of time from the 7 | ### dataset mcycle. Two extra columns are added to allow for 8 | ### strata with a different residual variance in each stratum. 9 | ### ~give succinct details here 10 | ### 11 | ### DATA DESCRIPTION: 12 | ### This data frame contains the following columns: 13 | ### times: The time in milliseconds since impact. 14 | ### accel: The recorded head acceleration (in g). 15 | ### strata: A numeric column indicating to which of the three 16 | ### strata (numbered 1, 2 and 3) the observations belong. 17 | ### v: An estimate of the residual variance for the observation. 18 | ### v is constant within the strata but a different estimate 19 | ### is used for each of the three strata. 20 | ### 21 | ### SOURCE: 22 | ### The data were obtained from 23 | ### 24 | ### Silverman, B.W. (1985) Some aspects of the spline 25 | ### smoothing approach to non-parametric curve fitting. 26 | ### Journal of the Royal Statistical Society, B, 47, 1-52. 27 | ### 28 | ### OTHER REFERENCES: 29 | ### Davison, A.C. and Hinkley, D.V. (1997) Bootstrap Methods 30 | ### and Their Application. Cambridge University Press. 31 | ### 32 | ### Venables, W.N. and Ripley, B.D. (1994) Modern Applied 33 | ### Statistics with S-Plus. Springer-Verlag. 34 | ### 35 | ### 36 | # times accel strata v 37 | 2.4 0 1 3.7 38 | 2.6 -1.3 1 3.7 39 | 3.2 -2.7 1 3.7 40 | 3.6 0 1 3.7 41 | 4 -2.7 1 3.7 42 | 6.2 -2.7 1 3.7 43 | 6.6 -2.7 1 3.7 44 | 6.8 -1.3 1 3.7 45 | 7.8 -2.7 1 3.7 46 | 8.2 -2.7 1 3.7 47 | 8.8 -1.3 1 3.7 48 | 9.6 -2.7 1 3.7 49 | 10 -2.7 1 3.7 50 | 10.2 -5.4 1 3.7 51 | 10.6 -2.7 1 3.7 52 | 11 -5.4 1 3.7 53 | 11.4 0 1 3.7 54 | 13.2 -2.7 2 607 55 | 13.6 -2.7 2 607 56 | 13.8 0 2 607 57 | 14.6 -13.3 2 607 58 | 14.8 -2.7 2 607 59 | 15.4 -22.8 2 607 60 | 15.6 -40.2 2 607 61 | 15.8 -21.5 2 607 62 | 16 -42.9 2 607 63 | 16.2 -21.5 2 607 64 | 16.4 -5.4 2 607 65 | 16.6 -59 2 607 66 | 16.8 -71 2 607 67 | 17.6 -37.5 2 607 68 | 17.8 -99.1 2 607 69 | 18.6 -112.5 2 607 70 | 19.2 -123.1 2 607 71 | 19.4 -85.6 2 607 72 | 19.6 -127.2 2 607 73 | 20.2 -123.1 2 607 74 | 20.4 -117.9 2 607 75 | 21.2 -134 2 607 76 | 21.4 -101.9 2 607 77 | 21.8 -108.4 2 607 78 | 22 -123.1 2 607 79 | 23.2 -123.1 2 607 80 | 23.4 -128.5 2 607 81 | 24 -112.5 2 607 82 | 24.2 -95.1 2 607 83 | 24.6 -53.5 2 607 84 | 25 -64.4 2 607 85 | 25.4 -72.3 2 607 86 | 25.6 -26.8 2 607 87 | 26 -5.4 2 607 88 | 26.2 -107.1 2 607 89 | 26.4 -65.6 2 607 90 | 27 -16 2 607 91 | 27.2 -45.6 2 607 92 | 27.6 4 2 607 93 | 28.2 12 2 607 94 | 28.4 -21.5 2 607 95 | 28.6 46.9 2 607 96 | 29.4 -17.4 2 607 97 | 30.2 36.2 2 607 98 | 31 75 2 607 99 | 31.2 8.1 2 607 100 | 32 54.9 2 607 101 | 32.8 46.9 2 607 102 | 33.4 16 2 607 103 | 33.8 45.6 2 607 104 | 34.4 1.3 2 607 105 | 34.8 75 2 607 106 | 35.2 -16 2 607 107 | 35.4 69.6 2 607 108 | 35.6 34.8 2 607 109 | 36.2 -37.5 2 607 110 | 38 46.9 2 607 111 | 39.2 5.4 2 607 112 | 39.4 -1.3 2 607 113 | 40 -21.5 2 607 114 | 40.4 -13.3 2 607 115 | 41.6 30.8 3 138 116 | 42.4 29.4 3 138 117 | 42.8 0 3 138 118 | 43 14.7 3 138 119 | 44 -1.3 3 138 120 | 44.4 0 3 138 121 | 45 10.7 3 138 122 | 46.6 10.7 3 138 123 | 47.8 -26.8 3 138 124 | 48.8 -13.3 3 138 125 | 50.6 0 3 138 126 | 52 10.7 3 138 127 | 53.2 -14.7 3 138 128 | 55 -2.7 3 138 129 | 55.4 -2.7 3 138 130 | 57.6 10.7 3 138 131 | -------------------------------------------------------------------------------- /homeworks/stress_strain.txt: -------------------------------------------------------------------------------- 1 | #ex Sx 2 | 0 -29.48976 3 | 2.5164936854652E-4 11.7796283840872 4 | 5.03298737093041E-4 45.1952976506456 5 | 7.54948105639372E-4 49.495215031653 6 | 0.00100659747418589 42.2830189989765 7 | 0.00125824684273241 8.39627801145883 8 | 0.00150989621127893 -14.9703958119621 9 | 0.00176021410168495 55.3105636596187 10 | 0.00201186347023147 50.113480322398 11 | 0.00226351283877799 68.4330850444656 12 | 0.00251516220732451 1.04052569276414 13 | 0.00276681157587084 -29.2324317858972 14 | 0.00301846094441736 39.2371370116006 15 | 0.00327011031296388 15.4971650774774 16 | 0.0035217596815104 41.0067497627649 17 | 0.00377340905005692 31.6183238730225 18 | 0.00402505841860345 64.4817377012023 19 | 0.00427670778714997 29.425150822175 20 | 0.0045283571556963 56.4349966953239 21 | 0.00478000652424282 66.1917054446446 22 | 0.00503165589278934 84.3842221292984 23 | 0.00528330526133586 64.1342970527042 24 | 0.00553495462988238 17.6867196662124 25 | 0.0057866039984289 8.13290149658074 26 | 0.00603825336697542 71.8241717207333 27 | 0.00628990273552175 54.1055783102847 28 | 0.00654155210406827 133.931108835378 29 | 0.00679320147261479 44.7249405405434 30 | 0.00704485084116131 125.128779002674 31 | 0.00729516873156733 83.6289890177075 32 | 0.00754681810011385 59.9909515875781 33 | 0.00779846746866037 63.1652438835513 34 | 0.00805011683720689 53.3057398753294 35 | 0.00830176620575322 106.065299771458 36 | 0.00855341557429974 44.917078689686 37 | 0.00880506494284626 29.3249550027426 38 | 0.00905671431139278 38.6533693793094 39 | 0.0093083636799393 79.2366880926789 40 | 0.00956001304848582 81.5432698622676 41 | 0.00981166241703215 49.5697232235053 42 | 0.0100633117855787 112.988818441033 43 | 0.0103149611541252 98.4813516631601 44 | 0.0105666105226717 131.802079883606 45 | 0.0108182598912182 58.7807827592345 46 | 0.0110699092597648 80.8508618648309 47 | 0.0113215586283113 54.0698122307717 48 | 0.0115732079968576 107.105407949948 49 | 0.0118248573654041 122.227116109879 50 | 0.0120765067339506 76.0335649161192 51 | 0.0123281561024972 57.1990216982966 52 | 0.0125798054710437 117.085160625522 53 | 0.0128301233614497 121.911929686706 54 | 0.0130817727299962 90.23021140325 55 | 0.0133334220985427 103.016180177177 56 | 0.0135850714670891 75.3143938305953 57 | 0.0138367208356356 111.568127085613 58 | 0.0140883702041821 110.347003373033 59 | 0.0143400195727286 110.81023259711 60 | 0.0145916689412752 122.961535258353 61 | 0.0148433183098217 121.973229742014 62 | 0.0150949676783682 134.242950226038 63 | 0.0153466170469145 126.600107385021 64 | 0.0155982664154611 120.670177255677 65 | 0.0158499157840076 96.0067519096865 66 | 0.0161015651525541 70.5697015147201 67 | 0.0163532145211006 117.093177748809 68 | 0.0166048638896471 133.495843146748 69 | 0.0168565132581937 161.221141802961 70 | 0.01710816262674 131.783605030189 71 | 0.0173598119952865 150.450575954101 72 | 0.017611461363833 109.865424348669 73 | 0.0178631107323795 171.200107708156 74 | 0.0181147601009261 128.126906279963 75 | 0.0183664094694726 122.188080151985 76 | 0.0186167273598786 84.2766301429149 77 | 0.0188683767284251 139.851697810697 78 | 0.0191200260969715 90.8836470928417 79 | 0.019371675465518 153.445701744208 80 | 0.0196233248340645 167.503086569715 81 | 0.019874974202611 163.554996183722 82 | 0.0201266235711575 104.035052070505 83 | 0.0203782729397041 147.658806783849 84 | 0.0206299223082504 181.461734939786 85 | 0.0208815716767969 116.44747449374 86 | 0.0211332210453434 123.577866839318 87 | 0.02138487041389 135.954284550276 88 | 0.0216365197824365 141.888679057609 89 | 0.021888169150983 148.134100417192 90 | 0.0221398185195295 147.513401996935 91 | 0.0223914678880758 142.374027977623 92 | 0.0226431172566224 109.572491893775 93 | 0.0228947666251689 117.770137642804 94 | 0.0231464159937154 165.917813256934 95 | 0.0233980653622619 137.274739211222 96 | 0.0236497147308084 162.401800833733 97 | 0.023901364099355 165.014800191218 98 | 0.024151681989761 164.255033377749 99 | 0.0244033313583073 137.727658821774 100 | 0.0246549807268538 190.05311948471 101 | 0.0249066300954004 151.527356398606 102 | 0.0251582794639469 190.569557456292 103 | 0.0254099288324934 198.775855793889 104 | 0.0256615782010399 170.898850558517 105 | 0.0259132275695864 157.547044750416 106 | 0.0261648769381328 181.392302483799 107 | 0.0264165263066793 187.018727575624 108 | 0.0266681756752258 163.380605482812 109 | 0.0269198250437723 193.306157928012 110 | 0.0271714744123189 156.27548711341 111 | 0.0274231237808654 183.597261766786 112 | 0.0276747731494119 159.232871275662 113 | 0.0279264225179582 197.358237069794 114 | 0.0281780718865047 155.888251139298 115 | 0.0284297212550513 188.588912682128 116 | 0.0286813706235978 143.341865507948 117 | 0.0289330199921443 242.555034782026 118 | 0.0291846693606908 225.233307545857 119 | 0.0294363187292372 224.230023875977 120 | 0.0296866366196434 199.478272673254 121 | 0.0299382859881897 182.011989616633 122 | 0.0301899353567362 203.37494958373 123 | 0.0304415847252827 210.40730030333 124 | 0.0306932340938293 202.428666823603 125 | 0.0309448834623758 238.728445356684 126 | 0.0311965328309223 217.441466333187 127 | 0.0314481821994686 187.76453624755 128 | 0.0316998315680151 253.831648974793 129 | 0.0319514809365617 209.202413415543 130 | 0.0322031303051082 234.529399311493 131 | 0.0324547796736547 213.431535925771 132 | 0.0327064290422012 207.90654708578 133 | 0.0329580784107478 193.909503278717 134 | 0.0332097277792941 196.2293019944 135 | 0.0334613771478406 216.939977411526 136 | 0.0337130265163871 230.194136621798 137 | 0.0339646758849336 250.769189784166 138 | 0.0342163252534802 220.784385159826 139 | 0.0344679746220267 198.542547202945 140 | 0.0347196239905732 204.304455273311 141 | 0.0349712733591195 186.10854856223 142 | 0.035222922727666 157.985499517954 143 | 0.0354732406180721 233.606004480019 144 | 0.0357248899866186 220.32339709933 145 | 0.0359765393551651 247.055359143149 146 | 0.0362281887237116 226.318597596391 147 | 0.0364798380922582 204.647180558689 148 | 0.0367314874608047 191.676291250526 149 | 0.036983136829351 191.721935261294 150 | 0.0372347861978975 232.331999093137 151 | 0.037486435566444 200.933226165006 152 | 0.0377380849349906 224.189251032801 153 | 0.0379897343035371 209.857364129433 154 | 0.0382413836720836 200.832410267825 155 | 0.0384930330406301 229.5001461043 156 | 0.0387446824091765 231.461448213832 157 | 0.038996331777723 201.606562352178 158 | 0.0392479811462695 198.924550808659 159 | 0.039499630514816 211.276218382227 160 | 0.0397512798833625 271.955143437789 161 | 0.0400029292519091 205.267558434288 162 | 0.0402545786204554 223.17631451958 163 | 0.0405062279890019 207.388535135775 164 | 0.0407578773575484 196.34474650703 165 | 0.0410081952479544 242.120554011121 166 | 0.041259844616501 204.826947620643 167 | 0.0415114939850475 252.774711780837 168 | 0.041763143353594 213.726486945956 169 | 0.0420147927221405 218.08337950676 170 | 0.0422664420906869 224.55492917198 171 | 0.0425180914592334 261.533922919983 172 | 0.0427697408277799 266.537582195593 173 | 0.0430213901963264 247.088936333346 174 | 0.0432730395648729 249.636775976632 175 | 0.0435246889334195 243.421675375481 176 | 0.043776338301966 206.476053971579 177 | 0.0440279876705123 241.317787123938 178 | 0.0442796370390588 252.080562845992 179 | 0.0445312864076054 285.408292627402 180 | 0.0447829357761519 230.86785710249 181 | 0.0450345851446984 246.245417847387 182 | 0.0452862345132449 211.756574900815 183 | 0.0455378838817914 257.893757882291 184 | 0.0457895332503378 236.443081781297 185 | 0.0460411826188843 230.078803983025 186 | 0.0462928319874308 273.239053484351 187 | 0.0465431498778368 248.365677507365 188 | 0.0467947992463833 254.298954346642 189 | 0.0470464486149299 256.970290272147 190 | 0.0472980979834764 278.577513589531 191 | 0.0475497473520229 242.684649784984 192 | 0.0478013967205692 267.902891478262 193 | 0.0480530460891158 279.415628008663 194 | 0.0483046954576623 263.509377610682 195 | 0.0485563448262088 264.834219565341 196 | 0.0488079941947553 279.291581234821 197 | 0.0490596435633018 313.516244830767 198 | 0.0493112929318484 275.800598775752 199 | 0.0495629423003947 292.718976715863 200 | 0.0498145916689412 257.550371173814 201 | 0.0500662410374877 266.203813951018 202 | 0.0503178904060343 212.694076306157 203 | 0.0505695397745808 206.432679859889 204 | 0.0508211891431273 236.794508750759 205 | 0.0510728385116736 249.516508501791 206 | 0.0513244878802201 292.59946225673 207 | 0.0515761372487667 313.863834227224 208 | 0.0518277866173132 290.173808904954 209 | 0.0520794359858597 259.05001113741 210 | 0.0523297538762657 274.664058136546 211 | 0.0525814032448122 283.079422687481 212 | 0.0528330526133588 263.175333304692 213 | 0.0530847019819051 261.665735967632 214 | 0.0533363513504516 237.695429454506 215 | 0.0535880007189981 263.371370703834 216 | 0.0538396500875447 281.823801002574 217 | 0.0540912994560912 252.238859765118 218 | 0.0543429488246377 251.1460456284 219 | 0.0545945981931842 216.2096274208 220 | 0.0548462475617306 241.725596427132 221 | 0.0550978969302771 276.816797434355 222 | 0.0553495462988236 263.996711296391 223 | 0.0556011956673701 279.043827361075 224 | 0.0558528450359166 295.173125385371 225 | 0.0561044944044632 262.092984904694 226 | 0.0563561437730097 284.467497220228 227 | 0.056607793141556 250.950516697039 228 | 0.0568594425101025 294.712647002161 229 | 0.057111091878649 258.847151300004 230 | 0.0573627412471956 260.976219203916 231 | 0.0576143906157421 237.40901708157 232 | 0.0578647085061481 215.745580809944 233 | 0.0581163578746946 260.148369156249 234 | 0.0583680072432411 318.113628408578 235 | 0.0586196566117875 275.283222919222 236 | 0.058871305980334 286.216649583516 237 | 0.0591229553488805 286.255228467127 238 | 0.059374604717427 256.947402323402 239 | 0.0596262540859736 230.026253252434 240 | 0.0598779034545201 269.356004333502 241 | 0.0601295528230666 279.06906884118 242 | 0.0603812021916129 302.663619191635 243 | 0.0606328515601594 312.482515753181 244 | 0.060884500928706 270.77732205039 245 | 0.0611361502972525 301.318408456928 246 | 0.061387799665799 286.836089740658 247 | 0.0616394490343455 241.243432366081 248 | 0.0618910984028919 265.496890053498 249 | 0.0621427477714384 244.489175047344 250 | 0.0623943971399849 273.674168368339 251 | 0.0626460465085314 252.171542019933 252 | 0.0628976958770779 250.106327944403 253 | 0.0631493452456245 294.787844214194 254 | 0.0633996631360305 268.442981691731 255 | 0.063651312504577 302.778017280359 256 | 0.0639029618731233 283.13075328781 257 | 0.0641546112416699 220.938118042856 258 | 0.0644062606102164 311.76690199447 259 | 0.0646579099787629 254.720757656191 260 | 0.0649095593473094 242.2503210217 261 | 0.0651612087158559 281.095075907992 262 | 0.0654128580844025 284.875286230444 263 | 0.0656645074529488 259.369008430108 264 | 0.0659161568214953 280.255900024756 265 | 0.0661678061900418 253.104485891406 266 | 0.0664194555585883 247.096930586916 267 | 0.0666711049271349 288.945957105561 268 | 0.0669227542956814 271.109421079966 269 | 0.0671744036642279 312.417997224108 270 | 0.0674260530327742 311.35265358235 271 | 0.0676777024013208 280.945049567926 272 | 0.0679293517698673 266.344813872979 273 | 0.0681810011384138 304.614661051119 274 | 0.0684326505069603 303.988823212728 275 | 0.0686842998755068 250.682328344367 276 | 0.0689359492440532 281.903639468598 277 | 0.0691862671344594 326.263535940052 278 | 0.0694379165030057 258.720273512012 279 | 0.0696895658715522 280.558738426518 280 | 0.0699412152400987 267.024884937103 281 | 0.0701928646086453 247.516192259017 282 | 0.0704445139771918 245.742675984797 283 | 0.0706961633457383 274.838908270152 284 | 0.0709478127142848 301.565511887431 285 | 0.0711994620828312 306.579816677262 286 | 0.0714511114513777 319.556986674595 287 | 0.0717027608199242 296.883164847322 288 | 0.0719544101884707 258.428021140238 289 | 0.0722060595570172 280.052067474106 290 | 0.0724577089255638 210.04190235145 291 | 0.0727093582941101 279.120255050275 292 | 0.0729610076626566 261.011371947484 293 | 0.0732126570312031 293.252440693805 294 | 0.0734643063997497 295.157219835352 295 | 0.0737159557682962 271.394362531283 296 | 0.0739676051368427 281.217778660472 297 | 0.0742192545053892 237.271216917017 298 | 0.0744709038739356 245.932714222687 299 | 0.0747212217643416 299.381686706303 300 | 0.0749728711328881 257.709205584339 301 | 0.0752245205014346 291.077653130555 302 | 0.0754761698699811 286.335388147009 303 | 0.0757278192385276 252.04676690464 304 | 0.0759794686070742 266.822785146391 305 | 0.0762311179756207 303.474564517298 306 | 0.076482767344167 287.352187603582 307 | 0.0767344167127135 282.234891148172 308 | 0.0769860660812601 279.724467492041 309 | 0.0772377154498066 282.76671355452 310 | 0.0774893648183531 245.187816830951 311 | 0.0777410141868996 252.302582034258 312 | 0.0779926635554461 265.888927513376 313 | 0.0782443129239925 270.871877384302 314 | 0.078495962292539 283.12242450133 315 | 0.0787476116610855 269.551311454415 316 | 0.078999261029632 274.055525850527 317 | 0.0792509103981786 254.744890321903 318 | 0.0795025597667251 277.864487229257 319 | 0.0797542091352714 263.441547311327 320 | 0.0800058585038179 250.711376270934 321 | 0.0802561763942239 277.800902903122 322 | 0.0805078257627705 273.910841031534 323 | 0.080759475131317 283.088231320681 324 | 0.0810111244998635 272.36661023714 325 | 0.08126277386841 270.076081000019 326 | 0.0815144232369566 235.301399427135 327 | 0.0817660726055031 308.629918206277 328 | 0.0820177219740494 265.921057455076 329 | 0.0822693713425959 288.21434094391 330 | 0.0825210207111424 243.417516364015 331 | 0.082772670079689 220.308800613944 332 | 0.0830243194482355 234.494002004804 333 | 0.083275968816782 249.51679297032 334 | 0.0835276181853283 270.624771114895 335 | 0.0837792675538749 276.220082240363 336 | 0.0840309169224214 276.678660632982 337 | 0.0842825662909679 279.991979703534 338 | 0.0845342156595144 250.142680873841 339 | 0.0847858650280609 250.508739607534 340 | 0.0850375143966075 269.150880423785 341 | 0.0852891637651538 290.87924221664 342 | 0.0855408131337003 251.179535439975 343 | 0.0857924625022468 264.999176495777 344 | 0.0860427803926528 292.721055735911 345 | 0.0862944297611994 251.05898410334 346 | 0.0865460791297459 228.907822703008 347 | 0.0867977284982924 267.572669137336 348 | 0.0870493778668389 265.651304037697 349 | 0.0873010272353853 295.404774847519 350 | 0.0875526766039318 281.979491624756 351 | 0.0878043259724783 265.2510739584 352 | 0.0880559753410248 250.554252197496 353 | 0.0883076247095713 262.244785098389 354 | 0.0885592740781179 301.344881228399 355 | 0.0888109234466644 237.536593187352 356 | 0.0890625728152107 278.917320574541 357 | 0.0893142221837572 292.651399227331 358 | 0.0895658715523038 267.743120209063 359 | 0.0898175209208503 227.301517228179 360 | 0.0900691702893968 251.355682751366 361 | 0.0903208196579433 272.67322339122 362 | 0.0905724690264896 254.069064149995 363 | 0.0908241183950362 266.459805061964 364 | 0.0910757677635827 263.543297618597 365 | 0.0913274171321292 325.867813499512 366 | 0.0915777350225352 207.782362108989 367 | 0.0918293843910817 221.533273995805 368 | 0.0920810337596283 282.979298582048 369 | 0.0923326831281748 224.436813002912 370 | 0.0925843324967213 266.362562370092 371 | 0.0928359818652676 243.190508077268 372 | 0.0930876312338142 248.766264720701 373 | 0.0933392806023607 255.916355340324 374 | 0.0935909299709072 256.27622882304 375 | 0.0938425793394537 289.351687989871 376 | 0.0940942287080002 267.782260296854 377 | 0.0943458780765466 231.766850625374 378 | 0.0945975274450931 268.361034896507 379 | 0.0948491768136396 262.83431246759 380 | 0.0951008261821861 285.020901340239 381 | 0.0953524755507326 297.547639390522 382 | 0.0956041249192792 208.208453830764 383 | 0.0958557742878257 216.234669738415 384 | 0.096107423656372 217.829139501079 385 | 0.0963590730249185 231.992071201663 386 | 0.0966107223934651 245.725770446078 387 | 0.0968623717620116 288.964264621398 388 | 0.0971126896524176 273.190171966975 389 | 0.0973643390209641 232.021925971175 390 | 0.0976159883895106 247.413223955689 391 | 0.0978676377580572 240.140005846838 392 | 0.0981192871266035 303.864356410885 393 | 0.09837093649515 279.502246397883 394 | 0.0986225858636965 247.7241807668 395 | 0.098874235232243 247.187342842468 396 | 0.0991258846007896 241.410091589957 397 | 0.0993775339693361 226.463453086054 398 | 0.0996291833378826 232.49381569701 399 | 0.0998808327064289 268.176380703698 400 | 0.100132482074975 266.066973806138 401 | 0.100384131443522 275.954851248358 402 | 0.100635780812069 281.559296317107 403 | 0.100887430180615 297.442420463113 404 | 0.101139079549162 281.457463089248 405 | 0.101390728917708 231.157775629225 406 | 0.101642378286254 276.859937771317 407 | 0.101894027654801 209.355936067354 408 | 0.102145677023347 250.03292932705 409 | 0.102397326391894 236.736872237036 410 | 0.10264897576044 227.691479271602 411 | 0.102899293650847 166.24881218008 412 | 0.103150943019393 216.432961308815 413 | 0.10340259238794 218.22045909269 414 | 0.103654241756486 258.222621446435 415 | 0.103905891125032 276.87692713962 416 | 0.104157540493579 227.384060351693 417 | 0.104409189862125 248.629889656513 418 | 0.104660839230672 199.668895314073 419 | 0.104912488599218 245.101883680289 420 | 0.105164137967765 241.697608940606 421 | 0.105415787336311 272.728227434566 422 | 0.105667436704858 228.161173596249 423 | 0.105919086073404 245.67481556854 424 | 0.106170735441951 252.118031287177 425 | 0.106422384810497 237.599618486472 426 | 0.106674034179044 233.072592119171 427 | 0.10692568354759 236.935810508475 428 | 0.107177332916137 279.532981616899 429 | 0.107428982284683 268.693197336643 430 | 0.10768063165323 252.965376616682 431 | 0.107932281021776 216.035926917381 432 | 0.108183930390323 216.48895377036 433 | 0.108434248280729 196.833335242695 434 | 0.108685897649275 224.094732720883 435 | 0.108937547017822 252.57087362251 436 | 0.109189196386368 243.994429845055 437 | 0.109440845754915 240.064352736574 438 | 0.109692495123461 233.264477976744 439 | 0.109944144492008 202.381378886184 440 | 0.110195793860554 232.562371876758 441 | 0.110447443229101 211.923525633451 442 | 0.110699092597647 271.326425265842 443 | 0.110950741966194 222.936912269945 444 | 0.11120239133474 196.532643382867 445 | 0.111454040703287 156.880644527175 446 | 0.111705690071833 178.147070280634 447 | 0.11195733944038 225.349735809736 448 | 0.112208988808926 233.601437503268 449 | 0.112460638177473 245.483126168608 450 | 0.112712287546019 245.380042722719 451 | 0.112963936914566 201.294435379894 452 | 0.113215586283112 207.268997446378 453 | 0.113467235651659 193.679854796096 454 | 0.113718885020205 210.560150069716 455 | 0.113969202910611 186.070836046082 456 | 0.114220852279158 230.795122350581 457 | 0.114472501647704 210.453134519394 458 | 0.114724151016251 166.363951542164 459 | 0.114975800384797 212.485766573376 460 | 0.115227449753344 232.834714725679 461 | 0.11547909912189 221.929693734658 462 | 0.115730748490437 217.089552951926 463 | 0.115982397858983 220.698554318103 464 | 0.11623404722753 191.779214670059 465 | 0.116485696596076 197.704646377988 466 | 0.116737345964623 188.472216713478 467 | 0.116988995333169 198.817865588011 468 | 0.117240644701716 219.332599546006 469 | 0.117492294070262 196.425631494958 470 | 0.117743943438808 219.005549124672 471 | 0.117995592807355 282.505796141544 472 | 0.118247242175902 210.17391008912 473 | 0.118498891544448 226.210150401141 474 | 0.118750540912995 191.203615708136 475 | 0.119002190281541 216.96890763341 476 | 0.119253839650088 215.769184707086 477 | 0.119505489018634 230.927341785225 478 | 0.11975580690904 231.74657736456 479 | 0.120007456277586 231.550650577518 480 | 0.120259105646133 221.150760659386 481 | 0.12051075501468 241.750999879159 482 | 0.120762404383226 189.011606274537 483 | 0.121014053751773 195.651046209999 484 | 0.121265703120319 229.566000350407 485 | 0.121517352488865 178.96002012063 486 | 0.121769001857412 191.255722812567 487 | 0.122020651225958 208.939744278024 488 | 0.122272300594505 184.237942875002 489 | 0.122523949963051 200.371379520141 490 | 0.122775599331598 165.049328206967 491 | 0.123027248700144 179.621210785132 492 | 0.123278898068691 216.915794744107 493 | 0.123530547437237 196.727272450805 494 | 0.123782196805784 221.748390092569 495 | 0.12403384617433 208.959462745261 496 | 0.124285495542877 200.514072630582 497 | 0.124537144911423 167.35234314074 498 | 0.12478879427997 189.81911736022 499 | 0.125040443648516 163.165379753724 500 | 0.125290761538922 195.238036011774 501 | 0.125542410907469 217.208587889498 502 | 0.125794060276015 177.620234380932 503 | 0.126045709644562 241.547268340828 504 | 0.126297359013108 234.69071716217 505 | 0.126549008381655 216.659767949552 506 | 0.126800657750201 184.692754401694 507 | 0.127052307118748 208.856964806508 508 | 0.127303956487294 206.247925271608 509 | 0.127555605855841 155.585225936638 510 | 0.127807255224387 198.05917643242 511 | 0.128058904592934 203.643314400531 512 | 0.12831055396148 159.925475844524 513 | 0.128562203330027 206.59887209272 514 | 0.128813852698573 163.788317318255 515 | 0.12906550206712 177.946074334182 516 | 0.129317151435666 183.208172976767 517 | 0.129568800804213 211.013790770041 518 | 0.129820450172759 208.623701657443 519 | 0.130072099541306 205.731889519933 520 | 0.130323748909852 189.723229652946 521 | 0.130575398278399 204.90029142025 522 | 0.130825716168805 238.582962509207 523 | 0.131077365537351 199.828864274499 524 | 0.131329014905898 205.286539976374 525 | 0.131580664274444 202.396517923569 526 | 0.131832313642991 162.880298552903 527 | 0.132083963011537 151.089681516379 528 | 0.132335612380084 187.597861226416 529 | 0.13258726174863 219.610460622072 530 | 0.132838911117177 207.493967658377 531 | 0.133090560485723 182.165270358198 532 | 0.13334220985427 162.920954750365 533 | 0.133593859222816 166.381529398611 534 | 0.133845508591363 194.159645048771 535 | 0.134097157959909 182.141002187132 536 | 0.134348807328456 152.13449618845 537 | 0.134600456697002 199.202986668604 538 | 0.134852106065549 183.80632838553 539 | 0.135103755434095 167.48213713094 540 | 0.135355404802642 174.819190496423 541 | 0.135607054171188 159.490268601276 542 | 0.135858703539735 172.441573723084 543 | 0.136110352908281 173.862093013022 544 | 0.136362002276828 188.629869826674 545 | 0.136612320167234 183.710458588516 546 | 0.13686396953578 201.716033327606 547 | 0.137115618904327 137.448237555481 548 | 0.137367268272873 151.76506091083 549 | 0.13761891764142 155.611105671701 550 | 0.137870567009966 193.660639965188 551 | 0.138122216378513 154.478559467952 552 | 0.138373865747059 203.769078857945 553 | 0.138625515115606 240.762127401727 554 | 0.138877164484152 196.285273689914 555 | 0.139128813852699 192.804998903687 556 | 0.139380463221245 207.280832146244 557 | 0.139632112589792 176.483636302617 558 | 0.139883761958338 193.783551722079 559 | 0.140135411326885 202.103966850557 560 | 0.140387060695431 152.429626725969 561 | 0.140638710063978 194.331589239181 562 | 0.140890359432524 156.683432041361 563 | 0.14114200880107 157.773493152426 564 | 0.141393658169617 111.409492993573 565 | 0.141645307538163 173.184281410189 566 | 0.14189695690671 148.874724935843 567 | 0.142147274797116 237.599778760273 568 | 0.142398924165663 164.906290160434 569 | 0.142650573534209 161.679259955842 570 | 0.142902222902756 204.038926634644 571 | 0.143153872271302 165.902779175958 572 | 0.143405521639848 175.24132475698 573 | 0.143657171008395 224.324887321552 574 | 0.143908820376941 151.558089383519 575 | 0.144160469745488 150.654217025261 576 | 0.144412119114034 176.160707946693 577 | 0.144663768482581 183.256924651734 578 | 0.144915417851127 175.534401445478 579 | 0.145167067219674 133.768402158907 580 | 0.14541871658822 123.151079967569 581 | 0.145670365956767 166.798233103048 582 | 0.145922015325313 129.83742171704 583 | 0.14617366469386 206.459728878803 584 | 0.146425314062406 172.309473065709 585 | 0.146676963430953 168.094747923939 586 | 0.146928612799499 158.615013945774 587 | 0.147180262168046 186.754415930227 588 | 0.147431911536592 163.845783037795 589 | 0.147682229426998 185.072121862129 590 | 0.147933878795545 184.940623317082 591 | 0.148185528164091 159.362260607175 592 | 0.148437177532638 198.761503479912 593 | 0.148688826901184 190.19775806721 594 | 0.148940476269731 158.367400794153 595 | 0.149192125638277 150.141552752526 596 | 0.149443775006824 165.586904045333 597 | 0.14969542437537 145.550025416523 598 | 0.149947073743917 157.824124774705 599 | 0.150198723112463 198.682021377733 600 | 0.15045037248101 169.394572192729 601 | 0.150702021849556 186.347172351707 602 | 0.150953671218103 136.846880543109 603 | 0.151205320586649 125.360124815494 604 | 0.151456969955196 150.922179910145 605 | 0.151708619323742 170.329410261283 606 | 0.151960268692289 157.952940074991 607 | 0.152211918060835 156.940233974564 608 | 0.152463567429382 188.609210739644 609 | 0.152715216797928 193.277647449335 610 | 0.152966866166475 191.746748975746 611 | 0.153218515535021 170.084171404829 612 | 0.153468833425427 137.759166374029 613 | 0.153720482793974 179.641367870288 614 | 0.15397213216252 120.994084278784 615 | 0.154223781531067 164.225362941434 616 | 0.154475430899613 132.063220026382 617 | 0.15472708026816 162.339765438582 618 | 0.154978729636706 144.521238217267 619 | 0.155230379005253 136.062823887134 620 | 0.155482028373799 173.442477740655 621 | 0.155733677742346 122.732715390834 622 | 0.155985327110892 170.230621381514 623 | 0.156236976479439 150.421093318526 624 | 0.156488625847985 122.450054748382 625 | 0.156740275216532 194.346816774297 626 | 0.156991924585078 170.558675312672 627 | 0.157243573953625 131.920076397436 628 | 0.157495223322171 165.160222513362 629 | 0.157746872690718 139.705015118136 630 | 0.157998522059264 179.956665195104 631 | 0.158250171427811 161.172853210491 632 | 0.158501820796357 133.894158821194 633 | 0.158753470164904 148.781131821498 634 | 0.15900378805531 155.972327263846 635 | 0.159255437423856 183.224815787956 636 | 0.159507086792403 177.153655139315 637 | 0.159758736160949 115.758162465593 638 | 0.160010385529496 158.039996687797 639 | 0.160262034898042 169.390309724352 640 | 0.160513684266589 146.212230369209 641 | 0.160765333635135 177.229903533132 642 | 0.161016983003681 181.498180567943 643 | 0.161268632372228 161.813397153037 644 | 0.161520281740774 149.157529896732 645 | 0.161771931109321 153.534270084542 646 | 0.162023580477868 177.012563399906 647 | 0.162275229846414 147.187676839206 648 | 0.162526879214961 163.462314399158 649 | 0.162778528583507 124.263117092069 650 | 0.163030177952053 127.357846146401 651 | 0.1632818273206 124.75102803803 652 | 0.163533476689146 172.202535946416 653 | 0.163785126057693 122.203187865428 654 | 0.164036775426239 154.256250808622 655 | 0.164288424794786 168.13164116632 656 | 0.164538742685192 150.853959405716 657 | 0.164790392053738 178.755642212531 658 | 0.165042041422285 166.370873250875 659 | 0.165293690790831 121.305326056861 660 | 0.165545340159378 125.696879717222 661 | 0.165796989527924 138.680413744436 662 | 0.166048638896471 155.115995297784 663 | 0.166300288265017 160.363522117832 664 | 0.166551937633564 196.535117587755 665 | 0.16680358700211 186.336963978601 666 | 0.167055236370657 175.032523617669 667 | 0.167306885739203 136.79281291167 668 | 0.16755853510775 141.398939486879 669 | 0.167810184476296 118.355733867568 670 | 0.168061833844843 133.860207324447 671 | 0.168313483213389 127.003636894996 672 | 0.168565132581936 127.066787378555 673 | 0.168816781950482 164.854019194013 674 | 0.169068431319029 162.399375977491 675 | 0.169320080687575 152.821263525553 676 | 0.169571730056122 147.432303533307 677 | 0.169823379424668 160.211347211292 678 | 0.170075028793215 79.7468646397455 679 | 0.170325346683621 179.884869701094 680 | 0.170576996052167 78.2330800476954 681 | 0.170828645420714 126.967236295764 682 | 0.17108029478926 105.025997187011 683 | 0.171331944157807 151.940875812592 684 | 0.171583593526353 165.735671473388 685 | 0.1718352428949 186.241716816385 686 | 0.172086892263446 152.660282630843 687 | 0.172338541631993 167.422434370901 688 | 0.172590191000539 148.325043459927 689 | 0.172841840369086 168.926854514029 690 | 0.173093489737632 142.555726167344 691 | 0.173345139106179 126.04022952595 692 | 0.173596788474725 121.568241667951 693 | 0.173848437843272 165.067668995714 694 | 0.174100087211818 108.422432894691 695 | 0.174351736580365 176.184573519306 696 | 0.174603385948911 185.043910684583 697 | 0.174855035317458 159.74559848082 698 | 0.175106684686004 153.73029160031 699 | 0.175358334054551 91.6222973524578 700 | 0.175609983423097 151.608989652698 701 | 0.175860301313503 117.763813563557 702 | 0.17611195068205 121.167259025481 703 | 0.176363600050596 158.658437624515 704 | 0.176615249419143 143.601445944052 705 | 0.176866898787689 147.694577235701 706 | 0.177118548156236 113.544677254735 707 | 0.177370197524782 157.930778513214 708 | 0.177621846893329 134.8654798997 709 | 0.177873496261875 193.253624313752 710 | 0.178125145630422 105.605393413476 711 | 0.178376794998968 143.960526754919 712 | 0.178628444367515 141.931641681654 713 | 0.178880093736061 157.348191245807 714 | 0.179131743104608 130.285415117058 715 | 0.179383392473154 150.447604238486 716 | 0.179635041841701 179.106875077794 717 | 0.179886691210247 172.054132660184 718 | 0.180138340578794 132.840022105875 719 | 0.18038998994734 149.262373856245 720 | 0.180641639315886 114.017048795125 721 | 0.180893288684433 166.522134331963 722 | 0.181144938052979 127.128970463722 723 | 0.181395255943385 161.10113635222 724 | 0.181646905311932 112.288824013659 725 | 0.181898554680479 127.053693696054 726 | 0.182150204049025 119.536345021529 727 | 0.182401853417572 160.494086013247 728 | 0.182653502786118 122.398574484593 729 | 0.182905152154664 142.056509822728 730 | 0.183156801523211 175.373270184619 731 | 0.183408450891757 150.741760667182 732 | 0.183660100260304 135.201151740703 733 | 0.183911749628851 177.887740913591 734 | 0.184163398997397 123.136418899844 735 | 0.184415048365943 131.425468674656 736 | 0.18466669773449 113.17679940678 737 | 0.184918347103036 111.135246402342 738 | 0.185169996471583 107.891440608455 739 | 0.185421645840129 158.309098597741 740 | 0.185673295208676 167.634819699767 741 | 0.185924944577222 157.002048120549 742 | 0.186176593945769 165.123056274799 743 | 0.186428243314315 193.16749376217 744 | 0.186679892682862 139.561666735 745 | 0.186931542051408 140.75631220663 746 | 0.187181859941814 155.129712252925 747 | 0.187433509310361 123.726405299968 748 | 0.187685158678907 130.49057622102 749 | 0.187936808047454 140.613735434404 750 | 0.188188457416 167.485413172824 751 | 0.188440106784547 119.241778555462 752 | 0.188691756153093 163.641675445163 753 | 0.18894340552164 146.713278808555 754 | 0.189195054890186 146.722012313548 755 | 0.189446704258733 149.86241042364 756 | 0.189698353627279 178.863392874013 757 | 0.189950002995826 150.323681868008 758 | 0.190201652364372 150.390508734205 759 | 0.190453301732919 113.996865133457 760 | 0.190704951101465 123.333009242786 761 | 0.190956600470012 115.246338216101 762 | 0.191208249838558 150.548291817711 763 | 0.191459899207105 149.667383419511 764 | 0.191711548575651 203.910078561832 765 | 0.191963197944198 142.287948664973 766 | 0.192214847312744 187.189641471491 767 | 0.192466496681291 186.252020906745 768 | 0.192716814571697 165.51268033452 769 | 0.192968463940243 128.463201273435 770 | 0.19322011330879 134.127160839476 771 | 0.193471762677336 166.522742857371 772 | 0.193723412045883 132.376657989165 773 | 0.193975061414429 156.822360553426 774 | 0.194226710782976 128.804792854442 775 | 0.194478360151522 161.709112912507 776 | 0.194730009520069 90.4090865199464 777 | 0.194981658888615 153.342846624368 778 | 0.195233308257162 119.614613080889 779 | 0.195484957625708 118.017538424708 780 | 0.195736606994255 162.866317718039 781 | 0.195988256362801 182.418538694826 782 | 0.196239905731348 169.798583184981 783 | 0.196491555099894 135.620949310711 784 | 0.196743204468441 132.308788508968 785 | 0.196994853836987 127.546354841911 786 | 0.197246503205534 104.783969806054 787 | 0.19749815257408 129.30410113077 788 | 0.197749801942627 125.126356314482 789 | 0.198001451311173 137.986885151948 790 | 0.198251769201579 148.42515776708 791 | 0.198503418570126 168.951701508273 792 | 0.198755067938672 181.878499492038 793 | 0.199006717307219 157.753641538296 794 | 0.199258366675765 143.310336954836 795 | 0.199510016044312 125.570833252025 796 | 0.199761665412858 137.529676621539 797 | 0.200013314781405 115.83955221671 798 | 0.200264964149951 123.559593714793 799 | 0.200516613518497 140.562164962897 800 | 0.200768262887044 169.476341803785 801 | 0.20101991225559 146.345187423948 802 | 0.201271561624137 171.706439361563 803 | 0.201523210992684 168.460968768813 804 | 0.20177486036123 165.885613164647 805 | 0.202026509729777 118.040212830842 806 | 0.202278159098323 124.629957552683 807 | 0.202529808466869 132.516907802281 808 | 0.202781457835416 149.321221896145 809 | 0.203033107203962 181.286531868731 810 | 0.203284756572509 191.657003007115 811 | 0.203536405941056 127.593196554932 812 | 0.203788055309602 168.319226414234 813 | 0.204038373200008 113.873660155741 814 | 0.204290022568554 154.988270181161 815 | 0.204541671937101 159.563206455793 816 | 0.204793321305647 137.00835098914 817 | 0.205044970674194 187.010091959183 818 | 0.20529662004274 149.687245943302 819 | 0.205548269411287 185.219597443923 820 | 0.205799918779834 82.69963076297 821 | 0.20605156814838 130.900791160269 822 | 0.206303217516926 124.839035993776 823 | 0.206554866885473 145.409992936497 824 | 0.206806516254019 157.9360435536 825 | 0.207058165622566 149.016748138992 826 | 0.207309814991112 147.284977719022 827 | 0.207561464359659 147.519567128291 828 | 0.207813113728205 155.262311357735 829 | 0.208064763096752 102.498219590211 830 | 0.208316412465298 138.329496979035 831 | 0.208568061833845 156.906477314902 832 | 0.208819711202391 152.385051212925 833 | 0.209071360570938 160.640143741369 834 | 0.209323009939484 107.20749605135 835 | 0.20957332782989 138.181666653307 836 | 0.209824977198437 141.656008607087 837 | 0.210076626566983 123.052038505292 838 | 0.21032827593553 166.385342442512 839 | 0.210579925304076 165.005688364602 840 | 0.210831574672623 157.560871578593 841 | 0.211083224041169 142.170917601801 842 | 0.211334873409716 149.375486087035 843 | 0.211586522778262 155.836305390871 844 | 0.211838172146809 134.084790143204 845 | 0.212089821515355 157.851928242537 846 | 0.212341470883902 163.440193233288 847 | 0.212593120252448 121.805216663777 848 | 0.212844769620995 158.863294759165 849 | 0.213096418989541 129.731786731842 850 | 0.213348068358088 139.837929749068 851 | 0.213599717726634 184.43766534808 852 | 0.213851367095181 119.469357969613 853 | 0.214103016463727 136.808576156378 854 | 0.214354665832274 155.6727505555 855 | 0.21460631520082 143.684318056977 856 | 0.214857964569367 120.012179523091 857 | 0.215108282459773 163.517168688769 858 | 0.215359931828319 155.561884608391 859 | 0.215611581196866 137.287463978679 860 | 0.215863230565412 131.920746158233 861 | 0.216114879933959 120.465742415471 862 | 0.216366529302505 176.9707404617 863 | 0.216618178671052 126.864693725507 864 | 0.216869828039598 147.469064180938 865 | 0.217121477408145 150.192671927623 866 | 0.217373126776691 144.636441230828 867 | 0.217624776145238 125.476593286599 868 | 0.217876425513784 119.577962505992 869 | 0.218128074882331 153.305119918426 870 | 0.218379724250877 157.482938249568 871 | 0.218631373619424 114.512670595418 872 | 0.21888302298797 121.412397010731 873 | 0.219134672356517 124.484281997479 874 | 0.219386321725063 132.060236912829 875 | 0.21963797109361 122.682562622467 876 | 0.219889620462156 154.988826865906 877 | 0.220141269830702 123.426548587633 878 | 0.220392919199249 175.590461765229 879 | 0.220644568567795 146.231441370421 880 | 0.220894886458202 161.310087326494 881 | 0.221146535826748 163.149601895296 882 | 0.221398185195295 152.773134792168 883 | 0.221649834563841 181.361624353093 884 | 0.221901483932388 175.108348962506 885 | 0.222153133300934 171.420396154337 886 | 0.22240478266948 164.543530358769 887 | 0.222656432038027 143.434474762954 888 | 0.222908081406573 150.526172921569 889 | 0.22315973077512 153.191893776277 890 | 0.223411380143667 167.878015832268 891 | 0.223663029512213 141.442100510068 892 | 0.223914678880759 157.270224216229 893 | 0.224166328249306 133.51582523402 894 | 0.224417977617852 168.601383279862 895 | 0.224669626986399 139.942673389094 896 | 0.224921276354945 150.456137048252 897 | 0.225172925723492 160.116779446085 898 | 0.225424575092038 149.151264387632 899 | 0.225676224460585 160.684935449855 900 | 0.225927873829131 172.241983482318 901 | 0.226179523197678 170.967993422884 902 | 0.226429841088084 170.700575381921 903 | 0.22668149045663 202.412343399937 904 | 0.226933139825177 148.9853061072 905 | 0.227184789193723 161.878723673755 906 | 0.22743643856227 94.5574197868875 907 | 0.227688087930816 143.526656186675 908 | 0.227939737299363 148.714460761008 909 | 0.228191386667909 141.925933315736 910 | 0.228443036036456 154.551089576463 911 | 0.228694685405002 135.905850566561 912 | 0.228946334773549 157.187837896666 913 | 0.229197984142095 119.394171125465 914 | 0.229449633510642 150.343183792633 915 | 0.229701282879188 146.439407385304 916 | 0.229952932247735 172.46560593339 917 | 0.230204581616281 217.900934890344 918 | 0.230456230984828 127.490926092815 919 | 0.230707880353374 144.467564123752 920 | 0.230959529721921 137.171216767484 921 | 0.231211179090467 165.532289167278 922 | 0.231462828459014 133.681137448685 923 | 0.23171447782756 166.51497193051 924 | 0.231964795717966 155.776292703969 925 | 0.232216445086513 148.537730889486 926 | 0.232468094455059 155.148062482871 927 | 0.232719743823606 146.91574783018 928 | 0.232971393192152 182.966323092648 929 | 0.233223042560699 123.848693865982 930 | 0.233474691929245 151.261623045719 931 | 0.233726341297792 143.789757754542 932 | 0.233977990666338 135.609932494581 933 | 0.234229640034885 123.822471961148 934 | 0.234481289403431 131.125919984188 935 | 0.234732938771978 165.110717692855 936 | 0.234984588140524 155.426097522933 937 | 0.235236237509071 149.112090347988 938 | 0.235487886877617 122.130794782098 939 | 0.235739536246164 142.083580923505 940 | 0.23599118561471 98.4590312645436 941 | 0.236242834983257 120.598570098842 942 | 0.236494484351803 148.556690586245 943 | 0.23674613372035 111.852731512758 944 | 0.236997783088896 107.569637823572 945 | 0.237249432457443 99.4611036038197 946 | 0.237501081825989 159.692342290678 947 | 0.237751399716395 164.476483022646 948 | 0.238003049084942 168.732536493879 949 | 0.238254698453488 170.985890223506 950 | 0.238506347822035 123.288535735402 951 | 0.238757997190581 118.308858577303 952 | 0.239009646559128 118.081459213787 953 | 0.239261295927674 122.962535555234 954 | 0.239512945296221 152.411581193988 955 | 0.239764594664767 139.157332070706 956 | 0.240016244033313 131.74423360705 957 | 0.24026789340186 172.855527343396 958 | 0.240519542770407 162.226643665124 959 | 0.240771192138953 127.297942312136 960 | 0.2410228415075 105.88259409261 961 | 0.241274490876046 149.859369424148 962 | 0.241526140244593 114.056347698745 963 | 0.241777789613139 105.944141300993 964 | 0.242029438981685 124.019479303088 965 | 0.242281088350232 137.750136229458 966 | 0.242532737718778 126.950255059171 967 | 0.242784387087325 120.654815499211 968 | 0.243036036455872 118.418438001452 969 | 0.243286354346278 133.062783103675 970 | 0.243538003714824 154.501070684428 971 | 0.24378965308337 163.701903950647 972 | 0.244041302451917 154.092882869534 973 | 0.244292951820463 165.912491336602 974 | 0.24454460118901 125.646245681078 975 | 0.244796250557556 112.085541720633 976 | 0.245047899926103 112.61778076936 977 | 0.24529954929465 133.857619915696 978 | 0.245551198663196 117.237426821058 979 | 0.245802848031742 158.594394615661 980 | 0.246054497400289 154.210931081654 981 | 0.246306146768835 165.86052320139 982 | 0.246557796137382 137.72649094319 983 | 0.246809445505928 143.853013755475 984 | 0.247061094874475 120.317896465095 985 | 0.247312744243021 134.576677696731 986 | 0.247564393611568 156.641332119643 987 | 0.247816042980114 118.289534643511 988 | 0.248067692348661 130.00600490052 989 | 0.248319341717207 139.330321684122 990 | 0.248570991085754 159.123119769675 991 | 0.24882130897616 130.58332735294 992 | 0.249072958344706 169.83352544641 993 | 0.249324607713253 154.214687702421 994 | 0.249576257081799 171.972901535773 995 | 0.249827906450346 136.107015608772 996 | 0.250079555818892 146.032072179877 997 | 0.250331205187439 130.731721425448 998 | 0.250582854555985 115.896652810193 999 | 0.250834503924532 125.218603011159 1000 | 0.251086153293078 118.189280598834 1001 | 0.251337802661625 142.209075311945 1002 | 0.251589452030171 153.353895102072 1003 | -------------------------------------------------------------------------------- /lectures/catalysis.csv: -------------------------------------------------------------------------------- 1 | Time,NO3,NO2,N2,NH3,N2O 2 | 0,500.00,0.00,0.00,0.00,0.00 3 | 30,250.95,107.32,18.51,3.33,4.98 4 | 60,123.66,132.33,74.85,7.34,20.14 5 | 90,84.47,98.81,166.19,13.14,42.10 6 | 120,30.24,38.74,249.78,19.54,55.98 7 | 150,27.94,10.42,292.32,24.07,60.65 8 | 180,13.54,6.11,309.50,27.26,62.54 -------------------------------------------------------------------------------- /lectures/challenger_data.csv: -------------------------------------------------------------------------------- 1 | Date,Temperature,Damage Incident 2 | 04/12/1981,66,0 3 | 11/12/1981,70,1 4 | 3/22/82,69,0 5 | 6/27/82,80,NA 6 | 01/11/1982,68,0 7 | 04/04/1983,67,0 8 | 6/18/83,72,0 9 | 8/30/83,73,0 10 | 11/28/83,70,0 11 | 02/03/1984,57,1 12 | 04/06/1984,63,1 13 | 8/30/84,70,1 14 | 10/05/1984,78,0 15 | 11/08/1984,67,0 16 | 1/24/85,53,1 17 | 04/12/1985,67,0 18 | 4/29/85,75,0 19 | 6/17/85,70,0 20 | 7/29/85,81,0 21 | 8/27/85,76,0 22 | 10/03/1985,79,0 23 | 10/30/85,75,1 24 | 11/26/85,76,0 25 | 01/12/1986,58,1 26 | 1/28/86,31,Challenger Accident 27 | -------------------------------------------------------------------------------- /lectures/coal_mining_disasters.csv: -------------------------------------------------------------------------------- 1 | year,disasters 2 | 1851,4 3 | 1852,5 4 | 1853,4 5 | 1854,0 6 | 1855,1 7 | 1856,4 8 | 1857,3 9 | 1858,4 10 | 1859,0 11 | 1860,6 12 | 1861,3 13 | 1862,3 14 | 1863,4 15 | 1864,0 16 | 1865,2 17 | 1866,6 18 | 1867,3 19 | 1868,3 20 | 1869,5 21 | 1870,4 22 | 1871,5 23 | 1872,3 24 | 1873,1 25 | 1874,4 26 | 1875,4 27 | 1876,1 28 | 1877,5 29 | 1878,5 30 | 1879,3 31 | 1880,4 32 | 1881,2 33 | 1882,5 34 | 1883,2 35 | 1884,2 36 | 1885,3 37 | 1886,4 38 | 1887,2 39 | 1888,1 40 | 1889,3 41 | 1890,2 42 | 1891,2 43 | 1892,1 44 | 1893,1 45 | 1894,1 46 | 1895,1 47 | 1896,3 48 | 1897,0 49 | 1898,0 50 | 1899,1 51 | 1900,0 52 | 1901,1 53 | 1902,1 54 | 1903,0 55 | 1904,0 56 | 1905,3 57 | 1906,1 58 | 1907,0 59 | 1908,3 60 | 1909,2 61 | 1910,2 62 | 1911,0 63 | 1912,1 64 | 1913,1 65 | 1914,1 66 | 1915,0 67 | 1916,1 68 | 1917,0 69 | 1918,1 70 | 1919,0 71 | 1920,0 72 | 1921,0 73 | 1922,2 74 | 1923,1 75 | 1924,0 76 | 1925,0 77 | 1926,0 78 | 1927,1 79 | 1928,1 80 | 1929,0 81 | 1930,2 82 | 1931,3 83 | 1932,3 84 | 1933,1 85 | 1934,1 86 | 1935,2 87 | 1936,1 88 | 1937,1 89 | 1938,1 90 | 1939,1 91 | 1940,2 92 | 1941,4 93 | 1942,2 94 | 1943,0 95 | 1944,0 96 | 1945,1 97 | 1946,4 98 | 1947,0 99 | 1948,0 100 | 1949,0 101 | 1950,1 102 | 1951,0 103 | 1952,0 104 | 1953,0 105 | 1954,0 106 | 1955,0 107 | 1956,1 108 | 1957,0 109 | 1958,0 110 | 1959,1 111 | 1960,0 112 | 1961,1 113 | -------------------------------------------------------------------------------- /lectures/coin_flipping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/coin_flipping.png -------------------------------------------------------------------------------- /lectures/coin_toss_bayes_1: -------------------------------------------------------------------------------- 1 | digraph coin_toss_bayes_1 { 2 | theta [label=<θ>] 3 | X1 [label=1> style=filled] 4 | theta -> X1 5 | } 6 | -------------------------------------------------------------------------------- /lectures/coin_toss_bayes_2: -------------------------------------------------------------------------------- 1 | digraph coin_toss_bayes_2 { 2 | theta [label=<θ>] 3 | X1 [label=1> style=filled] 4 | X2 [label=2> style=filled] 5 | theta -> X1 6 | theta -> X2 7 | } 8 | -------------------------------------------------------------------------------- /lectures/coin_toss_bayes_3: -------------------------------------------------------------------------------- 1 | digraph coin_toss_bayes_3 { 2 | theta [label=<θ>] 3 | X1 [label=1> style=filled] 4 | X2 [label=2> style=filled] 5 | X3 [label=3> style=filled] 6 | theta -> X1 7 | theta -> X2 8 | theta -> X3 9 | } 10 | -------------------------------------------------------------------------------- /lectures/coin_toss_bayes_plate: -------------------------------------------------------------------------------- 1 | digraph coin_toss_bayes_plate { 2 | theta [label=<θ>] 3 | subgraph cluster_0 { 4 | xn [label=n> style=filled] 5 | label="n=1,...,N" 6 | labelloc=b 7 | } 8 | theta -> xn 9 | } 10 | -------------------------------------------------------------------------------- /lectures/coin_toss_g: -------------------------------------------------------------------------------- 1 | digraph coin_toss_g { 2 | omega0 [label=<ω0>] 3 | v0 [label=0>] 4 | g [style=filled] 5 | X 6 | g -> X 7 | v0 -> X 8 | omega0 -> X 9 | } 10 | -------------------------------------------------------------------------------- /lectures/demos.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/demos.zip -------------------------------------------------------------------------------- /lectures/demos/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Initialize the demos. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 3/13/2018 9 | 6/6/2014 10 | 11 | """ 12 | 13 | 14 | from ._utils import * 15 | from ._cache import * 16 | from ._numpy_array_cache import * 17 | from ._cached_function import * 18 | from ._model import * 19 | from . import catalysis 20 | from . import diffusion 21 | -------------------------------------------------------------------------------- /lectures/demos/_cache.py: -------------------------------------------------------------------------------- 1 | """ 2 | A generic class representing a cache. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 6/6/2014 9 | 10 | """ 11 | 12 | 13 | __all__ = ['Cache'] 14 | 15 | 16 | class Cache(object): 17 | 18 | """ 19 | A generic class representing a cache. 20 | """ 21 | 22 | # The maximum size of the cache 23 | _max_size = None 24 | 25 | # A name for the object 26 | __name__ = None 27 | 28 | @property 29 | def max_size(self): 30 | """ 31 | :getter: The max size. 32 | """ 33 | return self._max_size 34 | 35 | def __init__(self, max_size=256, name='Cache'): 36 | """ 37 | Initialize the object. 38 | """ 39 | assert isinstance(max_size, int) 40 | assert max_size > 0 41 | self._max_size = max_size 42 | self.__name__ = name 43 | 44 | @property 45 | def is_empty(self): 46 | """ 47 | Is the cache empty? 48 | """ 49 | return self.size == None 50 | 51 | @property 52 | def size(self): 53 | """ 54 | :getter: The current size of the cache. 55 | """ 56 | raise NotImplementedError('Implement me!') 57 | 58 | def drop_one(self): 59 | """ 60 | Drops one value from the cache assuming that there is at least one 61 | element. 62 | """ 63 | raise NotImplementedError('Implement me!') 64 | 65 | def _do_append(self, value): 66 | """ 67 | Adds ``value`` to the cache assuming the maximum size has not been 68 | reached. 69 | """ 70 | raise NotImplementedError('Implement me!') 71 | 72 | def append(self, value): 73 | """ 74 | Adds ``value`` to the cache. 75 | """ 76 | if self.size == self.max_size: 77 | self.drop_one() 78 | self._do_append(value) 79 | 80 | def get_index_of(self, value): 81 | """ 82 | Return the index of ``value``. 83 | 84 | It should return -1 if ``value`` is not stored in the cashe. 85 | """ 86 | raise NotImplementedError('Implement me!') 87 | 88 | def _get_value_at(self, i): 89 | """ 90 | Return the value of the cache corresponding to index ``i``. 91 | """ 92 | raise NotImplementedError('Implement me!') 93 | 94 | def __getitem__(self, i): 95 | """ 96 | Return the value of the cashe stored at index ``i``. 97 | """ 98 | assert i >= 0 and i < self.size 99 | return self._get_value_at(i) 100 | 101 | def __str__(self): 102 | """ 103 | Return a string representation of the object. 104 | """ 105 | s = 'Name: ' + self.__name__ 106 | return s 107 | -------------------------------------------------------------------------------- /lectures/demos/_cached_function.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implementation of a cached function of a numpy array input. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 6/6/2014 9 | 10 | """ 11 | 12 | 13 | __all__ = ['CachedFunction'] 14 | 15 | 16 | from . import Cache 17 | from . import NumpyArrayCache 18 | 19 | 20 | class CachedFunction(object): 21 | 22 | """ 23 | A class representing a cached function. 24 | """ 25 | 26 | # The input cache 27 | _input_cashe = None 28 | 29 | # The output cache 30 | _output_cache = None 31 | 32 | # The underlying function 33 | _f = None 34 | 35 | # The object that implements the function (if any) 36 | _obj = None 37 | 38 | def __init__(self, f, 39 | input_cache_type=NumpyArrayCache, 40 | input_cache_args={'name': 'Input Cache'}, 41 | output_cache_type=NumpyArrayCache, 42 | output_cache_args={'name': 'Output Cache'}): 43 | """ 44 | Initialize the object. 45 | """ 46 | self._count = 0 47 | self._count_eval = 0 48 | self._f = f 49 | self._input_cache = input_cache_type(**input_cache_args) 50 | self._output_cache = output_cache_type(**output_cache_args) 51 | 52 | def __get__(self, obj, type=None): 53 | return self.__class__(self._f.__get__(obj, type)) 54 | 55 | def __call__(self, *args, **kw): 56 | """ 57 | Call the function at x. 58 | """ 59 | x = args[0] 60 | # Look for x in the cache 61 | i = self._input_cache.get_index_of(x) 62 | self._count += 1 63 | if i == -1: 64 | # Not found in cache, so evaluate 65 | y = self._f(*args, **kw) 66 | self._count_eval += 1 67 | self._input_cache.append(x) 68 | self._output_cache.append(y) 69 | else: 70 | # Found in cache, recover 71 | y = self._output_cache[i] 72 | return y 73 | 74 | def __str__(self): 75 | """ 76 | Return a string representation of the object. 77 | """ 78 | s = 'Cached function:\n' 79 | s = ('Evaluations = ' + str(self._count) + 80 | ' (' + str(self._count_eval) + ' actual)') 81 | return s 82 | -------------------------------------------------------------------------------- /lectures/demos/_model.py: -------------------------------------------------------------------------------- 1 | """ 2 | The Model class. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 5/21/2014 9 | 10 | """ 11 | 12 | 13 | import numpy as np 14 | from . import call_many 15 | from . import CachedFunction 16 | 17 | 18 | class Model(object): 19 | 20 | """ 21 | Defines the concept of a model. 22 | 23 | Only the ``_eval()`` method of this class should be implemented by the children. 24 | See :meth:`vuq.Model._eval()` for further details on implementing this method. 25 | The bottom line is that it should evaluate the model and its first derivatives 26 | with respect to every parameter. 27 | 28 | """ 29 | 30 | # A name for the model 31 | __name__ = None 32 | 33 | # The number of input dimensions 34 | _num_input = None 35 | 36 | # The number of output dimensions 37 | _num_output = None 38 | 39 | # Can this model compute gradients? 40 | _compute_grad = None 41 | 42 | # Can this model compute hessians? 43 | _compute_hessian = None 44 | 45 | @property 46 | def num_input(self): 47 | """ 48 | :getter: The number of input dimensions. 49 | """ 50 | return self._num_input 51 | 52 | @property 53 | def num_output(self): 54 | """ 55 | :getter: The number of output dimensions. 56 | """ 57 | return self._num_output 58 | 59 | @property 60 | def compute_grad(self): 61 | """ 62 | :getter: ``True`` if this model can compute gradients. 63 | """ 64 | return self._compute_grad 65 | 66 | @property 67 | def compute_hessian(self): 68 | """ 69 | :getter: ``True`` if this model can compute hessian. 70 | """ 71 | return self._compute_hessian 72 | 73 | def __init__(self, num_input, num_output, name='Model', 74 | compute_grad=True, compute_hessian=True): 75 | """ 76 | Initialize the object. 77 | """ 78 | self._num_input = int(num_input) 79 | self._num_output = int(num_output) 80 | self.__name__ = str(name) 81 | self._compute_grad = compute_grad 82 | self._compute_hessian = compute_hessian 83 | if self.compute_hessian: 84 | assert self.compute_grad 85 | 86 | def _eval(self, x): 87 | """ 88 | Evaluate the model at a single input ``x``. 89 | 90 | :param x: The input at which we want to evaluate the model. It should have 91 | the :attr:`vuq.Model.num_input` dimensions. 92 | :type x: :class:`numpy.ndarray` 93 | :returns: A dictionary containing the following keys: 94 | + f: The output of the model, 1D array of num_output 95 | dimensions. 96 | + f_grad: The Jacobian of the model at x, a 2D array of size 97 | num_output x num_input dimensions. Return None, if 98 | you cannot compute the Jacobian. 99 | + f_grad_2: a 3D array of num_ouptut x num_input x num_input 100 | dimensions. Return None, if you cannot compute the 101 | Hessian. 102 | """ 103 | raise NotImplementedError('My children should implement this!') 104 | 105 | def __call__(self, x): 106 | """ 107 | Evaluate the model at many inputs ``x``. 108 | 109 | :param x: The input. It should always be a 2D matrix with the rows representing 110 | different points and the columns different inputs. The dimensions 111 | of x should be ``num_points x num_input``. 112 | :type x: :class:`numpy.ndarray` 113 | :returns: A dictionary containing the following keys: 114 | + f: The output at each row of x, 1D array of size x.shape[0]. 115 | Each element if of whatever type is the output of 116 | the model. 117 | + f_grad: A list of the Jacobians of each row of x. 118 | + f_grad_2: A list of the Hessians of each row of x. 119 | """ 120 | out = call_many(x, self._eval, return_numpy=False) 121 | state = {} 122 | state['f'] = np.array([s['f'] for s in out]) 123 | state['f_grad'] = [s['f_grad'] for s in out] 124 | state['f_grad_2'] = [s['f_grad_2'] for s in out] 125 | return state 126 | 127 | def __str__(self): 128 | """ 129 | Return a string representation of the object. 130 | """ 131 | s = 'Name: ' + self.__name__ + '\n' 132 | s += 'Number of input parameters: ' + str(self.num_input) + '\n' 133 | s += 'Number of output parameters: ' + str(self.num_output) + '\n' 134 | return s 135 | -------------------------------------------------------------------------------- /lectures/demos/_numpy_array_cache.py: -------------------------------------------------------------------------------- 1 | """ 2 | A cache specifically designed for numpy arrays. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 6/6/2014 9 | 10 | """ 11 | 12 | 13 | __all__ = ['NumpyArrayCache'] 14 | 15 | 16 | import numpy as np 17 | from . import Cache 18 | from . import euclidean_distance 19 | 20 | 21 | class NumpyArrayCache(Cache): 22 | 23 | """ 24 | A cache specifically designed for numpy arrays. 25 | 26 | See :class:`vuq.Cache` for the documentation of the overloaded functions. 27 | 28 | :param dist: The distance metric. 29 | :param tol: The tolerance below which two entries are considered to be 30 | identical. 31 | """ 32 | 33 | # The underlying cache (list of numpy arrays) 34 | _cache = None 35 | 36 | # The distance metric 37 | _dist = None 38 | 39 | def __init__(self, dist=euclidean_distance, tol=1e-16, 40 | max_size=256, name='Numpy Array Cache'): 41 | """ 42 | Initialize the object. 43 | """ 44 | super(NumpyArrayCache, self).__init__(max_size=max_size, name=name) 45 | self._dist = dist 46 | self._tol = tol 47 | self._cache = [] 48 | 49 | @property 50 | def size(self): 51 | return len(self._cache) 52 | 53 | def _do_append(self, value): 54 | self._cache.append(value) 55 | 56 | def drop_one(self): 57 | self._cache = self._cache[1:] 58 | 59 | def get_index_of(self, value): 60 | i = -1 61 | # We start iterating from the end of the sequence because it is more 62 | # probably that we will be asked to query a value from the top of the 63 | # cache 64 | for j in range(self.size - 1, -1, -1): 65 | d = self._dist(value, self._cache[i]) 66 | if d <= self._tol: 67 | i = j 68 | break 69 | return i 70 | 71 | def _get_value_at(self, i): 72 | return self._cache[i] 73 | -------------------------------------------------------------------------------- /lectures/demos/_utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Various utilities used throughout the code. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 5/19/2014 9 | 10 | """ 11 | 12 | 13 | __all__ = ['regularize_array', 'make_vector', 'call_many', 'view_as_column', 14 | 'euclidean_distance'] 15 | 16 | 17 | import numpy as np 18 | from scipy.spatial.distance import cdist 19 | 20 | 21 | def regularize_array(x): 22 | """ 23 | Regularize a numpy array. 24 | 25 | If x is 2D then nothing happens to it. If it is 1D then it is converted to 26 | 2D with 1 row and as many columns as the number of elements in x. 27 | 28 | :param x: The array to regularize. 29 | :type x: :class:`numpy.ndarray` 30 | :returns: Regularized version of x. 31 | :rtype: :class:`numpy.ndarray` 32 | 33 | .. note:: 34 | It does nothing if ``x`` is not a numpy array. 35 | """ 36 | if not isinstance(x, np.ndarray): 37 | return x 38 | if x.ndim == 1: 39 | x = x[None, :] 40 | return x 41 | 42 | 43 | def make_vector(x): 44 | """ 45 | Make a vector out of x. We will attempt to make an array out of x and then 46 | flatten it. 47 | """ 48 | return np.array(x).flatten() 49 | 50 | 51 | def call_many(x, func, return_numpy=True): 52 | """ 53 | Assuming the ``x`` is a 2D array, evaluate ``func(x[i, :])`` for each ``i`` 54 | and return the result as a numpy array. 55 | 56 | :param x: The evaluation points. 57 | :type x: :class:`numpy.ndarray` 58 | :param func: The function. 59 | :param return_numpy: If ``True``, then it puts all the outputs in a numpy array. 60 | Otherwise, it returns a list. 61 | """ 62 | x = regularize_array(x) 63 | out = [func(x[i, :]) for i in range(x.shape[0])] 64 | if return_numpy: 65 | return np.array(out) 66 | return out 67 | 68 | 69 | def view_as_column(x): 70 | """ 71 | View x as a column vector. 72 | """ 73 | if x.ndim == 1: 74 | x = x[:, None] 75 | elif x.ndim == 2 and x.shape[0] == 1: 76 | x = x.T 77 | return x 78 | 79 | 80 | def euclidean_distance(x, y): 81 | """ 82 | Returns the Euclidean distance between two numpy arrays. 83 | """ 84 | return cdist(regularize_array(x), regularize_array(y)) 85 | -------------------------------------------------------------------------------- /lectures/demos/advection/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Initialize the advection forward model module. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 10/15/2014 9 | 10 | """ 11 | 12 | from ._forward_advection import * 13 | -------------------------------------------------------------------------------- /lectures/demos/advection/_forward_advection.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem with one source. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Date: 7 | 10/15/2014 8 | 9 | """ 10 | 11 | 12 | import numpy as np 13 | import fipy as fp 14 | from transport_model import * 15 | import sys 16 | sys.path.insert(0,'../') 17 | from vuq import Model 18 | from vuq import view_as_column 19 | 20 | class ContaminantAdvectionModel(Model): 21 | """ 22 | A class representing the forward model of the contaminant transport problem 23 | using a advection equation. 24 | """ 25 | 26 | def __init__(self, name='Transport model'): 27 | """ 28 | Initialize the object 29 | """ 30 | super(ContaminantAdvectionModel, self).__init__(2, 40, name=name) 31 | 32 | def _eval(self, xs): 33 | """ 34 | Solves the advection equation for u and its derivatives for a given 35 | source location xs. 36 | """ 37 | xs = view_as_column(xs) 38 | assert xs.shape[0] == 2 39 | nx = 50 40 | ny = nx 41 | dx = 0.1 42 | dy = dx 43 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 44 | vx, vy = make_V_field(mesh) 45 | u = f(xs[:,0], mesh, vx, vy) 46 | du1 = df(xs[:,0], mesh, vx, vy, 1) 47 | du2 = df(xs[:,0], mesh, vx, vy, 2) 48 | d2u11 = df2(xs[:,0], mesh, vx, vy, 1, 1) 49 | d2u22 = df2(xs[:,0], mesh, vx, vy, 1, 1) 50 | d2u12 = df2(xs[:,0], mesh, vx, vy, 1, 2) 51 | dU = np.hstack([view_as_column(du1), view_as_column(du2)]) 52 | d2U = np.hstack([view_as_column(d2u11), view_as_column(d2u12), view_as_column(d2u12), view_as_column(d2u22)]) 53 | d2U = d2U.reshape((d2U.shape[0], 2, 2)) 54 | state = {} 55 | state['f'] = u #view_as_column(u) 56 | state['f_grad'] = dU 57 | state['f_grad_2'] = d2U 58 | return state 59 | 60 | def _eval_u(self, xs): 61 | """ 62 | Solves only the diffusion equation for u for a given 63 | source location xs. 64 | """ 65 | xs = view_as_column(xs) 66 | assert xs.shape[0] == 2 67 | nx = 50 68 | ny = nx 69 | dx = 0.1 70 | dy = dx 71 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 72 | vx, vy = make_V_field(mesh) 73 | u = f(xs[:,0], mesh, vx, vy) 74 | return u 75 | -------------------------------------------------------------------------------- /lectures/demos/advection/covarMatrix50.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/demos/advection/covarMatrix50.npy -------------------------------------------------------------------------------- /lectures/demos/advection/data_concentrations_advection.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/demos/advection/data_concentrations_advection.npy -------------------------------------------------------------------------------- /lectures/demos/advection/generate_data.py: -------------------------------------------------------------------------------- 1 | """ 2 | Generate data for the advection forward model. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 10/15/2014 9 | 10 | """ 11 | 12 | import numpy as np 13 | import fipy as fp 14 | from random import * 15 | import matplotlib.pyplot as plt 16 | 17 | ################################################################################ 18 | ######## --------- Make the hydraulic conductivity field ------------ ########## 19 | ################################################################################ 20 | 21 | A = np.load('covarMatrix50.npy') 22 | # Perform Cholesky decomposition of cov 23 | U = np.linalg.cholesky(A) 24 | # 25 | 26 | y = np.zeros((2500,1)) 27 | e = np.zeros((2500,1)) 28 | mu = -1.106 # normal random field mean 29 | for i in range(0,2500): 30 | e[i] = gauss(0,1) 31 | 32 | y = mu + np.dot(U,e) 33 | K = np.exp(y) 34 | 35 | ################################################################################ 36 | ################################################################################ 37 | ###### ----- Initialization and steady state equation for head ---------- ###### 38 | ################################################################################ 39 | ################################################################################ 40 | 41 | nx = 50 42 | ny = nx 43 | dx = 0.05 44 | dy = dx 45 | L = dx * nx 46 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 47 | 48 | phi = fp.CellVariable(name = "solution variable", mesh=mesh, value=0.) 49 | 50 | D = fp.CellVariable(mesh=mesh, value = 1.) 51 | for i in range(0,2500): 52 | D()[i] = K[i] 53 | 54 | coeffD = D()*fp.numerix.exp(0.01*phi) 55 | 56 | # --- Assign boundary conditions 57 | valueTop = -1.5 58 | valueGradBottom = 0. 59 | 60 | phi.constrain(valueTop, where=mesh.facesTop) 61 | phi.constrain(valueGradBottom, where=mesh.facesBottom) 62 | 63 | if __name__ == '__main__': 64 | viewer = fp.Viewer(vars=phi)#, datamin=-1.5, datamax=0.) 65 | viewer.plot() 66 | 67 | # --- Solve steady state groundwater flow equation 68 | eqSteady = (fp.DiffusionTerm(coeff=D()*fp.numerix.exp(0.01*phi)) + np.gradient(coeffD())[0]) 69 | eqSteady.solve(var=phi) 70 | print phi() 71 | if __name__ == '__main__': 72 | viewer.plot() 73 | 74 | if __name__ == '__main__': 75 | raw_input("Implicit steady-state diffusion. Press to proceed...") 76 | 77 | ################################################################################ 78 | ################################################################################ 79 | ##### -------------------- Solve the transport equation ------------------ ##### 80 | ################################################################################ 81 | 82 | 83 | theta = 0.4 # Effective porosity 84 | Rd = 2 # Retardation factor 85 | 86 | head = phi().reshape(50,50) 87 | Dhead = np.gradient(head) 88 | qx = Dhead[1] 89 | Vx = qx.reshape(2500) 90 | for i in range(0,2500): 91 | Vx[i] = -coeffD[i]*Vx[i]/theta 92 | print Vx 93 | 94 | qy = Dhead[0] + 1. 95 | Vy = qy.reshape(2500) 96 | for i in range(0,2500): 97 | Vy[i] = -coeffD[i]*Vy[i]/theta 98 | print Vy 99 | 100 | 101 | vx = fp.CellVariable(name="x-component velocity", mesh=mesh, value=Vx) 102 | vy = fp.CellVariable(name="y-component velocity", mesh=mesh, value=Vy) 103 | viewerVx = fp.Viewer(vars=vx, datamin=-0.1, datamax=0.1) 104 | viewerVx.plot() 105 | viewerVy = fp.Viewer(vars=vy, datamin=-4., datamax=0.) 106 | viewerVy.plot() 107 | 108 | if __name__ == '__main__': 109 | raw_input("Velocity field. Press to proceed") 110 | 111 | convCoeff = fp.CellVariable(mesh=mesh, rank=1) 112 | convCoeff()[0,:] = Vx 113 | convCoeff()[1,:] = Vy 114 | time = fp.Variable() 115 | 116 | 117 | var = fp.CellVariable(name = "variable", mesh=mesh) 118 | 119 | # --- Make source term 120 | rho = 0.35 121 | q0 = 1. / (np.pi * rho ** 2) 122 | T = 0.3 123 | xs_1 = np.array([1.25, 2.]) 124 | sourceTerm_1 = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 125 | for i in range(sourceTerm_1().shape[0]): 126 | sourceTerm_1()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs_1[0]) ** 2 127 | + (mesh.cellCenters[1]()[i] - xs_1[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 128 | 129 | 130 | 131 | eqC = fp.TransientTerm(Rd) == -fp.ConvectionTerm(coeff=convCoeff) + sourceTerm_1 132 | 133 | if __name__ == '__main__': 134 | viewer = fp.Viewer(vars=var, datamin = 0., datamax = 2.5) 135 | viewer.plot() 136 | 137 | data = [] 138 | timeStepDuration = 0.005 139 | steps = 400 140 | for step in range(steps): 141 | time.setValue(time() + timeStepDuration) 142 | eqC.solve(var=var, dt=timeStepDuration) 143 | if step == 99 or step == 199 or step == 299 or step == 399: 144 | data = np.hstack([data, np.array([var()[8*50+14], var()[8*50+34], 145 | var()[18*50+14], var()[18*50+34], 146 | var()[28*50+14], var()[28*50+34], 147 | var()[38*50+14], var()[38*50+34], 148 | var()[48*50+14], var()[48*50+34]])]) 149 | # data = np.hstack([data, np.array([dc, uc])]) 150 | if __name__ == '__main__': 151 | viewer.plot() 152 | 153 | if __name__ == '__main__': 154 | raw_input("Steady-state convection-diffusion equation. Press to continue...") 155 | 156 | np.save('data_concentrations_advection.npy',data) -------------------------------------------------------------------------------- /lectures/demos/advection/transport_model.py: -------------------------------------------------------------------------------- 1 | """ 2 | Solve numerically the transport equation using the fipy package. 3 | The velocity field is obtained after solving first the Richard's equation 4 | in steady state. 5 | 6 | Author: 7 | Panagiotis Tsilifis 8 | 9 | Date: 10 | 10/15/2014 11 | """ 12 | 13 | import numpy as np 14 | import fipy as fp 15 | from random import * 16 | import matplotlib.pyplot as plt 17 | 18 | 19 | 20 | def make_source(xs, mesh, time): 21 | """ 22 | Makes the source term of the transport equation 23 | """ 24 | #assert xs.shape[0] == 2 25 | sourceTerm = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 26 | rho = 0.35 27 | q0 = 1 / (np.pi * rho ** 2) 28 | T = 0.3 29 | for i in range(sourceTerm().shape[0]): 30 | sourceTerm()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs[0]) ** 2 31 | + (mesh.cellCenters[1]()[i] - xs[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 32 | return sourceTerm 33 | 34 | def make_source_der(xs, mesh, time, i): 35 | """ 36 | Make the source term of the transport equation for the derivatives of u. 37 | """ 38 | #assert xs.shape[0] == 2 39 | rho = 0.35 40 | sourceTerm = make_source(xs, mesh, time) 41 | for j in range(sourceTerm().shape[0]): 42 | sourceTerm()[j] = sourceTerm()[j] * (mesh.cellCenters[i-1]()[j] - xs[i-1]) / rho ** 2 43 | return sourceTerm 44 | 45 | def make_source_der_2(xs, mesh, time, i, j): 46 | """ 47 | Make the source term of the transport equation for the 2nd derivatives of u. 48 | """ 49 | #assert xs.shape[0] == 2 50 | rho = 0.35 51 | sourceTerm = make_source(xs, mesh, time) 52 | for k in range(sourceTerm().shape[0]): 53 | sourceTerm()[k] = sourceTerm()[k] * ( (mesh.cellCenters[i-1]()[k] - xs[i-1]) * (mesh.cellCenters[j-1]()[k] - xs[j-1]) / rho **2 54 | - (i == j)) / rho **2 55 | return sourceTerm 56 | 57 | def make_V_field(mesh): 58 | """ 59 | Solves Richards equation at steady state and uses head solution to calculate 60 | the velocity field. 61 | """ 62 | ############################################################################ 63 | ######## --------- Make the hydraulic conductivity field --------- ######### 64 | ############################################################################ 65 | 66 | A = np.load('covarMatrix50.npy') 67 | # Perform Cholesky decomposition of cov 68 | U = np.linalg.cholesky(A) 69 | # 70 | 71 | y = np.zeros((2500,1)) 72 | e = np.zeros((2500,1)) 73 | mu = -1.106 # normal random field mean 74 | for i in range(0,2500): 75 | e[i] = gauss(0,1) 76 | 77 | y = mu + np.dot(U,e) 78 | K = np.exp(y) 79 | 80 | ############################################################################ 81 | ############################################################################ 82 | ########## ---------- Steady state equation for head ----------- ########### 83 | ############################################################################ 84 | ############################################################################ 85 | 86 | phi = fp.CellVariable(name = "solution variable", mesh=mesh, value=0.) 87 | 88 | D = fp.CellVariable(mesh=mesh, value = 1.) 89 | for i in range(0,2500): 90 | D()[i] = K[i] 91 | 92 | coeffD = D()*fp.numerix.exp(0.01*phi) 93 | 94 | # --- Assign boundary conditions 95 | valueTop = -1.5 96 | valueGradBottom = 0. 97 | 98 | phi.constrain(valueTop, where=mesh.facesTop) 99 | phi.constrain(valueGradBottom, where=mesh.facesBottom) 100 | 101 | # --- Solve steady state groundwater flow equation 102 | eqSteady = (fp.DiffusionTerm(coeff=D()*fp.numerix.exp(0.01*phi)) + np.gradient(coeffD())[0]) 103 | eqSteady.solve(var=phi) 104 | 105 | theta = 0.4 # Effective porosity 106 | Rd = 2 # Retardation factor 107 | 108 | head = phi().reshape(50,50) 109 | Dhead = np.gradient(head) 110 | qx = Dhead[1] 111 | vx = qx.reshape(2500) 112 | for i in range(0,2500): 113 | vx[i] = -coeffD[i]*vx[i]/theta 114 | 115 | qy = Dhead[0] + 1. 116 | vy = qy.reshape(2500) 117 | for i in range(0,2500): 118 | vy[i] = -coeffD[i]*vy[i]/theta 119 | 120 | return vx, vy 121 | 122 | def f(xs, mesh, vx, vy): 123 | """ 124 | Evaluate the model for the concentration at the 10 locations of the domain 125 | at times ``t``. 126 | 127 | It returns a flatten version of the system, i.e.: 128 | y_1(t_1) 129 | ... 130 | y_10(t_1) 131 | ... 132 | y_1(t_4) 133 | ... 134 | y_10(t_4) 135 | """ 136 | ############################################################################ 137 | ############################################################################ 138 | ##### ------------------ Solve the transport equation ---------------- ##### 139 | ############################################################################ 140 | 141 | Rd = 2. # Retardation factor 142 | 143 | convCoeff = fp.CellVariable(mesh=mesh, rank=1) 144 | convCoeff()[0,:] = vx 145 | convCoeff()[1,:] = vy 146 | time = fp.Variable() 147 | # --- Make Source --- 148 | q = make_source(xs, mesh, time) 149 | # The solution variable 150 | var = fp.CellVariable(name = "variable", mesh=mesh, value=0.) 151 | # Define the equation 152 | eqC = fp.TransientTerm(Rd) == -fp.ConvectionTerm(coeff=convCoeff) + q 153 | 154 | # Solve 155 | U_sol = [] 156 | timeStepDuration = 0.005 157 | steps = 400 158 | for step in range(steps): 159 | time.setValue(time() + timeStepDuration) 160 | eqC.solve(var=var, dt=timeStepDuration) 161 | if step == 99 or step == 199 or step == 299 or step == 399: 162 | U_sol = np.hstack([U_sol, np.array([var()[8*50+14], var()[8*50+34], 163 | var()[18*50+14], var()[18*50+34], 164 | var()[28*50+14], var()[28*50+34], 165 | var()[38*50+14], var()[38*50+34], 166 | var()[48*50+14], var()[48*50+34]])]) 167 | 168 | return U_sol 169 | 170 | ############################## END ################################### 171 | def df(xs, mesh, vx, vy, i): 172 | """ 173 | Evaluate the model for the derivatives at the 10 locations of the domain 174 | at times ``t``. 175 | 176 | It returns a flatten version of the system, i.e.: 177 | y_1(t_1) 178 | ... 179 | y_10(t_1) 180 | ... 181 | y_1(t_4) 182 | ... 183 | y_10(t_4) 184 | """ 185 | assert i == 1 or i == 2 186 | ############################################################################ 187 | ############################################################################ 188 | ##### ------------------ Solve the transport equation ---------------- ##### 189 | ############################################################################ 190 | 191 | Rd = 2. # Retardation factor 192 | 193 | convCoeff = fp.CellVariable(mesh=mesh, rank=1) 194 | convCoeff()[0,:] = vx 195 | convCoeff()[1,:] = vy 196 | time = fp.Variable() 197 | # --- Make Source --- 198 | q0 = make_source_der(xs, mesh, time, i) 199 | # The solution variable 200 | var = fp.CellVariable(name = "variable", mesh=mesh, value=0.) 201 | # Define the equation 202 | eqC = fp.TransientTerm(Rd) == -fp.ConvectionTerm(coeff=convCoeff) + q0 203 | 204 | # Solve 205 | dU = [] 206 | timeStepDuration = 0.005 207 | steps = 400 208 | for step in range(steps): 209 | time.setValue(time() + timeStepDuration) 210 | eqC.solve(var=var, dt=timeStepDuration) 211 | if step == 99 or step == 199 or step == 299 or step == 399: 212 | dU = np.hstack([dU, np.array([var()[8*50+14], var()[8*50+34], 213 | var()[18*50+14], var()[18*50+34], 214 | var()[28*50+14], var()[28*50+34], 215 | var()[38*50+14], var()[38*50+34], 216 | var()[48*50+14], var()[48*50+34]])]) 217 | 218 | return dU 219 | 220 | ############################## END ################################### 221 | 222 | def df2(xs, mesh, vx, vy, i, j): 223 | """ 224 | Evaluate the model for the 2nd derivatives at the 10 locations of the domain 225 | at times ``t``. 226 | 227 | It returns a flatten version of the system, i.e.: 228 | y_1(t_1) 229 | ... 230 | y_10(t_1) 231 | ... 232 | y_1(t_4) 233 | ... 234 | y_10(t_4) 235 | """ 236 | assert i == 1 or i == 2 237 | assert j == 1 or j == 2 238 | ############################################################################ 239 | ############################################################################ 240 | ##### ------------------ Solve the transport equation ---------------- ##### 241 | ############################################################################ 242 | 243 | Rd = 2. # Retardation factor 244 | 245 | convCoeff = fp.CellVariable(mesh=mesh, rank=1) 246 | convCoeff()[0,:] = vx 247 | convCoeff()[1,:] = vy 248 | time = fp.Variable() 249 | # --- Make Source --- 250 | q0 = make_source_der_2(xs, mesh, time, i, j) 251 | # The solution variable 252 | var = fp.CellVariable(name = "variable", mesh=mesh, value=0.) 253 | # Define the equation 254 | eqC = fp.TransientTerm(Rd) == -fp.ConvectionTerm(coeff=convCoeff) + q0 255 | 256 | # Solve 257 | d2U = [] 258 | timeStepDuration = 0.005 259 | steps = 400 260 | for step in range(steps): 261 | time.setValue(time() + timeStepDuration) 262 | eqC.solve(var=var, dt=timeStepDuration) 263 | if step == 99 or step == 199 or step == 299 or step == 399: 264 | d2U = np.hstack([d2U, np.array([var()[8*50+14], var()[8*50+34], 265 | var()[18*50+14], var()[18*50+34], 266 | var()[28*50+14], var()[28*50+34], 267 | var()[38*50+14], var()[38*50+34], 268 | var()[48*50+14], var()[48*50+34]])]) 269 | 270 | return d2U 271 | 272 | ############################## END ################################### -------------------------------------------------------------------------------- /lectures/demos/catalysis/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Initialize the module. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | 8 | Date: 9 | 5/22/2014 10 | 3/13/2018 11 | 12 | """ 13 | 14 | from ._forward_model import * 15 | -------------------------------------------------------------------------------- /lectures/demos/catalysis/_forward_model.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | Date: 8 | 05/22/2014 9 | 03/13/2018 10 | """ 11 | 12 | 13 | import numpy as np 14 | from .model_1 import * 15 | from .model_2 import * 16 | import sys 17 | from .. import Model 18 | from .. import view_as_column 19 | 20 | 21 | 22 | class CatalysisModel(Model): 23 | """ 24 | A class representing the forward model of the catalysis problem. 25 | """ 26 | 27 | def __init__(self, name='Catalysis model'): 28 | """ 29 | Initialize the object 30 | """ 31 | #self._kappa = x 32 | super(CatalysisModel, self).__init__(5, 35, name=name) 33 | 34 | def _eval(self, x): 35 | """ 36 | Solves the dynamical system for given parameters x. 37 | """ 38 | x = view_as_column(x) 39 | # Points where the solution will be evaluated 40 | t = np.array([0., 30., 60., 90., 120., 150., 180.]) 41 | t = view_as_column(t) 42 | # Initial condition 43 | y0 = np.array([500., 0., 0., 0., 0., 0.]) 44 | y0 = view_as_column(y0) 45 | assert x.shape[0] == 5 46 | sol = f(x[:,0], y0[:,0], t[:,0]) 47 | J = df(x[:,0], y0[:,0], t[:,0]) 48 | H = df2(x[:,0], y0[:,0], t[:,0]) 49 | y = np.delete(sol.reshape((7,6)), 2, 1).flatten() # The 3rd species is unobserved 50 | dy = np.array([np.delete(J[:,i].reshape((7,6)), 2, 1).reshape(35) for i in range(J.shape[1])]) # Delete the 3rd species 51 | d2y = np.zeros((35, H.shape[1], H.shape[2])) 52 | for i in range(H.shape[1]): 53 | for j in range(H.shape[2]): 54 | d2y[:,i,j]= np.delete(H[:,i,j].reshape((7,6)), 2, 1).reshape(35) # Delete the 3rd species 55 | state = {} 56 | state['f'] = y 57 | state['f_grad'] = dy.T 58 | state['f_grad_2'] = d2y 59 | return state 60 | -------------------------------------------------------------------------------- /lectures/demos/catalysis/model_1.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implements the catalysis model as found in Yiannis paper. 3 | 4 | """ 5 | 6 | from .system import * 7 | 8 | 9 | def make_A(kappa): 10 | """ 11 | Make the matrix of the dynamical system from ``kappa``. 12 | """ 13 | A = np.array([[-kappa[0], 0, 0, 0, 0, 0], 14 | [kappa[0], -kappa[1]-kappa[3]-kappa[4], 0, 0, 0, 0], 15 | [0, kappa[1], -kappa[2], 0, 0, 0], 16 | [0, 0, kappa[2], 0, 0, 0], 17 | [0, kappa[4], 0, 0, 0, 0], 18 | [0, kappa[3], 0, 0, 0, 0]]) 19 | # The derivative of A with respect to kappa 20 | d = A.shape[0] 21 | s = kappa.shape[0] 22 | dA = np.zeros((d, d, s)) 23 | dA[0, 0, 0] = -1. 24 | dA[1, 0, 0] = 1. 25 | dA[1, 1, 1] = -1. 26 | dA[1, 1, 4] = -1. 27 | dA[1, 1, 3] = -1. 28 | dA[2, 1, 1] = 1. 29 | dA[2, 2, 2] = -1. 30 | dA[3, 2, 2] = 1. 31 | dA[4, 1, 4] = 1. 32 | dA[5, 1, 3] = 1. 33 | return A, dA 34 | 35 | def make_full_A(kappa): 36 | """ 37 | Make the matrix of the dynamical system from ``kappa``. 38 | """ 39 | assert kappa.shape[0] == 36 40 | A = kappa.reshape((6,6)) 41 | dA = np.zeros((36,36)) 42 | for i in range(36): 43 | dA[i,i] = 1 44 | dA = dA.reshape((6,6,36)) 45 | return A, dA 46 | 47 | def f(kappa, y0, t): 48 | """ 49 | Evaluate the model at ``kappa`` and at times ``t`` with initial 50 | conditions ``y0``. 51 | 52 | It returns a flatten version of the system, i.e.: 53 | y_1(t_1) 54 | ... 55 | y_d(t_1) 56 | ... 57 | y_1(t_K) 58 | ... 59 | y_d(t_K) 60 | """ 61 | A = make_A(kappa)[0] 62 | r = ode(f0) 63 | r.set_initial_value(y0, 0).set_f_params(A) 64 | y_m = [y0[None, :]] 65 | for tt in t[1:]: 66 | r.integrate(tt) 67 | y_m.append(r.y[None, :]) 68 | y_m = np.vstack(y_m) 69 | return y_m.flatten() 70 | 71 | def f_full(kappa, y0, t): 72 | """ 73 | Evaluate the model at ``kappa`` and at times ``t`` with initial 74 | conditions ``y0``. 75 | 76 | It returns a flatten version of the system, i.e.: 77 | y_1(t_1) 78 | ... 79 | y_d(t_1) 80 | ... 81 | y_1(t_K) 82 | ... 83 | y_d(t_K) 84 | """ 85 | A = make_full_A(kappa)[0] 86 | r = ode(f0) 87 | r.set_initial_value(y0, 0).set_f_params(A) 88 | y_m = [y0[None, :]] 89 | for tt in t[1:]: 90 | r.integrate(tt) 91 | y_m.append(r.y[None, :]) 92 | y_m = np.vstack(y_m) 93 | return y_m.flatten() 94 | 95 | def df(kappa, y0, t): 96 | """ 97 | Evaluate the derivative of the model derivatives at ``kappa`` and at times ``t`` 98 | with initial conditions ``y0``. 99 | 100 | This returns a matrix of the following form: 101 | dy_1(t_1) / dkappa_1 ... dy_1(t_1) / dkappa_s 102 | ... 103 | dy_d(t_1) / dkappa_1 ... dy_d(t_1) / dkappa_s 104 | ... 105 | dy_1(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 106 | ... 107 | dy_d(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 108 | """ 109 | A, dA = make_A(kappa) 110 | d = A.shape[0] 111 | r = ode(f1) # Look at system.py: this should be the f of the adjoint 112 | r.set_initial_value(np.hstack([y0, np.zeros((d ** 3, ))])).set_f_params(A) 113 | y_m = [r.y] 114 | for tt in t[1:]: 115 | r.integrate(tt) 116 | y_m.append(r.y[None, :]) 117 | y_m = np.vstack(y_m) 118 | # This is the Jacobian with respect to the full matrix A 119 | J = y_m[:, d:].reshape((t.shape[0], d, d, d)) 120 | # Now we apply the chain rule to compute the jacobian wrt kappa 121 | J_kappa = np.einsum('ijkl,klr', J, dA) 122 | return J_kappa.reshape((d * t.shape[0], kappa.shape[0])) 123 | 124 | def df_full(kappa, y0, t): 125 | """ 126 | Evaluate the derivative of the model derivatives at ``kappa`` and at times ``t`` 127 | with initial conditions ``y0``. 128 | 129 | This returns a matrix of the following form: 130 | dy_1(t_1) / dkappa_1 ... dy_1(t_1) / dkappa_s 131 | ... 132 | dy_d(t_1) / dkappa_1 ... dy_d(t_1) / dkappa_s 133 | ... 134 | dy_1(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 135 | ... 136 | dy_d(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 137 | """ 138 | A, dA = make_full_A(kappa) 139 | d = A.shape[0] 140 | r = ode(f1) # Look at system.py: this should be the f of the adjoint 141 | r.set_initial_value(np.hstack([y0, np.zeros((d ** 3, ))])).set_f_params(A) 142 | y_m = [r.y] 143 | for tt in t[1:]: 144 | r.integrate(tt) 145 | y_m.append(r.y[None, :]) 146 | y_m = np.vstack(y_m) 147 | # This is the Jacobian with respect to the full matrix A 148 | J = y_m[:, d:].reshape((t.shape[0], d, d, d)) 149 | # Now we apply the chain rule to compute the jacobian wrt kappa 150 | J_kappa = np.einsum('ijkl,klr', J, dA) 151 | return J_kappa.reshape((d * t.shape[0], kappa.shape[0])) 152 | 153 | def ndf(kappa0, y0, t, h=1e-3): 154 | """ 155 | Numerically compute the Jacobian of f at x0. 156 | """ 157 | y = f(kappa0, y0, t) 158 | kappa = kappa0.copy() 159 | J = np.zeros((y.shape[0], kappa0.shape[0])) 160 | for i in range(kappa0.shape[0]): 161 | kappa[i] += h 162 | py = f(kappa, y0, t) 163 | kappa[i] -= h 164 | J[:, i] = (py - y) / h 165 | return J 166 | 167 | 168 | if __name__ == '__main__': 169 | # Test if what we are doing make sense 170 | import matplotlib.pyplot as plt 171 | # Some kappas to evaluate the model at: 172 | kappa = np.array([0.0216, 0.0292, 0.0219, 0.0021, 0.0048]) 173 | # The observed data (to read the initial conditions): 174 | obs_data = np.loadtxt('obs_data.txt') 175 | # Remember that the column of the data that has all zeros contains X 176 | # which is not observed. This is the 4th column. 177 | # The observed times 178 | t = obs_data[:, 0] 179 | # The intial points 180 | y0 = obs_data[0, 1:] 181 | # Here is how we evaluate the model 182 | y = f(kappa, y0, t) 183 | # Here is how we evaluate the derivatives of the model 184 | dy = df(kappa, y0, t) 185 | # Now evaluate the numerical derivatives at kappa 186 | J = ndf(kappa, y0, t) 187 | for i in range(kappa.shape[0]): 188 | plt.plot(dy[:, i], 'r', linewidth=2) 189 | plt.plot(J[:, i], '--g', linewidth=2) 190 | plt.legend(['Adjoint derivative', 'Numerical Derivative']) 191 | plt.show() 192 | -------------------------------------------------------------------------------- /lectures/demos/catalysis/model_2.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implements the catalysis model as found in Yiannis paper. 3 | 4 | """ 5 | 6 | from .system import * 7 | 8 | 9 | def make_A(kappa): 10 | """ 11 | Make the matrix of the dynamical system from ``kappa``. 12 | """ 13 | A = np.array([[-kappa[0], 0, 0, 0, 0, 0], 14 | [kappa[0], -kappa[1]-kappa[3]-kappa[4], 0, 0, 0, 0], 15 | [0, kappa[1], -kappa[2], 0, 0, 0], 16 | [0, 0, kappa[2], 0, 0, 0], 17 | [0, kappa[4], 0, 0, 0, 0], 18 | [0, kappa[3], 0, 0, 0, 0]]) 19 | # The derivative of A with respect to kappa 20 | d = A.shape[0] 21 | s = kappa.shape[0] 22 | dA = np.zeros((d, d, s)) 23 | dA[0, 0, 0] = -1. 24 | dA[1, 0, 0] = 1. 25 | dA[1, 1, 1] = -1. 26 | dA[1, 1, 4] = -1. 27 | dA[1, 1, 3] = -1. 28 | dA[2, 1, 1] = 1. 29 | dA[2, 2, 2] = -1. 30 | dA[3, 2, 2] = 1. 31 | dA[4, 1, 4] = 1. 32 | dA[5, 1, 3] = 1. 33 | return A, dA 34 | 35 | def make_full_A(kappa): 36 | """ 37 | Make the matrix of the dynamical system from ``kappa``. 38 | """ 39 | assert kappa.shape[0] == 36 40 | A = kappa.reshape((6,6)) 41 | dA = np.zeros((36,36)) 42 | for i in range(36): 43 | dA[i,i] = 1 44 | dA = dA.reshape((6,6,36)) 45 | return A, dA 46 | 47 | def f(kappa, y0, t): 48 | """ 49 | Evaluate the model at ``kappa`` and at times ``t`` with initial 50 | conditions ``y0``. 51 | 52 | It returns a flatten version of the system, i.e.: 53 | y_1(t_1) 54 | ... 55 | y_d(t_1) 56 | ... 57 | y_1(t_K) 58 | ... 59 | y_d(t_K) 60 | """ 61 | A = make_A(kappa)[0] 62 | r = ode(f0) 63 | r.set_initial_value(y0, 0).set_f_params(A) 64 | y_m = [y0[None, :]] 65 | for tt in t[1:]: 66 | r.integrate(tt) 67 | y_m.append(r.y[None, :]) 68 | y_m = np.vstack(y_m) 69 | return y_m.flatten() 70 | 71 | def f_full(kappa, y0, t): 72 | """ 73 | Evaluate the model at ``kappa`` and at times ``t`` with initial 74 | conditions ``y0``. 75 | 76 | It returns a flatten version of the system, i.e.: 77 | y_1(t_1) 78 | ... 79 | y_d(t_1) 80 | ... 81 | y_1(t_K) 82 | ... 83 | y_d(t_K) 84 | """ 85 | A = make_full_A(kappa)[0] 86 | r = ode(f0) 87 | r.set_initial_value(y0, 0).set_f_params(A) 88 | y_m = [y0[None, :]] 89 | for tt in t[1:]: 90 | r.integrate(tt) 91 | y_m.append(r.y[None, :]) 92 | y_m = np.vstack(y_m) 93 | return y_m.flatten() 94 | 95 | 96 | def df2(kappa, y0, t): 97 | """ 98 | Evaluate the derivative of the model derivatives at ``kappa`` and at times ``t`` 99 | with initial conditions ``y0``. 100 | 101 | This returns a matrix of the following form: 102 | dy_1(t_1) / dkappa_1 ... dy_1(t_1) / dkappa_s 103 | ... 104 | dy_d(t_1) / dkappa_1 ... dy_d(t_1) / dkappa_s 105 | ... 106 | dy_1(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 107 | ... 108 | dy_d(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 109 | """ 110 | A, dA = make_A(kappa) 111 | d = A.shape[0] 112 | r = ode(f2) # Look at system.py: this should be the f of the adjoint 113 | r.set_initial_value(np.hstack([y0, np.zeros((d ** 3 + d ** 5, ))])).set_f_params(A) 114 | y_m = [r.y] 115 | for tt in t[1:]: 116 | r.integrate(tt) 117 | y_m.append(r.y[None, :]) 118 | y_m = np.vstack(y_m) 119 | # This is the Jacobian with respect to the full matrix A 120 | J = y_m[:, d:d+d**3].reshape((t.shape[0], d, d, d)) 121 | # Now we apply the chain rule to compute the jacobian wrt kappa 122 | J_kappa = np.einsum('ijkl,klr', J, dA) 123 | 124 | J2 = y_m[:,d+d**3:d+d**3+d**5].reshape((t.shape[0], d, d**2, d**2)) 125 | #print len(J2[0,0,0,:]) 126 | ## Trying something New 127 | JJ = np.zeros((t.shape[0],d,kappa.shape[0],d**2)) 128 | for j in range(t.shape[0]): 129 | for k in range(d): 130 | for i in range(d**2): 131 | JJ[j,k,:,i] = np.einsum('kl,klr',J2[j,k,i,:].reshape(d,d), dA) 132 | J_kappa2 = np.einsum('ijklr,lrs', JJ.reshape(t.shape[0],d,kappa.shape[0],d,d), dA) 133 | return J_kappa2.reshape((d * t.shape[0], kappa.shape[0], kappa.shape[0])) 134 | 135 | def df2_full(kappa, y0, t): 136 | """ 137 | Evaluate the derivative of the model derivatives at ``kappa`` and at times ``t`` 138 | with initial conditions ``y0``. 139 | 140 | This returns a matrix of the following form: 141 | dy_1(t_1) / dkappa_1 ... dy_1(t_1) / dkappa_s 142 | ... 143 | dy_d(t_1) / dkappa_1 ... dy_d(t_1) / dkappa_s 144 | ... 145 | dy_1(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 146 | ... 147 | dy_d(t_K) / dkappa_1 ... dy_d(t_K) / dkappa_s 148 | """ 149 | A, dA = make_full_A(kappa) 150 | d = A.shape[0] 151 | r = ode(f2) # Look at system.py: this should be the f of the adjoint 152 | r.set_initial_value(np.hstack([y0, np.zeros((d ** 3 + d ** 5, ))])).set_f_params(A) 153 | y_m = [r.y] 154 | for tt in t[1:]: 155 | r.integrate(tt) 156 | y_m.append(r.y[None, :]) 157 | y_m = np.vstack(y_m) 158 | # This is the Jacobian with respect to the full matrix A 159 | J = y_m[:, d:d+d**3].reshape((t.shape[0], d, d, d)) 160 | # Now we apply the chain rule to compute the jacobian wrt kappa 161 | J_kappa = np.einsum('ijkl,klr', J, dA) 162 | 163 | J2 = y_m[:,d+d**3:d+d**3+d**5].reshape((t.shape[0], d, d**2, d**2)) 164 | #print len(J2[0,0,0,:]) 165 | ## Trying something New 166 | JJ = np.zeros((t.shape[0],d,kappa.shape[0],d**2)) 167 | for j in range(t.shape[0]): 168 | for k in range(d): 169 | for i in range(d**2): 170 | JJ[j,k,:,i] = np.einsum('kl,klr',J2[j,k,i,:].reshape(d,d), dA) 171 | J_kappa2 = np.einsum('ijklr,lrs', JJ.reshape(t.shape[0],d,kappa.shape[0],d,d), dA) 172 | return J_kappa2.reshape((d * t.shape[0], kappa.shape[0], kappa.shape[0])) 173 | 174 | def ndf2(kappa0, y0 ,t, h=1e-3): 175 | 176 | y = f(kappa0, y0, t) 177 | kappa = kappa0.copy() 178 | J = np.zeros((y.shape[0], kappa0.shape[0], kappa0.shape[0])) 179 | for i in range(kappa0.shape[0]): 180 | for j in range(kappa0.shape[0]): 181 | kappa[i] += h 182 | kappa[j] += h 183 | py1 = f(kappa, y0, t) 184 | kappa[j] -= 2*h 185 | py2 = f(kappa, y0 ,t) 186 | kappa[j] += 2*h 187 | kappa[i] -= 2*h 188 | py3 = f(kappa, y0, t) 189 | kappa[j] -= 2*h 190 | py4 = f(kappa, y0, t) 191 | kappa[i] += h 192 | kappa[j] += h 193 | J[:, i, j] = (py1 - py2 - py3 + py4) / (4*h**2) 194 | return J 195 | 196 | if __name__ == '__main__': 197 | # Test if what we are doing make sense 198 | import matplotlib.pyplot as plt 199 | # Some kappas to evaluate the model at: 200 | kappa = np.array([0.0216, 0.0292, 0.0219, 0.0021, 0.0048]) 201 | # The observed data (to read the initial conditions): 202 | obs_data = np.loadtxt('obs_data.txt') 203 | # Remember that the column of the data that has all zeros contains X 204 | # which is not observed. This is the 4th column. 205 | # The observed times 206 | t = obs_data[:, 0] 207 | # The intial points 208 | y0 = obs_data[0, 1:] 209 | # Here is how we evaluate the model 210 | y = f(kappa, y0, t) 211 | # Here is how we evaluate the derivatives of the model 212 | ddy = df2(kappa, y0, t) 213 | # Now evaluate the numerical derivatives at kappa 214 | J = ndf2(kappa, y0, t) 215 | for i in range(kappa.shape[0]): 216 | for j in range(kappa.shape[0]): 217 | plt.plot(ddy[:, i, j], 'r', linewidth=2) 218 | plt.plot(J[:, i, j], '--g', linewidth=2) 219 | plt.legend(['Adjoint derivative', 'Numerical Derivative']) 220 | plt.show() 221 | -------------------------------------------------------------------------------- /lectures/demos/catalysis/system.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implementation of a generic adjoint solver for linear dynamical systems. 3 | 4 | Author: 5 | Ilias Bilionis 6 | 7 | Date: 8 | 4/23/2014 9 | """ 10 | 11 | 12 | import numpy as np 13 | from scipy.integrate import ode 14 | 15 | 16 | def f0(t, y, A): 17 | """ 18 | The nomral dynamics. 19 | """ 20 | return np.dot(A, y) 21 | 22 | def f1(t, y, A): 23 | """ 24 | The normal dynamics and of the 1st derivatives wrt A. 25 | """ 26 | d = A.shape[0] 27 | y_0 = y[:d] 28 | f_y_0 = f0(t, y_0, A) 29 | y_1 = y[d:d + d**3].reshape((d, d, d)) 30 | f_y_1 = np.einsum('ij,jkl->ikl', A, y_1) 31 | for i in range(d): 32 | f_y_1[i, i, :] += y_0 33 | return np.hstack([f_y_0, f_y_1.flatten()]) 34 | 35 | def f2(t, y, A): 36 | """ 37 | The normal dynamics and of the 2st derivatives wrt A. 38 | """ 39 | d = A.shape[0] 40 | y_0 = y[:d] 41 | f_y_0 = f0(t, y_0, A) 42 | y_1 = y[d:d + d**3].reshape((d, d, d)) 43 | f_y_1 = np.einsum('ij,jkl->ikl', A, y_1) 44 | for i in range(d): 45 | f_y_1[i, i, :] += y_0 46 | #return np.hstack([f_y_0, f_y_1.flatten()]) 47 | y_2 = y[d+d**3:d+d**3+d**5].reshape((d,d**2,d**2)) 48 | f_y_2 = np.einsum('ij,jkl->ikl',A, y_2) 49 | for i in range(d): 50 | for j in range(i*d,(i+1)*d): 51 | f_y_2[i,j,:] += y_1[j-i*d,:,:].flatten() 52 | for i in range(d): 53 | for j in range(i*d,(i+1)*d): 54 | f_y_2[i,:,j] += y_1[j-i*d,:,:].flatten() 55 | return np.hstack([f_y_0, f_y_1.flatten(), f_y_2.flatten()]) 56 | 57 | 58 | def loss(A, t, y): 59 | r = ode(f0) 60 | r.set_initial_value(y[0, :], t[0]).set_f_params(A) 61 | y_m = np.zeros((t.shape[0], r.y.shape[0])) 62 | y_m[0, :] = r.y 63 | for i in range(1, t.shape[0]): 64 | r.integrate(t[i]) 65 | y_m[i, :] = r.y 66 | tmp = y_m - y 67 | return np.dot([1, 1, 0, 1, 1, 1], np.linalg.norm(tmp, axis=0) ** 2) 68 | 69 | 70 | def dloss(A, t, y): 71 | r = ode(f1) 72 | d = A.shape[0] 73 | r.set_initial_value(np.hstack([y[0, :], np.zeros((d ** 3, ))])).set_f_params(A) 74 | y_m = np.zeros((t.shape[0], r.y.shape[0])) 75 | y_m[0, :] = r.y 76 | for i in range(1, t.shape[0]): 77 | r.integrate(t[i]) 78 | y_m[i, :] = r.y 79 | y_m0 = y_m[:, :d] 80 | y_m1 = y_m[:, d:d + d**3].reshape((t.shape[0], d, d, d)) 81 | tmp = y_m0 - y 82 | return 2. * np.einsum('ij,j,ijkl->kl', tmp, np.array([1, 1, 0, 1, 1, 1]), y_m1) 83 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Initialize the diffusion forward model module. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 6/12/2014 9 | 10 | """ 11 | 12 | from ._forward_diffusion import * 13 | from ._forward_diffusion_left import * 14 | from ._forward_diffusion_upperleft import * 15 | from ._forward_diffusion_centers import * 16 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/_forward_diffusion.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem with one source. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | Date: 8 | 06/12/2014 9 | 3/13/2018 10 | 11 | """ 12 | 13 | 14 | import numpy as np 15 | import fipy as fp 16 | from .transport_model import * 17 | import sys 18 | from .. import Model 19 | from .. import view_as_column 20 | 21 | 22 | class ContaminantTransportModel(Model): 23 | """ 24 | A class representing the forward model of the contaminta transport problem 25 | using a diffusion equation. 26 | """ 27 | 28 | def __init__(self, name='Transport model'): 29 | """ 30 | Initialize the object 31 | """ 32 | super(ContaminantTransportModel, self).__init__(2, 16, name=name) 33 | 34 | def _eval(self, xs): 35 | """ 36 | Solves the diffusion equations for u and its derivatives for a given 37 | source location xs. 38 | """ 39 | xs = view_as_column(xs) 40 | assert xs.shape[0] == 2 41 | nx = 25 42 | ny = nx 43 | dx = 0.04 44 | dy = dx 45 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 46 | u = f(xs[:,0], mesh) 47 | du1 = df(xs[:,0], mesh, 1) 48 | du2 = df(xs[:,0], mesh, 2) 49 | d2u11 = df2(xs[:,0], mesh, 1, 1) 50 | d2u22 = df2(xs[:,0], mesh, 1, 1) 51 | d2u12 = df2(xs[:,0], mesh, 1, 2) 52 | dU = np.hstack([view_as_column(du1), view_as_column(du2)]) 53 | d2U = np.hstack([view_as_column(d2u11), view_as_column(d2u12), view_as_column(d2u12), view_as_column(d2u22)]) 54 | d2U = d2U.reshape((d2U.shape[0], 2, 2)) 55 | state = {} 56 | state['f'] = u #view_as_column(u) 57 | state['f_grad'] = dU 58 | state['f_grad_2'] = d2U 59 | return state 60 | 61 | def _eval_u(self, xs): 62 | """ 63 | Solves only the diffusion equation for u for a given 64 | source location xs. 65 | """ 66 | xs = view_as_column(xs) 67 | assert xs.shape[0] == 2 68 | nx = 25 69 | ny = nx 70 | dx = 0.04 71 | dy = dx 72 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 73 | u = f(xs[:,0], mesh) 74 | return u 75 | 76 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/_forward_diffusion.pyo: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/demos/diffusion/_forward_diffusion.pyo -------------------------------------------------------------------------------- /lectures/demos/diffusion/_forward_diffusion_centers.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem with one sources. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | 8 | Date: 9 | 09/8/2014 10 | 03/13/2018 11 | 03/09/2020 12 | """ 13 | 14 | 15 | import numpy as np 16 | import fipy as fp 17 | from . transport_model_centers import * 18 | import sys 19 | from .. import Model 20 | from .. import view_as_column 21 | 22 | 23 | class ContaminantTransportModelCenter(Model): 24 | """ 25 | A class representing the forward model of the contaminant transport problem 26 | using a diffusion equation. 27 | """ 28 | 29 | def __init__(self, name='Transport model'): 30 | """ 31 | Initialize the object 32 | """ 33 | super(ContaminantTransportModelCenter, self).__init__(2, 8, name=name) 34 | 35 | def _eval(self, xs): 36 | """ 37 | Solves the diffusion equations for u and its derivatives for a given 38 | source location xs. 39 | """ 40 | xs = view_as_column(xs) 41 | assert xs.shape[0] == 2 42 | nx = 25 43 | ny = nx 44 | dx = 0.04 45 | dy = dx 46 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 47 | u = f(xs[:,0], mesh) 48 | du1 = df(xs[:,0], mesh, 1) 49 | du2 = df(xs[:,0], mesh, 2) 50 | d2u11 = df2(xs[:,0], mesh, 1, 1) 51 | d2u22 = df2(xs[:,0], mesh, 1, 1) 52 | d2u12 = df2(xs[:,0], mesh, 1, 2) 53 | dU = np.hstack([view_as_column(du1), view_as_column(du2)]) 54 | d2U = np.hstack([view_as_column(d2u11), view_as_column(d2u12), view_as_column(d2u12), view_as_column(d2u22)]) 55 | d2U = d2U.reshape((d2U.shape[0], 2, 2)) 56 | state = {} 57 | state['f'] = u #view_as_column(u) 58 | state['f_grad'] = dU 59 | state['f_grad_2'] = d2U 60 | return state 61 | 62 | def _eval_u(self, xs): 63 | """ 64 | Solves only the diffusion equation for u for a given 65 | source location xs. 66 | """ 67 | xs = view_as_column(xs) 68 | assert xs.shape[0] == 2 69 | xs_1 = xs[:2,0] 70 | xs_2 = xs[2:,0] 71 | nx = 25 72 | ny = nx 73 | dx = 0.04 74 | dy = dx 75 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 76 | u = f(xs[:,0], mesh) 77 | return u 78 | 79 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/_forward_diffusion_left.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem with two sources. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | 8 | Date: 9 | 06/12/2014 10 | 3/18/2018 11 | 12 | """ 13 | 14 | 15 | import numpy as np 16 | import fipy as fp 17 | from .transport_model_left_corner import * 18 | import sys 19 | sys.path.insert(0,'../') 20 | from .. import Model 21 | from .. import view_as_column 22 | 23 | class ContaminantTransportModelLeft(Model): 24 | """ 25 | A class representing the forward model of the contaminant transport problem 26 | using a diffusion equation. 27 | """ 28 | 29 | def __init__(self, name='Transport model'): 30 | """ 31 | Initialize the object 32 | """ 33 | super(ContaminantTransportModelLeft, self).__init__(2, 8, name=name) 34 | 35 | def _eval(self, xs): 36 | """ 37 | Solves the diffusion equations for u and its derivatives for a given 38 | source location xs. 39 | """ 40 | xs = view_as_column(xs) 41 | assert xs.shape[0] == 2 42 | nx = 25 43 | ny = nx 44 | dx = 0.04 45 | dy = dx 46 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 47 | u = f(xs[:,0], mesh) 48 | du1 = df(xs[:,0], mesh, 1) 49 | du2 = df(xs[:,0], mesh, 2) 50 | d2u11 = df2(xs[:,0], mesh, 1, 1) 51 | d2u22 = df2(xs[:,0], mesh, 1, 1) 52 | d2u12 = df2(xs[:,0], mesh, 1, 2) 53 | dU = np.hstack([view_as_column(du1), view_as_column(du2)]) 54 | d2U = np.hstack([view_as_column(d2u11), view_as_column(d2u12), view_as_column(d2u12), view_as_column(d2u22)]) 55 | d2U = d2U.reshape((d2U.shape[0], 2, 2)) 56 | state = {} 57 | state['f'] = u #view_as_column(u) 58 | state['f_grad'] = dU 59 | state['f_grad_2'] = d2U 60 | return state 61 | 62 | def _eval_u(self, xs): 63 | """ 64 | Solves only the diffusion equation for u for a given 65 | source location xs. 66 | """ 67 | xs = view_as_column(xs) 68 | assert xs.shape[0] == 2 69 | xs_1 = xs[:2,0] 70 | xs_2 = xs[2:,0] 71 | nx = 25 72 | ny = nx 73 | dx = 0.04 74 | dy = dx 75 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 76 | u = f(xs[:,0], mesh) 77 | return u 78 | 79 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/_forward_diffusion_upperleft.py: -------------------------------------------------------------------------------- 1 | """ 2 | Forward problem with two sources. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | Ilias Bilionis 7 | 8 | Date: 9 | 06/16/2014 10 | 03/18/2018 11 | 12 | """ 13 | 14 | 15 | import numpy as np 16 | import fipy as fp 17 | from .transport_model_upperleft import * 18 | import sys 19 | sys.path.insert(0,'../') 20 | from .. import Model 21 | from .. import view_as_column 22 | 23 | class ContaminantTransportModelUpperLeft(Model): 24 | """ 25 | A class representing the forward model of the contaminant transport problem 26 | using a diffusion equation. 27 | """ 28 | 29 | def __init__(self, name='Transport model'): 30 | """ 31 | Initialize the object 32 | """ 33 | super(ContaminantTransportModelUpperLeft, self).__init__(2, 4, name=name) 34 | 35 | def _eval(self, xs): 36 | """ 37 | Solves the diffusion equations for u and its derivatives for a given 38 | source location xs. 39 | """ 40 | xs = view_as_column(xs) 41 | assert xs.shape[0] == 2 42 | nx = 25 43 | ny = nx 44 | dx = 0.04 45 | dy = dx 46 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 47 | u = f(xs[:,0], mesh) 48 | du1 = df(xs[:,0], mesh, 1) 49 | du2 = df(xs[:,0], mesh, 2) 50 | d2u11 = df2(xs[:,0], mesh, 1, 1) 51 | d2u22 = df2(xs[:,0], mesh, 1, 1) 52 | d2u12 = df2(xs[:,0], mesh, 1, 2) 53 | dU = np.hstack([view_as_column(du1), view_as_column(du2)]) 54 | d2U = np.hstack([view_as_column(d2u11), view_as_column(d2u12), view_as_column(d2u12), view_as_column(d2u22)]) 55 | d2U = d2U.reshape((d2U.shape[0], 2, 2)) 56 | state = {} 57 | state['f'] = u #view_as_column(u) 58 | state['f_grad'] = dU 59 | state['f_grad_2'] = d2U 60 | return state 61 | 62 | def _eval_u(self, xs): 63 | """ 64 | Solves only the diffusion equation for u for a given 65 | source location xs. 66 | """ 67 | xs = view_as_column(xs) 68 | assert xs.shape[0] == 2 69 | xs_1 = xs[:2,0] 70 | xs_2 = xs[2:,0] 71 | nx = 25 72 | ny = nx 73 | dx = 0.04 74 | dy = dx 75 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 76 | u = f(xs[:,0], mesh) 77 | return u 78 | 79 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/generate_data.py: -------------------------------------------------------------------------------- 1 | """ 2 | Generate data for the diffusion forward model. 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 6/12/2014 9 | 10 | """ 11 | 12 | import numpy as np 13 | import fipy as fp 14 | import os 15 | import matplotlib.pyplot as plt 16 | 17 | 18 | # Make the source 19 | 20 | nx = 101 21 | ny = nx 22 | dx = 1./101 23 | dy = dx 24 | rho = 0.05 25 | q0 = 1. / (np.pi * rho ** 2) 26 | T = 0.3 27 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 28 | xs_1 = np.array([0.91, 0.23]) 29 | #xs_2 = np.array([0.89, 0.75]) 30 | time = fp.Variable() 31 | sourceTerm_1 = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 32 | #sourceTerm_2 = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 33 | for i in range(sourceTerm_1().shape[0]): 34 | sourceTerm_1()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs_1[0]) ** 2 35 | + (mesh.cellCenters[1]()[i] - xs_1[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 36 | #sourceTerm_2()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs_2[0]) ** 2 37 | # + (mesh.cellCenters[1]()[i] - xs_2[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 38 | 39 | # The equation 40 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=1.) + sourceTerm_1# + sourceTerm_2 41 | 42 | # The solution variable 43 | phi = fp.CellVariable(name = "Concentration", mesh=mesh, value=0.) 44 | 45 | 46 | #if __name__ == '__main__': 47 | # viewer = fp.Viewer(vars=phi, datamin=0., datamax=3.) 48 | # viewer.plot() 49 | 50 | x = np.arange(0,101.)/101 51 | y = x 52 | 53 | data = [] 54 | dt = 0.005 55 | steps = 60 56 | for step in range(steps): 57 | time.setValue(time() + dt) 58 | eq.solve(var=phi, dt=dt) 59 | # if __name__ == '__main__': 60 | # viewer.plot() 61 | if step == 14 or step == 29 or step == 44 or step == 59: 62 | dc = phi()[50] 63 | #dr = phi()[109] 64 | uc = phi()[10150] 65 | #ur = phi()[12099] 66 | #data = np.hstack([data, np.array([dl, dr, ul, ur])]) 67 | data = np.hstack([data, np.array([dc, uc])]) 68 | 69 | fig = plt.figure() 70 | plt.contourf(x, y, phi().reshape(101,101),200) 71 | plt.colorbar() 72 | # fig.suptitle('Concentration at t = ' + str(time())) 73 | plt.xlabel('x') 74 | plt.ylabel('y') 75 | #png_file = os.path.join('figures', 'concentration'+'t.png') 76 | #plt.savefig(png_file) 77 | plt.show() 78 | 79 | 80 | 81 | #if __name__ == '__main__': 82 | # raw_input("Transient diffusion with source term. Press to proceed") 83 | 84 | #np.save('data_concentrations_upperlowercenters.npy',data) -------------------------------------------------------------------------------- /lectures/demos/diffusion/transport_model.py: -------------------------------------------------------------------------------- 1 | """ 2 | Solve numerically the diffusion equation using the fipy package 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 6/12/2014 9 | """ 10 | 11 | import numpy as np 12 | import fipy as fp 13 | import matplotlib.pyplot as plt 14 | 15 | 16 | 17 | def make_source(xs, mesh, time): 18 | """ 19 | Makes the source term of the diffusion equation 20 | """ 21 | #assert xs.shape[0] == 2 22 | sourceTerm = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 23 | rho = 0.05 24 | q0 = 1 / (np.pi * rho ** 2) 25 | T = 0.3 26 | for i in range(sourceTerm().shape[0]): 27 | sourceTerm()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs[0]) ** 2 28 | + (mesh.cellCenters[1]()[i] - xs[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 29 | return sourceTerm 30 | 31 | def make_source_der(xs, mesh, time, i): 32 | """ 33 | Make the source term of the diffusion equation for the derivatives of u. 34 | """ 35 | #assert xs.shape[0] == 2 36 | rho = 0.05 37 | sourceTerm = make_source(xs, mesh, time) 38 | for j in range(sourceTerm().shape[0]): 39 | sourceTerm()[j] = sourceTerm()[j] * (mesh.cellCenters[i-1]()[j] - xs[i-1]) / rho ** 2 40 | return sourceTerm 41 | 42 | def make_source_der_2(xs, mesh, time, i, j): 43 | """ 44 | Make the source term of the diffusion equation for the 2nd derivatives of u. 45 | """ 46 | #assert xs.shape[0] == 2 47 | rho = 0.05 48 | sourceTerm = make_source(xs, mesh, time) 49 | for k in range(sourceTerm().shape[0]): 50 | sourceTerm()[k] = sourceTerm()[k] * ( (mesh.cellCenters[i-1]()[k] - xs[i-1]) * (mesh.cellCenters[j-1]()[k] - xs[j-1]) / rho **2 51 | - (i == j)) / rho **2 52 | return sourceTerm 53 | 54 | def f(xs, mesh): 55 | """ 56 | Evaluate the model for the concentration at the four corners of the domain 57 | at times ``t``. 58 | 59 | It returns a flatten version of the system, i.e.: 60 | y_1(t_1) 61 | ... 62 | y_4(t_1) 63 | ... 64 | y_1(t_4) 65 | ... 66 | y_4(t_4) 67 | """ 68 | time = fp.Variable() 69 | q = make_source(xs, mesh, time) 70 | D = 1. 71 | # Define the equation 72 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q 73 | # Boundary conditions 74 | 75 | # The solution variable 76 | phi = fp.CellVariable(name = "Concentration", mesh=mesh, value=0.) 77 | 78 | # Solve 79 | dt = 0.005 80 | steps = 60 81 | U_sol = [] 82 | for step in range(steps): 83 | eq.solve(var=phi, dt=dt) 84 | if step == 14 or step == 29 or step == 44 or step == 59: 85 | dl = phi()[0] 86 | dr = phi()[24] 87 | ul = phi()[600] 88 | ur = phi()[624] 89 | U_sol = np.hstack([U_sol, np.array([dl, dr, ul, ur])]) 90 | 91 | return U_sol 92 | 93 | def df(xs, mesh, i): 94 | """ 95 | Evaluate the model for the derivatives at the four corners of the domain 96 | at times ``t``. 97 | 98 | It returns a flatten version of the system, i.e.: 99 | y_1(t_1) 100 | ... 101 | y_4(t_1) 102 | ... 103 | y_1(t_4) 104 | ... 105 | y_4(t_4) 106 | """ 107 | assert i == 1 or i == 2 108 | time = fp.Variable() 109 | q0 = make_source_der(xs, mesh, time, i) 110 | D = 1. 111 | # Define the equation 112 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 113 | # Boundary conditions 114 | 115 | # The solution variable 116 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 117 | 118 | # Solve 119 | dt = 0.005 120 | steps = 60 121 | dU = [] 122 | for step in range(steps): 123 | eq.solve(var=phi, dt=dt) 124 | if step == 14 or step == 29 or step == 44 or step == 59: 125 | dl = phi()[0] 126 | dr = phi()[24] 127 | ul = phi()[600] 128 | ur = phi()[624] 129 | dU = np.hstack([dU, np.array([dl, dr, ul, ur])]) 130 | 131 | return dU 132 | 133 | def df2(xs, mesh, i, j): 134 | """ 135 | Evaluate the model for the 2nd derivatives at the four corners of the domain 136 | at times ``t``. 137 | 138 | It returns a flatten version of the system, i.e.: 139 | y_1(t_1) 140 | ... 141 | y_4(t_1) 142 | ... 143 | y_1(t_4) 144 | ... 145 | y_4(t_4) 146 | """ 147 | assert i == 1 or i == 2 148 | assert j == 1 or j == 2 149 | nx = 25 150 | ny = nx 151 | dx = 0.04 152 | dy = dx 153 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 154 | time = fp.Variable() 155 | q0 = make_source_der_2(xs, mesh, time, i, j) 156 | D = 1. 157 | # Define the equation 158 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 159 | # Boundary conditions 160 | 161 | # The solution variable 162 | phi = fp.CellVariable(name = "Concentration", mesh=mesh, value=0.) 163 | 164 | # Solve 165 | dt = 0.005 166 | steps = 60 167 | d2U = [] 168 | for step in range(steps): 169 | eq.solve(var=phi, dt=dt) 170 | if step == 14 or step == 29 or step == 44 or step == 59: 171 | dl = phi()[0] 172 | dr = phi()[24] 173 | ul = phi()[600] 174 | ur = phi()[624] 175 | d2U = np.hstack([d2U, np.array([dl, dr, ul, ur])]) 176 | 177 | return d2U 178 | -------------------------------------------------------------------------------- /lectures/demos/diffusion/transport_model.pyo: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/demos/diffusion/transport_model.pyo -------------------------------------------------------------------------------- /lectures/demos/diffusion/transport_model_centers.py: -------------------------------------------------------------------------------- 1 | """ 2 | Solve numerically the diffusion equation using the fipy package 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 9/8/2014 9 | """ 10 | 11 | import numpy as np 12 | import fipy as fp 13 | import matplotlib.pyplot as plt 14 | 15 | 16 | 17 | def make_source(xs, mesh, time): 18 | """ 19 | Makes the source term of the diffusion equation 20 | """ 21 | #assert xs.shape[0] == 2 22 | sourceTerm = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 23 | rho = 0.05 24 | q0 = 1 / (np.pi * rho ** 2) 25 | T = 0.3 26 | for i in range(sourceTerm().shape[0]): 27 | sourceTerm()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs[0]) ** 2 28 | + (mesh.cellCenters[1]()[i] - xs[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 29 | return sourceTerm 30 | 31 | def make_source_der(xs, mesh, time, i): 32 | """ 33 | Make the source term of the diffusion equation for the derivatives of u. 34 | """ 35 | #assert xs.shape[0] == 2 36 | rho = 0.05 37 | sourceTerm = make_source(xs, mesh, time) 38 | for j in range(sourceTerm().shape[0]): 39 | sourceTerm()[j] = sourceTerm()[j] * (mesh.cellCenters[i-1]()[j] - xs[i-1]) / rho ** 2 40 | return sourceTerm 41 | 42 | def make_source_der_2(xs, mesh, time, i, j): 43 | """ 44 | Make the source term of the diffusion equation for the 2nd derivatives of u. 45 | """ 46 | #assert xs.shape[0] == 2 47 | rho = 0.05 48 | sourceTerm = make_source(xs, mesh, time) 49 | for k in range(sourceTerm().shape[0]): 50 | sourceTerm()[k] = sourceTerm()[k] * ( (mesh.cellCenters[i-1]()[k] - xs[i-1]) * (mesh.cellCenters[j-1]()[k] - xs[j-1]) / rho **2 51 | - (i == j)) / rho **2 52 | return sourceTerm 53 | 54 | def f(xs, mesh): 55 | """ 56 | Evaluate the model for the concentration at the four corners of the domain 57 | at times ``t``. 58 | 59 | It returns a flatten version of the system, i.e.: 60 | y_1(t_1) 61 | ... 62 | y_4(t_1) 63 | ... 64 | y_1(t_4) 65 | ... 66 | y_4(t_4) 67 | """ 68 | time = fp.Variable() 69 | q = make_source(xs, mesh, time) 70 | D = 1. 71 | # Define the equation 72 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q 73 | # Boundary conditions 74 | 75 | # The solution variable 76 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 77 | 78 | # Solve 79 | dt = 0.005 80 | steps = 60 81 | U_sol = [] 82 | for step in range(steps): 83 | eq.solve(var=phi, dt=dt) 84 | if step == 14 or step == 29 or step == 44 or step == 59: 85 | dc = phi()[12] 86 | #dr = phi()[24] 87 | uc = phi()[612] 88 | #ur = phi()[624] 89 | #U_sol = np.hstack([U_sol, np.array([dl, dr, ul, ur])]) 90 | U_sol = np.hstack([U_sol, np.array([dc, uc])]) 91 | 92 | return U_sol 93 | 94 | def df(xs, mesh, i): 95 | """ 96 | Evaluate the model for the derivatives at the four corners of the domain 97 | at times ``t``. 98 | 99 | It returns a flatten version of the system, i.e.: 100 | y_1(t_1) 101 | ... 102 | y_4(t_1) 103 | ... 104 | y_1(t_4) 105 | ... 106 | y_4(t_4) 107 | """ 108 | assert i == 1 or i == 2 109 | time = fp.Variable() 110 | q0 = make_source_der(xs, mesh, time, i) 111 | D = 1. 112 | # Define the equation 113 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 114 | # Boundary conditions 115 | 116 | # The solution variable 117 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 118 | 119 | # Solve 120 | dt = 0.005 121 | steps = 60 122 | dU = [] 123 | for step in range(steps): 124 | eq.solve(var=phi, dt=dt) 125 | if step == 14 or step == 29 or step == 44 or step == 59: 126 | dc = phi()[12] 127 | #dr = phi()[24] 128 | uc = phi()[612] 129 | #ur = phi()[624] 130 | #dU = np.hstack([dU, np.array([dl, dr, ul, ur])]) 131 | dU = np.hstack([dU, np.array([dc, uc])]) 132 | 133 | return dU 134 | 135 | def df2(xs, mesh, i, j): 136 | """ 137 | Evaluate the model for the 2nd derivatives at the four corners of the domain 138 | at times ``t``. 139 | 140 | It returns a flatten version of the system, i.e.: 141 | y_1(t_1) 142 | ... 143 | y_4(t_1) 144 | ... 145 | y_1(t_4) 146 | ... 147 | y_4(t_4) 148 | """ 149 | assert i == 1 or i == 2 150 | assert j == 1 or j == 2 151 | nx = 25 152 | ny = nx 153 | dx = 0.04 154 | dy = dx 155 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 156 | time = fp.Variable() 157 | q0 = make_source_der_2(xs, mesh, time, i, j) 158 | D = 1. 159 | # Define the equation 160 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 161 | # Boundary conditions 162 | 163 | # The solution variable 164 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 165 | 166 | # Solve 167 | dt = 0.005 168 | steps = 60 169 | d2U = [] 170 | for step in range(steps): 171 | eq.solve(var=phi, dt=dt) 172 | if step == 14 or step == 29 or step == 44 or step == 59: 173 | dc = phi()[12] 174 | #dr = phi()[24] 175 | uc = phi()[612] 176 | #ur = phi()[624] 177 | #d2U = np.hstack([d2U, np.array([dl, dr, ul, ur])]) 178 | d2U = np.hstack([d2U, np.array([dc, uc])]) 179 | 180 | return d2U -------------------------------------------------------------------------------- /lectures/demos/diffusion/transport_model_left_corner.py: -------------------------------------------------------------------------------- 1 | """ 2 | Solve numerically the diffusion equation using the fipy package 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 6/12/2014 9 | """ 10 | 11 | import numpy as np 12 | import fipy as fp 13 | import matplotlib.pyplot as plt 14 | 15 | 16 | 17 | def make_source(xs, mesh, time): 18 | """ 19 | Makes the source term of the diffusion equation 20 | """ 21 | #assert xs.shape[0] == 2 22 | sourceTerm = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 23 | rho = 0.05 24 | q0 = 1 / (np.pi * rho ** 2) 25 | T = 0.3 26 | for i in range(sourceTerm().shape[0]): 27 | sourceTerm()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs[0]) ** 2 28 | + (mesh.cellCenters[1]()[i] - xs[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 29 | return sourceTerm 30 | 31 | def make_source_der(xs, mesh, time, i): 32 | """ 33 | Make the source term of the diffusion equation for the derivatives of u. 34 | """ 35 | #assert xs.shape[0] == 2 36 | rho = 0.05 37 | sourceTerm = make_source(xs, mesh, time) 38 | for j in range(sourceTerm().shape[0]): 39 | sourceTerm()[j] = sourceTerm()[j] * (mesh.cellCenters[i-1]()[j] - xs[i-1]) / rho ** 2 40 | return sourceTerm 41 | 42 | def make_source_der_2(xs, mesh, time, i, j): 43 | """ 44 | Make the source term of the diffusion equation for the 2nd derivatives of u. 45 | """ 46 | #assert xs.shape[0] == 2 47 | rho = 0.05 48 | sourceTerm = make_source(xs, mesh, time) 49 | for k in range(sourceTerm().shape[0]): 50 | sourceTerm()[k] = sourceTerm()[k] * ( (mesh.cellCenters[i-1]()[k] - xs[i-1]) * (mesh.cellCenters[j-1]()[k] - xs[j-1]) / rho **2 51 | - (i == j)) / rho **2 52 | return sourceTerm 53 | 54 | def f(xs, mesh): 55 | """ 56 | Evaluate the model for the concentration at the four corners of the domain 57 | at times ``t``. 58 | 59 | It returns a flatten version of the system, i.e.: 60 | y_1(t_1) 61 | ... 62 | y_4(t_1) 63 | ... 64 | y_1(t_4) 65 | ... 66 | y_4(t_4) 67 | """ 68 | time = fp.Variable() 69 | q = make_source(xs, mesh, time) 70 | D = 1. 71 | # Define the equation 72 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q 73 | # Boundary conditions 74 | 75 | # The solution variable 76 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 77 | 78 | # Solve 79 | dt = 0.005 80 | steps = 60 81 | U_sol = [] 82 | for step in range(steps): 83 | eq.solve(var=phi, dt=dt) 84 | if step == 14 or step == 29 or step == 44 or step == 59: 85 | dl = phi()[0] 86 | #dr = phi()[24] 87 | ul = phi()[600] 88 | #ur = phi()[624] 89 | #U_sol = np.hstack([U_sol, np.array([dl, dr, ul, ur])]) 90 | U_sol = np.hstack([U_sol, np.array([dl, ul])]) 91 | 92 | return U_sol 93 | 94 | def df(xs, mesh, i): 95 | """ 96 | Evaluate the model for the derivatives at the four corners of the domain 97 | at times ``t``. 98 | 99 | It returns a flatten version of the system, i.e.: 100 | y_1(t_1) 101 | ... 102 | y_4(t_1) 103 | ... 104 | y_1(t_4) 105 | ... 106 | y_4(t_4) 107 | """ 108 | assert i == 1 or i == 2 109 | time = fp.Variable() 110 | q0 = make_source_der(xs, mesh, time, i) 111 | D = 1. 112 | # Define the equation 113 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 114 | # Boundary conditions 115 | 116 | # The solution variable 117 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 118 | 119 | # Solve 120 | dt = 0.005 121 | steps = 60 122 | dU = [] 123 | for step in range(steps): 124 | eq.solve(var=phi, dt=dt) 125 | if step == 14 or step == 29 or step == 44 or step == 59: 126 | dl = phi()[0] 127 | #dr = phi()[24] 128 | ul = phi()[600] 129 | #ur = phi()[624] 130 | #dU = np.hstack([dU, np.array([dl, dr, ul, ur])]) 131 | dU = np.hstack([dU, np.array([dl, ul])]) 132 | 133 | return dU 134 | 135 | def df2(xs, mesh, i, j): 136 | """ 137 | Evaluate the model for the 2nd derivatives at the four corners of the domain 138 | at times ``t``. 139 | 140 | It returns a flatten version of the system, i.e.: 141 | y_1(t_1) 142 | ... 143 | y_4(t_1) 144 | ... 145 | y_1(t_4) 146 | ... 147 | y_4(t_4) 148 | """ 149 | assert i == 1 or i == 2 150 | assert j == 1 or j == 2 151 | nx = 25 152 | ny = nx 153 | dx = 0.04 154 | dy = dx 155 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 156 | time = fp.Variable() 157 | q0 = make_source_der_2(xs, mesh, time, i, j) 158 | D = 1. 159 | # Define the equation 160 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 161 | # Boundary conditions 162 | 163 | # The solution variable 164 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 165 | 166 | # Solve 167 | dt = 0.005 168 | steps = 60 169 | d2U = [] 170 | for step in range(steps): 171 | eq.solve(var=phi, dt=dt) 172 | if step == 14 or step == 29 or step == 44 or step == 59: 173 | dl = phi()[0] 174 | #dr = phi()[24] 175 | ul = phi()[600] 176 | #ur = phi()[624] 177 | #d2U = np.hstack([d2U, np.array([dl, dr, ul, ur])]) 178 | d2U = np.hstack([d2U, np.array([dl, ul])]) 179 | 180 | return d2U -------------------------------------------------------------------------------- /lectures/demos/diffusion/transport_model_upperleft.py: -------------------------------------------------------------------------------- 1 | """ 2 | Solve numerically the diffusion equation using the fipy package 3 | 4 | Author: 5 | Panagiotis Tsilifis 6 | 7 | Date: 8 | 6/16/2014 9 | """ 10 | 11 | import numpy as np 12 | import fipy as fp 13 | import matplotlib.pyplot as plt 14 | 15 | 16 | 17 | def make_source(xs, mesh, time): 18 | """ 19 | Makes the source term of the diffusion equation 20 | """ 21 | #assert xs.shape[0] == 2 22 | sourceTerm = fp.CellVariable(name = "Source term", mesh=mesh, value = 0.) 23 | rho = 0.05 24 | q0 = 1 / (np.pi * rho ** 2) 25 | T = 0.3 26 | for i in range(sourceTerm().shape[0]): 27 | sourceTerm()[i] = q0 * np.exp( - ((mesh.cellCenters[0]()[i] - xs[0]) ** 2 28 | + (mesh.cellCenters[1]()[i] - xs[1]) ** 2 ) / (2 * rho **2)) * (time() < T) 29 | return sourceTerm 30 | 31 | def make_source_der(xs, mesh, time, i): 32 | """ 33 | Make the source term of the diffusion equation for the derivatives of u. 34 | """ 35 | #assert xs.shape[0] == 2 36 | rho = 0.05 37 | sourceTerm = make_source(xs, mesh, time) 38 | for j in range(sourceTerm().shape[0]): 39 | sourceTerm()[j] = sourceTerm()[j] * (mesh.cellCenters[i-1]()[j] - xs[i-1]) / rho ** 2 40 | return sourceTerm 41 | 42 | def make_source_der_2(xs, mesh, time, i, j): 43 | """ 44 | Make the source term of the diffusion equation for the 2nd derivatives of u. 45 | """ 46 | #assert xs.shape[0] == 2 47 | rho = 0.05 48 | sourceTerm = make_source(xs, mesh, time) 49 | for k in range(sourceTerm().shape[0]): 50 | sourceTerm()[k] = sourceTerm()[k] * ( (mesh.cellCenters[i-1]()[k] - xs[i-1]) * (mesh.cellCenters[j-1]()[k] - xs[j-1]) / rho **2 51 | - (i == j)) / rho **2 52 | return sourceTerm 53 | 54 | def f(xs, mesh): 55 | """ 56 | Evaluate the model for the concentration at the four corners of the domain 57 | at times ``t``. 58 | 59 | It returns a flatten version of the system, i.e.: 60 | y_1(t_1) 61 | ... 62 | y_4(t_1) 63 | ... 64 | y_1(t_4) 65 | ... 66 | y_4(t_4) 67 | """ 68 | time = fp.Variable() 69 | q = make_source(xs, mesh, time) 70 | D = 1. 71 | # Define the equation 72 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q 73 | # Boundary conditions 74 | 75 | # The solution variable 76 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 77 | 78 | # Solve 79 | dt = 0.005 80 | steps = 60 81 | U_sol = [] 82 | for step in range(steps): 83 | eq.solve(var=phi, dt=dt) 84 | if step == 14 or step == 29 or step == 44 or step == 59: 85 | #dl = phi()[0] 86 | #dr = phi()[24] 87 | ul = phi()[600] 88 | #ur = phi()[624] 89 | #U_sol = np.hstack([U_sol, np.array([dl, dr, ul, ur])]) 90 | U_sol = np.hstack([U_sol, np.array([ul])]) 91 | 92 | return U_sol 93 | 94 | def df(xs, mesh, i): 95 | """ 96 | Evaluate the model for the derivatives at the four corners of the domain 97 | at times ``t``. 98 | 99 | It returns a flatten version of the system, i.e.: 100 | y_1(t_1) 101 | ... 102 | y_4(t_1) 103 | ... 104 | y_1(t_4) 105 | ... 106 | y_4(t_4) 107 | """ 108 | assert i == 1 or i == 2 109 | time = fp.Variable() 110 | q0 = make_source_der(xs, mesh, time, i) 111 | D = 1. 112 | # Define the equation 113 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 114 | # Boundary conditions 115 | 116 | # The solution variable 117 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 118 | 119 | # Solve 120 | dt = 0.005 121 | steps = 60 122 | dU = [] 123 | for step in range(steps): 124 | eq.solve(var=phi, dt=dt) 125 | if step == 14 or step == 29 or step == 44 or step == 59: 126 | #dl = phi()[0] 127 | #dr = phi()[24] 128 | ul = phi()[600] 129 | #ur = phi()[624] 130 | #dU = np.hstack([dU, np.array([dl, dr, ul, ur])]) 131 | dU = np.hstack([dU, np.array([ul])]) 132 | 133 | return dU 134 | 135 | def df2(xs, mesh, i, j): 136 | """ 137 | Evaluate the model for the 2nd derivatives at the four corners of the domain 138 | at times ``t``. 139 | 140 | It returns a flatten version of the system, i.e.: 141 | y_1(t_1) 142 | ... 143 | y_4(t_1) 144 | ... 145 | y_1(t_4) 146 | ... 147 | y_4(t_4) 148 | """ 149 | assert i == 1 or i == 2 150 | assert j == 1 or j == 2 151 | nx = 25 152 | ny = nx 153 | dx = 0.04 154 | dy = dx 155 | mesh = fp.Grid2D(dx=dx, dy=dy, nx=nx, ny=ny) 156 | time = fp.Variable() 157 | q0 = make_source_der_2(xs, mesh, time, i, j) 158 | D = 1. 159 | # Define the equation 160 | eq = fp.TransientTerm() == fp.DiffusionTerm(coeff=D) + q0 161 | # Boundary conditions 162 | 163 | # The solution variable 164 | phi = fp.CellVariable(name = "Concentraion", mesh=mesh, value=0.) 165 | 166 | # Solve 167 | dt = 0.005 168 | steps = 60 169 | d2U = [] 170 | for step in range(steps): 171 | eq.solve(var=phi, dt=dt) 172 | if step == 14 or step == 29 or step == 44 or step == 59: 173 | #dl = phi()[0] 174 | #dr = phi()[24] 175 | ul = phi()[600] 176 | #ur = phi()[624] 177 | #d2U = np.hstack([d2U, np.array([dl, dr, ul, ur])]) 178 | d2U = np.hstack([d2U, np.array([ul])]) 179 | 180 | return d2U -------------------------------------------------------------------------------- /lectures/infer_normal_mu: -------------------------------------------------------------------------------- 1 | digraph infer_normal_mu { 2 | g 3 | g0 [label=0> style=filled] 4 | s0 [label=0> style=filled] 5 | g0 -> g 6 | s0 -> g 7 | sigma [label=<σ> style=filled] 8 | subgraph cluster_0 { 9 | Xn [label=n> style=filled] 10 | label="n=1,...,N" 11 | labelloc=b 12 | } 13 | g -> Xn 14 | sigma -> Xn 15 | } 16 | -------------------------------------------------------------------------------- /lectures/motor.dat: -------------------------------------------------------------------------------- 1 | ### Motorcycle Data 2 | ### Data from a Simulated Motorcycle Accident 3 | ### 4 | ### SUMMARY: 5 | ### The motor data frame has 94 rows and 4 columns. The rows 6 | ### are obtained by removing replicate values of time from the 7 | ### dataset mcycle. Two extra columns are added to allow for 8 | ### strata with a different residual variance in each stratum. 9 | ### ~give succinct details here 10 | ### 11 | ### DATA DESCRIPTION: 12 | ### This data frame contains the following columns: 13 | ### times: The time in milliseconds since impact. 14 | ### accel: The recorded head acceleration (in g). 15 | ### strata: A numeric column indicating to which of the three 16 | ### strata (numbered 1, 2 and 3) the observations belong. 17 | ### v: An estimate of the residual variance for the observation. 18 | ### v is constant within the strata but a different estimate 19 | ### is used for each of the three strata. 20 | ### 21 | ### SOURCE: 22 | ### The data were obtained from 23 | ### 24 | ### Silverman, B.W. (1985) Some aspects of the spline 25 | ### smoothing approach to non-parametric curve fitting. 26 | ### Journal of the Royal Statistical Society, B, 47, 1-52. 27 | ### 28 | ### OTHER REFERENCES: 29 | ### Davison, A.C. and Hinkley, D.V. (1997) Bootstrap Methods 30 | ### and Their Application. Cambridge University Press. 31 | ### 32 | ### Venables, W.N. and Ripley, B.D. (1994) Modern Applied 33 | ### Statistics with S-Plus. Springer-Verlag. 34 | ### 35 | ### 36 | # times accel strata v 37 | 2.4 0 1 3.7 38 | 2.6 -1.3 1 3.7 39 | 3.2 -2.7 1 3.7 40 | 3.6 0 1 3.7 41 | 4 -2.7 1 3.7 42 | 6.2 -2.7 1 3.7 43 | 6.6 -2.7 1 3.7 44 | 6.8 -1.3 1 3.7 45 | 7.8 -2.7 1 3.7 46 | 8.2 -2.7 1 3.7 47 | 8.8 -1.3 1 3.7 48 | 9.6 -2.7 1 3.7 49 | 10 -2.7 1 3.7 50 | 10.2 -5.4 1 3.7 51 | 10.6 -2.7 1 3.7 52 | 11 -5.4 1 3.7 53 | 11.4 0 1 3.7 54 | 13.2 -2.7 2 607 55 | 13.6 -2.7 2 607 56 | 13.8 0 2 607 57 | 14.6 -13.3 2 607 58 | 14.8 -2.7 2 607 59 | 15.4 -22.8 2 607 60 | 15.6 -40.2 2 607 61 | 15.8 -21.5 2 607 62 | 16 -42.9 2 607 63 | 16.2 -21.5 2 607 64 | 16.4 -5.4 2 607 65 | 16.6 -59 2 607 66 | 16.8 -71 2 607 67 | 17.6 -37.5 2 607 68 | 17.8 -99.1 2 607 69 | 18.6 -112.5 2 607 70 | 19.2 -123.1 2 607 71 | 19.4 -85.6 2 607 72 | 19.6 -127.2 2 607 73 | 20.2 -123.1 2 607 74 | 20.4 -117.9 2 607 75 | 21.2 -134 2 607 76 | 21.4 -101.9 2 607 77 | 21.8 -108.4 2 607 78 | 22 -123.1 2 607 79 | 23.2 -123.1 2 607 80 | 23.4 -128.5 2 607 81 | 24 -112.5 2 607 82 | 24.2 -95.1 2 607 83 | 24.6 -53.5 2 607 84 | 25 -64.4 2 607 85 | 25.4 -72.3 2 607 86 | 25.6 -26.8 2 607 87 | 26 -5.4 2 607 88 | 26.2 -107.1 2 607 89 | 26.4 -65.6 2 607 90 | 27 -16 2 607 91 | 27.2 -45.6 2 607 92 | 27.6 4 2 607 93 | 28.2 12 2 607 94 | 28.4 -21.5 2 607 95 | 28.6 46.9 2 607 96 | 29.4 -17.4 2 607 97 | 30.2 36.2 2 607 98 | 31 75 2 607 99 | 31.2 8.1 2 607 100 | 32 54.9 2 607 101 | 32.8 46.9 2 607 102 | 33.4 16 2 607 103 | 33.8 45.6 2 607 104 | 34.4 1.3 2 607 105 | 34.8 75 2 607 106 | 35.2 -16 2 607 107 | 35.4 69.6 2 607 108 | 35.6 34.8 2 607 109 | 36.2 -37.5 2 607 110 | 38 46.9 2 607 111 | 39.2 5.4 2 607 112 | 39.4 -1.3 2 607 113 | 40 -21.5 2 607 114 | 40.4 -13.3 2 607 115 | 41.6 30.8 3 138 116 | 42.4 29.4 3 138 117 | 42.8 0 3 138 118 | 43 14.7 3 138 119 | 44 -1.3 3 138 120 | 44.4 0 3 138 121 | 45 10.7 3 138 122 | 46.6 10.7 3 138 123 | 47.8 -26.8 3 138 124 | 48.8 -13.3 3 138 125 | 50.6 0 3 138 126 | 52 10.7 3 138 127 | 53.2 -14.7 3 138 128 | 55 -2.7 3 138 129 | 55.4 -2.7 3 138 130 | 57.6 10.7 3 138 131 | -------------------------------------------------------------------------------- /lectures/omega_X: -------------------------------------------------------------------------------- 1 | digraph omega_X { 2 | omega [label=<ω>] 3 | X 4 | omega -> X 5 | } 6 | -------------------------------------------------------------------------------- /lectures/sample_smc.py: -------------------------------------------------------------------------------- 1 | from collections import OrderedDict 2 | 3 | import numpy as np 4 | import logging 5 | from scipy.special import logsumexp 6 | from fastprogress.fastprogress import progress_bar 7 | import multiprocessing as mp 8 | import warnings 9 | from theano import function as theano_function 10 | import pymc3 as pm 11 | from pymc3.model import modelcontext, Point 12 | from pymc3.parallel_sampling import _cpu_count 13 | from pymc3.theanof import inputvars, make_shared_replacements 14 | from pymc3.vartypes import discrete_types 15 | from pymc3.sampling import sample_prior_predictive 16 | from pymc3.theanof import floatX, join_nonshared_inputs 17 | from pymc3.step_methods.arraystep import metrop_select 18 | from pymc3.step_methods.metropolis import MultivariateNormalProposal 19 | from pymc3.backends.ndarray import NDArray 20 | from pymc3.backends.base import MultiTrace 21 | from pymc3.util import is_transformed_name 22 | 23 | EXPERIMENTAL_WARNING = ( 24 | "Warning: SMC-ABC methods are experimental step methods and not yet" 25 | " recommended for use in PyMC3!" 26 | ) 27 | 28 | 29 | class SMC: 30 | def __init__( 31 | self, 32 | draws=1000, 33 | kernel="metropolis", 34 | n_steps=25, 35 | parallel=False, 36 | start=None, 37 | cores=None, 38 | tune_steps=True, 39 | p_acc_rate=0.99, 40 | threshold=0.5, 41 | epsilon=1.0, 42 | dist_func="absolute_error", 43 | sum_stat=False, 44 | progressbar=False, 45 | model=None, 46 | random_seed=-1, 47 | ): 48 | 49 | self.draws = draws 50 | self.kernel = kernel 51 | self.n_steps = n_steps 52 | self.parallel = parallel 53 | self.start = start 54 | self.cores = cores 55 | self.tune_steps = tune_steps 56 | self.p_acc_rate = p_acc_rate 57 | self.threshold = threshold 58 | self.epsilon = epsilon 59 | self.dist_func = dist_func 60 | self.sum_stat = sum_stat 61 | self.progressbar = progressbar 62 | self.model = model 63 | self.random_seed = random_seed 64 | 65 | self.model = modelcontext(model) 66 | 67 | if self.random_seed != -1: 68 | np.random.seed(self.random_seed) 69 | 70 | if self.cores is None: 71 | self.cores = _cpu_count() 72 | 73 | self.beta = 0 74 | self.max_steps = n_steps 75 | self.proposed = draws * n_steps 76 | self.acc_rate = 1 77 | self.acc_per_chain = np.ones(self.draws) 78 | self.model.marginal_log_likelihood = 0 79 | self.variables = inputvars(self.model.vars) 80 | dimension = sum(v.dsize for v in self.variables) 81 | self.scalings = np.ones(self.draws) * min(1, 2.38 ** 2 / dimension) 82 | self.discrete = np.concatenate( 83 | [[v.dtype in discrete_types] * (v.dsize or 1) for v in self.variables] 84 | ) 85 | self.any_discrete = self.discrete.any() 86 | self.all_discrete = self.discrete.all() 87 | 88 | def initialize_population(self): 89 | """ 90 | Create an initial population from the prior distribution 91 | """ 92 | population = [] 93 | var_info = OrderedDict() 94 | if self.start is None: 95 | init_rnd = sample_prior_predictive( 96 | self.draws, var_names=[v.name for v in self.model.unobserved_RVs], model=self.model, 97 | ) 98 | else: 99 | init_rnd = self.start 100 | 101 | init = self.model.test_point 102 | 103 | for v in self.variables: 104 | var_info[v.name] = (init[v.name].shape, init[v.name].size) 105 | 106 | for i in range(self.draws): 107 | 108 | point = Point({v.name: init_rnd[v.name][i] for v in self.variables}, model=self.model) 109 | population.append(self.model.dict_to_array(point)) 110 | 111 | self.posterior = np.array(floatX(population)) 112 | self.var_info = var_info 113 | 114 | def setup_kernel(self): 115 | """ 116 | Set up the likelihood logp function based on the chosen kernel 117 | """ 118 | shared = make_shared_replacements(self.variables, self.model) 119 | self.prior_logp = logp_forw([self.model.varlogpt], self.variables, shared) 120 | 121 | if self.kernel.lower() == "abc": 122 | warnings.warn(EXPERIMENTAL_WARNING) 123 | if len(self.model.observed_RVs) != 1: 124 | warnings.warn("SMC-ABC only works properly with models with one observed variable") 125 | simulator = self.model.observed_RVs[0] 126 | self.likelihood_logp = PseudoLikelihood( 127 | self.epsilon, 128 | simulator.observations, 129 | simulator.distribution.function, 130 | self.model, 131 | self.var_info, 132 | self.variables, 133 | self.dist_func, 134 | self.sum_stat, 135 | ) 136 | elif self.kernel.lower() == "metropolis": 137 | self.likelihood_logp = logp_forw([self.model.datalogpt], self.variables, shared) 138 | 139 | def initialize_logp(self): 140 | """ 141 | initialize the prior and likelihood log probabilities 142 | """ 143 | if self.parallel and self.cores > 1: 144 | self.pool = mp.Pool(processes=self.cores) 145 | priors = self.pool.starmap(self.prior_logp, [(sample,) for sample in self.posterior]) 146 | likelihoods = self.pool.starmap( 147 | self.likelihood_logp, [(sample,) for sample in self.posterior] 148 | ) 149 | else: 150 | priors = [self.prior_logp(sample) for sample in self.posterior] 151 | likelihoods = [self.likelihood_logp(sample) for sample in self.posterior] 152 | 153 | self.priors = np.array(priors).squeeze() 154 | self.likelihoods = np.array(likelihoods).squeeze() 155 | 156 | def update_weights_beta(self): 157 | """ 158 | Calculate the next inverse temperature (beta), the importance weights based on current beta 159 | and tempered likelihood and updates the marginal likelihood estimation 160 | """ 161 | low_beta = old_beta = self.beta 162 | up_beta = 2.0 163 | rN = int(len(self.likelihoods) * self.threshold) 164 | 165 | while up_beta - low_beta > 1e-6: 166 | new_beta = (low_beta + up_beta) / 2.0 167 | log_weights_un = (new_beta - old_beta) * self.likelihoods 168 | log_weights = log_weights_un - logsumexp(log_weights_un) 169 | ESS = int(np.exp(-logsumexp(log_weights * 2))) 170 | if ESS == rN: 171 | break 172 | elif ESS < rN: 173 | up_beta = new_beta 174 | else: 175 | low_beta = new_beta 176 | if new_beta >= 1: 177 | new_beta = 1 178 | log_weights_un = (new_beta - old_beta) * self.likelihoods 179 | log_weights = log_weights_un - logsumexp(log_weights_un) 180 | 181 | ll_max = np.max(log_weights_un) 182 | self.model.marginal_log_likelihood += ll_max + np.log( 183 | np.exp(log_weights_un - ll_max).mean() 184 | ) 185 | self.beta = new_beta 186 | self.weights = np.exp(log_weights) 187 | 188 | def resample(self): 189 | """ 190 | Resample particles based on importance weights 191 | """ 192 | resampling_indexes = np.random.choice( 193 | np.arange(self.draws), size=self.draws, p=self.weights 194 | ) 195 | self.posterior = self.posterior[resampling_indexes] 196 | self.priors = self.priors[resampling_indexes] 197 | self.likelihoods = self.likelihoods[resampling_indexes] 198 | self.tempered_logp = self.priors + self.likelihoods * self.beta 199 | self.acc_per_chain = self.acc_per_chain[resampling_indexes] 200 | self.scalings = self.scalings[resampling_indexes] 201 | 202 | def update_proposal(self): 203 | """ 204 | Update proposal based on the covariance matrix from tempered posterior 205 | """ 206 | cov = np.cov(self.posterior, bias=False, rowvar=0) 207 | cov = np.atleast_2d(cov) 208 | cov += 1e-6 * np.eye(cov.shape[0]) 209 | if np.isnan(cov).any() or np.isinf(cov).any(): 210 | raise ValueError('Sample covariances not valid! Likely "draws" is too small!') 211 | self.proposal = MultivariateNormalProposal(cov) 212 | 213 | def tune(self): 214 | """ 215 | Tune scaling and n_steps based on the acceptance rate. 216 | """ 217 | ave_scaling = np.exp(np.log(self.scalings.mean()) + (self.acc_per_chain.mean() - 0.234)) 218 | self.scalings = 0.5 * ( 219 | ave_scaling + np.exp(np.log(self.scalings) + (self.acc_per_chain - 0.234)) 220 | ) 221 | 222 | if self.tune_steps: 223 | acc_rate = max(1.0 / self.proposed, self.acc_rate) 224 | self.n_steps = min( 225 | self.max_steps, max(2, int(np.log(1 - self.p_acc_rate) / np.log(1 - acc_rate))), 226 | ) 227 | 228 | self.proposed = self.draws * self.n_steps 229 | 230 | def mutate(self): 231 | """ 232 | Perform mutation step, i.e. apply selected kernel 233 | """ 234 | parameters = ( 235 | self.proposal, 236 | self.scalings, 237 | self.any_discrete, 238 | self.all_discrete, 239 | self.discrete, 240 | self.n_steps, 241 | self.prior_logp, 242 | self.likelihood_logp, 243 | self.beta, 244 | ) 245 | if self.parallel and self.cores > 1: 246 | results = self.pool.starmap( 247 | metrop_kernel, 248 | [ 249 | ( 250 | self.posterior[draw], 251 | self.tempered_logp[draw], 252 | self.priors[draw], 253 | self.likelihoods[draw], 254 | draw, 255 | *parameters, 256 | ) 257 | for draw in range(self.draws) 258 | ], 259 | ) 260 | else: 261 | iterator = range(self.draws) 262 | if self.progressbar: 263 | iterator = progress_bar(iterator, display=self.progressbar) 264 | results = [ 265 | metrop_kernel( 266 | self.posterior[draw], 267 | self.tempered_logp[draw], 268 | self.priors[draw], 269 | self.likelihoods[draw], 270 | draw, 271 | *parameters 272 | ) 273 | for draw in iterator 274 | ] 275 | posterior, acc_list, priors, likelihoods = zip(*results) 276 | self.posterior = np.array(posterior) 277 | self.priors = np.array(priors) 278 | self.likelihoods = np.array(likelihoods) 279 | self.acc_per_chain = np.array(acc_list) 280 | self.acc_rate = np.mean(acc_list) 281 | 282 | def posterior_to_trace(self): 283 | """ 284 | Save results into a PyMC3 trace 285 | """ 286 | lenght_pos = len(self.posterior) 287 | varnames = [v.name for v in self.variables] 288 | 289 | with self.model: 290 | strace = NDArray(self.model) 291 | strace.setup(lenght_pos, 0) 292 | for i in range(lenght_pos): 293 | value = [] 294 | size = 0 295 | for var in varnames: 296 | shape, new_size = self.var_info[var] 297 | value.append(self.posterior[i][size: size + new_size].reshape(shape)) 298 | size += new_size 299 | strace.record({k: v for k, v in zip(varnames, value)}) 300 | return MultiTrace([strace]) 301 | 302 | 303 | def metrop_kernel( 304 | q_old, 305 | old_tempered_logp, 306 | old_prior, 307 | old_likelihood, 308 | draw, 309 | proposal, 310 | scalings, 311 | any_discrete, 312 | all_discrete, 313 | discrete, 314 | n_steps, 315 | prior_logp, 316 | likelihood_logp, 317 | beta, 318 | ): 319 | """ 320 | Metropolis kernel 321 | """ 322 | deltas = np.squeeze(proposal(n_steps) * scalings[draw]) 323 | 324 | accepted = 0 325 | for n_step in range(n_steps): 326 | delta = deltas[n_step] 327 | 328 | if any_discrete: 329 | if all_discrete: 330 | delta = np.round(delta, 0).astype("int64") 331 | q_old = q_old.astype("int64") 332 | q_new = (q_old + delta).astype("int64") 333 | else: 334 | delta[discrete] = np.round(delta[discrete], 0) 335 | q_new = floatX(q_old + delta) 336 | else: 337 | q_new = floatX(q_old + delta) 338 | 339 | ll = likelihood_logp(q_new) 340 | pl = prior_logp(q_new) 341 | 342 | new_tempered_logp = pl + ll * beta 343 | 344 | q_old, accept = metrop_select(new_tempered_logp - old_tempered_logp, q_new, q_old) 345 | 346 | if accept: 347 | accepted += 1 348 | old_prior = pl 349 | old_likelihood = ll 350 | old_tempered_logp = new_tempered_logp 351 | 352 | return q_old, accepted / n_steps, old_prior, old_likelihood 353 | 354 | 355 | def logp_forw(out_vars, vars, shared): 356 | """Compile Theano function of the model and the input and output variables. 357 | Parameters 358 | ---------- 359 | out_vars: List 360 | containing :class:`pymc3.Distribution` for the output variables 361 | vars: List 362 | containing :class:`pymc3.Distribution` for the input variables 363 | shared: List 364 | containing :class:`theano.tensor.Tensor` for depended shared data 365 | """ 366 | out_list, inarray0 = join_nonshared_inputs(out_vars, vars, shared) 367 | f = theano_function([inarray0], out_list[0]) 368 | f.trust_input = True 369 | return f 370 | 371 | 372 | class PseudoLikelihood: 373 | """ 374 | Pseudo Likelihood 375 | """ 376 | 377 | def __init__( 378 | self, epsilon, observations, function, model, var_info, variables, distance, sum_stat 379 | ): 380 | """ 381 | epsilon: float 382 | Standard deviation of the gaussian pseudo likelihood. 383 | observations: array-like 384 | observed data 385 | function: python function 386 | data simulator 387 | model: PyMC3 model 388 | var_info: dict 389 | generated by ``SMC.initialize_population`` 390 | distance: str 391 | Distance function. Available options are ``absolute_error`` (default) and 392 | ``sum_of_squared_distance``. 393 | sum_stat: bool 394 | Whether to use or not a summary statistics. 395 | """ 396 | self.epsilon = epsilon 397 | self.observations = observations 398 | self.function = function 399 | self.model = model 400 | self.var_info = var_info 401 | self.variables = variables 402 | self.varnames = [v.name for v in self.variables] 403 | self.unobserved_RVs = [v.name for v in self.model.unobserved_RVs] 404 | self.kernel = self.gauss_kernel 405 | self.dist_func = distance 406 | self.sum_stat = sum_stat 407 | self.get_unobserved_fn = self.model.fastfn(self.model.unobserved_RVs) 408 | 409 | if distance == "absolute_error": 410 | self.dist_func = self.absolute_error 411 | elif distance == "sum_of_squared_distance": 412 | self.dist_func = self.sum_of_squared_distance 413 | else: 414 | raise ValueError("Distance metric not understood") 415 | 416 | def posterior_to_function(self, posterior): 417 | model = self.model 418 | var_info = self.var_info 419 | 420 | varvalues = [] 421 | samples = {} 422 | size = 0 423 | for var in self.variables: 424 | shape, new_size = var_info[var.name] 425 | varvalues.append(posterior[size: size + new_size].reshape(shape)) 426 | size += new_size 427 | point = {k: v for k, v in zip(self.varnames, varvalues)} 428 | for varname, value in zip(self.unobserved_RVs, self.get_unobserved_fn(point)): 429 | if not is_transformed_name(varname): 430 | samples[varname] = value 431 | return samples 432 | 433 | def gauss_kernel(self, value): 434 | epsilon = self.epsilon 435 | return (-(value ** 2) / epsilon ** 2 + np.log(1 / (2 * np.pi * epsilon ** 2))) / 2.0 436 | 437 | def absolute_error(self, a, b): 438 | if self.sum_stat: 439 | return np.abs(a.mean() - b.mean()) 440 | else: 441 | return np.mean(np.atleast_2d(np.abs(a - b))) 442 | 443 | def sum_of_squared_distance(self, a, b): 444 | if self.sum_stat: 445 | return np.sum(np.atleast_2d((a.mean() - b.mean()) ** 2)) 446 | else: 447 | return np.mean(np.sum(np.atleast_2d((a - b) ** 2))) 448 | 449 | def __call__(self, posterior): 450 | func_parameters = self.posterior_to_function(posterior) 451 | sim_data = self.function(**func_parameters) 452 | value = self.dist_func(self.observations, sim_data) 453 | return self.kernel(value) 454 | 455 | 456 | def sample_smc( 457 | draws=1000, 458 | kernel="metropolis", 459 | n_steps=25, 460 | parallel=False, 461 | start=None, 462 | cores=None, 463 | tune_steps=True, 464 | p_acc_rate=0.99, 465 | threshold=0.5, 466 | epsilon=1.0, 467 | dist_func="absolute_error", 468 | sum_stat=False, 469 | progressbar=False, 470 | model=None, 471 | random_seed=-1, 472 | ): 473 | r""" 474 | Sequential Monte Carlo based sampling 475 | Parameters 476 | ---------- 477 | draws: int 478 | The number of samples to draw from the posterior (i.e. last stage). And also the number of 479 | independent chains. Defaults to 1000. 480 | kernel: str 481 | Kernel method for the SMC sampler. Available option are ``metropolis`` (default) and `ABC`. 482 | Use `ABC` for likelihood free inference togheter with a ``pm.Simulator``. 483 | n_steps: int 484 | The number of steps of each Markov Chain. If ``tune_steps == True`` ``n_steps`` will be used 485 | for the first stage and for the others it will be determined automatically based on the 486 | acceptance rate and `p_acc_rate`, the max number of steps is ``n_steps``. 487 | parallel: bool 488 | Distribute computations across cores if the number of cores is larger than 1. 489 | Defaults to False. 490 | start: dict, or array of dict 491 | Starting point in parameter space. It should be a list of dict with length `chains`. 492 | When None (default) the starting point is sampled from the prior distribution. 493 | cores: int 494 | The number of chains to run in parallel. If ``None`` (default), it will be automatically 495 | set to the number of CPUs in the system. 496 | tune_steps: bool 497 | Whether to compute the number of steps automatically or not. Defaults to True 498 | p_acc_rate: float 499 | Used to compute ``n_steps`` when ``tune_steps == True``. The higher the value of 500 | ``p_acc_rate`` the higher the number of steps computed automatically. Defaults to 0.99. 501 | It should be between 0 and 1. 502 | threshold: float 503 | Determines the change of beta from stage to stage, i.e.indirectly the number of stages, 504 | the higher the value of `threshold` the higher the number of stages. Defaults to 0.5. 505 | It should be between 0 and 1. 506 | epsilon: float 507 | Standard deviation of the gaussian pseudo likelihood. Only works with `kernel = ABC` 508 | dist_func: str 509 | Distance function. Available options are ``absolute_error`` (default) and 510 | ``sum_of_squared_distance``. Only works with ``kernel = ABC`` 511 | sum_stat: bool 512 | Whether to use or not a summary statistics. Defaults to False. Only works with 513 | ``kernel = ABC`` 514 | progressbar: bool 515 | Flag for displaying a progress bar. Defaults to False. 516 | model: Model (optional if in ``with`` context)). 517 | random_seed: int 518 | random seed 519 | Notes 520 | ----- 521 | SMC works by moving through successive stages. At each stage the inverse temperature 522 | :math:`\beta` is increased a little bit (starting from 0 up to 1). When :math:`\beta` = 0 523 | we have the prior distribution and when :math:`\beta` =1 we have the posterior distribution. 524 | So in more general terms we are always computing samples from a tempered posterior that we can 525 | write as: 526 | .. math:: 527 | p(\theta \mid y)_{\beta} = p(y \mid \theta)^{\beta} p(\theta) 528 | A summary of the algorithm is: 529 | 1. Initialize :math:`\beta` at zero and stage at zero. 530 | 2. Generate N samples :math:`S_{\beta}` from the prior (because when :math `\beta = 0` the 531 | tempered posterior is the prior). 532 | 3. Increase :math:`\beta` in order to make the effective sample size equals some predefined 533 | value (we use :math:`Nt`, where :math:`t` is 0.5 by default). 534 | 4. Compute a set of N importance weights W. The weights are computed as the ratio of the 535 | likelihoods of a sample at stage i+1 and stage i. 536 | 5. Obtain :math:`S_{w}` by re-sampling according to W. 537 | 6. Use W to compute the covariance for the proposal distribution. 538 | 7. For stages other than 0 use the acceptance rate from the previous stage to estimate the 539 | scaling of the proposal distribution and `n_steps`. 540 | 8. Run N Metropolis chains (each one of length `n_steps`), starting each one from a different 541 | sample in :math:`S_{w}`. 542 | 9. Repeat from step 3 until :math:`\beta \ge 1`. 543 | 10. The final result is a collection of N samples from the posterior. 544 | References 545 | ---------- 546 | .. [Minson2013] Minson, S. E. and Simons, M. and Beck, J. L., (2013), 547 | Bayesian inversion for finite fault earthquake source models I- Theory and algorithm. 548 | Geophysical Journal International, 2013, 194(3), pp.1701-1726, 549 | `link `__ 550 | .. [Ching2007] Ching, J. and Chen, Y. (2007). 551 | Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating, Model Class 552 | Selection, and Model Averaging. J. Eng. Mech., 10.1061/(ASCE)0733-9399(2007)133:7(816), 553 | 816-832. `link `__ 555 | """ 556 | 557 | smc = SMC( 558 | draws=draws, 559 | kernel=kernel, 560 | n_steps=n_steps, 561 | parallel=parallel, 562 | start=start, 563 | cores=cores, 564 | tune_steps=tune_steps, 565 | p_acc_rate=p_acc_rate, 566 | threshold=threshold, 567 | epsilon=epsilon, 568 | dist_func=dist_func, 569 | sum_stat=sum_stat, 570 | progressbar=progressbar, 571 | model=model, 572 | random_seed=random_seed, 573 | ) 574 | 575 | _log = logging.getLogger("pymc3") 576 | _log.info("Sample initial stage: ...") 577 | stage = 0 578 | smc.initialize_population() 579 | smc.setup_kernel() 580 | smc.initialize_logp() 581 | 582 | while smc.beta < 1: 583 | smc.update_weights_beta() 584 | _log.info( 585 | "Stage: {:3d} Beta: {:.3f} Steps: {:3d} Acce: {:.3f}".format( 586 | stage, smc.beta, smc.n_steps, smc.acc_rate 587 | ) 588 | ) 589 | smc.resample() 590 | smc.update_proposal() 591 | if stage > 0: 592 | smc.tune() 593 | smc.mutate() 594 | stage += 1 595 | 596 | if smc.parallel and smc.cores > 1: 597 | smc.pool.close() 598 | smc.pool.join() 599 | 600 | trace = smc.posterior_to_trace() 601 | 602 | return trace, smc -------------------------------------------------------------------------------- /lectures/trailer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/trailer.png -------------------------------------------------------------------------------- /lectures/trailer_g: -------------------------------------------------------------------------------- 1 | digraph Trailer { 2 | k 3 | m 4 | y0 [label=0>] 5 | omega [label=<ω>] 6 | v 7 | L 8 | X 9 | v -> omega 10 | L -> omega 11 | y0 -> X 12 | omega -> X 13 | k -> X 14 | m -> X 15 | } 16 | -------------------------------------------------------------------------------- /lectures/trailer_m_g: -------------------------------------------------------------------------------- 1 | digraph Trailer { 2 | k 3 | m 4 | y0 [label=0>] 5 | omega [label=<ω>] 6 | v 7 | L 8 | X 9 | v -> omega 10 | L -> omega 11 | y0 -> X 12 | omega -> X 13 | k -> X 14 | m -> X 15 | Xm [label=m> style=filled] 16 | X -> Xm 17 | } 18 | -------------------------------------------------------------------------------- /lectures/urn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/urn.png -------------------------------------------------------------------------------- /lectures/urn1_graph: -------------------------------------------------------------------------------- 1 | digraph Urn1 { 2 | reds [label="# red balls" style=filled] 3 | blues [label="# blue balls" style=filled] 4 | first [label="1st draw"] 5 | reds -> first 6 | blues -> first 7 | } 8 | -------------------------------------------------------------------------------- /lectures/urn2_graph: -------------------------------------------------------------------------------- 1 | digraph Urn2 { 2 | reds [label="# red balls" style=filled] 3 | blues [label="# blue balls" style=filled] 4 | first [label="1st draw"] 5 | second [label="2nd draw"] 6 | reds -> first 7 | blues -> first 8 | first -> second 9 | reds -> second 10 | blues -> second 11 | } 12 | -------------------------------------------------------------------------------- /lectures/urn3_graph: -------------------------------------------------------------------------------- 1 | digraph Urn3 { 2 | reds [label="# red balls" style=filled] 3 | blues [label="# blue balls" style=filled] 4 | first [label="1st draw" style=filled] 5 | second [label="2nd draw"] 6 | reds -> first 7 | blues -> first 8 | first -> second 9 | reds -> second 10 | blues -> second 11 | } 12 | -------------------------------------------------------------------------------- /lectures/urn4_graph: -------------------------------------------------------------------------------- 1 | digraph Urn4 { 2 | reds [label="# red balls" style=filled] 3 | blues [label="# blue balls" style=filled] 4 | first [label="1st draw"] 5 | second [label="2nd draw" style=filled] 6 | reds -> first 7 | blues -> first 8 | first -> second 9 | reds -> second 10 | blues -> second 11 | } 12 | -------------------------------------------------------------------------------- /lectures/venn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/venn.png -------------------------------------------------------------------------------- /lectures/venn_sum_rule.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/uq-course/10d937ccd3bcc10e57fe3653f6fe3d49076b1839/lectures/venn_sum_rule.png --------------------------------------------------------------------------------