├── .gitignore ├── LICENSE ├── README.md ├── learn_smc_proposals ├── __init__.py ├── cde.py ├── examples │ ├── __init__.py │ ├── factorial_hmm.py │ └── multilevel_poisson.py └── utils.py ├── notebooks ├── Factorial-HMM.ipynb ├── Linear-Regression.ipynb └── Multilevel-Poisson.ipynb └── saved ├── trained_hmm_params.rar ├── trained_poisson_params.rar └── trained_poisson_theta.rar /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | .* 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | {one line to give the program's name and a brief idea of what it does.} 635 | Copyright (C) {year} {name of author} 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | {project} Copyright (C) {year} {fullname} 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Learn proposal distributions for importance sampling and SMC 2 | 3 | This code provides a [PyTorch](http://pytorch.org/) implementation of the method described in [this paper](https://arxiv.org/abs/1602.06701) for training neural networks which can be used as proposal distributions for importance sampling or sequential Monte Carlo: 4 | 5 | Paige, B., & Wood, F. (2016). Inference Networks for Sequential Monte Carlo in Graphical Models. In _Proceedings of the 33rd International Conference on Machine Learning_. JMLR W&CP 48: 3040-3049. 6 | 7 | The largest section of re-usable code is an implementation of a conditional variant of [MADE](https://arxiv.org/abs/1502.03509) as a PyTorch module, found in `learn_smc_proposals.cde`. 8 | This can be used to fit a conditional density estimator. 9 | There is a version for real-valued data and a version for binary data. 10 | 11 | * The [linear regression notebook](notebooks/Linear-Regression.ipynb) provides an end-to-end usage example. This notebook defines a generative model using [PyMC](https://github.com/pymc-devs/pymc), a non-conjugate regression model. It then goes through the process of defining a network which will represent the inverse, training it on samples from the joint distribution, and then using it for inference. 12 | 13 | Two more involved examples are implemented in `learn_smc_proposals.examples`; pre-trained weights are included in this repository. Figures and inference are shown in two notebooks: 14 | 15 | * A [multilevel Poisson](notebooks/Multilevel-Poisson.ipynb) model demonstrates use of [divide-and-conquer SMC](https://arxiv.org/abs/1406.4993) for inference in a Bayesian heirarchical model; 16 | * A [factorial HMM](notebooks/Factorial-HMM.ipynb) demonstrates the binary data implementation and the repeated use of a single learned inverse in a particle filtering context. 17 | -------------------------------------------------------------------------------- /learn_smc_proposals/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tbrx/compiled-inference/93ee7f4282d44b5c62907c6a531b303340abaf55/learn_smc_proposals/__init__.py -------------------------------------------------------------------------------- /learn_smc_proposals/cde.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.tensor as tensor 4 | import torch.nn.init as init 5 | import torch.nn.functional as F 6 | from torch.autograd import Variable 7 | 8 | import numpy as np 9 | 10 | def sample_mask_indices(D, H, simple=False): 11 | if simple: 12 | return np.random.randint(0, D, size=(H,)) 13 | else: 14 | mk = np.linspace(0,D-1,H) 15 | ints = np.array(mk, dtype=int) 16 | ints += (np.random.rand() < mk - ints) 17 | return ints 18 | 19 | def create_mask(D_observed, D_latent, H, num_layers): 20 | m_input = np.concatenate((np.zeros(D_observed), 1+np.arange(D_latent))) 21 | m_w = [sample_mask_indices(D_latent, H) for i in range(num_layers)] 22 | m_v = np.arange(D_latent) 23 | M_A = (1.0*(np.atleast_2d(m_v).T >= np.atleast_2d(m_input))) 24 | M_W = [(1.0*(np.atleast_2d(m_w[0]).T >= np.atleast_2d(m_input)))] 25 | for i in range(1, num_layers): 26 | M_W.append(1.0*(np.atleast_2d(m_w[i]).T >= np.atleast_2d(m_w[i-1]))) 27 | M_V = (1.0*(np.atleast_2d(m_v).T >= np.atleast_2d(m_w[-1]))) 28 | 29 | return M_W, M_V, M_A 30 | 31 | 32 | class MaskedLinear(nn.Linear): 33 | def __init__(self, in_features, out_features, mask, bias=True): 34 | """ Linear layer, but with a mask. 35 | Mask should be a tensor of size (out_features, in_features). """ 36 | super(MaskedLinear, self).__init__(in_features, out_features, bias) 37 | self.register_buffer('mask', mask) 38 | 39 | def forward(self, input): 40 | mask = Variable(self.mask, requires_grad=False) 41 | if self.bias is None: 42 | return F.linear(input, self.weight*mask) 43 | else: 44 | return F.linear(input, self.weight*mask, self.bias) 45 | 46 | 47 | class AbstractConditionalMADE(nn.Module): 48 | def __init__(self, D_observed, D_latent, H, num_layers): 49 | super(AbstractConditionalMADE, self).__init__() 50 | self.D_in = D_observed + D_latent 51 | self.D_out = D_latent 52 | assert num_layers >= 1 53 | 54 | # create masks 55 | M_W, M_V, M_A = create_mask(D_observed, D_latent, H, num_layers) 56 | self.M_W = [torch.FloatTensor(M) for M in M_W] 57 | self.M_V = torch.FloatTensor(M_V) 58 | self.M_A = torch.FloatTensor(M_A) 59 | 60 | # nonlinearities 61 | self.relu = nn.ReLU() 62 | 63 | def forward(self, x): 64 | raise NotImplementedError() 65 | 66 | def sample(self, parents): 67 | raise NotImplementedError() 68 | 69 | def logpdf(self, parents, values): 70 | raise NotImplementedError() 71 | 72 | def propose(self, parents, ns=1): 73 | """ Given a setting of the observed (parent) random variables, sample values of 74 | the latents; returns a tuple of both the values and the log probability under 75 | the proposal. 76 | 77 | If ns > 1, each of the tensors has an added first dimension of ns, each 78 | containing a sample of size [batch_size, D_latent] and [batch_size] """ 79 | original_batch_size = parents.size(0) 80 | if ns > 1: 81 | parents = parents.repeat(ns,1) 82 | values = self.sample(parents) 83 | ln_q = self.logpdf(parents, values) 84 | if ns > 1: 85 | values = values.resize(ns, original_batch_size, self.D_out) 86 | ln_q = ln_q.resize(ns, original_batch_size) 87 | return values, ln_q 88 | 89 | 90 | class ConditionalBinaryMADE(AbstractConditionalMADE): 91 | def __init__(self, D_observed, D_latent, H, num_layers): 92 | super(ConditionalBinaryMADE, self).__init__(D_observed, D_latent, H, num_layers) 93 | 94 | # layers 95 | layers = [MaskedLinear(self.D_in, H, self.M_W[0])] 96 | for i in xrange(1,num_layers): 97 | layers.append(MaskedLinear(H, H, self.M_W[i])) 98 | self.layers = nn.ModuleList(layers) 99 | self.skip_p = MaskedLinear(self.D_in, self.D_out, self.M_A, bias=False) 100 | self.skip_q = MaskedLinear(self.D_in, self.D_out, self.M_A, bias=False) 101 | self.p = MaskedLinear(H, self.D_out, self.M_V) 102 | self.q = MaskedLinear(H, self.D_out, self.M_V) 103 | self.loss = nn.BCELoss(size_average=True) 104 | 105 | # initialize parameters 106 | for param in self.parameters(): 107 | if len(param.size()) == 1: 108 | init.normal(param, std=0.01) 109 | else: 110 | init.uniform(param, a=-0.01, b=0.01) 111 | 112 | def forward(self, x): 113 | h = x 114 | for i, layer in enumerate(self.layers): 115 | h = self.relu(layer(h)) 116 | epsilon = 1e-8 117 | logp = self.p(h) + self.skip_p(x) + epsilon 118 | logq = self.q(h) + self.skip_q(x) + epsilon 119 | A = torch.max(logp, logq, keepdim=True) 120 | normalizer = ((logp - A).exp_() + (logq - A).exp_()).log() + A 121 | # assert (1 - np.isfinite(normalizer.data.numpy())).sum() == 0 122 | logp -= normalizer 123 | return logp.exp_() 124 | 125 | def sample(self, parents): 126 | """ Given a setting of the observed (parent) random variables, sample values of the latents """ 127 | assert parents.size(1) == self.D_in - self.D_out 128 | batch_size = parents.size(0) 129 | FloatTensor = torch.cuda.FloatTensor if parents.is_cuda else torch.FloatTensor 130 | latent = Variable(FloatTensor(batch_size, self.D_out)) 131 | randvals = Variable(FloatTensor(batch_size, self.D_out)) 132 | torch.rand(batch_size, self.D_out, out=randvals.data) 133 | for d in xrange(self.D_out): 134 | full_input = torch.cat((parents, latent), 1) 135 | latent = torch.ge(self(full_input), randvals).float() 136 | return latent 137 | 138 | def logpdf(self, parents, values): 139 | """ Return the conditional log probability `ln p(values|parents)` """ 140 | p = self.forward(torch.cat((parents, values), 1)) 141 | p = torch.clamp(p, 1e-6, 1.0 - 1e-6) 142 | return -self.loss(p, values)*values.size(1) 143 | 144 | 145 | 146 | class ConditionalRealValueMADE(AbstractConditionalMADE): 147 | def __init__(self, D_observed, D_latent, H, num_layers, num_components): 148 | super(ConditionalRealValueMADE, self).__init__(D_observed, D_latent, H, num_layers) 149 | 150 | self.K = num_components 151 | self.softplus = nn.Softplus() 152 | 153 | layers = [MaskedLinear(self.D_in, H, self.M_W[0])] 154 | for i in xrange(1,num_layers): 155 | layers.append(MaskedLinear(H, H, self.M_W[i])) 156 | self.layers = nn.ModuleList(layers) 157 | skip_alpha,skip_mu,skip_sigma,mu,alpha,sigma = [],[],[],[],[],[] 158 | for k in xrange(num_components): 159 | skip_alpha.append(MaskedLinear(self.D_in, self.D_out, self.M_A, bias=False)) 160 | skip_mu.append(MaskedLinear(self.D_in, self.D_out, self.M_A, bias=False)) 161 | skip_sigma.append(MaskedLinear(self.D_in, self.D_out, self.M_A, bias=False)) 162 | alpha.append(MaskedLinear(H, self.D_out, self.M_V)) 163 | mu.append(MaskedLinear(H, self.D_out, self.M_V)) 164 | sigma.append(MaskedLinear(H, self.D_out, self.M_V)) 165 | self.skip_alpha = nn.ModuleList(skip_alpha) 166 | self.skip_mu = nn.ModuleList(skip_mu) 167 | self.skip_sigma = nn.ModuleList(skip_sigma) 168 | self.alpha = nn.ModuleList(alpha) 169 | self.mu = nn.ModuleList(mu) 170 | self.sigma = nn.ModuleList(sigma) 171 | 172 | # initialize parameters 173 | for param in self.parameters(): 174 | if len(param.size()) == 1: 175 | init.normal(param, std=0.01) 176 | else: 177 | init.uniform(param, a=-0.01, b=0.01) 178 | 179 | def forward(self, x): 180 | h = x 181 | for i, layer in enumerate(self.layers): 182 | h = self.relu(layer(h)) 183 | log_alpha = [self.alpha[k](h) + self.skip_alpha[k](x) for k in xrange(self.K)] 184 | mu = [self.mu[k](h) + self.skip_mu[k](x) for k in xrange(self.K)] 185 | sigma = [self.softplus(self.sigma[k](h) + self.skip_sigma[k](x)) for k in xrange(self.K)] 186 | 187 | alpha = torch.cat(map(lambda x: x.unsqueeze(2), log_alpha), 2) 188 | A, _ = torch.max(alpha, 2, keepdim=True) 189 | tmp = torch.sum((alpha - A.expand(alpha.size())).exp_(), 2, keepdim=True).log() 190 | log_normalizer = tmp + A 191 | alpha = (alpha - log_normalizer.expand(alpha.size())).exp_() 192 | mu = torch.cat(map(lambda x: x.unsqueeze(2), mu), 2) 193 | sigma = torch.cat(map(lambda x: x.unsqueeze(2), sigma), 2) 194 | sigma = torch.clamp(sigma, min=1e-6) 195 | 196 | return alpha, mu, sigma 197 | 198 | def sample(self, parents, ns=1): 199 | """ Given a setting of the observed (parent) random variables, sample values of the latents. 200 | 201 | If ns > 1, returns a tensor whose first dimension is ns, each containing a sample 202 | of size [batch_size, D_latent] """ 203 | assert parents.size(1) == self.D_in - self.D_out 204 | original_batch_size = parents.size(0) 205 | if ns > 1: 206 | parents = parents.repeat(ns,1) 207 | batch_size = parents.size(0) 208 | 209 | 210 | # sample noise variables 211 | FloatTensor = torch.cuda.FloatTensor if parents.is_cuda else torch.FloatTensor 212 | latent = Variable(torch.zeros(batch_size, self.D_out)) 213 | randvals = Variable(torch.FloatTensor(batch_size, self.D_out)) 214 | torch.randn(batch_size, self.D_out, out=randvals.data); 215 | gumbel = Variable(torch.rand(batch_size, self.D_out, self.K).log_().mul_(-1).log_().mul_(-1)) 216 | if parents.is_cuda: 217 | latent = latent.cuda() 218 | randvals = randvals.cuda() 219 | gumbel = gumbel.cuda() 220 | 221 | for d in xrange(self.D_out): 222 | full_input = torch.cat((parents, latent), 1) 223 | alpha, mu, sigma = self(full_input) 224 | _, z = torch.max(alpha.log() + gumbel, 2, keepdim=False) 225 | one_hot = torch.zeros(alpha.size()) 226 | if parents.is_cuda: one_hot = one_hot.cuda() 227 | one_hot = one_hot.scatter_(2, z.data.unsqueeze(-1), 1).squeeze_().byte() 228 | tmp = randvals.data * sigma.data[one_hot].view(z.size()) 229 | latent = Variable(tmp + mu.data[one_hot].view(z.size())) 230 | if ns > 1: 231 | latent = latent.resize(ns, original_batch_size, self.D_out) 232 | return latent 233 | 234 | def logpdf(self, parents, values): 235 | """ Return the conditional log probability `ln p(values|parents)` """ 236 | full_input = torch.cat((parents, values), 1) 237 | alpha, mu, sigma = self(full_input) 238 | eps = 1e-6 # need to prevent hard zeros 239 | alpha = torch.clamp(alpha, eps, 1.0-eps) 240 | 241 | const = sigma.pow(2).mul_(2*np.pi).log().mul_(0.5) 242 | normpdfs = (values[:,:,None].expand(mu.size()) - mu).div(sigma).pow(2).div_(2).add_(const).mul_(-1) 243 | lw = normpdfs + alpha.log() 244 | # print "norm", normpdfs, normpdfs.sum() 245 | # print "alph", alpha.log(), alpha.log().sum() 246 | # need to do log-sum-exp along dimension 2 247 | A, _ = torch.max(lw, 2, keepdim=True) 248 | weighted_normal = (torch.sum((lw - A.expand(lw.size())).exp(), 2, keepdim=True).log() + A).squeeze(2) 249 | return torch.sum(weighted_normal, 1, keepdim=True) 250 | -------------------------------------------------------------------------------- /learn_smc_proposals/examples/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tbrx/compiled-inference/93ee7f4282d44b5c62907c6a531b303340abaf55/learn_smc_proposals/examples/__init__.py -------------------------------------------------------------------------------- /learn_smc_proposals/examples/factorial_hmm.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from scipy import stats 3 | 4 | import torch 5 | from torch.autograd import Variable 6 | 7 | import os 8 | import shutil 9 | 10 | from .. import cde 11 | from ..utils import systematic_resample, ESS 12 | 13 | 14 | RESAMPLE_THRESH = 0.5 15 | SIGMA = 20.0 16 | P_INIT = 0.1 17 | TRANSITION = np.array([[0.95, 0.05], 18 | [0.05, 0.95]]) 19 | 20 | 21 | def gen_devices(num_devices=20): 22 | """ Produce a synthetic set of "devices", i.e. a set of mean parameters for different 23 | hypothetical applicances """ 24 | np.random.seed(0) 25 | return np.random.permutation(np.array(np.round(np.linspace(3,50,num_devices)), dtype=int)*10) 26 | 27 | 28 | def gen_dataset(devices, T=200): 29 | """ Sample a time series of synthetic data for a given set of devices """ 30 | devices = np.array(devices, dtype=float) 31 | D = len(devices) 32 | X = np.zeros((T, D), dtype=int) 33 | Y = np.zeros((T,)) 34 | sample_additive = lambda x: max(0, np.dot(x, devices) + np.random.randn()*SIGMA) 35 | X[0] = np.random.rand(D) < P_INIT 36 | Y[0] = sample_additive(X[0]) 37 | for t in xrange(1,T): 38 | X[t] = (TRANSITION[X[t-1]].cumsum(1) < np.random.rand(D,1)).sum(1) 39 | Y[t] = sample_additive(X[t]) 40 | 41 | return X, Y 42 | 43 | 44 | def get_training_data(batch_size, devices): 45 | """ return synthetic training data (inputs, outputs) """ 46 | D = len(devices) 47 | T = 51 48 | batch_size /= (T-1) 49 | data = np.empty((batch_size*(T-1), 2*D+1)) 50 | for i in xrange(batch_size): 51 | X, Y = gen_dataset(devices, T) 52 | inputs = np.hstack((X[:-1],Y[1:,None])) 53 | outputs = X[1:] 54 | data[i*(T-1):(i+1)*(T-1),:] = np.hstack((inputs, outputs)) 55 | return data[:,:D+1], data[:,D+1:] 56 | 57 | 58 | 59 | def _iterate_minibatches(inputs, outputs, batch_size): 60 | for start_idx in range(0, len(inputs) - batch_size + 1, batch_size): 61 | excerpt = slice(start_idx, start_idx + batch_size) 62 | yield Variable(torch.FloatTensor(inputs[excerpt])), Variable(torch.FloatTensor(outputs[excerpt])) 63 | 64 | def training_step(optimizer, dist_est, devices, dataset_size, batch_size, max_local_iters=10, misstep_tolerance=0, verbose=False): 65 | """ Training function for fitting density estimator to simulator output """ 66 | # Train 67 | USE_GPU = dist_est.parameters().next().is_cuda 68 | synthetic_ins, synthetic_outs = get_training_data(dataset_size, devices) 69 | ordering = np.random.permutation(synthetic_ins.shape[0]) 70 | synthetic_ins = synthetic_ins[ordering] 71 | synthetic_outs = synthetic_outs[ordering] 72 | if max_local_iters > 1: 73 | validation_size = dataset_size/10 74 | validation_ins, validation_outs = [Variable(torch.FloatTensor(v)) for v in get_training_data(validation_size, devices)] 75 | if USE_GPU: 76 | validation_ins = validation_ins.cuda() 77 | validation_outs = validation_outs.cuda() 78 | validation_err = -torch.mean(dist_est.logpdf(validation_ins, validation_outs)).data[0] 79 | else: 80 | validation_err = None 81 | 82 | missteps = 0 83 | num_batches = float(synthetic_ins.shape[0])/batch_size 84 | 85 | for local_iter in xrange(max_local_iters): 86 | 87 | train_err = 0 88 | for inputs, outputs in _iterate_minibatches(synthetic_ins, synthetic_outs, batch_size): 89 | optimizer.zero_grad() 90 | if USE_GPU: 91 | loss = -torch.mean(dist_est.logpdf(inputs.cuda(), outputs.cuda())) 92 | else: 93 | loss = -torch.mean(dist_est.logpdf(inputs, outputs)) 94 | loss.backward() 95 | params_before = [np.isnan(p.data.numpy()).sum() for p in dist_est.parameters()] 96 | assert sum(params_before) == 0 97 | optimizer.step() 98 | params_after = [np.isnan(p.data.numpy()).sum() for p in dist_est.parameters()] 99 | assert sum(params_after) == 0 100 | train_err += loss.data[0]/num_batches 101 | 102 | if max_local_iters > 1: 103 | next_validation_err = -torch.mean(dist_est.logpdf(validation_ins, validation_outs)).data[0] 104 | if next_validation_err > validation_err: 105 | missteps += 1 106 | validation_err = next_validation_err 107 | if missteps > misstep_tolerance: 108 | break 109 | 110 | if verbose: 111 | print train_err, validation_err, "(", local_iter+1, ")" 112 | return train_err, validation_err, local_iter+1 113 | 114 | 115 | def baseline_proposal(x, y): 116 | next_x = np.zeros_like(x) 117 | K, D = x.shape 118 | for k in xrange(K): 119 | next_x[k] = (TRANSITION[x[k]].cumsum(1) < np.random.rand(D,1)).sum(1) 120 | ln_q = np.sum(np.log(TRANSITION[0,0]) * (next_x == x) + 121 | np.log(TRANSITION[0,1]) * (next_x != x), 1) 122 | return next_x, ln_q 123 | 124 | def make_nn_proposal(dist_est): 125 | USE_GPU = dist_est.parameters().next().is_cuda 126 | def nn_proposal(x, y): 127 | K, D = x.shape 128 | state = np.concatenate((x,y*np.ones((K,1))), axis=1) 129 | if USE_GPU: 130 | state = Variable(torch.cuda.FloatTensor(state)) 131 | else: 132 | state = Variable(torch.FloatTensor(state)) 133 | val, ln_q = dist_est.propose(state) 134 | return val.data.cpu().numpy(), ln_q.data.cpu().numpy().squeeze() 135 | return nn_proposal 136 | 137 | def run_smc(devices, Y, K, proposal, verbose=True): 138 | """ Run an SMC algorithm using K particles, and proposal distribution `proposal`, 139 | which returns a value and its proposal log probability. 140 | 141 | `factorial_hmm.baseline_proposal` samples from the transition dynamics. 142 | `factorial_hmm.make_nn_proposal` generates a proposal using a learned network. """ 143 | 144 | T = len(Y) 145 | X_hat = np.zeros((K,T,len(devices)), dtype=int) 146 | ancestry = np.empty((K,T), dtype=int) 147 | ln_q = 0.0 148 | X_hat[:,0], ln_q = proposal(1*(np.random.rand(K, len(devices)) < P_INIT), Y[0]) 149 | log_weights = stats.norm(Y[0], SIGMA).logpdf(np.dot(X_hat[:,0], devices)) - ln_q 150 | ESS_history = np.empty((T,)) 151 | ESS_history[0] = ESS(log_weights) 152 | if ESS_history[0] < K*RESAMPLE_THRESH: 153 | X_hat = X_hat[systematic_resample(log_weights)] 154 | log_weights[:] = 0.0 # np.log(np.mean(np.exp(log_weights))) 155 | ancestry[:,0] = np.arange(K) 156 | for t in xrange(1,len(Y)): 157 | X_hat[:,t], ln_q = proposal(X_hat[:,t-1], Y[t]) 158 | ln_p_trans = np.sum(np.log(TRANSITION[0,0]) * (X_hat[:,t] == X_hat[:,t-1]) + 159 | np.log(TRANSITION[0,1]) * (X_hat[:,t] != X_hat[:,t-1]), 1) 160 | # assert np.isfinite(ln_q) # TODO add back 161 | log_weights += stats.norm(Y[t], SIGMA).logpdf(np.dot(X_hat[:,t], devices)) + ln_p_trans - ln_q 162 | ESS_history[t] = ESS(log_weights) 163 | if ESS_history[t] < K*RESAMPLE_THRESH: 164 | if verbose: 165 | print "RESAMPLE", t, ESS_history[t] 166 | indices = systematic_resample(log_weights) 167 | X_hat = X_hat[indices] 168 | log_weights[:] = 0.0 # np.log(np.mean(np.exp(log_weights))) 169 | ancestry = ancestry[indices] 170 | ancestry[:,t] = np.arange(K) 171 | indices = systematic_resample(log_weights) 172 | ancestry = ancestry[indices] 173 | ancestry[:,-1] = np.arange(K) 174 | return X_hat[indices], ancestry, ESS_history 175 | 176 | if __name__ == '__main__': 177 | USE_GPU = torch.cuda.is_available() 178 | print "Using GPU?", USE_GPU 179 | devices = gen_devices() 180 | trace_train = [] 181 | trace_validation = [] 182 | 183 | dist_est = cde.ConditionalBinaryMADE(len(devices)+1, len(devices), H=300, num_layers=4) 184 | if USE_GPU: 185 | dist_est.cuda() 186 | optimizer = torch.optim.Adam(dist_est.parameters(), lr=0.001) 187 | num_iterations = 2000 188 | dataset_size = 5000 189 | batch_size = 500 190 | 191 | outfile = 'trained_hmm_params.rar' 192 | if os.path.exists(outfile): 193 | shutil.copyfile(outfile, '{}.backup'.format(outfile)) 194 | 195 | for i in xrange(num_iterations): 196 | verbose = True 197 | print "["+str(1+len(trace_train))+"]", 198 | t,v,_ = training_step(optimizer, dist_est, devices, dataset_size, batch_size, max_local_iters=10, verbose=True) 199 | trace_train.append(t) 200 | trace_validation.append(v) 201 | torch.save(dist_est.cpu().state_dict(), outfile) 202 | if USE_GPU: dist_est.cuda() 203 | -------------------------------------------------------------------------------- /learn_smc_proposals/examples/multilevel_poisson.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import torch 3 | from scipy import stats 4 | from torch.autograd import Variable 5 | 6 | import pymc 7 | import os 8 | import shutil 9 | 10 | from learn_smc_proposals import cde 11 | 12 | num_points = 10 # number of points in the synthetic dataset we train on 13 | 14 | def gamma_poisson(x, t): 15 | """ x: number of failures (N vector) 16 | t: operation time, thousands of hours (N vector) """ 17 | 18 | if x is not None: 19 | N = x.shape 20 | else: 21 | N = num_points 22 | 23 | # place an exponential prior on t, for when it is unknown 24 | t = pymc.Exponential('t', beta=1.0/50.0, value=t, size=N, observed=(t is not None)) 25 | 26 | alpha = pymc.Exponential('alpha', beta=1.0, value=1.0) 27 | beta = pymc.Gamma('beta', alpha=0.1, beta=1.0, value=1.0) 28 | 29 | theta = pymc.Gamma('theta', alpha=alpha, beta=beta, size=N) 30 | 31 | @pymc.deterministic 32 | def mu(theta=theta, t=t): 33 | return theta*t 34 | 35 | x = pymc.Poisson('x', mu=mu, value=x, observed=(x is not None)) 36 | 37 | return locals() 38 | 39 | sample_test_points = lambda: num_points 40 | 41 | def get_input_theta(model): 42 | return np.atleast_2d(np.vstack((model.x.value[:1], model.t.value[:1]))) 43 | # return np.atleast_2d(np.vstack((model.x.value, model.t.value))) 44 | 45 | def get_target_theta(model): 46 | return np.atleast_2d(model.theta.value[:1]) 47 | # return np.atleast_2d(model.theta.value) 48 | 49 | #get_input_params = get_target_theta 50 | 51 | 52 | def get_input_params(model): 53 | return np.atleast_2d(model.theta.value) 54 | 55 | def get_target_params(model): 56 | return np.atleast_2d([model.alpha.value, model.beta.value]) 57 | 58 | 59 | def generate_synthetic(model, size=100): 60 | # N = sample_test_points() 61 | ins_theta, outs_theta = None, None 62 | ins_params, outs_params = None, None 63 | #while len(ins_params) < size: 64 | for i in xrange(size-1): 65 | try: 66 | model.draw_from_prior() 67 | if np.min(get_target_params(model).ravel()) < 1e-5 or \ 68 | np.max(get_target_params(model).ravel()) > 1e5 or \ 69 | np.min(get_input_params(model).ravel()) < 1e-5 or \ 70 | np.max(get_input_params(model).ravel()) > 1e5: 71 | # filter out garbage 72 | continue 73 | if ins_theta is None: 74 | ins_theta, outs_theta = get_input_theta(model).T, get_target_theta(model).T 75 | ins_params, outs_params = get_input_params(model), get_target_params(model) 76 | else: 77 | ins_theta = np.vstack((ins_theta, get_input_theta(model).T)) 78 | outs_theta = np.vstack((outs_theta, get_target_theta(model).T)) 79 | ins_params = np.vstack((ins_params, get_input_params(model))) 80 | outs_params = np.vstack((outs_params, get_target_params(model))) 81 | except Exception as e: 82 | #print e 83 | pass 84 | theta = (ins_theta, outs_theta) 85 | params = (ins_params, outs_params) 86 | #theta = (np.log(ins_theta), np.log(outs_theta)) 87 | #params = (np.log(ins_params), np.log(outs_params)) 88 | return theta, params 89 | 90 | def _iterate_minibatches(inputs, outputs, batchsize): 91 | for start_idx in range(0, len(inputs) - batchsize + 1, batchsize): 92 | excerpt = slice(start_idx, start_idx + batchsize) 93 | yield Variable(torch.FloatTensor(inputs[excerpt])), Variable(torch.FloatTensor(outputs[excerpt])) 94 | 95 | def training_step(target_model, batch_size, max_local_iters, misstep_tolerance=0, verbose=False, generator=generate_synthetic): 96 | """ Training function for fitting density estimator to simulator output """ 97 | # Train 98 | synthetic_data = generator(M_train, batch_size*10)[target_model] 99 | dataset_size = synthetic_data[0].shape[0] 100 | print ("(size=%d):\t" % dataset_size), 101 | validation_size = dataset_size/10 102 | validation_data = [Variable(torch.FloatTensor(t)) for t in generator(M_train, validation_size)[target_model]] 103 | USE_GPU = estimators[target_model].parameters().next().is_cuda 104 | if USE_GPU: 105 | validation_data = [d.cuda() for d in validation_data] 106 | 107 | missteps = 0 108 | num_batches = float(dataset_size)/batch_size 109 | 110 | # backup = estimators[target_model].state_dict() 111 | validation_err = -estimators[target_model].logpdf(validation_data[0], validation_data[1]).mean() 112 | validation_err = validation_err.data[0] 113 | for local_iter in xrange(max_local_iters): 114 | train_err = 0 115 | for inputs, outputs in _iterate_minibatches(synthetic_data[0], synthetic_data[1], batch_size): 116 | optimizers[target_model].zero_grad() 117 | if USE_GPU: 118 | loss = -torch.mean(estimators[target_model].logpdf(inputs.cuda(), outputs.cuda())) 119 | else: 120 | loss = -torch.mean(estimators[target_model].logpdf(inputs, outputs)) 121 | loss.backward() 122 | optimizers[target_model].step() 123 | train_err += loss.data[0]/num_batches 124 | 125 | 126 | next_validation_err = -estimators[target_model].logpdf(*validation_data).mean() 127 | # if np.isnan(train_err) or np.isnan(next_validation_err.data[0]): 128 | # estimators[target_model].load_state_dict(backup) 129 | # break 130 | if next_validation_err > validation_err: 131 | missteps += 1 132 | validation_err = next_validation_err.data[0] 133 | if missteps > misstep_tolerance: 134 | break 135 | 136 | if verbose: 137 | print round(train_err, 4), round(validation_err, 4), "(", local_iter+1, ")", 138 | 139 | return train_err, validation_err, local_iter+1 140 | 141 | def get_estimators(): 142 | theta_est = cde.ConditionalRealValueMADE(2, 1, 500, 2, 10) 143 | params_est = cde.ConditionalRealValueMADE(10, 2, 500, 2, 10) 144 | return theta_est, params_est 145 | 146 | 147 | if __name__ == '__main__': 148 | M_train = pymc.Model(gamma_poisson(None, None)) 149 | theta_est, params_est = get_estimators() 150 | USE_GPU = torch.cuda.is_available() 151 | if USE_GPU: 152 | theta_est.cuda() 153 | params_est.cuda() 154 | estimators = [theta_est, params_est] 155 | optimizers = [torch.optim.Adam(model.parameters()) for model in estimators] 156 | 157 | trace_train = ([], []) 158 | trace_validation = ([], []) 159 | trace_local_iters = ([], []) 160 | 161 | num_steps = 500 162 | batch_size = 2500 163 | max_local_iters = 10 164 | 165 | file1 = 'trained_poisson_theta.rar' 166 | file2 = 'trained_poisson_params.rar' 167 | if os.path.exists(file1): 168 | shutil.copyfile(file1, '{}.backup'.format(file1)) 169 | if os.path.exists(file2): 170 | shutil.copyfile(file2, '{}.backup'.format(file2)) 171 | 172 | for i in xrange(num_steps): 173 | for target_model in [0,1]: 174 | verbose = True # (i+1) % 1 == 0 175 | if verbose: 176 | print "["+("thetas","params")[target_model]+" "+str(1+len(trace_train[target_model]))+"]", 177 | t,v,l = training_step(target_model, batch_size, max_local_iters, verbose=verbose) 178 | trace_train[target_model].append(t) 179 | trace_validation[target_model].append(v) 180 | if verbose: print '\t', 181 | trace_local_iters[target_model].append(l) 182 | if verbose: 183 | print 184 | torch.save(theta_est.cpu().state_dict(), file1) 185 | torch.save(params_est.cpu().state_dict(), file2) 186 | if USE_GPU: 187 | theta_est.cuda() 188 | params_est.cuda() 189 | 190 | -------------------------------------------------------------------------------- /learn_smc_proposals/utils.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | def systematic_resample(log_weights): 4 | A = log_weights.max() 5 | normalizer = np.log(np.exp(log_weights - A).sum()) + A 6 | weights = np.exp(log_weights - normalizer) 7 | ns = len(weights) 8 | cdf = np.cumsum(weights) 9 | cutoff = (np.random.rand() + np.arange(ns))/ns 10 | return np.digitize(cutoff, cdf) 11 | 12 | def ESS(log_weights): 13 | A = log_weights.max() 14 | normalizer = np.log(np.exp(log_weights - A).sum()) + A 15 | log_normalized = 2*(log_weights - normalizer) 16 | B = log_normalized.max() 17 | log_denominator = np.log(np.sum(np.exp(log_normalized - B))) + B 18 | return np.exp(-log_denominator) 19 | 20 | -------------------------------------------------------------------------------- /notebooks/Multilevel-Poisson.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": false, 8 | "deletable": true, 9 | "editable": true 10 | }, 11 | "outputs": [ 12 | { 13 | "name": "stdout", 14 | "output_type": "stream", 15 | "text": [ 16 | "Couldn't import dot_parser, loading of dot files will not be possible.\n" 17 | ] 18 | } 19 | ], 20 | "source": [ 21 | "import numpy as np\n", 22 | "import torch\n", 23 | "from scipy import stats\n", 24 | "from torch.autograd import Variable\n", 25 | "import sys, inspect\n", 26 | "sys.path.insert(0, '..')\n", 27 | "\n", 28 | "%matplotlib inline\n", 29 | "import pymc\n", 30 | "import matplotlib.pyplot as plt\n", 31 | "\n", 32 | "from learn_smc_proposals import cde\n", 33 | "from learn_smc_proposals.examples import multilevel_poisson\n", 34 | "\n", 35 | "import seaborn as sns\n", 36 | "sns.set_context(\"notebook\", font_scale=1.5, rc={\"lines.markersize\": 12})\n", 37 | "sns.set_style('ticks')" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": { 43 | "deletable": true, 44 | "editable": true 45 | }, 46 | "source": [ 47 | "# Learn a multilevel model\n", 48 | "\n", 49 | "We're going to learn the model for the PUMPS data:\n", 50 | "\n", 51 | "http://www.openbugs.net/Examples/Pumps.html\n", 52 | "\n", 53 | "This model has local parameters $\\theta_i$ and global parameters $\\alpha, \\beta$." 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": 2, 59 | "metadata": { 60 | "collapsed": false, 61 | "deletable": true, 62 | "editable": true 63 | }, 64 | "outputs": [], 65 | "source": [ 66 | "theta_est, params_est = multilevel_poisson.get_estimators()\n", 67 | "theta_est.load_state_dict(torch.load('../saved/trained_poisson_theta.rar'))\n", 68 | "params_est.load_state_dict(torch.load('../saved/trained_poisson_params.rar'))" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 3, 74 | "metadata": { 75 | "collapsed": true, 76 | "deletable": true, 77 | "editable": true 78 | }, 79 | "outputs": [], 80 | "source": [ 81 | "true_t = np.array([94.3, 15.7, 62.9, 126, 5.24, 31.4, 1.05, 1.05, 2.1, 10.5])\n", 82 | "true_x = np.array([5, 1, 5, 14, 3, 19, 1, 1, 4, 22])" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": { 88 | "deletable": true, 89 | "editable": true 90 | }, 91 | "source": [ 92 | "### Training; synthetic data\n", 93 | "\n", 94 | "We can use our model to define synthetic data, which we will use to train the inference network.\n", 95 | "\n", 96 | "Each \"minibatch\" will be an unconditioned sample from the graphical model." 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": { 102 | "deletable": true, 103 | "editable": true 104 | }, 105 | "source": [ 106 | "Values on true data, as reported in the George et al. (1993) are:\n", 107 | " \n", 108 | " theta = [0.060 0.102 0.089 0.116 0.602 0.609 0.891 0.894 1.588 1.994 ]\n", 109 | " alpha = 0.7 \n", 110 | " beta = 0.9 \n", 111 | " \n", 112 | "We'll run MCMC here to get benchmark estimates of the posterior distributions." 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 4, 118 | "metadata": { 119 | "collapsed": false, 120 | "deletable": true, 121 | "editable": true 122 | }, 123 | "outputs": [], 124 | "source": [ 125 | "real_data = Variable(torch.FloatTensor(np.vstack((true_x, true_t)).T))" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": 5, 131 | "metadata": { 132 | "collapsed": false 133 | }, 134 | "outputs": [], 135 | "source": [ 136 | "M_train = pymc.Model(multilevel_poisson.gamma_poisson(None,None))\n", 137 | "M_test = pymc.Model(multilevel_poisson.gamma_poisson(true_x, true_t))" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 6, 143 | "metadata": { 144 | "collapsed": true, 145 | "deletable": true, 146 | "editable": true 147 | }, 148 | "outputs": [], 149 | "source": [ 150 | "def estimate_MCMC(data_x, data_t, ns, iters=10000, burn=0.5):\n", 151 | " \"\"\" MCMC estimate of weight distribution \"\"\"\n", 152 | " mcmc_est = pymc.MCMC(multilevel_poisson.gamma_poisson(data_x, data_t))\n", 153 | " mcmc_est.sample(iters, burn=burn*iters, thin=np.ceil(burn*iters/ns))\n", 154 | " trace_theta = mcmc_est.trace('theta').gettrace()[:ns]\n", 155 | " trace_alpha = mcmc_est.trace('alpha').gettrace()[:ns]\n", 156 | " trace_beta = mcmc_est.trace('beta').gettrace()[:ns]\n", 157 | " return trace_theta, trace_alpha, trace_beta" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": 7, 163 | "metadata": { 164 | "collapsed": false, 165 | "deletable": true, 166 | "editable": true 167 | }, 168 | "outputs": [ 169 | { 170 | "name": "stdout", 171 | "output_type": "stream", 172 | "text": [ 173 | " [-----------------100%-----------------] 500000 of 500000 complete in 69.2 sec\n", 174 | "\n", 175 | "MCMC Estimated theta: [ 0.059 0.101 0.09 0.115 0.517 0.56 1.168 0.794 1.731 1.909]\n", 176 | "\n", 177 | "MCMC Estimated (alpha, beta): 0.683 0.905\n" 178 | ] 179 | } 180 | ], 181 | "source": [ 182 | "mcmc_theta, mcmc_alpha, mcmc_beta = estimate_MCMC(true_x, true_t, ns=1000, iters=500000)\n", 183 | "# print \"MCMC MSE\", np.mean((mcmc_theta.mean(0) - true_theta)**2)\n", 184 | "\n", 185 | "true_alpha = mcmc_alpha.mean()\n", 186 | "true_beta = mcmc_beta.mean()\n", 187 | "true_theta = mcmc_theta.mean(0)\n", 188 | "\n", 189 | "print \"\\n\\nMCMC Estimated theta:\", true_theta.round(3)\n", 190 | "print \"\\nMCMC Estimated (alpha, beta):\", true_alpha.round(3), true_beta.round(3)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "## Comparison: samples from the proposal, and the MCMC posterior" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": 8, 203 | "metadata": { 204 | "collapsed": false, 205 | "deletable": true, 206 | "editable": true 207 | }, 208 | "outputs": [], 209 | "source": [ 210 | "def draw_inverse(ns=100):\n", 211 | " tmp = theta_est.sample(real_data, ns=ns).squeeze(2)\n", 212 | " samples = params_est.sample(tmp, ns=1)\n", 213 | " return tmp.cpu().data.numpy(), samples.cpu().data.numpy()" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": 9, 219 | "metadata": { 220 | "collapsed": false, 221 | "deletable": true, 222 | "editable": true 223 | }, 224 | "outputs": [], 225 | "source": [ 226 | "nn_raw_theta, nn_raw_params = draw_inverse(1000)" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": 10, 232 | "metadata": { 233 | "collapsed": false, 234 | "deletable": true, 235 | "editable": true 236 | }, 237 | "outputs": [ 238 | { 239 | "data": { 240 | "image/png": "iVBORw0KGgoAAAANSUhEUgAABYkAAAGYCAYAAAAHjm0BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8jPf+//9ngrG2xBZBtUISHxwklhxbq9ISWgctjjWq\n1dYWlKrY0lVrKUVSS6n2RFCnJXpy6lTP0e/p4bRF0NZSJF2OULE0SoUkkszvDz9hZJFJZuaa5XG/\n3eZ2y7znPe/rNVeuec11veaa9+VlNpvNAgAAAAAAAAB4JG+jAwAAAAAAAAAAGIciMQAAAAAAAAB4\nMIrEAAAAAAAAAODBKBIDAAAAAAAAgAejSAwAAAAAAAAAHowiMQAAAAAAAAB4MIrEbuzXX3/VlStX\nJEkjRoxQ9+7dbTr+5cuXlZ6ebtMxb5eTk6Ply5crLCxMrVu31sCBA/Xll1/adZkAysYdcs+tkpKS\n1KxZM508edJhywRgPXfIPefOnVNUVJQ6deqkli1bqmfPnlq7dq3MZrNdlwugdNwh75w/f14zZsxQ\n165dFRwcrKeeekoHDx606zIBlI075J5b5eXlafDgwTZ/HbAeRWI39cUXXyg8PNxub+xDhw6pV69e\nSk5Otsv4N7z++utaunSpOnfurKioKEnS008/rX379tl1uQBKx11yzw2//PKLpkyZQoEGcHLukHuy\ns7P1xBNP6B//+If69++vmTNnyt/fX/Pnz9cbb7xht+UCKB13yDtXr17VyJEj9dlnn2nQoEGaMmWK\nTp8+rWHDhuno0aN2Wy6A0nOH3HO7999/XwcOHHDY8lA0isRu6rvvvtOlS5fsNv7x48d19uxZu40v\nST/99JM2bNigMWPG6JVXXtGQIUO0bt06NWjQQAsXLrTrsgGUjjvknhv279+vP//5zzpz5oxDlgeg\n9Nwh92zcuFEpKSlaunSppk2bpqFDh2rFihV69NFHtW7dOv3yyy92XT4A67hD3vnoo4+UkpKixYsX\nKzIyUiNGjNC6detUrlw5rVy50q7LBlA67pB7bvXTTz9p6dKlqlChgsOWiaJRJIbT2rZtm8xms4YM\nGZLfVqlSJQ0YMEAHDhzgYAmA3axYsUJDhw5VxYoV1bt3b6PDAeAB9uzZIx8fH3Xr1s2ivVevXsrL\ny9O3335rTGAA3FZWVpaCg4P1wAMP5LfVqlVL/v7+OnbsmIGRAfAEeXl5mjlzpkJCQhQcHGx0OBBF\nYrcUFRWl2NhYSVJYWJhGjBiR/9iuXbv02GOP6Q9/+IO6deum5cuXKy8vz+L5KSkpGj9+vNq1a6fW\nrVtr8ODB2rlzZ/7jMTExmjFjhiQpIiLCYt6Yr776SqNHj1ZoaKhatGihrl27Kjo6ulTfdB06dEi1\natVSvXr1LNqbN28uSTp8+LDVYwKwH3fJPdL1b9AjIiK0detW+fv7l2oMAI7hLrnnlVde0bp16wq0\nX7hwQZJUvnx5q8cEYB/ukndGjx6tDz74wKLtypUrOnHihOrXr2/1eADsy11yzw1xcXE6evSoXn31\n1VKPAdsq99JLL71kdBCwrVq1aunixYv68ccfNWPGDPXs2VNJSUn68ccftX37dvXq1UuPPvqoTp8+\nrYSEBN19991q06aNJOnYsWMaMmSIMjMzFRERoc6dO+vo0aNau3at/P39FRAQoOrVq8tsNuvw4cMa\nM2aMBgwYIH9/f+3atUujR49WnTp1NHz4cN1///3Kzs5WYmKiUlNT1atXL6tex9q1a1W1alUNHjzY\nov3q1avauHGj2rRpkx83AOO5S+6RpO7du6tbt24ymUzas2eP9uzZo5EjR+ruu++29WoDUEbuknuq\nVKmiWrVqWbSZzWZFR0fr119/1fTp01WtWjWbrTcApecueedWv//+uw4ePKjZs2frxIkTeuWVV9Sw\nYUNbrTIANuBOued///ufJk2apKlTp6pr165KSEjQ77//rpEjR9p6tcEaZrilZcuWmQMDA82pqalm\ns9lsHj58uDkwMND82Wef5ff5/fffzSEhIeahQ4fmtw0fPtz80EMPmTMyMvLbrl27Zh46dKi5U6dO\n5qysLLPZbDZv3rzZHBgYaP7666/z+z311FPmBx98ML/PDYMGDTIHBwdb/Rp69OhhHjx4cIH2n3/+\n2RwYGGhesWKF1WMCsC93yD13ek0AnI875h6z2WyOjY01BwYGmqOiomwyHgDbcbe88+yzz5oDAwPN\ngYGB5ldffdV87dq1Mo0HwD7cIffk5eWZhw4dah4yZIg5Ly8vP74HH3zQ6rFgW0w34UEqV65s8XOB\natWqyd/fX+fPn5d0/eeMe/bs0QMPPKDMzEylp6crPT1dly5d0sMPP6zz58/r4MGDRY6/atUqbd68\nWSaTKb/twoULqlatmq5cuVKqmL28vEr1GADn4Yq5B4Drc/Xcs379ei1btkz+/v6aOXNmmccDYH+u\nnHcGDRqk2NhYDRkyROvXr9fEiRPLNB4Ax3G13BMXF6eDBw9q7ty51HWcDJObeZAaNWqoXLlyFm2V\nKlXSr7/+KklKTU2VJK1bt67QOfEk6fTp00WOX65cOaWmpmrp0qVKSUnRiRMndObMmVLHW6VKFWVm\nZhZov9HGTy4B1+BquQeAe3Dl3LNmzRotXLhQDRo00Lvvvqu77rrLJuMCsC9Xzjs3CkwPP/yw7r77\nbq1atUpfffWVOnbsaJPxAdiPK+We1NRUvfXWWxo5cqSqV6+u9PR0SdK1a9eUl5en9PR0Va5cWZUr\nVy7V+CgbisQexNu7+BPHc3NzJUnDhg3TQw89VGifpk2bFvn8d999VwsWLFDjxo3Vrl079ejRQ61b\nt9a6deuUmJhodbx+fn6Ffpt19uxZSZKvr6/VYwJwPFfLPQDcg6vmnrfeeksrV65Uo0aN9Je//IWL\nRwEuxFXzzu3Cw8O1atUqHTlyhCIx4AJcKffs3btXV69e1TvvvKN33nmnwOMdO3bUhAkTFBkZadW4\nsA2KxMjXoEEDSde/JerUqZPFYykpKTp58mSR3+ZkZWUpJiZGoaGhWrt2rcUVuJcuXVqqeJo3b64d\nO3bo/Pnzql27dn77999/L0n6wx/+UKpxATgXZ8s9ADyDM+aet99+WytXrlTTpk313nvvqW7duqUe\nC4Dzcba8M378eP32229av369RXtGRoak62ciAnB9zpR7unTpovfee69A+7x58/Trr79q4cKFuuee\ne6weF7bBnMRu6sY3SWazucTPqVu3rlq2bKmEhASLnw5cu3ZNM2fO1MSJE5WTk2Mxfl5enqTrU0Bc\nvXpV9913n0XS+P7777Vnzx5Jyn9uSfXo0UOSLH4OkZmZqc2bN6tt27acSQw4IXfIPQBcjzvknq++\n+koxMTG67777FBcXR4EYcHLukHd8fX2VlJSkAwcO5LeZzWa9//77KleunLp27WrVeADsz9VzT926\nddWpU6cCt+rVq6tixYrq1KkTRWIDcSaxm6pZs6ak63Pa3X///SV+3uzZszVy5Eg9/vjjGjJkiGrU\nqKFPPvlE3377raZOnSofHx+L8Tdu3Kjz58+rT58+at26tbZs2aJq1aqpcePGSk5O1ocffpifZDIy\nMlS9evUSxxIYGKjHHntMq1at0qVLl9SsWTNt3rxZp06d0htvvFHicQA4jjvkHgCuxx1yz+LFi2U2\nm/Xggw9q165dBR4PCQnhoAlwIu6QdyZMmKDPPvtMzz77rEaMGCEfHx9t375de/bs0aRJk9SoUaMS\njwXAMdwh98B5eZmt+foBLuPSpUuaNGmSkpKS1LBhQ9WuXVunTp3S559/btFvxIgRBdoPHz6smJgY\nJSUlKScnR40bN1ZERIT69++f3+fatWuaNm2a/t//+3+qWLGidu7cqfT0dL3xxhvavXu3srOz1aBB\nAz3yyCNq0qSJIiMjtWzZMvXs2dOq15Gdna2YmBglJCTo8uXLCgoK0nPPPac//vGPZVtBAOzCXXLP\nrWJiYhQbG6sdO3aoYcOGpR4HgP24eu65evWqgoODiz0r6I033tBjjz1m5ZoBYC+unnduOHHihN58\n80199dVXyszMVEBAgEaNGqU+ffqUbQUBsAt3yT23KyxeOB5FYgAAAAAAAADwYMxJDAAAAAAAAAAe\njDmJ4VDnzp0rUb8qVaqoatWqdo4GgKcg9wAwArkHgKORdwAYgdzjHphuAg4VFBRUon4TJkxQZGSk\nnaMB4CnIPQCMQO4B4GjkHQBGIPe4B6crEmdmZurQoUOqU6eOypUrZ3Q4sLF9+/aVqJ+fn5/q169v\n52iQm5urc+fOqWXLlqpUqZLR4RiK3OPeyD3OhdxzHXnH/ZF7nAu55zpyj3sj7zgX8s515B33R+5x\nLqXNPU433cShQ4c0bNgwo8MAPMr69evVrl07o8MwFLkHcDxPzz3kHcAY5B5yD+Bo5B3yDmAEa3OP\n0xWJ69SpI+n6C6lXr57B0QDuLS0tTcOGDct/33kycg/gOOSe68g7gGORe64j9wCOQ965jrwDOFZp\nc4/TFYlv/PSgXr16atiwocHRAJ6Bn/yQewAjeHruIe8AxiD3kHsARyPvkHcAI1ibe7ztFAcAAAAA\nAAAAwAVQJAYAAAAAAAAAD0aRGAAAAAAAAAA8GEViAAAAAAAAAPBgTnfhOriPPlM/tvo5iYv62iES\nwH6K2s7ZlgEAtlSS/So+ewDPVZpjrxvIHQDHdYDEmcQAAAAAAAAA4NE4kxiAS/vwww+1Zs0anT59\nWk2bNtW0adPUsWNHo8MCAAAArFKWs4EBACgrziQG4LISEhL08ssv6+mnn1ZiYqLat2+vcePG6eTJ\nk0aHBsANREdHa9asWRZtu3btUt++fdWqVSv16dNHX3zxhUHRAQAAAIDtUCQG4JLMZrNiYmL09NNP\na8CAAbr33ns1ffp0NWrUSAcOHDA6PAAuzGw2a+nSpdq0aZNFe0pKisaOHavw8HAlJCQoLCxM48eP\nV3JyskGRAgAAAIBtMN0EAJf0448/6tSpU+rdu3d+m7e3tz7+mJ/pASi91NRUzZw5U8nJyapfv77F\nY3FxcWrTpo3Gjh0rSZo8ebL27dunuLg4vfrqq0aECzcxb+fyEvRqYPc4AAAA4Lk4kxiAS/r5558l\nSZcuXVJERIQ6duyoYcOGaf/+/cYGBsCl7d+/X35+fkpMTFTDhg0tHktKSlKHDh0s2kJDQ5WUlOTI\nEAEAAADA5jiTGE7FHhdrSFzU1+ZjwniXL1+WJEVFRWnixIny9/fXhx9+qJEjR2rr1q1q0qRJkc+N\niYlRbGyso0KFwYy6CAy5xzX17dtXffsW/r9LS0uTr6+vRVvdunWVlpbmiNAssF0DAGylrJ8pfDa4\nnujoaOXm5mru3LlF9pk0aZI+/fRTi7aOHTvq/ffft3N0cDXsl7oPziQGCtG9e3cFBQVpw4YNhT4+\nevRoBQUFWUxtcOjQIT333HPq0qWLWrdurd69e+udd95RdnZ2fp+YmBgFBQVp4MCBhY67bds2BQUF\n6YknnrBov3z5spYsWaJevXqpdevWuv/++zV16lT99NNPZX+xLqpChQqSpDFjxqhPnz5q0aKFXnzx\nRd13333auHFjsc+NjIzUsWPHLG47duxwRNhAscg9zi0zM1Mmk8mizWQyKSsr647PvfE/uPUWFhZm\nr1CdCts1AOB2fDYYo6jrLhTm+PHjmjp1qnbt2pV/W7p0qQOiBOyH3FM8ziRGiRj1zZCRKlSooO3b\nt2vo0KEW7b/99pu+/vpri7a///3vioqK0mOPPably5fLx8dH3333nd544w3t3btXq1atkre3d/64\nBw8e1OnTp+Xn52cxzrZt2+Tl5WXRdv78eQ0dOlRVqlTR1KlTFRQUpPPnz2vFihUaPHiw4uPjFRAQ\nYIc14Nzq1q0rSQoMDMxv8/Lykr+/v06ePGlUWECZkXucV8WKFXXt2jWLtuzsbFWuXPmOz42MjFRk\nZKRF28mTJz2mUMx2DQC4HZ8NjlXcdRdul52drRMnTqhVq1aqU6eOgyIEHIPcUzTOJAaK8Mc//lF7\n9+5Venq6Rfs///lPtW7dOv/+mTNnNGfOHEVEROiVV15Rq1atdM899+iRRx7R22+/rf/85z8WP9Px\n8/NTo0aNtH37dotxMzIytHPnToWEhFi0v/TSSzKbzYqPj9dDDz2ke+65R8HBwXr77bfl6+ur+fPn\n2+HVO78WLVqoSpUqOnjwYH6b2WzWDz/8oHvuucfAyICyIfc4Lz8/P509e9ai7ezZswWmoEBBbNcA\ngNvx2eBYxV134XY//vijcnJyip3CD3BV5J6iUSQGihAcHKzatWvrX//6l0X7P/7xD/Xu3Tv/fmJi\norKysjRmzJgCY7Ru3VpxcXHq2rWrRXt4eHiB+Z0+//xzNW3aVI0aNcpvO3funHbs2KGRI0eqWrVq\nFv0rVKigRYsWafbs2aV+ja6scuXKGjlypJYsWaLPPvtMP//8s9544w2dOHFCQ4YMMTo8oNTIPc6r\nbdu22rt3r0Xb7t271a5dO4Mich1s1wCA2/HZ4Fh9+/bVggULSnRm8PHjx1WhQgXFxMSoW7du6tmz\np9566y2m2IJbIPcUjSIxUAQvLy/16NHD4lug9PR07d27Vz179sxvO3z4sPz9/XX33XcXOk5oaKju\nuusui7bw8HB98803OnPmTH7b7QlJkr7//nvl5eVZfJt1q4CAAN13333WvjS3MWnSJD311FN6/fXX\n1adPH33zzTdau3at/P39jQ4NKDVyj/MaPny4kpKStGzZMv3www9aunSpvv32W40cOdLo0Jwe2zUA\n4HZ8NjivlJQUSZK/v79WrVqlCRMm6KOPPlJ0dPQdn8v1X+DsyD1Fo0gMFCM8PFy7d+/WxYsXJUmf\nffaZQkJCVLt27fw+ly5dKpAY7qR58+a655579M9//lPS9cnKd+3apV69eln0u3TpkiQVmZQ8nZeX\nl5599ln9+9//1sGDB/XXv/6VM/rgFsg9zikoKEixsbHavn27+vXrp88//1wrV67kp5glxHZdNqaA\nfcXeAMAV8dngnCZPnqxdu3bpiSeeUFBQkPr06aNZs2Zp69atunDhgtHhAWVG7ikcRWKgGG3btpWP\nj0/+N5+FfQNUo0aN/MRijfDw8Pxvrv71r3+pefPmBS4g4OPjI0mlGh+A6yL3OId169Zp7ty5Fm3d\nunXTJ598ooMHD+rjjz9Wp06dDIrO9bBdF2/v4bRibwDgjvhscE7e3t6qUaOGRduNC4anpfGZBNdH\n7imc1UXi6OhozZo1y6JtwIABBeacub0P4Iq8vLzUs2dPbd++Xenp6dq/f78efvhhiz7BwcH66aef\ninxzT58+XevXry/QHh4erqSkJJ0/f16ffvppgW+WJKlly5YqX768vvnmm0LHTkxM1OTJk0s0NxQA\n10HugTtiuwYA3I7PBuc0adIkjR8/3qLt0KFDMplMFvOqeoI+Uz8u9AbXRu4pXImLxGazWUuXLtWm\nTZsKtKekpOjNN9/Url278m8zZsywebCAEcLDw/Xll19q69at6tChg2rWrFng8cqVK2vVqlUFnrtv\n3z5t3bq1wETkktSiRQs1aNBAH3/8sb766iuFh4cX6FO9enU9/PDD+stf/qKMjAyLx7KysrR69Wr9\n9ttvqlixYhlfJQBnQ+6BO2K7BgDcjs8G42VnZ+vcuXPKzs6WJPXs2VM7duzQe++9pxMnTujTTz/V\n/Pnz9eSTT6pq1aoGRwvYBrmnoPIl6ZSamqqZM2cqOTm5wCnSqampunr1qtq0aVOiq2QCriYkJETV\nq1dXbGxsoWfI165dW3PmzNHMmTN1+fJlDRo0SNWqVdOePXu0ePFihYWF6dFHHy107PDwcL399tv6\nwx/+IF9f30L7REVFaejQoRo2bJgmTpyogIAAnTp1Sm+//bbOnDmjJUuW2PT1wjYK+3Y5cVFfAyKB\nqyL3wB2xXQMAbsdng/EOHDigiIgIxcXFKTQ0VL1791Z2drbeffddvfXWW6pVq5YiIiL07LPPGh0q\nYDPknoJKVCTev3+//Pz8tHjxYk2ZMsXisePHj6tSpUpq0KCBXQKEZ3HGIpq3t7d69uypTZs2Ffj5\nwQ39+/dXvXr1tHbtWj3zzDPKyMhQo0aN9Mwzz2j48OEqV65coc8LDw/X6tWrC/35wQ316tXTpk2b\ntGrVKr3++us6e/asatasqQ4dOmju3Lke93MfwFrOmFdKgtyD4rBdF8R2DcDdlfQildnJba0a1+jP\nFD4bHG/dunUW90NDQ3Xs2DGLtn79+qlfv36ODAsuyugcUlrknoK8zGaz2ZonjBgxQo0aNcq/kMuK\nFSu0fv16dejQQXv27JGPj48ee+wxjRw5Ut7e1l8X7+TJkwoLC9OOHTvUsGFDq58P+3DUnDuumlxc\nFe+3m0q7Lqx5b7B9A9eRe65jPeCGeTuXl/nidNnJbfmcuQPec9exHpzXnfYr3bVI7M54v13nCuvB\n2poH7xs4s9K+50p0JnFxUlJSdOXKFXXp0kXPPvus9u/frwULFuj333/XxIkTi31uTEyMYmNjyxoC\nADjUvJ3L8/82BRR9UG/tDjoAAAAAAIARylwknj9/vq5cuaK7775bkhQUFKTff/9dK1euVGRkpLy8\nvIp8bmRkpCIjIy3ablS7AQAAAAAAAAD2Z/18ELcpX758foH4hqCgIGVkZOj3338v6/AAAAAAAAAA\nADsq85nEgwYNUqtWrTR79uz8toMHD6pu3boFiscAAAAAAHiCW6coK4nipjEDAMDeylwkfvjhh7Vs\n2TK1bNlSISEh2r17t9asWaNZs2bZIj4AAAAAAFzCrRe/ougLAHAlZS4Sjx49WuXLl9eKFSv0yy+/\nqH79+poxY4YGDhxoi/gAAAAAwGNER0crNzdXc+fOzW/btWuXFi5cqJ9++kn33nuvnn/+eT3wwAMG\nRgkAANyN1UXidevWWdz38vLSqFGjNGrUKJsFBQAAAACexGw2a9myZdq0aZMGDBiQ356SkqKxY8dq\n3Lhx6tGjhxITEzV+/HglJCQoICDAwIgBAIA7KfOF6wAAAAAApZeamqqIiAht3LhR9evXt3gsLi5O\nbdq00dixY9WkSRNNnjxZwcHBiouLMyhaAADgjigSAwAAAICB9u/fLz8/PyUmJqphw4YWjyUlJalD\nhw4WbaGhoUpKSnJkiAAAwM2VeU5ioKRMAfvu2GfezlM2X25U13FWP6d79+7y9vZWYmKiKleubPHY\niBEj1KhRI82dO1dBQUEKDg7Whg0b5O3tXWCMAQMGaNw465cPwHasvbK4rZB7YE9s14B76du3r/r2\n7VvoY2lpafL19bVoq1u3rtLSuCiaUYrLwUZerM676sX/P4Y7H3fdqqzHYHw2wJXcenFJ2Ab7pe6D\nM4nd0Lydy0t0Q/FSU1O1ePHiO/Y7cOAAP/cDYDPkHrgjtmug9DIzM2UymSzaTCaTsrKy7vjcmJgY\nBQUFWdzCwsLsFSpgFT4bABiB3FM0isRAEe655x7Fx8dr//79d+y3ZMkSpaamOigyAO6M3AN3xHYN\nlF7FihV17do1i7bs7OwCZ0AVJjIyUseOHbO47dixw16hAlbhswGAEcg9RaNIDBShf//+Cg4O1qxZ\ns4o9U+OZZ55R3bp1NWvWLJnNZgdGCMAdkXvgjtiugdLz8/PT2bNnLdrOnj1bYAoKwNXw2QDACOSe\nolEkBorg5eWluXPn6tSpU4qJiSmyX8WKFTV37lzt2bNHH3zwgQMjBOCOyD1wR2zXQOm1bdtWe/fu\ntWjbvXu32rVrZ1BEgG3w2QDACOSeolEkBorRuHFjTZw4UWvXrtWhQ4eK7Ne+fXsNGTJECxcu1OnT\npx0YIQB3RO6BO2K7Bkpn+PDhSkpK0rJly/TDDz9o6dKl+vbbbzVy5EijQwPKjM8GAEYg9xSOIjFw\nB6NGjVKLFi00Y8aMAvPB3er5559XjRo1NGfOHAdGB8BdkXvgjtiuAesFBQUpNjZW27dvV79+/fT5\n559r5cqVatKkidGhATbBZwMAI5B7CipvdADArfYeTrP6Oe1b1LNDJDeVK1dOr7/+uvr376+VK1cW\n2a9q1ap69dVX9eSTT2rLli12jQmA+yP3wB2xXQN3tm7dugJt3bp1U7du3RwfDOAAfDYAMAK5pyDO\nJAZKICAgQGPHjtWqVat04sSJIvt17txZjz/+uObNm6fLly87MEIA7ojcA3fEdg0AuB2fDQCMQO6x\nRJEYKKFnnnlGTZo0UVpa8Wc7z5gxQxUrVtTFixcdFBkAd0bugTtiuwYA3I7PBgBGIPfcxHQTHmze\nzuUl6hfVdZxNlped3NYm49wuakxfu4x7uwoVKuiNN97QwIEDi+1311136ZVXXtGYMWMcEheA4tkq\nhxmF3IPCsF2jMH2mfnzHPomLHLPfBMDx8jKqS7L+uOu/yaVbnq3zCZ8NgGtiv9R9UCQGCvH5558X\n2t68eXMdPnw4//6xY8cK7ffggw8W+RgAFIXcA3fEdg0AjmUK2FeifvY6iack+GwAYARyT/GYbgIA\nAAAAAAAAPBhFYgAAAAAAAADwYEw3gTuat3O5TAHFT+AtGftzJQAAAAAAAAClw5nEAAAAAAAAAODB\nKBIDcHnffPONmjdvrt27dxsdCgAAAAAAgMuhSAzApV25ckUvvPCCcnNzjQ4FAAAAAADAJVEkBuDS\n5s2bJ19fX6PDAAAAAAAAcFlcuA6Ay/riiy/073//W6tXr9af/vQno8MBAAAAXIYpYF+J+nGBcvcT\nHR2t3NxczZ07t8g+Bw8e1Ny5c/X999/L19dX48aNU79+/RwYJQBH40xiAC4pPT1ds2bN0muvvabq\n1atb9dyYmBgFBQVZ3MLCwuwUKQAAAAAYz2w2a+nSpdq0aVOx/dLT0zV69Gi1aNFCW7Zs0YgRIzRr\n1izt2rXLQZECMAJnEgNwSS+++KK6d++u+++/X2lpaVY9NzIyUpGRkRZtJ0+epFAMAAAAwC2lpqZq\n5syZSk5OVv369Yvt++GHH6patWqaNWuWvL291aRJEx05ckRr165Vly5dHBQxAEfjTGIALichIUFH\njhzR9OmrtvWUAAAgAElEQVTTjQ4FAAAAAJze/v375efnp8TERDVs2LDYvklJSWrfvr28vW+WjDp0\n6KD9+/fLbDbbO1QABuFMYgAuZ8uWLTpz5kz+t9g3dlSefvpp9evXT6+88oqR4QEAAACAU+nbt6/6\n9u1bor5paWlq3ry5RVvdunV19epVXbhwQTVr1izyuTExMYqNjS1TrACMYXWRuLAJznft2qWFCxfq\np59+0r333qvnn39eDzzwgE0DBYAb3nzzTWVmZubfP3funIYNG6bXXntNnTt3NjAyAAAAAHBtmZmZ\nMplMFm037mdnZxf7XKb2A1xXiYvEZrNZy5Yt06ZNmzRgwID89pSUFI0dO1bjxo1Tjx49lJiYqPHj\nxyshIUEBAQF2CRqAZ/P19bW4X7Fixfz2WrVqGRFSifSZ+nGBtsRFJfs2HwAAAAAcoVKlSgWKwTfu\nV65c2YiQADhAiYrExU1wHhcXpzZt2mjs2LGSpMmTJ2vfvn2Ki4vTq6++avuIAQAAAAAAYBf16tXT\nuXPnLNrOnj2rKlWq6K677jIoqtKZt3O5xX1TQOEXPc9ObuuIcACnVqIi8Y0JzhcvXqwpU6ZYPJaU\nlKRevXpZtIWGhuqTTz6xXZQAUIx69erp2LFjRocBAAAAAC6vbdu22rJli8xms7y8vCRJu3fvVkhI\niMXF7DxZYb8SlfilKFxbiYrExU1wnpaWVuCn33Xr1lVaWuHfzqD0bv8GDAAAAM7PEftwpoB9d+zD\nWVIAgMJkZ2fr4sWLql69ukwmkwYMGKA1a9boxRdf1MiRI/Xll1/q73//u1avXm10qADsqMxfARU1\noXlWVtYdnxsTE6OgoCCLG5OZAwAAAAAAOMaBAwfUpUsXHThwQJJUu3ZtrVmzRkeOHFG/fv0UHx+v\n+fPnq2PHjgZHCsCeSnzhuqJUrFhR165ds2jLzs4u0WTmXPUSAAAAAADAcdatW2dxPzQ0tMD0fW3a\ntNFHH33kyLAAGKzMZxL7+fnp7NmzFm1nz54tMAUFAAAAAAAAAMD5lPlM4rZt22rv3r0Wbbt371a7\ndu3KOjQAAAAAAA7BNWAAAJ6szGcSDx8+XElJSVq2bJl++OEHLV26VN9++61Gjhxpi/gAAAAAAAAA\nAHZU5iJxUFCQYmNjtX37dvXr10+ff/65Vq5cqSZNmtgiPgAAAAAAAACAHVk93cTtE5xLUrdu3dSt\nWzdbxAMAAAAAAAAAcKAyn0kMAAAAAAAAAHBdZb5wHQAAgKdISUnRI488UqB9/fr1XLQXAAAAgMui\nSAwAAFBCx48fl4+PjxITEy3aa9SoYVBEAAAAAFB2FIlhM6aAfSXql53c1s6RAM6hJO+JeTtPKarr\nOAdEA8AWjh8/rqZNm6pOnTpGhwIAAAAANsOcxAAAACWUnJwsf39/o8MAAAAAAJuiSAwAAFBCycnJ\n+uWXXzRo0CB17txZTzzxhL777jujwwIAAACAMmG6CQAAgBLIzMxUamqqatasqRdeeEEmk0nx8fEa\nPny4EhIS1KRJkyKfGxMTo9jYWAdGCwAAAAAlR5HYCczbudxhy9p7OM1hywIAwJ1UqlRJe/fulclk\nkslkkiTNmzdPhw8f1oYNGzRnzpwinxsZGanIyEiLtpMnTyosLMyuMQMAAABASVAkBgAAKKFq1apZ\n3Pf29lbTpk11+vRpgyICAABAWZXkouM3ZCe3tWMkgHGYkxgAAKAEDh06pJCQEB06dCi/LTc3V0eP\nHlVAQICBkQEAAABA2VAkBgAAKIFmzZqpQYMGio6O1rfffqvk5GTNmDFDFy5cUEREhNHhAQAAAECp\nUSQGAAAogfLly2vNmjVq3LixxowZo4EDB+r8+fOKj49XrVq1jA4PAAAAAEqNOYkBAABKyNfXV4sW\nLTI6DAAAAACwKYrEAAAAAODEUlJS9MgjjxRoX79+vdq1a2dARAAAwN1QJAYAAAAAJ3b8+HH5+Pgo\nMTHRor1GjRoGReTe9h5OMzoEAAAcjiIxAAAAADix48ePq2nTpqpTp47RoQAAADdFkRgAAAAAnFhy\ncrL8/f2NDsNlzdu53OgQ3FKfqR+X+rmJi/raMBIAgC1QJAYAAAAAJ5acnKysrCwNGjRIp06dUkBA\ngKZMmaJWrVoV+7yYmBjFxsY6KEoAAODKKBIDAAAAgJPKzMxUamqqatasqRdeeEEmk0nx8fEaPny4\nEhIS1KRJkyKfGxkZqcjISIu2kydPKiwszN5hw42YAvaVqF92cls7RwIAsCeKxAAAAADgpCpVqqS9\ne/fKZDLJZDJJkubNm6fDhw9rw4YNmjNnjsERAgAAd0CRGAAAAACcWLVq1Szue3t7q2nTpjp9+rRB\nEQEAAHdDkRgurzQXTOBCCQAAwBOVZL+J/STncujQIUVERCguLk4tW7aUJOXm5uro0aMKDw83ODoA\nAOAuKBIDcFnnz5/XwoUL9d///leZmZlq3bq1pk+frsDAQKNDAwAAsIlmzZqpQYMGio6O1osvvqgq\nVapo9erVunDhgiIiIowODwBwi6K+jOULWLgCisR2NG/ncqNDANxWXl6eJkyYILPZrOXLl6tKlSqK\niYnRE088oU8++UQ+Pj5GhwgAAFBm5cuX15o1a7RgwQKNGTNGV69eVUhIiOLj41WrVi2jwwMAAG6C\nIjEAl3T06FEdOHBA27Zty7+q98KFC9WhQwd98cUX6tevn8ERAgAA2Iavr68WLVpkdBgAXFhubq6W\nLFmihIQEZWRkqGvXroqOjlbt2rUL7T9p0iR9+umnFm0dO3bU+++/74BoARiBIjEAl+Tn56dVq1ap\ncePG+W1eXl6SpIsXLxoVFgAAAAA4nZiYGCUkJGj+/PmqUaOGXn75ZUVGRmrjxo2F9j9+/LimTp2q\n/v3757eZTCZHhVtmN6Z9MAWkGRwJ4DooEsPhTAH7StQvO7mtnSOBK/Px8VG3bt0s2tatW6fMzEx1\n6dLFmKAAACjE3sMcoAIAjJOdna24uDjNnj1bnTt3liQtXrxYYWFh2r9/v0JCQgr0P3HihFq1aqU6\ndeoYETIAA9ikSJySkqJHHnmkQPv69evVrl07WywCAIq1Y8cOLV68WKNGjcqffqIoMTExio2NdVBk\nAAAAAGCco0ePKiMjQx06dMhva9iwoRo0aKCkpKQCReIff/xROTk5dzyuAuBebFIkPn78uHx8fJSY\nmGjRXqNGDVsMDwDF2rJli+bMmaPevXtr2rRpd+wfGRmpyMhIi7aTJ08qLCzMXiECAAAAgCHS0q7/\nosXX19eivW7duvmP3er48eOqUKGCYmJi9J///EcVK1ZUeHi4xo0bp4oVKxa7LE84IYdfR8Nd2axI\n3LRpU36GAMDhVqxYoSVLlmj48OGaPXt2/rzEAAAAAADp6tWr8vb2VoUKFSzaTSaTsrKyCvRPSUmR\nJPn7+2vYsGE6fvy45s2bp7S0NM2fP7/YZXFCDuC6bFIkTk5Olr+/vy2GAoASW716tZYsWaKJEydq\n/PjxRocDAAAAeCzOrnRelSpVUl5ennJyclS+/M0yUHZ2tipXrlyg/+TJk/Xkk0/m/zo8KChI5cqV\n03PPPaeoqCj5+Pg4LHYAjmOzInFWVpYGDRqkU6dOKSAgQFOmTFGrVq2KfZ4n/AwBgH0cPXpUb731\nlh5//HENGjRI586dy3+satWqqlKlioHRAQAAAIBz8PPzkySdO3cu/29JOnv2bIEpKCTJ29u7wPSh\ngYGBkq5PXUGRGHBPZS4SZ2ZmKjU1VTVr1tQLL7wgk8mk+Ph4DR8+XAkJCcVOdM7PEACU1rZt25Sb\nm6vNmzdr8+bNFo9NmjRJ48aNMygyAACcU0nO8uMMPwBwP82aNVPVqlW1Z88e9e3bV9L12supU6fU\nvn37Av0nTZqknJwcvf322/lthw4dkslkUqNGjRwWNwDHKnORuFKlStq7d69MJpNMJpMkad68eTp8\n+LA2bNigOXPmlDlIALjdlClTNGXKFKPDKLO9h9PUZ+vHFm2Ji/oaFA0AAAAAd2MymTR06FAtWLBA\nPj4+qlWrll5++WV16NBBbdq0UXZ2ti5evKjq1avLZDKpZ8+emjJlit577z2FhYXpyJEjmj9/vp58\n8klVrVrV6JcDwE5sMt1EtWrVLO57e3uradOmOn36tC2GBwAAAAAAQClNnjxZOTk5mjZtmnJyctS1\na1dFR0dLkg4cOKCIiAjFxcUpNDRUvXv3VnZ2tt5991299dZbqlWrliIiIvTss88a/CoA2FOZi8SH\nDh3KTyYtW7aUJOXm5uro0aMKDw8vc4AAAAAAAAAovfLlyysqKkpRUVEFHgsNDdWxY8cs2vr166d+\n/fo5KjwATqDMReJmzZqpQYMGio6O1osvvqgqVapo9erVunDhgiIiImwRIwAAAAAAACBJmrdzebGP\nmwLSHBQJ4D68yzpA+fLltWbNGjVu3FhjxozRwIEDdf78ecXHx6tWrVq2iBEAAAAAAAAAYCc2mZPY\n19dXixYtssVQLuFO31gBAAAAAAAAgKuwSZEYAAAA8EScPAAAAAB3QJEYTssUsK9E/bKT29o5EgAA\nAAAAANsr6RfOUV3H2TkSeLoyz0kMAAAAAAAAAHBdnEkMAAAAIF+fqR/fsU/ior4OiAQAANd166+j\n5+08ZWAkQMlQJIZHKsnBz+04GAIAAAAAAIA7okgMAAAAAHA5XDgSAADboUgMAAAAAAAcpjS/7LyB\nX3gCgH1QJAYAAAAAAA5x6zytxclObmvnSAAAt/I2OgAAAAAAAAAAgHEoEgMAAAAAAACAB2O6CQBw\nMoXN0cbcawAAAAAAwF4oEruwvYfTjA4BAAAAAAAAgIujSAwABivJxTvm7TylqK7jHBANAAAAAADw\nNBSJ4fK4Oi4AAAAAAABQehSJAQAAAAAAADsparrQ9i3qOTgSoGjeRgcAAAAAAAAAADAOZxIDJdRn\n6sdWPydxUV87RAIAAGAfd5rGi+m7AAAwxrydy0vUj2vZoLQ4kxgAAAAAAAAAPBhFYgAAAAAAAADw\nYEw3AQAAAABwO0VdKAoAABREkRgAAAAAADiVouZIn7fzlMV95l8FANugSAyPcacLsdxgywuylOZi\ndxIXvENBew+nqc/WgtsT2woAAHAVJb3oElCc288QL2wfuSjsOwNA0SgSA7cxopgMAAAAAAAAGIUi\n8S34ZhsAAAAAAADuztY1MKZ+cX0UiQHARRR2lvvtc7JJfDgDAAAAhSntdIASU1XAdbjLCZAlfR0c\n/9qORxSJ3eUNAgAAAADOhuMtGInpAgHnQFHX9dmkSJybm6slS5YoISFBGRkZ6tq1q6Kjo1W7dm1b\nDA8AhSL3FLxwhySpq+PjADwFeQeAEcg9AMrK2jxy8OBBzZ07V99//718fX01btw49evXz8FRwx3x\nxaLzskmROCYmRgkJCZo/f75q1Kihl19+WZGRkdq4caMthgeAQpF7ClfYz+j4eRxgG+Qdz8OBDJwB\nuQdAWVmTR9LT0zV69Gg9+uijmjt3rr788kvNmjVLtWvXVpcuXQyI3vMUejKQpPYt6jk4EniSMheJ\ns7OzFRcXp9mzZ6tz586SpMWLFyssLEz79+9XSEhImYMsCjvtMJKtf9Z063iFzTN7Az/NuM7I3OPs\nmLsYsA/yDnBTSeb15AtK2yD3AI5V1HHe7fvTrrQvbW0e+fDDD1WtWjXNmjVL3t7eatKkiY4cOaK1\na9dSJLaxoorB1vb35OIx01zYTpmLxEePHlVGRoY6dOiQ39awYUM1aNBASUlJ7LTA45W0mAzrkHsA\nOBp5ByjZfs2NL8jLcoGoW3l6sdnI3MNJOYB7sDaPJCUlqX379vL29s5v69Chg15++WWZzWZ5eXnZ\nLVbyDuyFYvKdlblInJZ2/VsMX19fi/a6devmP2aN3Nxci3GL8/v5S1aP706yL2YYHQLs5L9f/lDk\nY32+nFpoe6vAOhb3x7Yffsfl3Hif3XjfuRJnyT2u8j4sbJsqbFu6fTuyRkm2OXtYsTfekOWWlKet\nF3fOPUbmnZL8P43a1lyZo94nrvJZYSvXrqTbdLzwse/dsc+aWQ/fsQ+55zpPOt7ytPceilfi3FTE\ndnP7++HkyZN3HMpZ8o61eSQtLU3Nmzcv0Pfq1au6cOGCatasadXyjcg7nvb+d/V87UxmJcyz6Xgl\n3Ucv6X6pPY+3ylwkvnr1qry9vVWhQgWLdpPJpKysrGKfGxMTo9jY2EIfGzZsWFlDAzzK8dvuf6Q7\nH1DdcO7cOd177722DcjOyD32cft2ZA1rtjlP4mnrxZ1zj7PnHU/b1uDM/u7wJYZ9XvIDOnLPdZ6+\nzwNPVLbc5MrHW9bmkczMTJlMpgJ9petTVxSHvGOMshzHwb5svY9uz9xT5iJxpUqVlJeXp5ycHJUv\nf3O47OxsVa5cudjnRkZGKjIy0qItMzNThw4dUp06dVSuXLmyhucWwsLCtGPHDqPD8BietL5zc3N1\n7tw5tWzZ0uhQrOYsuceTthdXwP/DuRT1/3DV3OMsecfZ8L67iXVxkzOuC3LPdbbIPc74/3U3rGPH\nsPd6dpa8Y20eqVSpUoFi8I37zr7Pw3un7FiHtmHkeixt7ilzkdjPz0/S9er0jb8l6ezZswV+ylAS\nlSpVUrt27coalttp2LCh0SF4FE9a3650Js2tnCn3eNL24gr4fziXov4frph7nCnvOBvedzexLm5y\nxnVB7rFd7nHG/6+7YR07hr3XszPkHWvzSL169XTu3DmLtrNnz6pKlSq66667rF6+o/d5eO+UHevQ\nNoxcj6XJPd537lK8Zs2aqWrVqtqzZ09+28mTJ3Xq1Cm1b9++rMMDQKHIPQAcjbwDwAjkHgBlZW0e\nadu2rZKSkmQ2m/Pbdu/erZCQEIuL2QFwL2U+k9hkMmno0KFasGCBfHx8VKtWLb388svq0KGD2rRp\nY4sYAaAAcg8ARyPvADACuQdAWd0pj2RnZ+vixYuqXr26TCaTBgwYoDVr1ujFF1/UyJEj9eWXX+rv\nf/+7Vq9ebfRLAWBHZS4SS9LkyZOVk5OjadOmKScnR127dlV0dLQthgaAIpF7ADgaeQeAEcg9AMqq\nuDxy4MABRUREKC4uTqGhoapdu7bWrFmj1157Tf369VP9+vU1f/58dezY0eBXAcCebFIkLl++vKKi\nohQVFWWL4XCbCRMmGB2CR2F9uw5nyD1sL86F/4dzccf/hzPkHWfjjv/n0mJd3MS6sC1nyz38f+2P\ndewYnrSei8sjoaGhOnbsmEVbmzZt9NFHHzkqPJvxpP+pvbAObcMV16OX+dZJZgAAAAAAAAAAHoUZ\nxwEAAAAAAADAg1EkBgAAAAAAAAAPRpEYAAAAAAAAADwYRWIAAAAAAAAA8GAUiQEAAAAAAADAg1Ek\nNlhubq4WLVqkLl26KDg4WBMnTtT58+eL7H/w4EENHjxYrVu3Vo8ePbR161YHRuv6rF3fN5w4cULB\nwcFKS0tzQJRwBaXdlmBf0dHRmjVrltFheLTz589r+vTp6tKli9q1a6ennnpKx48fNzos2Mivv/6q\nSZMmqV27durYsaMWLlyonJycIvtfu3ZNsbGxeuihh9SmTRv1799f//rXvxwYse2wz3aTteti27Zt\n6tu3r9q0aaOHH35Y77zzjnJzcx0YMazBtu4YHJfYH7nKc3BsZhvsx9vWN998o+bNm2v37t1Gh1Ji\nFIkNFhMTo4SEBM2fP1/x8fFKS0tTZGRkoX3T09M1evRotWjRQlu2bNGIESM0a9Ys7dq1y8FRuy5r\n1vcNP/30k5588klduXLFQVHCFZRmW4L9mM1mLV26VJs2bTI6FI+Wl5enCRMm6Oeff9by5cv1wQcf\nqFq1anriiSd04cIFo8ODDURGRur8+fOKj4/XvHnztGXLFsXExBTZf8mSJfrggw80c+ZMffzxxwoP\nD1dkZKT27t3rwKhtg322m6xZF1988YWef/55DRw4UH/72980depUrV69WitXrnRw1CgptnXH4LjE\n/shVnoNjs7JjP962rly5ohdeeMH1vmgywzBZWVnm4OBg8+bNm/PbUlNTzYGBgeZ9+/YV6L9y5Upz\n9+7dzbm5ufltUVFR5lGjRjkkXldn7fo2m83m999/3xwcHGzu37+/OTAw0Hz69GlHhQsnVpptCfZz\n4sQJ8/Dhw82hoaHmbt26mWfOnGl0SB7r8OHD5sDAQHNKSkp+W1ZWlrl169bmhIQEAyODLezfv98c\nGBhoPnHiRH7bli1bzMHBweasrKwC/XNzc83t27c3r1+/3qI9IiLCHBUVZfd4bYl9tpusXRdjxowx\nT5o0yaItNjbW3L17d7vHCuuxrTsGxyX2R67yHByb2Qb78bY1Z84c8/Dhw82BgYHmr7/+2uhwSowz\niQ109OhRZWRkqEOHDvltDRs2VIMGDZSUlFSgf1JSktq3by9v75v/tg4dOmj//v0ym80OidmVWbu+\nJWnHjh169dVXNX36dEeFCRdQmm0J9rN//375+fkpMTFRDRs2NDocj+bn56dVq1apcePG+W1eXl6S\npIsXLxoVFmwkKSlJDRo00D333JPf1qFDB2VkZOj7778v0D8vL09LlixRjx49LNq9vb116dIlu8dr\nS+yz3WTtuhg7dqwmTJhg0eaK24CnYFt3DI5L7I9c5Tk4NrMN9uNt54svvtC///1vzZ492+hQrEaR\n2EA35pHy9fW1aK9bt26hc0ylpaUV2vfq1auc/l8C1q5vSYqLi9Mjjzxi99jgWkqzLcF++vbtqwUL\nFqhOnTpGh+LxfHx81K1bN4tiwbp165SZmakuXboYGBls4cyZM6pbt65F2437p0+fLtC/fPny6tSp\nk2rXrp3f9t133+nrr79W165d7RusjbHPdpO166JVq1Zq2rRp/v3Lly9r48aNLrcNeAq2dcfguMT+\nyFWeg2Mz22A/3jbS09M1a9Ysvfbaa6pevbrR4VitvNEBeLKrV6/K29tbFSpUsGg3mUzKysoq0D8z\nM1Mmk6lAX0nKzs62X6Buwtr1DRSFbQkomR07dmjx4sUaNWqUmjRpYnQ4uIOTJ08qLCys0MdMJpP+\n9Kc/qWLFihbtFSpUkJeXV4ly3//+9z9NmDBBrVq10uOPP26TmB2FfbabyvIZePXqVY0bN05ZWVma\nOnWqPcNEKbGtOwb7kvZHrvIcvJ/sg/340nnxxRfVvXt33X///S75JQVFYgNVqlRJeXl5ysnJUfny\nN/8V2dnZqly5cqH9b9/ZunG/sP6wZO36BorCtgTc2ZYtWzRnzhz17t1b06ZNMzoclICvr6+2bdtW\n6GPe3t6Kj48vsB9y7do1mc1mValSpdixDx06pGeffVY1a9bUypUrCxzIOTv22W4q7Wdgenq6xo0b\np5SUFK1du1YNGjRwRLiwEtu6Y7AvaX/kKs/B+8n22I8vnYSEBB05ckR/+9vfjA6l1JhuwkB+fn6S\npHPnzlm0nz17tsBPJSSpXr16hfatUqWK7rrrLvsF6iasXd9AUdiWgOKtWLFCM2bM0ODBg7VgwQKL\nn63BeVWoUEFNmjQp9Na4ceMi90Okgj/xvNWuXbs0YsQINWrUSPHx8fLx8bHr67AH9tluKs1n4MmT\nJzVkyBCdPHlS8fHxatWqld3jROmwrTsG+5L2R67yHLyfbIv9+NLbsmWLzpw5oy5duig4OFjh4eGS\npKefflrR0dEGR1cy/LcN1KxZM1WtWlV79uzJbzt58qROnTql9u3bF+jftm1bJSUlWVwEYvfu3QoJ\nCeGNWwLWrm+gKGxLQNFWr16tJUuWaOLEiZozZ07+BS/g+tq2bavU1FSL+Yd3796tqlWrqlmzZoU+\nJykpSWPHjlVoaKjee+89l5ybTWKf7VbWrotff/1VERERysvL08aNG4vcVuAc2NYdg31J+yNXeQ7e\nT7bDfnzZvPnmm/rkk0+0detWbd26VWvWrJEkvfbaa5o0aZLB0ZVMuZdeeuklo4PwVOXKldPvv/+u\nd999VwEBAbp8+bJmzpype++9V+PGjVN2drbS09NVoUIFlStXTvfdd59Wr16tU6dOqVGjRvrkk0/0\n3nvv6aWXXrK40jgKZ+36vtWpU6eUkJCgUaNGqVq1aga9AjiLO21LME5CQoKqV69e5LyqsK+jR4/q\nueee02OPPabRo0frypUr+TcvLy+Xm2IAlurVq6ddu3Zp+/bt+r//+z99//33euWVVxQREaFOnTpJ\nkjIyMnTx4kVVrVpV2dnZGj58uHx9fbVo0SLl5OTkbw85OTmqVKmSwa+o5Nhnu8nadREVFaVjx45p\nxYoV8vHxyd8Grl69esdpSuB4bOuOwXGJ/ZGrPAfHZrbBfnzZVatWTTVq1Mi/eXt76/3339eIESMU\nEBBgdHglwpzEBps8ebJycnI0bdo05eTkqGvXrvmnoR84cEARERGKi4tTaGioateurTVr1ui1115T\nv379VL9+fc2fP18dO3Y0+FW4DmvWN1Cc4rYlwFNt27ZNubm52rx5szZv3mzx2KRJk9hRd3FeXl6K\njY3VSy+9pGHDhqlq1aoaOHCgxo8fn99n7dq1io2N1bFjx7Rnzx6lpaUpLS1N3bp1sxirY8eOev/9\n9x37AsqIfbabSrouWrdurX/+85/Ky8vTwIEDLcYoV66cjhw5YkT4uAO2dcfguMT+yFWeg2OzsmM/\nHpLkZb71t0EAAAAAAAAAAI/CRFEAAAAAAAAA4MEoEgMAAAAAAACAB6NIDAAAAAAAAAAejCIxAAAA\nAAAAAHgwisQAAAAAAAAA4MEoEgMAAAAAAACAB6NIDAAAAAAAAAAejCIxAAAAAAAAAHgwisQAAAAA\nAAAA4MEoEgMAAAAAAACAB6NIDAAAAAAAAAAejCIxAAAAAAAAAHgwisQAAAAAAAAA4MEoEgMAAAAA\nAACAB6NIDAAAAAAAAAAejCIxAAAAAAAAAHgwisQAAAAAAAAA4MEoEgMAAAAAAACAB6NIDAAAAAAA\nAJsrzuwAACAASURBVAAejCIxAAAAAAAAAHgwisQAAAAAAAAA4MEoEruxX3/9VVeuXJEkjRgxQt27\nd7fp+JcvX1Z6erpNx7zd+vXrFRQUVOgtIyPDrssGUDrukHskacOGDXrkkUfUqlUr9ezZU2vXrlVu\nbq7dlwugdFw994wYMaLIfZ6goCBFRUXZbdkASsfV844kZWRk6LXXXlOXLl3UsmVLPfroo9q6datd\nlwmgbNwh9/z222+Kjo7WH//4R4WEhCgiIkJ79+616zJxZxSJ3dQXX3yh8PBwu72xDx06pF69eik5\nOdku49+QkpKiGjVqaMGCBQVuFStWtOuyAVjPXXLPW2+9pZdfflkBAQGaOXOmgoKCNH/+fMXExNh1\nuQBKxx1yz5gxYwrd3wkJCZEkPfjgg3ZbNgDruUPekaTIyEjFx8erW7dumjlzpnx8fDR9+nT99a9/\ntetyAZSOO+SezMxMjRw5Ups2bdKDDz6oKVOmSJKeeOIJff7553ZbLu6svNEBwD6+++47Xbp0yW7j\nHz9+XGfPnrXb+DckJycrMDBQffv2tfuyAJSdO+SeH374QatXr9aAAQM0d+5cSdLgwYM1YcIEvfvu\nuxozZowqVapk1xgAWMcdck/nzp0LtB08eFDfffedBg0apJ49e9p1+QCs4w5559tvv9V///tfjRgx\nQrNnz5Yk/fnPf1b//v21ZMkSDRgwQN7enFcGOBN3yD0bNmzQ0aNH9f+xd+9hUdb5/8dfchhRLM8g\nQraJpytLBQNqwb5tbGr+Mu245oEszVoNMw3DPBS7WoJJGnYwSwttta3Esm2zsvJwdTAO7eYBD7mW\nkCiIWSIyDczvDy4mpwHkMMzx+bgurks+9z0f3vc0vLp5zz2fe+bMmbr//vslSXfddZcmTpyolJQU\nxcbGclGgk5D4cGkHDx5Uz549nV0GAC/y7rvvSpLlHe0a9913n6ZMmaIzZ844oywAXiglJUXt2rVT\nUlKSs0sB4IGOHj0qSYqLi7OM+fr6KjY2VidPntTJkyedVRoAD/bpp5+qXbt2uueeeyxjvr6+uuee\ne1RUVKTPP//cidV5N5rEHig5OVkrVqyQJMXHx2vChAmWbTt37tStt96qK6+8Utddd52ef/55VVVV\nWT3+0KFDmjZtmq666ioNHDhQY8aM0Y4dOyzbMzIyNGfOHElSQkKC1fo3X3zxhSZPnqyYmBj1799f\nQ4YM0YIFC5r0TldJSYl++ukn9erVS5JUXl5uUysA1+Ep2ZOTk6O+ffuqc+fOkn7LnoEDByoxMVFd\nunRp9JwAWo6nZM/vffjhh/r222/14IMP6uKLL272fADsx1Ny59JLL5Uk/e9//7MaP3r0qAwGg9q3\nb9/oOQG0HE/JnuPHj+vSSy+VwWCwGq/JpL179zZ6TtiH7xNPPPGEs4uAfXXu3FmnT5/W4cOHNWfO\nHA0bNkzZ2dk6fPiwtmzZohtvvFE33XSTjh07pqysLF188cUaNGiQJGn//v266667dO7cOSUkJCg2\nNlb5+flavXq1evbsqd69e6t9+/Yym83as2ePHnjgAd1+++3q2bOndu7cqcmTJ6tr164aP368rr32\nWhmNRm3evFlHjx7VjTfe2Kjj+M9//qNNmzbp4osv1pIlS5SWlqY1a9bo2LFjuvrqq+Xv798STx+A\nJvKU7ElPT1evXr0UEBCgBx98UE899ZTWrFmjkydP6pprrpGvr29LPH0AmshTsuf3aq4eXrJkCbkD\nuBhPyZ3g4GD9+OOPevPNN3XppZeqdevWevvtt7V27VpNmjRJQ4YMaYmnD0ATeUr2vPXWWyovL7dq\nckvS4cOHtXHjRoWHh+u6666z19OGxjDDIz377LPmPn36mI8ePWo2m83m8ePHm/v06WP+8MMPLfv8\n8ssv5sjISPPYsWMtY+PHjzf/+c9/NpeVlVnGfv31V/PYsWPNf/zjH80VFRVms9lsfvvtt819+vQx\nf/nll5b9Jk2aZP7Tn/5k2afGnXfeaY6IiGj0Mbz22mvmPn36mP/85z+b169fb96yZYt57ty55j59\n+pgnTpxorqqqavScAFqWJ2TPgAEDzPHx8eaBAwealy9fbt6yZYv5scceM/fp08c8e/bsRs8HoOV5\nQvac75tvvjH36dPH/PzzzzdrHgAtx1Ny58CBA+bhw4eb+/TpY/maPn26ubKysknzAWhZnpA98+fP\nN/fr18+8b98+q/HFixeb+/TpY54zZ06j54R9sNyEF2nTpo3VxwXatWunnj17qqSkRJJ06tQp7dq1\nS//3f/+nc+fOqbS0VKWlpfr55591ww03qKSkRN9++22d869cuVJvv/221UcGTp06pXbt2uns2bON\nrveKK67QAw88oDfeeENjxozR0KFDtXDhQk2ZMkWff/65Pvvss0bPCcDx3C17jEajjh49qieeeELT\np0/X0KFDtWjRIt12223atGlTi99lHIB9uFv2nO+f//yn/P399Ze//KVZ8wBwLHfLnfz8fN15550q\nLS21fIx9woQJ+vDDD5WUlCSz2dzoOQE4nrtlz9133y1/f39NmzZN27Zt09GjR7VmzRpt3LhRfn5+\n8vPza/ScsA+eeS/SoUMHm48rBgQEWG5IUHPjgrVr12rt2rW1znHs2LE65/f19dXRo0e1fPlyHTp0\nSD/88IOOHz/e5HojIyMVGRlpM/6Xv/xFL730kr766iv96U9/avL8ABzD3bKnTZs2MplMuvnmm63G\nR48erbffflu7du1S7969mzw/AMdwt+ypYTab9dlnnykmJkadOnVq9nwAHMfdcmflypWqqKjQunXr\n1L9/f0nSDTfcoG7dumnJkiW68cYb9ec//7nJ8wNwDHfLnvDwcD333HNKTk7WlClTJEkhISFasmSJ\npkyZwnroTkST2Iv4+NR/4XhlZaUkady4cXWeDNTcRK42r7zyitLS0nTZZZfpqquu0tChQzVw4ECt\nXbtWmzdvbnrhv1NzM6nmXqUDwDHcLXuCg4N19uxZm7prsqesrKzRcwJwPHfLnhq7d+9WSUmJbrjh\nhibPAcA53C13Dhw4oPDwcEuDuMatt96qJUuWaNeuXTSJATfgbtkjSUOGDNGnn36qffv2yc/PT/36\n9dPRo0dlNpt1ySWXNGlONB9NYliEhoZKqn6X6I9//KPVtkOHDqmgoEBt2rSp9bEVFRXKyMhQTEyM\nVq9ebfXxgOXLlzepnscee0x79uzRpk2b1KpVK8v44cOHJUlhYWFNmheAa3G17Onfv7/ef/99nT59\n2upd7IKCAknV73IDcH+ulj01cnNzJUnXXHNNs+YB4HpcLXcMBoOMRqPNeE1DqaqqqknzAnAtrpY9\n3377rfbt26c777xTAwcOtIx//fXXkqTBgwc3aV40H2sSe6iad5Ias45UUFCQrrjiCmVlZVl9dODX\nX3/VY489punTp8tkMlnNX3PicO7cOZWXl+sPf/iDVWjs27dPu3btkiTLYxuqa9euys/P18cff2wZ\nM5vNeuGFF+Tn59fsu4YDsD9PyJ6RI0eqsrJSq1evtoyZzWZlZmYqICBA1157baPmA9DyPCF7auTn\n5yswMFA9evRo0uMBOIYn5E5sbKwOHTpkaczUeOONNyRJV199daPmA9DyPCF7/vvf/2r+/PnKy8uz\njP3000965ZVXFBMTo/Dw8EbNB/vhSmIPVbOG3csvv9yohsa8efN0991367bbbtNdd92lDh066F//\n+pf+85//aNasWerYsaPV/OvXr1dJSYlGjhypgQMHauPGjWrXrp0uu+wyHTx4UG+++aYlZMrKyhq1\ntszkyZO1efNmPfLIIxo/fry6d++uDz/8UF9++aVmzZrFRxAAF+QJ2fN///d/Gj58uF588UWdOHFC\nV155pT755BPt3LlTycnJrJEFuCBPyJ4a33//vUJCQqw+RQXA9XhC7kyePFlbtmzRlClTNHbsWPXo\n0UO7du3Se++9pyFDhig+Pr7BcwFwDE/InptuukmrVq3S9OnTdffdd6t169basGGDTp48qWeffbbB\n88D+Wpm5ZalH+vnnn/XQQw8pOztbYWFh6tKliwoLC/XJJ59Y7TdhwgSb8T179igjI0PZ2dkymUy6\n7LLLlJCQoFtuucWyz6+//qqkpCR9+umnat26tXbs2KHS0lI99dRT+uqrr2Q0GhUaGqr/9//+n8LD\nw5WYmKhnn31Ww4YNa9RxHDt2TE8//bR27typs2fPKjw8XBMnTtTo0aOb9wQBaBGekj2//vqrVq1a\npbffflsnTpzQpZdeqkmTJlnVAsB1eEr2SNKIESN00UUXWa7kA+CaPCV3Tp48qWeeeUaffvqpTp8+\nrW7duunmm2/WAw88IIPB0LwnCYDdeUr2HDlyREuWLFF2drak6iUmHn74YW4Q7mQ0iQEAAAAAAADA\ni7EmMQAAAAAAAAB4MdYkhkMVFxc3aL+2bdsqMDCwhasB4C3IHgDOQPYAcDRyB4AzkD2egeUm4FB9\n+/Zt0H4PPvigEhMTW7gaAN6C7AHgDGQPAEcjdwA4A9njGVyuSXzu3Dnt3r1bXbt2la+vr7PLgZ3l\n5OQ0aL+QkBB17969hatBZWWliouLdcUVVyggIMDZ5TgV2ePZyB7XQvZUI3c8H9njWsieamSPZyN3\nXAu5U43c8Xxkj2tpavbYfbmJb775RmPHjtWaNWsUExPT6Mfv3r1b48aNs3dZAOrx+uuv66qrrnJ2\nGU5F9gCO5+3ZQ+4AzkH2kD2Ao5E75A7gDI3NHrs2ic+ePavZs2ersrKyyXN07dpVUvWBdOvWzV6l\nAahFUVGRxo0bZ/m982ZkD+A4ZE81cgdwLLKnGtkDOA65U43cARyrqdlj1ybx4sWLFRwcrO+//77J\nc9R89KBbt24KCwuzV2kA6sFHfsgewBm8PXvIHcA5yB6yB3A0cofcAZyhsdnjY68fvG3bNn322Wea\nN2+evaYEAAAAAAAAALQwu1xJXFpaqrlz5+rJJ59U+/bt7TElAAAAAAAAAMAB7HIl8eOPP67rr79e\n1157baMel5GRob59+1p9xcfH26MkAG6msrJSS5cuVVxcnCIiIjR9+nSVlJTUuX9RUZGmT5+uiIgI\nXXPNNXriiSdUXl7uwIoBAAAAAAA8Q7OvJM7KytLevXv17rvvNvqxiYmJSkxMtBorKChwmUbxyFnv\n2IxtXjrKCZUAni8jI0NZWVlKTU1Vhw4dlJKSosTERK1fv95mX6PRqHvuuUddu3bV+vXr9dNPPyk5\nOVk+Pj5asGCBE6p3LbVl1++RZQDqQoYAQP1ZSAYCcAXkFOyt2U3ijRs36vjx44qLi5Mkmc1mSdJ9\n992n0aNH629/+1tzfwQAD2c0GpWZmal58+YpNjZWkpSenq74+Hjl5uYqMjLSav/NmzeruLhYGzZs\nsCxxU1dDGQAAAAAAAPVrdpP46aef1rlz5yzfFxcXa9y4cVq4cKGl2QMA9cnPz1dZWZmio6MtY2Fh\nYQoNDVV2drZNk3jnzp364x//aLUG+m233abbbrvNYTUDAAAAAAB4imY3iYODg62+b926tWW8c+fO\nzZ0egBcoKiqSZJsnQUFBlm3nO3LkiK6++motW7ZM7777rlq1aqWhQ4dqxowZlgwCAAAAAABAwzS7\nSQwAzVVeXi4fHx/5+/tbjRsMBlVUVNjsf+bMGb311lu69tprtXz5ch0/flx///vfdfLkSaWlpV3w\n52VkZGjFihV2qx8AAAAAAMCd2b1J3K1bN+3fv9/e0wLwYAEBAaqqqpLJZJKf32+xZDQa1aZNG5v9\n/fz81L59e6WlpcnX11dXXnmlTCaTHnroIc2ZM0cdO3as9+e5+k0zAQAAAAAAHMnH2QUAQEhIiKTq\nNc3Pd+LECZslKKTqZSnCw8Pl6+trGevVq5ckqbCwsAUrBQAAAAAA8Dw0iQE4Xb9+/RQYGKhdu3ZZ\nxgoKClRYWKioqCib/a+66irt27dPv/76q2XswIED8vX1VWhoqENqBgAAAAAA8BSsSQyHGTnrHaf8\n3M1LRzX6Mddff70KCwv1+OOPa+zYsTbbJ0+erB07digtLU2jRlXPv3v3br3yyiv6+uuv9csvvyg0\nNFSjR4/WxIkTZTAYJP22Fu6AAQP05ptv2sz7/vvv6+GHH9Y111yjV1991TJ+5swZvfzyy9qyZYt+\n/PFHtW/fXlFRUXrwwQd12WWXNfr4XI3BYNDYsWOVlpamjh07qnPnzkpJSVF0dLQGDRoko9Go06dP\nq3379jIYDBozZozWrl2rRx99VNOmTdPx48e1ZMkSjRo16oJLTcD7kD1kT0MtWLBAlZWVWrRoUZ37\nfPvtt1q0aJH27dun4OBgTZ06VaNHj3ZglXAH7pQ7EtkDeAp3yh5yB/AcZI/nZA9XEgN18Pf315Yt\nW2zGf/rpJ3355ZdWY++9957GjBmjiy66SM8//7zee+89TZs2TZmZmZo2bZqqqqqs5v3222917Ngx\nm7nff/99tWrVymqspKREt956qz777DPNmjVL7733npYvX65ffvlFY8aM0cGDB+10xM41Y8YMjRw5\nUklJSUpISFD37t21fPlySVJeXp7i4uKUl5cnSerSpYtef/11nT59WrfeeqtmzZqloUOHKiUlxZmH\nANgF2eN4ZrNZy5cv1xtvvFHvfqWlpZo8ebL69++vjRs3asKECZo7d6527tzpoEqBlkP2AHA0cgeA\nM5A9deNKYqAOV199tT7//HOVlpaqU6dOlvGPPvpIAwcOVHZ2tiTp+PHjmj9/vhISEjR79mzLfpdc\nconCwsJ055136oMPPtCIESMkVa+/26pVK23ZskUTJ0607F9WVqYdO3YoMjLSqo4nnnhCZrNZ69at\nU7t27SxzP/fcc7rtttuUmpqql19+uaWeBofx8/NTcnKykpOTbbbFxMTY3BCzV69eeuWVVxxVHuAw\nZI9jHT16VI899pgOHjyo7t2717vvm2++qXbt2mnu3Lny8fFReHi49u7dq9WrVysuLs5BFQMtg+wB\n4GjkDnBhzrpK15ORPXXjSmKgDhEREerSpYs+/vhjq/F///vflhCQpM2bN6uiokIPPPCAzRwDBw5U\nZmamhgwZYjU+fPhwffDBB1Zjn3zyiXr16qUePXpYxoqLi7V161bdfffdltCo4e/vr6VLl2revHlN\nPka4p5Gz3qn3C+6N7HGs3NxchYSEaPPmzQoLC6t33+zsbEVFRcnH57fTp+joaOXm5spsNrd0qUCL\nInsAOBq5A8AZyJ660SQG6tCqVSsNHTrU6mMIpaWl+vrrrzVs2DDL2J49e9SzZ09dfPHFtc4TExOj\niy66yGps+PDh+uabb3T8+HHL2O8DSZL27dunqqoqDRw4sNa5e/furT/84Q+NPTQALozscaxRo0Yp\nLS1NXbt2veC+RUVFCg4OthoLCgpSeXm5Tp061VIlAg5B9gBwNHIHaBmG3jlavOP5C355K7KnbjSJ\ngXoMHz5cX331lU6fPi1J+vDDDxUZGakuXbpY9vn5559tguFCLr/8cl1yySX66KOPJFUvVr5z507d\neOONVvv9/PPPklRnKAHwTGSPazp37pzl5hQ1ar43Go31PjYjI0N9+/a1+oqPj2+xWoGmIHsAOBq5\nA8AZyJ7a0SQG6jF48GB17NhRW7dulVT7O0AdOnSwBEtjDB8+3PLO1ccff6zLL7/cZj3Mjh07SlKT\n5gfgvsge1xQQEGDTDK75vk2bNvU+NjExUfv377f6qvnvC7gKsgeAo5E7QP3LCaJlkD21o0kM1KNV\nq1YaNmyYtmzZotLSUuXm5uqGG26w2iciIkL/+9//6vzlfvTRR/X666/bjA8fPlzZ2dkqKSnRBx98\nYPPOkiRdccUV8vPz0zfffFPr3Js3b9aMGTNUUVHRhKMD4KrIHtfUrVs3FRcXW42dOHFCbdu2bfRV\nBoArInsAOBq5A8AZyJ7a0SQGLmD48OH6/PPPtWnTJkVHR1vd/bJme5s2bbRy5Uqbx+bk5GjTpk02\nC5FLUv/+/RUaGqp33nlHX3zxhYYPH26zT/v27XXDDTfotddeU1lZmdW2iooKrVq1Sj/99JNat27d\nzKME4GrIHtczePBgZWdnW92k7quvvlJkZKTVzewAd0b2AHA0cgeAM5A9tviLBriAyMhItW/fXitW\nrLD5+IEkdenSRfPnz9eaNWu0YMEC7d69W0eOHNE///lPTZs2TfHx8brppptqnXv48OF67rnndOWV\nV9rcDKlGcnKyzGazxo0bp08++URHjx7Vl19+qcmTJ+v48eNasGCBXY8XgGsge5zPaDSquLjYsqTE\n7bffrtLSUj3++OP67rvvtHbtWr333nuaPHmykysF7IfsAeBo5A4AZyB7bPk5/CfCa21eOsrZJTSJ\nj4+Phg0bpjfeeMPm4wc1brnlFnXr1k2rV6/WlClTVFZWph49emjKlCkaP368fH19a33c8OHDtWrV\nqlo/flCjW7dueuONN7Ry5Uo9+eSTOnHihDp16qTo6GgtWrRIPXr0sMtxAp6K7LFF9jRMXl6eEhIS\nlJmZqZiYGHXp0kUvv/yyFi5cqNGjR6t79+5KTU3VNddc4+xS4WLcNXcksgdoiAutE+qsDHDX7CF3\n7KOyslLLli1TVlaWysrKNGTIEC1YsMDqRlw1JkyYoF27dtU6z7p16xQVFaVt27ZpypQpNtu3bdum\nbt262b1+uC+yx5a7Zk8r8/mfmXQBBQUFio+P19atWxUWFubUWhq6SLi7/kIArvT75mzu9FzY4wYG\n5BacyZ1+31qSqz4PDckYMgTuyFV/5xyN56Fh6stCQ++ceh8b1b+6gZY8ZKpda4L7ceTv27Jly/TW\nW28pNTVVHTp0UEpKinx9fbV+/XqbfX/66Sf9+uuvlu+rqqr0wAMPqF27dlqzZo38/Pz00ksv6d//\n/rdeeuklq8d27ty50ctskTv1s/cN6gy9cyw5VB8yynM19XeOK4kBAAAAAADclNFoVGZmpubNm6fY\n2FhJUnp6uuLj45Wbm6vIyEir/Tt06GD1/UsvvaSjR4/q3//+t/z8qttEBw8eVJ8+fdS1a1fHHAQA\np2NNYgAAAAAAADeVn5+vsrIyRUdHW8bCwsIUGhqq7Ozseh9bXFysF154QQ8//LBVQ/jgwYMKDw9v\nsZoBuB6uJAYAAAAAAHBTRUVFkmRzg6ygoCDLtrqsWrVKnTt31pgxYyxjlZWVOnz4sHbv3q2bb75Z\npaWluvLKK5WUlKSePXvWO19GRoZWrFjRxCOBvXy9p+7/7g1ZigLeiSuJAQAAAAAA3FR5ebl8fHzk\n7+9vNW4wGFRRUVHn486cOaO3335bkydPtroB1w8//KCKigoZjUYtXLhQy5Ytk9Fo1Lhx43Ty5Ml6\na0lMTNT+/futvrZu3dq8AwTgEFxJDAAAAAAA4KYCAgJUVVUlk8lkWVNYql6ruE2bNnU+buvWraqs\nrNTNN99sNX7ZZZfpq6++0sUXX2y5Sd2KFSt03XXX6Z133tG9997bMgcCwKm4khgAAAAAAMBNhYSE\nSKpeX/h8J06csFmC4nxbt27Vddddp7Zt29ps69Chg6VBLElt2rTRJZdcomPHjtmpagCuhiYxAAAA\nAACAm+rXr58CAwO1a9cuy1hBQYEKCwsVFRVV5+NycnJ09dVX24x//PHHioiIUGlpqWXszJkzOnLk\niHr37m3f4gG4DJabAAAAAAAAcFMGg0Fjx45VWlqaOnbsqM6dOyslJUXR0dEaNGiQjEajTp8+rfbt\n28tgMEiqvsq4pKREffr0sZkvKipK7dq1U1JSkpKSklRZWan09HR17NhRo0aNcvThAXAQmsQAAAAA\nAK+3eMfzln8behc5sRKg8WbMmCGTyaSkpCSZTCYNGTJECxYskCTl5eUpISFBmZmZiomJkfTb0hTt\n27e3mat9+/Z69dVXtWTJEiUkJMhkMik2NlavvfaaWrdu7biDAuBQNInPc/5JgVT3iYHx4GBHlONx\nfv/8OkrykKmNfsz1118vHx8fbd682Wah/wkTJqhHjx5atGiR+vbtq4iICP3jH/+wWq+pZo7bb79d\nU6c2/ucDsB+yB4CjuVPuSGSPM1RWVmrZsmXKyspSWVmZpZnTpUuXCz72/vvv19mzZ7V27VoHVAp3\n4k7ZQ+7Yn5+fn5KTk5WcnGyzLSYmRvv377ca69+/v83Y+cLDw/Xiiy/avU54HrLHc7AmMVCHo0eP\nKj09/YL75eXlKTMz0wEVAfAGZA8AZyB7HCsjI0NZWVlKTU3VunXrVFRUpMTExAs+bsOGDfrss89a\nvkDAAcgdAM5A9tSNJjFQh0suuUTr1q1Tbm7uBfdbtmyZjh496qDKAHgysgeAM5A9jmM0GpWZmamZ\nM2cqNjZW/fv3V3p6unJzc+t9/r///ns988wzioiIcGC1QMshdwA4A9lTN5rEQB1uueUWRUREaO7c\nuaqoqKhzvylTpigoKEhz586V2Wx2YIUAPBHZA8AZyB7Hyc/PV1lZmaKjoy1jYWFhCg0NVXZ2dq2P\nqays1KOPPqrJkycrPDzcUaUCLYrcAeAMZE/daBIDdWjVqpUWLVqkwsJCZWRk1Llf69attWjRIu3a\ntUsbNmxwYIUAPBHZA8AZyB7HKSqqvu9JcHCw1XhQUJBl2++tXLlSkjRp0qSWLQ5wIHIHgDOQPXWj\nSQzU47LLLtP06dO1evVq7d69u879oqKidNddd2nJkiU6duyYAysE4InIHgDOQPY4Rnl5uXx8fOTv\n7281bjAYar2iaffu3VqzZo1SU1Ntbp5zIRkZGerbt6/VV3x8fLPqB+yJ3AHgDGRP7fycXYAr+XpP\n7e/cw7vdc8892rJli+bMmaONGzfWud8jjzyibdu2af78+Xr55ZcdWCHQNA29C21T7hqL5iN7ADgD\n2dPyAgICVFVVJZPJJD+/3/4cMxqNNndar6io0OzZszVjxgxdeumljf5ZiYmJNjfEKygooFEs9nnO\njwAAIABJREFUaeSsd2zGDL35e9AZyB0AzkD22OJKYuACfH199eSTT+p///ufXnzxxTr3CwwM1N//\n/nft2LGj3oABgIYgewA4A9nT8kJCQiRJxcXFVuMnTpywWYLiP//5j7777js9/fTTioiIUEREhDZt\n2qTs7GxFREToxx9/dFjdQEshdwA4A9ljiyuJgQbo3bu3/vrXv+qFF15Q586d1aNHj1r3i42N1W23\n3abFixc7uEIAnojsAeAMZE/L6tevnwIDA7Vr1y6NGjVKUvXVvYWFhYqKirLad8CAAfrwww+txtLT\n0/Xjjz/q6aefVlBQkMPqBloSuQPAGcgea1xJDDTQlClTFB4eXucNRWrMmTNHrVu31unTpx1UGQBP\nRvYAcAayp+UYDAaNHTtWaWlp2r59u/bs2aOZM2cqOjpagwYNktFoVHFxsYxGowICAnTppZdafbVr\n184yfv5yFYC7I3cAOAPZ8xuaxEAD+fv766mnnrrgyfhFF12kv/3tbw6qCoCnI3sAOAPZ07JmzJih\nkSNHKikpSQkJCerevbuWL18uScrLy1NcXJzy8vKcXCXgWOQOAGcge37DW89wGHe6+dUnn3xS6/jl\nl1+uPXv2WL7fv39/rfv96U9/qnMbAMciewA4mjvljkT2OIOfn5+Sk5OVnJxssy0mJqbe53PRokUt\nWRrcmDtlD7kDNJ6hd46zS6gV2eM5uJIYAAAAAAAAALwYTWIAAAAAAAAA8GI0iQG4hMrKSi1dulRx\ncXGKiIjQ9OnTVVJS0qDH3n///ZowYUILVwgAAAAAAOCZWJMYgEvIyMhQVlaWUlNT1aFDB6WkpCgx\nMVHr16+v93EbNmzQZ599pujoaAdV6jyLdzwvSTL0rv+uq8aDgx1RDgAAAAAA8BA0iQE4ndFoVGZm\npubNm6fY2FhJUnp6uuLj45Wbm6vIyMhaH/f999/rmWeeUUREhCPLBQAAAADArdVchHQh7nRjOjQP\nTWIATpefn6+ysjKrq4HDwsIUGhqq7OzsWpvElZWVevTRRzV58mQdOXJEP/zwgyNLdmkNuevt4h2F\nDqgEAAAAAAC4A5rETfD7BkxdzRbebQEapqioevmE4OBgq/GgoCDLtt9buXKlJGnSpEmaP39+o35e\nRkaGVqxY0YRKAQAAAAAAPA9NYgBOV15eLh8fH/n7+1uNGwwGVVRU2Oy/e/durVmzRm+99ZZ8fBp/\n/83ExEQlJiZajRUUFCg+Pr7RcwEAAAAAALi7xndXAMDOAgICVFVVJZPJZDVuNBrVpk0bq7GKigrN\nnj1bM2bM0KWXXurIMgEAAAAAADwSVxIDcLqQkBBJUnFxseXfknTixAmbJSj+85//6LvvvtPTTz+t\np59+WlJ1M7mqqkoRERH617/+pe7duzuueAAAAAAAADdHkxiA0/Xr10+BgYHatWuXRo0aJal6+YfC\nwkJFRUVZ7TtgwAB9+OGHVmPp6en68ccf9fTTTysoKMhhdQMAAAAAAHgCmsQAnM5gMGjs2LFKS0tT\nx44d1blzZ6WkpCg6OlqDBg2S0WjU6dOn1b59ewUEBNgsM9GuXbtaxwEAAABH+3pP9Y2XR256x2bb\n5qWjHF0OAAANwprEAFzCjBkzNHLkSCUlJSkhIUHdu3fX8uXLJUl5eXmKi4tTXl6ek6sEAAAAAADw\nPFxJDMAl+Pn5KTk5WcnJyTbbYmJitH///jofu2jRopYsDQDwOyNn2V4dVxuumAMAAADcA1cSAwAA\nAAAAuLHKykotXbpUcXFxioiI0PTp01VSUlLn/g899JD69u1r9TVx4kTL9vLycs2fP18xMTG66qqr\nNG/ePJWVlTngSAA4C1cSAwAAAAAAuLGMjAxlZWUpNTVVHTp0UEpKihITE7V+/fpa9z9w4IBmzZql\nW265xTJmMBgs/16wYIH27NmjlStXymQy6bHHHtOCBQu0dOnSFj8WAM5BkxgAvFDNDVXqEtW/m4Mq\nAQAAANAcRqNRmZmZmjdvnmJjYyVJ6enpio+PV25uriIjI232/+GHHzRgwAB17drVZr6ioiK99957\nevXVVzVo0CBJ0sKFC5WQkKDZs2crODi45Q8KgMOx3AQAAAAAAICbys/PV1lZmaKjoy1jYWFhCg0N\nVXZ2ts3+hw8flslkUnh4eK3z5ebmysfHx6q5HBkZKV9fX+Xk5Nj/AAC4BK4kBgAAAAAAcFNFRdWf\nEvz9Fb5BQUGWbec7cOCA/P39lZGRoe3bt6t169YaPny4pk6dqtatW+v48ePq1KmT/P39LY/x8/NT\np06ddOzYsZY9GABOQ5MYAAAAAADATZWXl8vHx8eqqStVrzFcUVFhs/+hQ4ckST179tS4ceN04MAB\nLV68WEVFRUpNTVV5eblat25t87i65jtfRkaGVqxY0YyjAeAsNIkBAAAAAADcVEBAgKqqqmQymeTn\n91ubx2g0qk2bNjb7z5gxQ/fee686dOggSerbt698fX318MMPKzk5WQEBATIajTaPMxqNatu2bb21\nJCYmKjEx0WqsoKBA8fHxTTk0tID67k/DvWm8m13WJC4pKdGjjz6quLg4XXXVVZo0aZIOHDhgj6kB\nAAAAAABQh5CQEElScXGx1fiJEydqvcmcj4+PpUFco0+fPpKql67o1q2bSktLVVlZadluMplUWlqq\noKAge5cPwEU0u0lcVVWlBx98UEeOHNHzzz+vDRs2qF27dpo4caJOnTpljxpd3td7iqy+AAAAAAAA\nHKFfv34KDAzUrl27LGMFBQUqLCxUVFSUzf4PPfSQpk2bZjW2e/duGQwG9ejRQ4MHD5bJZFJeXp5l\ne05OjqqqqjR48OCWOxAATtXsJnF+fr7y8vL05JNPasCAAerVq5eWLFmis2fPatu2bfaoEQAAAAAA\nALUwGAwaO3as0tLStH37du3Zs0czZ85UdHS0Bg0aJKPRqOLiYssSEsOGDdPWrVu1Zs0a/fDDD/rg\ngw+Umpqqe++9V4GBgQoODtaNN96ouXPnKicnR9nZ2Zo/f75GjRpV65XJADxDs9ckDgkJ0cqVK3XZ\nZZdZxlq1aiVJOn36dHOnBwAAAAAAQD1mzJghk8mkpKQkmUwmDRkyRAsWLJAk5eXlKSEhQZmZmYqJ\nidGIESNkNBr1yiuv6JlnnlHnzp2VkJCg+++/3zLfwoULtXDhQk2ZMkV+fn4aNmyYHnvsMWcdHgAH\naHaTuGPHjrruuuusxtauXatz584pLi6uudMDAAAAAACgHn5+fkpOTlZycrLNtpiYGO3fv99qbPTo\n0Ro9enSd8wUGBuqpp57SU089ZfdaAbimZjeJf2/r1q1KT0/XPffco/Dw8Hr3zcjI0IoVK+xdAgAA\nAAAAAACggezaJN64caPmz5+vESNGKCkp6YL7JyYmKjEx0WqsoKBA8fHx9iwLANBI59+Ec+Smd2rd\nZ/PSUY4qBwAAAAAAtKBm37iuxgsvvKA5c+ZozJgxSktLk4+P3aYGAABoEZWVlVq6dKni4uIUERGh\n6dOnq6SkpM79H3roIfXt29fqa+LEiY4rGAAAAABagF2uJF61apWWLVum6dOna9q0afaYEgAAoMVl\nZGQoKytLqamp6tChg1JSUpSYmKj169fXuv+BAwc0a9Ys3XLLLZYxg8HgqHIBAAAAoEU0u0mcn5+v\nZ555RrfddpvuvPNOFRcXW7YFBgaqbdu2zf0RAAAnMvTOqXV88Y5Cq++Th0x1RDmA3RiNRmVmZmre\nvHmKjY2VJKWnpys+Pl65ubmKjIy02f+HH37QgAED1LVrV2eUDAAAAAAtotlrQrz//vuqrKzU22+/\nrbi4OKuvV1991Q4lAgAA2F9+fr7KysoUHR1tGQsLC1NoaKiys7Nt9j98+LBMJtMFb8wLAAAAAO6m\n2VcSz5w5UzNnzrRHLQAAAA5TVFR9g8bg4GCr8aCgIMu28x04cED+/v7KyMjQ9u3b1bp1aw0fPlxT\np05V69atHVIzAAAAALQEu6xJDAAA4G7Ky8vl4+Mjf39/q3GDwaCKigqb/Q8dOiRJ6tmzp8aNG6cD\nBw5o8eLFKioqUmpqar0/KyMjQytWrLBf8QAAAPAIdS3vBzgaTWIAAOCVAgICVFVVJZPJJD+/306J\njEaj2rRpY7P/jBkzdO+996pDhw6SpL59+8rX11cPP/ywkpOT1bFjxzp/VmJiohITE63GCgoKFB8f\nb6ejAQAAAICma/aaxAAAAO4oJCREkqxuuitJJ06csFmCQpJ8fHwsDeIaffr0kaRal6cAAAAAAHfB\nlcQAAMAr9evXT4GBgdq1a5dGjRolqfrq3sLCQkVFRdns/9BDD8lkMum5556zjO3evVsGg0E9evRw\nWN3ubPGO5xu0X/KQqS1cCQAAAIDz0SQGAABeyWAwaOzYsUpLS1PHjh3VuXNnpaSkKDo6WoMGDZLR\naNTp06fVvn17GQwGDRs2TDNnztSaNWsUHx+vvXv3KjU1Vffee68CAwOdfTgAABdT2zqji3cU2ozx\nxhgAwBXQJAYAAF5rxowZMplMSkpKkslk0pAhQ7RgwQJJUl5enhISEpSZmamYmBiNGDFCRqNRr7zy\nip555hl17txZCQkJuv/++518FAAAAADQPDSJAQCA1/Lz81NycrKSk5NttsXExGj//v1WY6NHj9bo\n0aMdVZ7bqOuu3LVdMQdAqqys1LJly5SVlaWysjLLG1RdunSpdf/3339fK1eu1Pfff6+uXbvqjjvu\n0KRJk+Tr6+vgyt1TbUvdGHqzljwAAOfjxnUAAAAA4EAZGRnKyspSamqq1q1bp6KiIiUmJta677Zt\n2/TII4/ojjvu0LvvvqtZs2Zp1apVevHFFx1cNQAA8GReeyXxyFnv2IwZejuhEAAAAABew2g0KjMz\nU/PmzVNsbKwkKT09XfHx8crNzVVkZKTV/hs2bNDQoUM1fvx4SVKPHj303XffaePGjZo2bZrD6wcA\nAJ7Ja5vEAAAAAOBo+fn5KisrU3R0tGUsLCxMoaGhys7OtmkS//Wvf1Xbtm2txnx8fPTzzz87pF4A\nAOAdaBIDAAAAgIMUFVWvhRscHGw1HhQUZNl2vgEDBlh9f+bMGa1fv15Dhgy54M/KyMjQihUrmlEt\n7O3rPbb/jUdu+u1TrpuXjnJkOQAAWNAkBgAAAAAHKS8vl4+Pj/z9/a3GDQaDKioqLvjYqVOnqqKi\nQrNmzbrgz0pMTLRZ67igoEDx8fGNLxwAAHg0j2wS17beMO/IAq6Nu3wDAABvEBAQoKqqKplMJvn5\n/fbnmNFoVJs2bep8XGlpqaZOnapDhw5p9erVCg0NdUS5AADAS/g4uwAAkLjLNwAA8A4hISGSpOLi\nYqvxEydO2CxBUaOgoEB33XWXCgoKtG7dOpslKAAAAJrLI68kBuBeuMs3AADwFv369VNgYKB27dql\nUaOqP+1YUFCgwsJCRUVF2ex/8uRJJSQkyNfXV+vXr9cll1zi6JIBAF6ivnXT+YS+56NJDMDpuMs3\nAADwFgaDQWPHjlVaWpo6duyozp07KyUlRdHR0Ro0aJCMRqNOnz6t9u3by2AwKCUlRadOndJrr72m\ngIAAyxXIrVq1qnNZLgCAa6ltWdQaht4OLASoh1c0iQ29c7R4R+HvxmzfHQHgHI68yzcAoOEMvXOc\nXQLgkWbMmCGTyaSkpCSZTCbLvRgkKS8vTwkJCcrMzNTAgQP10UcfqaqqSnfccYfVHL6+vtq7d68z\nygcAAB7IK5rEAFybI+/yLVWvf7xixYom1wsAaFmLdzzfoP2Sh0xt4UqAluHn56fk5GQlJyfbbIuJ\nidH+/fst3+/bt8+RpQEAAC/lNU3i2tZVAeAaHH2X78TERJub4hUUFCg+Pr5pBwAAAAAAAODGfJxd\nAABwl28AAAAAaLrKykotXbpUcXFxioiI0PTp01VSUlLn/u+//75GjRqlQYMG6YYbbtBLL72kyspK\ny/bXX39dffv2tfq6/PLLHXEoAJyEJjEApzv/Lt81GnKX76qqKq1fv179+vVzZLkAAAAA4FIyMjKU\nlZWl1NRUrVu3TkVFRTafnqyxbds2PfLII7rjjjv07rvvatasWVq1apVefPFFyz4HDhzQ9ddfr507\nd1q+tm/f7qjDAeAEXrPcBADXxV2+AQAAAKBpjEajMjMzNW/ePMXGxkqS0tPTFR8fr9zcXEVGRlrt\nv2HDBg0dOlTjx4+XJPXo0UPfffedNm7cqGnTpkmSDh48qKuvvlpdu3Z17MEAcBqaxABcAnf5BgAA\nAIDGy8/PV1lZmaKjoy1jYWFhCg0NVXZ2tk2T+K9//avatm1rNebj46Off/7Z8v2hQ4c0bty4li0c\ngEuhSQzAJXCXbwAAAABovKKiIkmyuZ9LUFCQZdv5fn8/lzNnzmj9+vUaMmSIJOn48eM6ffq0tm/f\nroyMDJWXlysqKkpJSUl13jOmRkZGhlasWNGcw4GLMfTOkSQt3lFY737JQ6Y6ohy0IJrEAAAAaBFf\n77H9w7Q2Uf27tXAlAAB4rvLycvn4+Mjf399q3GAwqKKi4oKPnTp1qioqKjRr1ixJ1UtNSNUX8jzz\nzDM6deqU0tPTNXHiRGVlZSkgIKDO+RITE23WQi4oKFB8fHxTDg2AA9EkBgAAAADASWqu0pPqv1KP\nq/RQl4CAAFVVVclkMsnP77c2j9FoVJs2bep8XGlpqaZOnapDhw5p9erVCg0NlSTFxcXpiy++UKdO\nnSz79urVS9dee622bdumYcOGtdzBAHAamsQAgCb5/RWCIze9Y/X95qWjHFkOgEZYvOP5OrcZejfs\n6l8AAOAaQkJCJEnFxcWWf0vSiRMn6lweoqCgQJMmTVJZWZnWrVunfv36WW0/v0EsVS9d0bFjRx07\ndszO1QNwFT7OLgAAAAAAAABN069fPwUGBmrXrl2WsYKCAhUWFioqKspm/5MnTyohIUFVVVVav369\nTYM4MzNTcXFx+vXXXy1jhYWFKi0tVe/evVvuQAA4FU1iAAAAAAAAN2UwGDR27FilpaVp+/bt2rNn\nj2bOnKno6GgNGjRIRqNRxcXFMhqNkqSUlBSdOnVKS5cuVUBAgIqLi1VcXKySkhJJ0nXXXaeysjLN\nnTtX3333nXJycpSYmKjBgwcrNjbWmYcKoAWx3AQAwC7OX09Pkm57MafW/YwHB1v+zZIUAAAAQPPN\nmDFDJpNJSUlJMplMGjJkiBYsWCBJysvLU0JCgjIzMzVw4EB99NFHqqqq0h133GE1h6+vr/bu3ase\nPXpozZo1Wrp0qe644w75+/vr+uuvV3JysjMODYCD0CQGAAAAAABwY35+fkpOTq61kRsTE6P9+/db\nvt+3b98F5xs0aJDWrl1r1xoBuDaaxAAAh+IO3gCa6kI3zKzBpxQAAACAxqFJ3ILqu3P4+WiEAPBW\nv2/4nG/kpndo9AAAAAAA4AA0iVtAbU2PqP7dnFAJAAAAAAAAANSPJjEAAAAAAC6OT6oCAFoSTWIA\nAAAAAFxAfUtx8elUAEBLokkMAAAAAICLq6+BLNFEBgA0D01iAAAAuCVD75xaxxfvKLT6no9eA95n\n5Kx3LP829K6/uQoAACQfZxcAAAAAAAAAAHAeriQGALgkQ+8cm6sBa8MVggAAAAAANA9NYgCA2/p6\nT5FGbnqn3n02Lx3loGoANNWF1tkEAAAA0LJYbgIAAAAAAAAAvBhXEgMA3FpdN66qUbNkBctSAAAA\nAHCUxTuet/ybG2jCHdAkBgC4LD6CDgAAAACu7/ymeH24eMd10SQGAACAR+OPFgAAAKB+NIkBAADg\nURr6KYSo/t2svqeZDAAAAG9FkxgAAABoBJrJgGuo73fRG9f/rHmDbOSmd2rdvnnpKEeWAwBwMzSJ\nAQAAAAAAgGYaOeu3N2m88c0quDeaxAAAAEAL4IpjAM5g6J1T6/jiHYVW35M9AIDzeUST+Px3agAA\nAAAAAAAADefj7AIAAAAAAAAAAM7jEVcSAwAAAAAAAGgZNTfH/L2o/t0cXAlaClcSAwAAAAAAAIAX\n40piAAAAoJnqurrm97jaBgAAeDNu7Ou6aBIDADxaTeNm5Kbab3K6eekoR5YDwMvV1kyuLZ/IJgD2\n9vv8OT97yByg4eprchp6N+xNY09S3xvlvDnuXmgSAwC8gqF3Tq3ji3cUWn3PO9aA92jI1b/8cQMA\nAABvYJcmcWVlpZYtW6asrCyVlZVpyJAhWrBggbp06WKP6etU8+6NO7xT8/s/QviDA7DW2Bz59ttv\ntWjRIu3bt0/BwcGaOnWqRo8e7eCqAbg7sgeuoLY3sX7/BlYN3sjyDGQPXMH52VNX5kjkjruwd66U\nl5frySef1IcffqjKykoNHz5cc+bMUWBgoKMOCYCD2aVJnJGRoaysLKWmpqpDhw5KSUlRYmKi1q9f\nb4/pAXiBxuRIaWmpJk+erJtuukmLFi3S559/rrlz56pLly6Ki4tzQvUA3BXZgwtp6FrDQGOQPQDs\nzd65smDBAu3Zs0crV66UyWTSY489pgULFmjp0qWOPjR4qYauXdxQvOF1Yc1uEhuNRmVmZmrevHmK\njY2VJKWnpys+Pl65ubmKjIxsdpEAPFtjc+TNN99Uu3btNHfuXPn4+Cg8PFx79+7V6tWr3fKPJXv/\nzw9Aw3h79gBwDrIHgL3ZO1eKior03nvv6dVXX9WgQYMkSQsXLlRCQoJmz56t4OBghx8jgJbX7CZx\nfn6+ysrKFB0dbRkLCwtTaGiosrOzaRI3AHd2hLdrbI5kZ2crKipKPj4+lrHo6GilpKTIbDarVatW\nDqsdgPvyxOzhTSfP521X1XjiebInZo+9kWVA49g7V3Jzc+Xj42P1uMjISPn6+ionJ0cjRoxo+YMC\nXJwnnqM0u0lcVFT9Ebzfv5MUFBRk2dYYlZWVVvPW55eSnyVJxtNljf45zlZTe2MUFBS0QCXwZjW/\nZzW/d86uo6E5UlRUpMsvv9xm3/Lycp06dUqdOnVqdA2NyZ4Xvl7X6PntyR0zz5X9Po/nZi12Sh1/\njRrvlJ/bUA193TfkOMieaq6aO2SMa2jKuWJTOCvz7K0h58lkTzVXzR57I8sap77McfWcaOg5lL1f\nz650zmPvXDl+/Lg6deokf39/y3Y/Pz916tRJx44da3R93pI7Etnze446n2kIZ2VZQ3t5rvD3VrOb\nxOXl5fLx8bEKD0kyGAyqqKio97EZGRlasWJFrdvGjRvX3NJc2oEmPOYtrbF7HYAkFRcX69JLL3Xa\nz29sjpw7d04Gg8FmX6n6o1YX4s3ZA1tNyeOW4CkZ35jj8KbsIXfQWK6STe6C7Kkd2YOGcufMcdY5\nlCvljr1zpby8XK1bt7Z5HH0eNJY7Z4u92DujWjJ7mt0kDggIUFVVlUwmk/z8fpvOaDSqTZs29T42\nMTFRiYmJVmPnzp3T7t271bVrV/n6+ja3PI8THx+vrVu3OrsMt8Zz+JvKykoVFxfriiuucGodjc2R\ngIAAmz+Kar6/UO5I7p89vIZdH/+N6ueN2eNOucPr1xrPhy13fU7Inmqumj324q6vT3fAc9t4jsod\ne+dKbdtr9mnbtm29tbhC7njza9Wbj13y7uM//9ibmj3NbhKHhIRIqu5O1/xbkk6cONGkxcwDAgJ0\n1VVXNbcsjxYWFubsEtwez+FvnHklTY3G5ki3bt1UXFxsNXbixAm1bdtWF110UZNqcLfs4TXs+vhv\nVD+yx7Vzh9evNZ4PW+76nJA9rp099uKur093wHPbeI7IHXvnSrdu3VRaWqrKykpLU9dkMqm0tFRB\nQUGNrs8ZuePNr1VvPnbJu4///GNvSvb4XHiX+vXr10+BgYHatWuXZaygoECFhYWKiopq7vQAvEBj\nc2Tw4MHKzs6W2Wy2jH311VeKjIy0uvkCANSH7AHgDGQPAHuzd64MHjxYJpNJeXl5lu05OTmqqqrS\n4MGDW/ZgADhNs88qDAaDxo4dq7S0NG3fvl179uzRzJkzFR0drUGDBtmjRgAe7kI5YjQaVVxcbPnI\n0+23367S0lI9/vjj+u6777R27Vq99957mjx5spOPBIA7IXsAOAPZA8De7J0rwcHBuvHGGzV37lzl\n5OQoOztb8+fP16hRo5r0iXEA7sEubz3PmDFDI0eOVFJSkhISEtS9e3ctX77cHlMD8BL15UheXp7i\n4uIs72R36dJFL7/8svbu3avRo0dr3bp1Sk1N1TXXXOPMQwDghsgeAM5A9gCwN3vnysKFCxUZGakp\nU6Zo2rRpuvrqq/XEE08449AAOEgr8/mfL4DLy8jIsFkEHo3Dcwh3x2vY9fHfCO6M1681ng9bPCdw\nZbw+Ww7PLdyFN79WvfnYJe8+fnscO01iAAAAAAAAAPBi3OkAAAAAAAAAALwYTWIAAAAAAAAA8GI0\niQEAAAAAAADAi9EkBgAAAAAAAAAvRpMYAAAAAAAAALwYTWIAAAAAAAAA8GI0id3MggULNHfuXGeX\n4XZKSkr06KOPKi4uTldddZUmTZqkAwcOOLssoMEqKyu1dOlSxcXFKSIiQtOnT1dJSYmzy8J5Dh06\npL59+9p8ZWdnO7s0wKKxWfLtt99qzJgxGjhwoIYOHapNmzY5sNqW19jzg9tvv93md9wTz8sam2ee\n/jqB6+M8qeVwfgN34s1ZQM+j2jfffKPLL79cX331lbNLcZg333xTw4YN04ABA3Trrbfqiy++aPJc\nNIndhNls1vLly/XGG284uxS3U1VVpQcffFBHjhzR888/rw0bNqhdu3aaOHGiTp065ezygAbJyMhQ\nVlaWUlNTtW7dOhUVFSkxMdHZZeE8Bw4cUMeOHbVz506rr4EDBzq7NMCiMVlSWlqqyZMnq3///tq4\ncaMmTJiguXPnaufOnQ6uumU09vzAbDbr0KFDevrpp61+x+fMmeOE6ltWY/LM018ncA+cJ7Uczm/g\nTrw1C+h5VDt79qxmz56tyspKZ5fiMFlZWUpJSdF9992nzZs3KyoqSlOnTlVBQUGT5vN89vjRAAAI\nKUlEQVSzc31oAUePHtVjjz2mgwcPqnv37s4ux+3k5+crLy9P77//vsLDwyVJS5YsUXR0tLZt26bR\no0c7uUKgfkajUZmZmZo3b55iY2MlSenp6YqPj1dubq4iIyOdXCGk6j+ievXqpa5duzq7FKBWjc2S\nN998U+3atdPcuXPl4+Oj8PBw7d27V6tXr1ZcXJwzDsGuGnt+cPToUZWXl2vQoEEe/3vemDzz9NcJ\nXB/nSS2L8xu4C2/OAnoe1RYvXqzg4GB9//33zi7FIcxmszIyMnTffffp9ttvlyQ9+uij+vLLL5WX\nl6ewsLBGz8mVxG4gNzdXISEh2rx5c5P+I3u7kJAQrVy5UpdddpllrFWrVpKk06dPO6ssoMHy8/NV\nVlam6Ohoy1hYWJhCQ0P5qJ8LOXjwoHr27OnsMoA6NTZLsrOzFRUVJR+f304Xo6OjlZubK7PZ7JCa\nW1Jjzw8OHDiggIAAhYaGOqxGZ2lMnnn66wSuj/OklsX5DdyFN2cBPQ9p27Zt+uyzzzRv3jxnl+Iw\nhw8fVmFhoUaMGGEZ8/Hx0TvvvKORI0c2aU6axG5g1KhRSktL493bJurYsaOuu+46qz9e1q5dq3Pn\nznGFC9xCUVGRJCk4ONhqPCgoyLINznfw4EH9+OOPuvPOOxUbG6uJEyfqv//9r7PLAiwamyVFRUW1\n7lteXu4RH11s7PnBwYMHddFFF+mRRx5RXFycRo4cqTVr1qiqqsqRZTtEY/LM018ncH2cJ7Uszm/g\nLrw5C7y951FaWqq5c+dq4cKFat++vbPLcZgjR45Ikn7++WclJCTommuu0bhx45Sbm9vkOWkSw+ts\n3bpV6enpuueeeywfxQBcWXl5uXx8fOTv7281bjAYVFFR4aSqcL5z587p6NGjOnPmjGbPnq0XXnhB\nQUFBGj9+vL777jtnlwdIanyWnDt3TgaDwWZfqfojnZ7mQucHhw4d0tmzZxUXF6dXXnlFY8eO1bPP\nPqsVK1Y4odqW09g887bXCVwP50kth/MbuBOy4Dfe1vN4/PHHdf311+vaa691dikOdebMGUlScnKy\n7rjjDr388svq3bu37r777iZnNGsSw6ts3LhR8+fP14gRI5SUlOTscoAGCQgIUFVVlUwmk/z8fott\no9GoNm3aOLEy1AgICNDXX38tg8FgaY4sXrxYe/bs0T/+8Q/Nnz/fyRUCjc+SgIAAmyZfzfeelj0N\nOT9ITU3V2bNndfHFF0uS+vbtq19++UUvvviiEhMTLR/rdHeNzTNvep3ANXGe1HI4v4E7IQuqeVvP\nIysrS3v37tW7777r7FIcruYNkQceeMCyvMTll1+unJwcrV+/vklLb3AlMbzGCy+8oDlz5mjMmDFK\nS0uz+igG4MpCQkIkScXFxVbjJ06csPk4FZynXbt2VlfT+fj4qFevXjp27JgTqwJ+09gs6datW637\ntm3bVhdddFHLFepgDT0/8PPzszSIa/Tt21dlZWX65Zf/3979hbL3x3Ecf80ojcIupMVITeTKkBty\n4Ua5c8EVF+LGheWCULSLuUHiSsy/3HBlbuZCuVEUuVB2MRfuNtmFPzUh0b43v5/vT99fX5GZ7Twf\ndW4+q9P71Kf3Xnuf7Sz6HaV+m4/0M6PsE/xc5KT4It8gWdALjDnz2NzcVCQSUX19vaqqqtTc3CxJ\n6unp0djYWIKri6/8/HxJUllZ2euayWRSaWmpQqHQp86Z+jsGkOT1ejUzM6O+vj6Njo6mzLd9YAzl\n5eXKysrS0dHR61ooFFI4HFZtbW0CK8O/AoGAnE6nAoHA69rLy4uCwaAcDkcCKwN++2gvqa6u1vHx\n8Zs/Hzs8PJTT6UyZDx0fyQdtbW3yeDxv1k5PT5Wfn//H8DiZfbSfGWGf4GcjJ8UP+QbJxOi9wKgz\nj6mpKfn9fm1tbWlra0uLi4uSJI/HI5fLleDq4quyslIWi0Wnp6eva7FYTOfn5yoqKvrUOc1ut9v9\nRfXhG/h8PuXk5KipqSnRpSSNYDCo/v5+tba2qru7W/f396+HyWT645lFwE9jNpsVjUa1tLQkh8Oh\nu7s7jYyMqLi4WL29vYkuD5KsVqu2t7e1t7en8vJyRaNRTUxMKBgManJyUhaLJdElAu/2kqenJ11f\nXysjI0Nms1klJSXyer0Kh8Oy2+3y+/1aWVmR2+3+dPD8Sd7LBy8vL7q5uVFmZqbS0tJ0e3ur5eVl\n2Ww2WSwW7ezsaHZ2VgMDA6qsrEz05XyZ9/pZenq6ofYJfj5yUvyQb5BMjNwLjDzzyM7OVm5u7uuR\nlpam1dVVdXR0pPzNrIyMDD0+Psrr9aq4uFhms1lzc3Pa39/X+Pi48vLyPnxOU+y/t/3x43V0dMhu\nt2t8fDzRpSSN6elpzc/P/+9rLpcr5d8wkBqen581NTUln8+n5+dnNTQ0aGxsTFarNdGl4R+RSEQT\nExM6ODjQw8ODnE6nhoaG3vz8B0i0v/WSw8NDdXZ2am1tTXV1dZKkk5MTeTwenZ2dyWazqa+vTy0t\nLQm+iq/xXj4oKCjQ8PCwdnd3VVhYqFgsptXVVW1sbOji4kI2m01dXV1qb2//5srj72/9zGj7BMmB\nnBQ/5BskE6P2AmYev11eXqqxsfFNTkllsVhMCwsLWl9f19XVlSoqKjQ4OKiamppPnY8hMQAAAAAA\nAAAYGA8KAwAAAAAAAAADY0gMAAAAAAAAAAbGkBgAAAAAAAAADIwhMQAAAAAAAAAYGENiAAAAAAAA\nADAwhsQAAAAAAAAAYGAMiQEAAAAAAADAwBgSAwAAAAAAAICB/QLdIlxv54a81gAAAABJRU5ErkJg\ngg==\n", 241 | "text/plain": [ 242 | "" 243 | ] 244 | }, 245 | "metadata": {}, 246 | "output_type": "display_data" 247 | } 248 | ], 249 | "source": [ 250 | "plt.figure(figsize=(20,6))\n", 251 | "for i in xrange(10):\n", 252 | " plt.subplot(2,5,i+1)\n", 253 | " plt.hist(mcmc_theta[:,i], normed=True);\n", 254 | " plt.hist(nn_raw_theta[:,i], alpha=0.7, normed=True, bins=20);\n", 255 | " plt.title(\"theta_\"+str(i))\n", 256 | " plt.legend(['MCMC', 'NN'])\n", 257 | " \n", 258 | "plt.tight_layout()" 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": 11, 264 | "metadata": { 265 | "collapsed": false, 266 | "deletable": true, 267 | "editable": true 268 | }, 269 | "outputs": [ 270 | { 271 | "data": { 272 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnEAAAD4CAYAAACDrbABAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtUlWXC/vGLrYKChoioKDpTHjAyFEKoN/J1dEpyegc0\ncznqmJppaoiHTA3E0dLUSbEwHUesd5AsMsXGTtbo28E1o2noSnQULUt0QmFILVQI3L8/+rmnHWfY\nm2c/m+9nLdca7n3vh+tB5+7iOW0Pq9VqFQAAAEzFYnQAAAAA1B0lDgAAwIQocQAAACZEiQMAADAh\nShwAAIAJUeIAAABMiBIHAABgQpQ4AKilw4cPKyQkRPv37zc6CgBQ4gCgNq5cuaInn3xS5eXlRkcB\nAEmUOAColeXLl6tjx45GxwAAm+ZGB6jJtWvXlJOTo4CAADVr1szoOABcRHl5uQoKCtSnTx+1bNnS\nqd/ro48+0ocffqiNGzfqt7/9bb23w3oGoDL1Xc9cvsTl5ORozJgxRscA4KJeeeUVRUREOG37RUVF\nSkxM1LJly+Tr61vr96Wmpmrt2rVOywXA/dR1PXP5EhcQECDpxx3r1KmTwWkAuIr8/HyNGTPGtkY4\ny6JFizRo0CANGDBA+fn5tX5ffHy84uPj7ca+/vpr3XfffaxnAOzUdz1z+RJ345RDp06dFBQUZHAa\nAK7Gmacls7KydOzYMf31r391yPZYzwBUp67rmcuXOAAwyvbt23X+/HlFR0dLkqxWqyTp0UcfVVxc\nnJYsWWJkPABNHCUOAKrw3HPP6dq1a7avCwoKNGbMGD3zzDO6++67673dvxz5WjflX6t5YhXiI3rU\n+70A3AclDgCq8PNHinh5ednG/f39jYgEADY8Jw4AAMCEOBIHALXUqVMnnThxwugYACCJI3EAAACm\nxJG4n1iyaV+VryU/cmcjJgEAAKgeR+IAAABMiBIHAABgQpQ4AAAAE6LEAQAAmBA3NgAA0IRUdxOf\nM3GDoONxJA4AALikQYMGKTg4WFu2bKn09UmTJik4OFhvvvmmbSwnJ0ezZs1SdHS0+vbtq6FDh+rP\nf/6zSktLbXNSU1MVHByshx56qNLtvvPOOwoODtb48ePtxr///nutWbNG999/v/r27asBAwZozpw5\nOn36dMN3th44EldLVf3mwm8WAAA4T4sWLbRr1y6NHj3abvzixYvat8/+v81vvfWW5s+fr+HDh2vd\nunXy8/PT559/rmeffVYHDhzQhg0bZLFYbNs9cuSIvvnmGwUGBtpt55133pGHh4fdWGFhoUaPHi1v\nb2/NmTNHwcHBKiws1Pr16zVq1ChlZGSoZ8+eTvgJVI0jcQAAwGXdeeedOnDggIqKiuzGP/jgA/Xt\n29f29fnz57Vw4UKNGzdOS5YsUWhoqLp27arf/OY3evHFF/Xxxx/rvffes80PDAxUt27dtGvXLrvt\nFhcX65NPPlF4eLjd+B/+8AdZrVZlZGTo17/+tbp27aqwsDC9+OKL6tixo1asWOGEva8eJQ4AALis\nsLAwtW/fXn/729/sxt99910NHTrU9vXOnTtVUlKixx57rMI2+vbtq/T0dN1zzz124zExMXbFTpL2\n7NmjHj16qFu3braxgoIC7d69Ww8//LBat25tN79FixZatWqVkpKS6r2P9UWJAwAALsvDw0P33Xef\n3RGzoqIiHThwQEOGDLGNHT16VLfccotuuummSrcTFRWlNm3a2I3FxMTo8OHDOn/+vG3s5+VQkv75\nz3/q+vXrdkf+fqpnz5765S9/WdddazBKHAAAcGkxMTHav3+/Ll26JEl6//33FR4ervbt29vmXL58\nuUJJq0lISIi6du2qDz74QNKPNy7s3btX999/v928y5cvS1KVBdEolDgAAODS7rjjDvn5+Wn37t2S\nKj9a1rZtW1vJq4uYmBjbUb6//e1vCgkJUefOne3m+Pn5SVK9tu9MlDgAAODSPDw8NGTIEO3atUtF\nRUXKzs7WvffeazcnLCxMp0+frrJozZs3T6+88kqF8ZiYGB08eFCFhYV67733KhyFk6Q+ffqoefPm\nOnz4cKXb3rlzp2bOnKmSkpJ67F39UeIAAIDLi4mJ0d///nft2LFDkZGRateuXYXXW7VqpQ0bNlR4\n72effaYdO3ZUuClBkm677TZ16dJFb775pv7xj38oJiamwhxfX1/de++9+stf/qLi4mK710pKSrRx\n40ZdvHhRXl5eDdzLuqlziUtOTlZiYmK1cxISEhQcHGz35+cPzAMAAKit8PBw+fr6au3atRVOpUpS\n+/bttXDhQr388stKTk5WTk6OvvrqK73++uuaPn26Bg8erAceeKDSbcfExOjFF1/U7bffro4dO1Y6\nZ/78+bJarRozZoz27NmjvLw87du3T5MmTdL58+eVnJzs0P2tjVo/7NdqteqFF15QZmamRowYUe3c\n3NxczZkzR8OGDbONeXp61j8lAABwCLM+pN5isWjIkCHKzMyscCr1hmHDhqlTp0566aWXNHnyZBUX\nF6tbt26aPHmyxo4dq2bNmlX6vpiYGG3cuLHSU6k3dOrUSZmZmdqwYYOWLVumCxcuqF27doqMjNTS\npUvtHknSWGpV4vLy8vTUU0/p5MmTFS72+7nS0lKdOXNGoaGhCggIcEhIAADQ9OzZs8fu64ULF2rh\nwoV2YydOnLD7+q677tJdd91V7Xbj4+MVHx9v+7pPnz4VtrN8+fIK7wsICFBSUpIhz4SrTK1Op2Zn\nZyswMFA7d+5UUFBQtXO//PJLlZWVqXv37g4JCAAAgIpqdSQuNjZWsbGxtdpgbm6uWrRoodTUVH38\n8cfy8vJSTEyMpk2bVuMFf6mpqVq7dm2tvg8AAEBTVutr4mrr1KlTkqRbbrlFY8aMUW5urpYvX678\n/PwaP1fs54c3Jens2bMaPHiwo2MCAACYmsNL3MyZMzVx4kS1bdtWkhQcHKxmzZpp1qxZmj9/vu2B\neQAAAKg/hz8nzmKx2ArcDb169ZIk5efnO/rbAQAANEkOL3EJCQmaPn263VhOTo48PT0Nuf0WAADA\nHTW4xJWWlqqgoEClpaWSpCFDhmj37t16+eWXdebMGb333ntasWKFJk6cKB8fnwYHBgAAgAOuiTt0\n6JDGjRun9PR0RUVFaejQoSotLdWmTZuUkpIif39/jRs3TlOmTHFEXgAAAKgeJW7z5s12X0dFRVV4\nQF5cXJzi4uIalgwAAABVcvjdqQAAwHUde2aZId83JOmpOr9n0KBBslgs2rlzp1q1amX32u9//3t1\n69ZNS5cuVXBwsMLCwrRlyxZZLJYK2xgxYoSmTZvWoPyuyOE3NgAAADhKXl6eVq9eXeO8Q4cOKT09\nvRESuQ5KHAAAcFldu3ZVRkaGsrOza5y3Zs0a5eXlNVIy41HiAACAyxo2bJjCwsKUmJiokpKSKudN\nnjxZHTp0UGJioqxWayMmNA4lDgCqkZ+frxkzZigyMlIRERGaNWuWzp8/b3QsoMnw8PDQ0qVLde7c\nOaWmplY5z8vLS0uXLtWnn36q1157rRETGocSBwBVsFqtmjx5si5fvqz09HRlZGSooKBAU6dONToa\n0KTcfPPNmjFjhl566SXl5ORUOa9///763e9+pz/+8Y/65ptvGjGhMZrk3alLNu0zOgIAEygsLFT3\n7t01Z84cBQUFSZLGjx+v6dOn69KlS/L19TU4IdB0TJgwQbt27dKCBQu0ffv2Kuc98cQT+uijj7Rw\n4UKlpaU1YsLGx5E4AKhCQECAUlJSbAUuPz9fmZmZuv322ylwQCNr1qyZli1bptOnT+tPf/pTlfN8\nfHz09NNP65NPPqm27LmDJnkkDgDqatq0adq9e7d8fX1r9RiD1NRUrV27thGSAU1Hz549NXXqVK1f\nv17+/v5Vfib73XffrQcffFDLly9v5ISNiyNxAFALCQkJ2rp1q8LDwzVhwoQab26Ij4/XiRMn7P7s\n3r27kdIC7mvy5Mnq3r278vPzq523YMECeXl56dKlS42UrPFxJA4AaiE4OFiSlJKSooEDByorK0uP\nPfaYwamAuqvPJye4khYtWujZZ5/VQw89VO28Nm3aaMmSJW79/1NKHABUobCwUPv379dvfvMb21ir\nVq3UtWtXHjMCNII9e/ZUOh4SEqKjR4/avv75Z7jf8Ktf/arK19wBp1MBoAr/+te/NHv2bB05csQ2\n9t133+n06dPq0aOHgckAgBIHAFXq06ePIiIilJSUpM8//1zHjh3TzJkz1a5dO8XFxRkdD0ATR4kD\ngCpYLBalpqbq1ltv1ZQpUzR27Fj5+PgoIyNDPj4+RscD0MRxTRwAVKNdu3Zu/5gCAObEkTgAAAAT\nosQBAACYECUOAADAhChxAAAAJkSJAwAAMCHuTgUAoAlJPXjKkO8bH8EDsh2tSZe40P3ba5zzedTw\nRkgCAAB+btCgQTp37pzta4vFIh8fH/Xr109PPPGEevfuXen7goODtXLlSsXGxjZWVEM06RIHAABc\n26OPPqqHH35YknT9+nUVFhbq6aef1oQJE/TBBx+odevWFd6zd+9e3XTTTY0dtdFxTRwAAHBZ3t7e\nCggIUEBAgDp27KjbbrtN8+bNU1FRkfbt21fpewICAuTl5dXISRsfJQ4AAJhKs2bNJEmenp4KDg7W\n888/rwEDBmjAgAEqKChQcHCw3nzzTdv8N954Qw888IBCQ0N17733KiMjw/ba9u3bNWTIEP3hD3/Q\nHXfcoSeffLLR96e+OJ0KAABMIy8vT6tWrVJAQIDCw8MlSVu3btXGjRv1ww8/KCAgwG7+yy+/rDVr\n1igpKUn9+/fXvn37tGzZMpWWlmrixImSpK+++kq33367duzYodLS0kbfp/qixAEAAJe1bt06bdy4\nUZL0ww8/qKysTCEhIVq7dq3terhhw4bp1ltvrfBeq9WqtLQ0Pfzww3rooYckSb/85S+Vl5entLQ0\nTZgwwTZ32rRp6tq1ayPskeNQ4gAAgMsaM2aMRo8eLenH06ht27atcDNDVeWrqKhIhYWFCgsLsxvv\n37+/0tLS9O9//1uS5OHhoaCgICekdy5KHAAAcFm+vr76xS9+Ue2cqm5iqGq8vLxcktS8+Y81yGKx\nyNPTswEpjcGNDQAAwC21bt1anTp1UnZ2tt34Z599poCAAPn6+hqUzDE4EgcAANzW1KlT9eyzz6pb\nt26KjIzU/v37lZGRoRkzZsjDw8PoeA1CiQMAoAlpah9/NWrUKF27dk0bNmzQ4sWL1bVrV82fP992\nnZ2ZUeIAAIBL2rNnT41zTpw4UePY+PHjNX78+ErfP3z4cA0fbs6P2OSaOAAAABPiSFwDLdlU+Ud+\nJD9yZyMnAQAATQlH4gAAAEyoziUuOTlZiYmJ1c45cuSIRo0apb59++q+++7Tjh076h0QAAAAFdW6\nxFmtVj3//PPKzMysdl5RUZEmTZqk2267Tdu3b9fvf/97JSYmau/evQ0OCwAAgB/V6pq4vLw8PfXU\nUzp58qQ6d+5c7dytW7eqdevWSkxMlMViUffu3XXs2DG99NJLio6OdkhoAACApq5WR+Kys7MVGBio\nnTt31vjZYgcPHlT//v1lsfxn05GRkcrOzpbVam1YWgAAAEiq5ZG42NhYxcbG1mqD+fn5CgkJsRvr\n0KGDrl69qm+//Vbt2rWre0oAAADYcfgjRq5du1bhQ2RvfF1aWlrte1NTU7V27VpHRwIAAHA7Di9x\nLVu2rFDWbnzdqlWrat8bHx+v+Ph4u7GzZ89q8ODBjg0JAABgcg5/TlynTp1UUFBgN3bhwgV5e3ur\nTZs2jv52AAAATZLDS9wdd9yhgwcP2t3EsH//foWHh9vd7AAAAID6a3CrKi0tVUFBge2U6YgRI1RU\nVKRFixbpiy++0ObNm/XWW29p0qRJDQ4LAACAHzW4xB06dEjR0dE6dOiQJKl9+/ZKS0vTsWPHFBcX\np4yMDK1YsUJ33XVXg8MCQGMrLCzUvHnzFB0drYiICD3yyCPKzc01OhYA1P3Ghs2bN9t9HRUVpRMn\nTtiN9evXT2+88UbDkgGAwa5fv67HH39cVqtV69atk7e3t1JTUzV+/Hi9/fbb8vPzMzoigCaMi9QA\noArHjx/XoUOHtGzZMoWGhqpHjx764x//qCtXruijjz4yOh6AJs7hjxhxN6H7t9c45/Oo4Y2QBEBj\nCwwM1IYNG3TzzTfbxjw8PCRJly5dMioWAEiixAFAlfz8/DRw4EC7sc2bN+vatWt8FjQAw1HiAKCW\ndu/erdWrV2vChAnq3r17tXP5BBoAzmaaEveXI1/rpvxrtZ4fH9HDiWkANDXbt2/XwoULNXToUM2d\nO7fG+XwCDQBn48YGAKjB+vXrtWDBAo0aNUorV67kweUAXIJpjsQBgBE2btyoNWvWaMaMGZo+fbrR\ncQDAhhIHAFU4fvy4UlJS9OCDD2rkyJF2nwvt4+Mjb29vA9MBaOrctsQde2ZZla+FnrnYiEkAmNU7\n77yj8vJybdu2Tdu2bbN7LSEhQdOmTTMoGQC4cYkDgIaaPXu2Zs+ebXQMAKgUV+cCAACYECUOAADA\nhDid6iRLNu2rdDz5kTsbOQkAAHBHlDgH4PNVAQBAY+N0KgAAgAlR4gAAAEyIEgcAAGBClDgAAAAT\nosQBAACYECUOAADAhChxAAAAJkSJAwAAMCFKHAAAgAlR4gAAAEyIEgcAAGBCbvvZqTu6h1f5WnHn\nHyqM9d/3iTPjAAAAOJTbljhJKr5asawBAAC4A06nAgAAmBAlDgAAwIQocQAAACZEiQMAADAhShwA\nAIAJUeIAAABMiBIHAABgQm79nDhXtGTTvkrHkx+5s5GTAAAAM6PE/X8H7rynzu/hUx4AAIBROJ0K\nAABgQrUqceXl5Vq1apWio6MVFhamGTNmqLCwsMr5CQkJCg4Otvszfvx4R2UGAABo8mp1OjU1NVVZ\nWVlasWKF2rZtq8WLFys+Pl6vvvpqpfNzc3M1Z84cDRs2zDbm6enpmMQAAACoucSVlpYqPT1dSUlJ\nuvvuuyVJq1ev1uDBg5Wdna3w8PAK88+cOaPQ0FAFBAQ4JzUAAEATV+Pp1OPHj6u4uFiRkZG2saCg\nIHXp0kUHDx6sMP/LL79UWVmZunfv7tikAAAAsKnxSFx+fr4kqWPHjnbjHTp0sL32U7m5uWrRooVS\nU1P18ccfy8vLSzExMZo2bZq8vLwcFBsAAKBpq7HEXb16VRaLRS1atLAb9/T0VElJSYX5p06dkiTd\ncsstGjNmjHJzc7V8+XLl5+drxYoV1X6v1NRUrV27ti75AaDRJCcnq7y8XEuXLjU6CgDUXOJatmyp\n69evq6ysTM2b/2d6aWmpWrVqVWH+zJkzNXHiRLVt21aSFBwcrGbNmmnWrFmaP3++/Pz8qvxe8fHx\nio+Ptxs7e/asBg8eXOsdclWh+7dX+/qxb/YoJOmpRkoDoC6sVqteeOEFZWZmasSIEUbHAQBJtbgm\nLjAwUJJUUFBgN37hwoUKp1glyWKx2ArcDb169ZKkSk+/AoAry8vL07hx4/Tqq6+qc+fORscBAJsa\nS1zv3r3l4+OjTz/91DZ29uxZnTt3Tv37968wPyEhQdOnT7cby8nJkaenp7p16+aAyADQeLKzsxUY\nGKidO3cqKCjI6DgAYFPj6VRPT0+NHj1aK1eulJ+fn/z9/bV48WJFRkaqX79+Ki0t1aVLl+Tr6ytP\nT08NGTJEs2fP1ssvv6zBgwfr2LFjWrFihSZOnCgfH5/G2CcAcJjY2FjFxsYaHQMAKqjVw35nzpyp\nsrIyzZ07V2VlZbrnnnuUnJwsSTp06JDGjRun9PR0RUVFaejQoSotLdWmTZuUkpIif39/jRs3TlOm\nTHHqjhiBz1sFUBVu1ALgbLUqcc2bN9f8+fM1f/78Cq9FRUXpxIkTdmNxcXGKi4tzTEIAMCF3vlEL\ngGuo1WenAgAAwLVQ4gAAAEyIEgcAAGBCtbomDgBgfks27av3e5MfudOBSQA4AiXOhRx7ZlmNc/hU\nB8A4mzdvNjoCANhQ4gDAZGrzC19lQs9crDD2edTwWr2Xo3iA66HEAQCcigIIOIcpS9x3J08ZHQEA\n3ELo/u0O21Ztj+oBcAzuTgUAADAhUx6JM7OqPqrrWKsWlY7HfZHtzDgAAMCkOBIHAABgQpQ4AAAA\nE+J0qsnwLDkAroqbJIDGRYkDABPJPXNRXSt53huApofTqQAAACZEiQMAADAhShwAAIAJcU2ciyi+\n+kOl4zu6h9d5WyENDQMAAFweR+IAAABMyC2OxFV1FAsAAMBduUWJAwC4lxvPnNvRgGfP9erWludm\nwq1R4lxcVUcZfar4rFVJSj14qk7fIz6iR53mAwAA41HiAKCRFZ/+Sh4Xv6vXe2/i8hEA/x83NgAA\nAJgQR+Lc0Hcnaz6d2qYnp1ABADAzShzqfA2dxHV0AFxf7pmLemPTvjq/L/mRO52QBnA8TqcCAACY\nECUOAADAhChxAAAAJsQ1cagXrqMDAMBYlDgAAH5iST1uhriBmyLQmChxTZQRjyHhkyQAuDsKIBoT\nJQ4A4LZCG/DZqz/3edRwh20LcARKHAAYoKrPRa6NA3fe0+Dv33/fJw3eBgBjUeLgsrh5AgCAqpmm\nxDXkA6PdUVW/xfu0atHISVwLxQ+As3BqFq6G58QBAACYUK2OxJWXl2vNmjXKyspScXGx7rnnHiUn\nJ6t9+/aVzj9y5IiWLl2qf/7zn+rYsaOmTZumuLg4hwZH5Rr7CJ0Rd7k6GnfNojp1Xf/MguvqjFXZ\nUb1j3+yp17ZCkp5qaByYVK1KXGpqqrKysrRixQq1bdtWixcvVnx8vF599dUKc4uKijRp0iQ98MAD\nWrp0qf7+978rMTFR7du3V3R0tMN3AA1TXemrTUGrDVcreg3NwynbpqUu6x/QELlnLtbvfZOfVK9u\nbRv8/SmD5lNjiSstLVV6erqSkpJ09913S5JWr16twYMHKzs7W+Hh4Xbzt27dqtatWysxMVEWi0Xd\nu3fXsWPH9NJLL1Hi0GRR/MyprutfU8PRPMBYNZa448ePq7i4WJGRkbaxoKAgdenSRQcPHqywiB08\neFD9+/eXxfKfy+0iIyO1ePFiWa1WeXh4ODA+aqshjzOAMepT/OqKoli9uq5/qDuKoGPU9yie3TYm\nP+mAJD+q6cYNHmzsGDWWuPz8fElSx44d7cY7dOhge+3n80NCQirMvXr1qr799lu1a9euTgHLy8sl\nScWXq/4HeqWkrE7bRM2KL9VtvrdXw250/v7Tf0uq+u+yoduvK2tBm0b9fkZZ+u55oyPU2/f/LpT0\nnzXCGeq6/tXkp+uZB+uWw3x4621GR3ApoYcONHgbn4f1b9gGLp+oNsdLH2U0bPv1dN+y+fV+7wuv\nH6r3e2eMDKv29RvrSV3Xsxr/y3j16lVZLBa1aGF/Ybynp6dKSkoqzL927Zo8PT0rzJV+PDVRndTU\nVK1du7bS1957+YWaogJoggoKCvSLX/zCKduu6/r3U6xnMMq7jtjI/g8avAmH5HCwFYMbvl/18eaG\n2s2r63pWY4lr2bKlrl+/rrKyMjVv/p/ppaWlatWqVaXzf17Wbnxd2fyfio+PV3x8vN3YtWvX1Ldv\nX73//vtq1qxZTXFNY/Dgwdq9e7fRMRzG3fZHcr99crf9KS8v13333ac+ffo47XvUdf37qaa0njmL\nu/2bdSZ+VrXnij+r8vJyFRQU1Hk9q7HEBQYGSvqxHd7435J04cKFCqcYJKlTp04qKCiwG7tw4YK8\nvb3Vpk3dT1G1bNlSkpz2m7aRgoKCjI7gUO62P5L77ZO77Y/0nzXCGeq6/tXEndczZ3HHf7POws+q\n9lzxZ1WfdaHGh/327t1bPj4++vTTT21jZ8+e1blz59S/f8Vz5nfccYcOHjwoq9VqG9u/f7/Cw8Pt\nbnYAAFdX1/UPABpTja3K09NTo0eP1sqVK/Xxxx/r6NGjmj17tiIjI9WvXz+VlpaqoKDAdsp0xIgR\nKioq0qJFi/TFF19o8+bNeuuttzRp0iSn7wwAOFJN6x8AGKlWt/zNnDlTZWVlmjt3rsrKymxPLJek\nQ4cOady4cUpPT1dUVJTat2+vtLQ0PfPMM4qLi1Pnzp21YsUK3XXXXU7dEQBwhurWPwAwUq1KXPPm\nzTV//nzNn1/x1tyoqCidOHHCbqxfv3564403HJNQ0uOPP+6wbbkKd9snd9sfyf32yd32R2qcfapu\n/asPd/x7cBZ+VrXHz6r23Oln5WH96cVrAAAAMAXuNAAAADAhShwAAIAJUeIAAABMiBIHAABgQpQ4\nAAAAE6LEAQAAmJBLl7jy8nKtWrVK0dHRCgsL04wZM1RYWGh0LIdITk5WYmKi0TEapLCwUPPmzVN0\ndLQiIiL0yCOPKDc31+hYDZKfn68ZM2YoMjJSERERmjVrls6fP290LIc4fPiwQkJCtH//fqOjNMip\nU6cUHBxc4c/BgweNjlYld17LnM0d1kpnccc12JnccX136RKXmpqqrKwsrVixQhkZGcrPz1d8fLzR\nsRrEarXq+eefV2ZmptFRGuT69et6/PHH9dVXX2ndunV67bXX1Lp1a40fP17ffvut0fHqxWq1avLk\nybp8+bLS09OVkZGhgoICTZ061ehoDXblyhU9+eSTKi8vNzpKg+Xm5srPz0979+61+9O3b1+jo1XJ\nHdcyZ3OXtdJZ3HENdia3Xd+tLqqkpMQaFhZm3bZtm20sLy/P2qtXL+tnn31mYLL6O3PmjHXs2LHW\nqKgo68CBA61PPfWU0ZHq7ejRo9ZevXpZT506ZRsrKSmx9u3b15qVlWVgsvq7cOGCdebMmda8vDzb\n2AcffGDt1auX9eLFiwYma7iFCxdax44da+3Vq5d13759RsdpkJSUFOuYMWOMjlFr7riWOZs7rZXO\n4o5rsDO56/ruskfijh8/ruLiYkVGRtrGgoKC1KVLF5c+bVKd7OxsBQYGaufOnQoKCjI6ToMEBgZq\nw4YNuvnmm21jHh4ekqRLly4ZFatBAgIClJKSYvu7yc/PV2Zmpm6//Xb5+voanK7+PvroI3344YdK\nSkoyOop66+jsAAAH70lEQVRDnDx5UrfccovRMWrNHdcyZ3OntdJZ3HENdiZ3Xd9r9dmpRsjPz5ck\ndezY0W68Q4cOttfMJjY2VrGxsUbHcAg/Pz8NHDjQbmzz5s26du2aoqOjjQnlQNOmTdPu3bvl6+ur\n9PR0o+PUW1FRkRITE7Vs2TJTL1Q/dfLkSZWUlGjkyJE6d+6cevbsqdmzZys0NNToaJVyx7XM2dxp\nrXQWd1+Dncld1nfJha+Ju3r1qiwWi1q0aGE37unpqZKSEoNSoSq7d+/W6tWrNWHCBHXv3t3oOA2W\nkJCgrVu3Kjw8XBMmTDDtxa+LFi3SoEGDNGDAAKOjOMS1a9eUl5en77//Xk8++aTWr1+vDh06aOzY\nsfriiy+Mjlcp1jI0Bndbg53JXdZ3yYVLXMuWLXX9+nWVlZXZjZeWlqpVq1YGpUJltm/frhkzZuj+\n++/X3LlzjY7jEMHBwQoNDVVKSoquX7+urKwsoyPVWVZWlo4dO6Z58+YZHcVhWrZsqQMHDig9PV0R\nEREKDQ3V8uXL1bVrV23ZssXoeJViLYOzueMa7EzusL7f4LIlLjAwUJJUUFBgN37hwoUKpyVgnPXr\n12vBggUaNWqUVq5cKYvFZf9J1aiwsFBvv/223VirVq3UtWtXU/6mtn37dp0/f972WIuYmBhJ0qOP\nPqrk5GSD09Vf69at5enpafvaYrGoR48e+uabbwxMVTXWMjiTO63BzuRu6/sNLvu33bt3b/n4+OjT\nTz+1jZ09e1bnzp1T//79DUyGGzZu3Kg1a9ZoxowZWrhwoe2iWrP617/+pdmzZ+vIkSO2se+++06n\nT59Wjx49DExWP88995zefvtt7dixQzt27FBaWpok6ZlnnlFCQoLB6eonJydH4eHhysnJsY2Vl5fr\n+PHj6tmzp4HJqsZaBmdxtzXYmdxtfb/BZW9s8PT01OjRo7Vy5Ur5+fnJ399fixcvVmRkpPr162d0\nvCbv+PHjSklJ0YMPPqiRI0faHWXw8fGRt7e3genqp0+fPoqIiFBSUpKefvppNW/eXKtWrVK7du0U\nFxdndLw6+/lRHi8vL9u4v7+/EZEarHfv3urSpYuSk5O1aNEieXt7a+PGjfr22281btw4o+NVirUM\nzuCOa7Azudv6foPLljhJmjlzpsrKyjR37lyVlZXpnnvuMfVpIHfyzjvvqLy8XNu2bdO2bdvsXktI\nSNC0adMMSlZ/FotFqampWrlypaZMmaKSkhJFR0crIyNDPj4+RseDpObNmystLU0rV67UY489pqtX\nryo8PFwZGRkuXUxZy+Bo7rgGO5O7ru8eVqvVanQIAAAA1I3LXhMHAACAqlHiAAAATIgSBwAAYEKU\nOAAAABOixAEAAJgQJQ4AAMCEKHEAAAAmRIkDAAAwIUocTOGvf/2rfvvb3yo0NFT/8z//o88++0xl\nZWX61a9+pddee83oeABQa+Xl5dqwYYN+/etfq0+fPvrv//5v/fnPfzY6FkyIEgeX99xzz2nu3LkK\nDQ3VvHnzVFxcrMTERL377rv64YcfNHz4cKMjAkCtLVu2TGvWrNF//dd/KSkpSbfeeqtWrVqlvXv3\nGh0NJuPSn50K7Nu3Txs3btTjjz+u+Ph4SZK3t7cWLFigDRs26OGHH5anp6fBKQGgdr766itt2bJF\nM2fO1JQpUyRJsbGxioiI0OHDhxUdHW1wQpgJR+Lg0tLT0+Xv769HH33UNtalSxdZrVbl5+frd7/7\nnYHpAKBu3n//fVksFo0ePdo21qJFC1mtVrVs2dLAZDAjShxcVllZmf7xj39o4MCBlS5uo0ePVuvW\nrQ1IBgD1k5OTo+DgYLVp08Y2dujQIZWXlys4ONjAZDAjShxc1tdff60rV64oJCTEbryoqEiSNGbM\nGCNiAUC9nThxokJZ27Rpk2666SZFREQYlApmRYmDy7pR1nx9fW1jVqvVdheXn5+fIbkAoD5KSkp0\n5swZ9erVS8XFxcrOzlZCQoL+7//+T1OnTlWrVq2MjgiT4cYGuKwbpxtyc3NtY6+//rqOHj0q6ccF\nkZsaAJjFqVOndP36dQUHB2vdunVKS0uTJN16660aOXKkwelgRpQ4uKyePXuqc+fO+t///V95eXmp\nrKxMaWlpGjJkiHbt2qXVq1dr3Lhxuvnmm42OCgA1uvELaXBwsNq1a6fbbrtNhw4d0uuvv67Ro0cr\nKytLzZo1MzglzITTqXBZzZo1U2pqqnr37q0//elPevXVVzV16lSlpKTo/vvvV2Zmpr788kujYwJA\nreTm5srf31/+/v7q3bu3hg4dqsTERD3xxBM6ceKETp06ZXREmAxH4uDS+vTpo61bt1YYX7NmjQFp\nAKD+Tp48WekdqNevX5fFYlFAQIABqWBmlDgAABpBbm6u2rRpo9LSUtv1vFevXtXrr7+uqKgotWvX\nzuCEMBsPq9VqNToEAADu7PLly+rfv78kKSwsTLGxsbpy5Yp27Nihc+fOKTMzUz179jQ4JcyGa+IA\nAHCyGzc1jBw5Ut98842WLl2qV155Rb169dIbb7xBgUO9cCQOAAAn27Jli5YsWaLs7Gx5e3sbHQdu\ngiNxAAA42cmTJ9W5c2cKHByKEgcAgJPl5uaqe/fuRseAm6HEAQDgZKdOnVKPHj2MjgE3wzVxAAAA\nJsSROAAAABOixAEAAJgQJQ4AAMCEKHEAAAAmRIkDAAAwIUocAACACVHiAAAATOj/Ad85LzAuUfmz\nAAAAAElFTkSuQmCC\n", 273 | "text/plain": [ 274 | "" 275 | ] 276 | }, 277 | "metadata": {}, 278 | "output_type": "display_data" 279 | } 280 | ], 281 | "source": [ 282 | "bins = 20\n", 283 | "a = .7\n", 284 | "\n", 285 | "plt.figure(figsize=(9,3.75))\n", 286 | "plt.subplot(121)\n", 287 | "plt.hist(mcmc_alpha, normed=True, bins=bins, color=sns.color_palette()[0], histtype='stepfilled', linewidth=2.0, alpha=a);\n", 288 | "plt.hist(nn_raw_params[:,0], color=sns.color_palette()[2], normed=True, bins=bins, histtype='stepfilled', linewidth=2.0, alpha=a);\n", 289 | "# plt.hist(nn_raw_samples[:,0], normed=True, bins=bins, histtype='step', linewidth=2.0);\n", 290 | "plt.hist(stats.expon(scale=1.0).rvs(10000), color=sns.color_palette()[5], normed=True, bins=2*bins, histtype='stepfilled', linewidth=2.0, alpha=a);\n", 291 | "plt.xlim((0,5))\n", 292 | "# plt.xlim((0,plt.xlim()[1]))\n", 293 | "plt.xlabel(\"$\\\\alpha$\")\n", 294 | "plt.subplot(122)\n", 295 | "plt.hist(mcmc_beta, normed=True, bins=bins, color=sns.color_palette()[0], histtype='stepfilled', linewidth=2.0, alpha=a);\n", 296 | "# plt.hist(nn_raw_samples[:,1], normed=True, bins=bins, histtype='step', linewidth=2.0);\n", 297 | "plt.hist(nn_raw_params[:,1], normed=True, color=sns.color_palette()[2], bins=bins, histtype='stepfilled', linewidth=2.0, alpha=a);\n", 298 | "plt.hist(stats.gamma(a=0.1, scale=1.0).rvs(10000), color=sns.color_palette()[5], normed=True, bins=2*bins, histtype='stepfilled', linewidth=2.0, alpha=a);\n", 299 | "plt.xlim((0,3.5))\n", 300 | "# plt.xlim((0,plt.xlim()[1]))\n", 301 | "plt.ylim(0,4)\n", 302 | "plt.legend(['MCMC', 'NN', 'Prior'])\n", 303 | "plt.xlabel(\"$\\\\beta$\")\n", 304 | "plt.tight_layout()\n", 305 | "# plt.savefig(\"poisson-params-alt.pdf\")" 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": { 311 | "deletable": true, 312 | "editable": true 313 | }, 314 | "source": [ 315 | "## Simple benchmark: likelihood weighting\n", 316 | "\n", 317 | "Using the prior as a proposal, relative to using the learned network, what do we gain in terms of effective sample size?" 318 | ] 319 | }, 320 | { 321 | "cell_type": "code", 322 | "execution_count": 12, 323 | "metadata": { 324 | "collapsed": false, 325 | "deletable": true, 326 | "editable": true 327 | }, 328 | "outputs": [], 329 | "source": [ 330 | "def bootstrap_propose(model):\n", 331 | " success = False\n", 332 | " while success == False:\n", 333 | " try:\n", 334 | " model.alpha.rand()\n", 335 | " model.beta.rand()\n", 336 | " model.theta.rand()\n", 337 | " success = True\n", 338 | " except:\n", 339 | " pass\n", 340 | " try:\n", 341 | " weight = model.x.logp\n", 342 | " except:\n", 343 | " weight = -np.Inf\n", 344 | " return weight\n", 345 | "\n", 346 | "def bootstrap_IS(model, ns=100):\n", 347 | " theta = np.empty((ns,10))\n", 348 | " params = np.empty((ns,2))\n", 349 | " log_w = np.empty((ns,))\n", 350 | " for n in xrange(ns):\n", 351 | " log_w[n] = bootstrap_propose(model)\n", 352 | " theta[n] = model.theta.value\n", 353 | " params[n] = [model.alpha.value, model.beta.value]\n", 354 | " return theta, params, log_w\n", 355 | "\n", 356 | "def normalize_log_weights(log_w):\n", 357 | " safe_log_w = np.array(log_w)\n", 358 | " safe_log_w[np.isnan(log_w)] = np.NINF\n", 359 | " A = safe_log_w.max()\n", 360 | " log_denom = np.log(np.sum(np.exp(safe_log_w - A))) + A\n", 361 | " return np.exp(safe_log_w - log_denom)\n", 362 | "\n", 363 | "def systematic_resample(weights):\n", 364 | " ns = len(weights)\n", 365 | " cdf = np.cumsum(weights)\n", 366 | " cutoff = (np.random.rand() + np.arange(ns))/ns\n", 367 | " return np.digitize(cutoff, cdf)" 368 | ] 369 | }, 370 | { 371 | "cell_type": "code", 372 | "execution_count": 13, 373 | "metadata": { 374 | "collapsed": true 375 | }, 376 | "outputs": [], 377 | "source": [ 378 | "def nn_IS(model, ns=100):\n", 379 | " # theta\n", 380 | " theta, log_q = theta_est.propose(real_data, ns=ns)\n", 381 | " theta = theta.data.cpu().numpy().squeeze(2)\n", 382 | " log_w = -log_q.data.cpu().numpy().sum(1)\n", 383 | " # alpha, beta\n", 384 | " params, log_q = params_est.propose(Variable(torch.Tensor(theta)))\n", 385 | " alpha = params.data.cpu().numpy()[:,0] #np.empty((ns,))\n", 386 | " beta = params.data.cpu().numpy()[:,1] # np.empty((ns,))\n", 387 | " log_w -= log_q.data.cpu().numpy().squeeze(1)\n", 388 | " # likelihood\n", 389 | " log_w += stats.poisson(model.t.value * theta).logpmf(model.x.value).sum(1)\n", 390 | " log_w += stats.gamma(a=alpha[:,None], scale=1.0/beta[:,None]).logpdf(theta).sum(1)\n", 391 | " log_w += stats.expon(scale=1.0).logpdf(alpha)\n", 392 | " log_w += stats.gamma(a=0.1, scale=1.0).logpdf(beta)\n", 393 | " log_w[np.isnan(log_w)] = np.NINF\n", 394 | " return theta, params.data.cpu().numpy(), log_w" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": 14, 400 | "metadata": { 401 | "collapsed": false 402 | }, 403 | "outputs": [], 404 | "source": [ 405 | "nn_theta,nn_params,nn_log_w = nn_IS(M_test, ns=10000)" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": 15, 411 | "metadata": { 412 | "collapsed": false 413 | }, 414 | "outputs": [], 415 | "source": [ 416 | "boostrap_theta,bootstrap_params,bootstrap_log_w = bootstrap_IS(M_test, ns=10000)" 417 | ] 418 | }, 419 | { 420 | "cell_type": "code", 421 | "execution_count": 16, 422 | "metadata": { 423 | "collapsed": false 424 | }, 425 | "outputs": [ 426 | { 427 | "name": "stdout", 428 | "output_type": "stream", 429 | "text": [ 430 | "Effective sample size (benchmark): 1.00000175273\n", 431 | "Effective sample size (NN): 323.396998175\n" 432 | ] 433 | } 434 | ], 435 | "source": [ 436 | "print \"Effective sample size (benchmark):\", 1.0/np.sum(normalize_log_weights(bootstrap_log_w)**2)\n", 437 | "print \"Effective sample size (NN):\", 1.0/np.sum(normalize_log_weights(nn_log_w)**2)" 438 | ] 439 | }, 440 | { 441 | "cell_type": "markdown", 442 | "metadata": {}, 443 | "source": [ 444 | "### Define a method for running divide-and-conquer SMC\n", 445 | "\n", 446 | "This algorithm first proposes values across all thetas, then resamples prior to proposing alpha and beta." 447 | ] 448 | }, 449 | { 450 | "cell_type": "code", 451 | "execution_count": 17, 452 | "metadata": { 453 | "collapsed": true 454 | }, 455 | "outputs": [], 456 | "source": [ 457 | "def run_DCSMC(model, ns=100, return_extras=False):\n", 458 | " # stage one\n", 459 | " num_points = 10\n", 460 | " theta, log_q = theta_est.propose(real_data, ns=ns)\n", 461 | " theta = theta.data.cpu().numpy().squeeze(2)\n", 462 | " log_w = -log_q.data.cpu().numpy()\n", 463 | " \n", 464 | " log_w += stats.poisson(model.t.value * theta).logpmf(model.x.value)\n", 465 | " \n", 466 | " log_w[np.isnan(log_w)] = np.NINF\n", 467 | " \n", 468 | " Z_est = np.log(np.exp(log_w).mean(0))\n", 469 | " \n", 470 | "# print Z_est\n", 471 | " \n", 472 | " for i in xrange(num_points):\n", 473 | " indices = systematic_resample(normalize_log_weights(log_w[:,i]))\n", 474 | " np.random.shuffle(indices)\n", 475 | " theta[:,i] = theta[indices,i]\n", 476 | "\n", 477 | " # Now: we have unweighted samples for each theta[:,i]\n", 478 | " # ... and we have Z estimates for each i. What is next?\n", 479 | " \n", 480 | " # Sample values of alpha, beta\n", 481 | " params, log_q = params_est.propose(Variable(torch.Tensor(theta)))\n", 482 | " alpha = params.data.cpu().numpy()[:,0] #np.empty((ns,))\n", 483 | " beta = params.data.cpu().numpy()[:,1] # np.empty((ns,))\n", 484 | " log_w = -log_q.data.cpu().numpy().squeeze(1)\n", 485 | " \n", 486 | " # Merge\n", 487 | " tmp = stats.gamma(a=alpha[:,None], scale=1.0/beta[:,None])\n", 488 | " log_w += tmp.logpdf(theta).sum(1)\n", 489 | " log_w += stats.expon(scale=1.0).logpdf(alpha)\n", 490 | " log_w += stats.gamma(a=0.1, scale=1.0).logpdf(beta)\n", 491 | " \n", 492 | " assert(log_w.shape == (ns,))\n", 493 | " \n", 494 | " indices = systematic_resample(normalize_log_weights(log_w))\n", 495 | " \n", 496 | " log_w[np.isnan(log_w)] = np.NINF\n", 497 | " \n", 498 | " Z_est = Z_est.sum() + np.log(np.exp(log_w).mean())\n", 499 | " \n", 500 | " if return_extras:\n", 501 | " alpha_orig = np.array(alpha)\n", 502 | " beta_orig = np.array(beta)\n", 503 | " \n", 504 | " alpha = alpha[indices]\n", 505 | " beta = beta[indices]\n", 506 | " theta = theta[indices]\n", 507 | " \n", 508 | " if return_extras:\n", 509 | " return theta, alpha, beta, Z_est, alpha_orig, beta_orig\n", 510 | " else:\n", 511 | " return theta, alpha, beta, Z_est" 512 | ] 513 | }, 514 | { 515 | "cell_type": "markdown", 516 | "metadata": {}, 517 | "source": [ 518 | "# Compare importance sampling with NN proposals, proposals from prior, and divide-and-conquer SMC with NN proposals" 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "execution_count": 18, 524 | "metadata": { 525 | "collapsed": true 526 | }, 527 | "outputs": [], 528 | "source": [ 529 | "sizes = [5, 10, 50, 100, 500, 1000, 5000, 10000]\n", 530 | "\n", 531 | "replications = 10" 532 | ] 533 | }, 534 | { 535 | "cell_type": "code", 536 | "execution_count": 19, 537 | "metadata": { 538 | "collapsed": false 539 | }, 540 | "outputs": [], 541 | "source": [ 542 | "dcsmc_results_Z = np.empty((replications, len(sizes)))\n", 543 | "dcsmc_results_L2 = np.empty((replications, len(sizes)))\n", 544 | "\n", 545 | "for c, size in enumerate(sizes):\n", 546 | " for rep in xrange(replications):\n", 547 | " tmp_theta, tmp_alpha, tmp_beta, Z_est = run_DCSMC(M_test, size)\n", 548 | "\n", 549 | " dcsmc_results_L2[rep,c] = np.sqrt(np.mean((tmp_theta.mean(0) - true_theta)**2))\n", 550 | " dcsmc_results_Z[rep,c] = Z_est" 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "execution_count": 20, 556 | "metadata": { 557 | "collapsed": false 558 | }, 559 | "outputs": [ 560 | { 561 | "name": "stderr", 562 | "output_type": "stream", 563 | "text": [ 564 | "/Library/Python/2.7/site-packages/ipykernel/__main__.py:7: RuntimeWarning: divide by zero encountered in log\n" 565 | ] 566 | } 567 | ], 568 | "source": [ 569 | "lwis_results_Z = np.empty((replications, len(sizes)))\n", 570 | "lwis_results_L2 = np.empty((replications, len(sizes)))\n", 571 | "\n", 572 | "for c, size in enumerate(sizes):\n", 573 | " for rep in xrange(replications):\n", 574 | " bootstrap_theta, _, bootstrap_log_w = bootstrap_IS(M_test, ns=size)\n", 575 | " Z_est = np.log(np.exp(bootstrap_log_w).mean())\n", 576 | " mean_est = np.dot(bootstrap_theta.T, normalize_log_weights(bootstrap_log_w))\n", 577 | " lwis_results_L2[rep,c] = np.sqrt(np.mean((mean_est - true_theta)**2))\n", 578 | " lwis_results_Z[rep,c] = Z_est\n" 579 | ] 580 | }, 581 | { 582 | "cell_type": "code", 583 | "execution_count": 21, 584 | "metadata": { 585 | "collapsed": true 586 | }, 587 | "outputs": [], 588 | "source": [ 589 | "nnis_results_Z = np.empty((replications, len(sizes)))\n", 590 | "nnis_results_L2 = np.empty((replications, len(sizes)))\n", 591 | "\n", 592 | "for c, size in enumerate(sizes):\n", 593 | " for rep in xrange(replications):\n", 594 | " nnis_theta, _, nnis_log_w = nn_IS(M_test, ns=size)\n", 595 | " Z_est = np.log(np.exp(nnis_log_w).mean())\n", 596 | " mean_est = np.dot(nnis_theta.T, normalize_log_weights(nnis_log_w))\n", 597 | " nnis_results_L2[rep,c] = np.sqrt(np.mean((mean_est - true_theta)**2))\n", 598 | " nnis_results_Z[rep,c] = Z_est\n" 599 | ] 600 | }, 601 | { 602 | "cell_type": "code", 603 | "execution_count": 22, 604 | "metadata": { 605 | "collapsed": false 606 | }, 607 | "outputs": [ 608 | { 609 | "name": "stderr", 610 | "output_type": "stream", 611 | "text": [ 612 | "/usr/local/lib/python2.7/site-packages/numpy/lib/nanfunctions.py:1304: RuntimeWarning: invalid value encountered in subtract\n", 613 | " np.subtract(arr, avg, out=arr, casting='unsafe')\n" 614 | ] 615 | }, 616 | { 617 | "data": { 618 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAADsCAYAAABaHo0zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlclNX+wPHPDDAMsoig4IIroJamgoCaqLlkds3Uq3U1\n5aq5ZUp2+1lquK+huWu5oJZLud2sNM1Ky+RqGVbmvmaGiigoCAIDM/P7Y2BkHDZhYAb8vl8vXs48\n5zzn+c5wnPlynvOcR6HX6/UIIYQQQtggpbUDEEIIIYTIjyQqQgghhLBZkqgIIYQQwmZJoiKEEEII\nmyWJihBCCCFsliQqQgghhLBZkqgIIYQQwmZJopJNq9WyYMECQkNDCQgI4I033uD27dvWDksIIYR4\nrEmikm3ZsmXs3LmTyMhINm3aRFxcHOHh4dYOSwghhHisKWRlWtBoNLRu3ZpJkybxz3/+E4DY2Fg6\nd+7Mp59+SmBgoJUjFEIIIR5PMqICnD17ltTUVEJCQozbfHx8qFWrFjExMVaMTAghhHi82Vs7AFsQ\nFxcHgLe3t8l2Ly8vY9mjSE9P5+TJk1SrVg07OzuLxCiEEELYOq1Wy61bt2jatClqtdoibUqiAqSl\npaFUKnFwcDDZrlKpyMjIKHDfZcuWsXz58tIMTwghhChXNm/eTFBQkEXakkQFUKvV6HQ6srKysLd/\n8JZoNBqcnJwK3Dc8PNxs0u1ff/1F165d2bx5M9WrVy+VmEXFFXnkPADj2zS0ciRCCPFo4uLiGDBg\nANWqVbNYm5KoADVq1ADg1q1bxscA8fHxZqeDiiLndE/16tXx8fGxTJDiseFc9S6A9B0hRLllyWkP\nMpkWaNy4Mc7Ozhw9etS4LTY2lmvXrhEcHGzFyIQQQojHm4yoYJiL8sorrzBv3jyqVKmCp6cn06dP\nJyQkhBYtWlg7PCGEEOKxJYlKtjfffJOsrCzefvttsrKyaNeuHVOmTLF2WEIIIcRjTRKVbPb29kyY\nMIEJEyZYOxQhhBBCZJM5KkIIIYSwWZKoCCGEEMJmSaIihBBCCJsliYoQQgghbJYkKkIIIYSwWZKo\nCCGEEMJmSaIihBBCCJsliYoQQgghbJYkKkIIIYSwWZKoCCGEEMJmSaIihBBCCJsliYoQQgghbJYk\nKkIIIYSwWXL3ZCvTZmSQcORnMuLjUXt749E6BDtHR2uHJazkfuo9PP/4GqekRKITGxLUrS9qZxdr\nh2V0P/Uex/buIPXGdVxq1rK5+CoaW3+/bT2+isbW3+/7qff4Ze92i7er0Ov1eou3+piLjY2lc+fO\n7N+/Hx8fnzzrvLx1FN4JmfQ4eBfn9Ae/glS1gl0d3Fn2elRZhVsgW/+PUZGcifmRa+8vxylNa9yW\n5mRHrXFjeCKovRUjM7D1+CoaW3+/bT2+isbW3++c+FKT0njn8vkCv/8elSQqpaAoicr/evZBqwC7\nPN59rQLabv3EqiMr/+vZh9tudlTK0FEp40GQ9x0V3HdU0nPjNqvFVhH13/waQ768bZK05khVK2i/\n7iOrJogHe/dB46Aw6Qs57jsqaLfeuvFVNOmpKUQPfdXkSylHmpMdoWvXlcr7rdfrIfvH+NhQYFIW\n3X8gmSoFTnn0hzSVgtYffIC6krNhg0LxUI0Hz02KHq5ntl/uIkWR6uUuUxSx3sPtFbhfGbFWfyiq\nkV/8jz6fLMY5XcdtjUYSlfKgKInKj7365JmkGD3RAGVNb7BTgFKJXqFAocx+rFSAQgnZz03+VSjA\nToleoTQ8NqmT/aPIbkNpXkef3e7lDR9T65YWJ03eH0SMHYDSQYVCr0ev1xk/xPQ6w3NFrsfoMXzA\n6bLrYSgzfOjpDJ+FOp3Zh6FeZ6hreJyrndz76fWgMzw37E/2Y4ztKXL2h+y6uT6Aje3msY9eD3oM\nj7PjUOiyt2PYbmyP3LHlfqxHoX+ozNjWg8fE38HjnvmHUI577o7onNXGNhTGYxraVhiP/6BMYTwe\nD+LIvd24X+72zOsp9ECWFvsC+qtWgaEf5VAoDO/Zgw357wzoC/sueKjcPBSFyVZ9Mb5cCovB7JiF\nxlT89u0zdagz828x3UGBzk6JguzfUU5IuX932Y2alOkx7GNSt7DfjshP7t9QoX24CPXM+5ihskKn\nL/D7Iuvh/38FBlBwTy1OX1DoQZndbGkkKjJHxUqU/2gHXx3Kv8KZy+jOXDbZZMmMMr/OmLPdr4B9\nnTR6mL/JgtGIwrjezUB/NwO9wvBBpwfI/jfnS9lku0kdhfG5+b4Ykgrlgw9QvSL7q0yhMG5zSlPi\nkq7LN777aiX3K9mR3TSmPSyn55qX5BQX9OFYaA5TyH+MvPYvzj4l2l9fSPnDMvN/rwFQKtA62mf/\n/gwt5v7dGw6ieHBohWnylvt3nRNQ7v6CInc9hUk5KFAnpOCemn+Md90cSPOoVOCXosl7UND7+VAb\n+b53Bb3HBcVRxA9W83r6PB+aHTufXQz18m8jN1VyGi55jLbmyHCyQ+PyYAS+0JdUaAd8tHRFlZyG\nS1ohfbYEJFGxksu//4xvQeVP16daYCDodA/+ItfpUOj0D7YZH+f6y16nR6HTZZfps8t02Y91D9V7\nUGb4C//B49tnT1M9MSvf+FK8XKnUoB6gMI7kKBTZIzo5oz8KhbFckTPCk5355zw32SdndEehRGEs\nU2bXN23XWGbcpsxuRglKZfahldltKnPtr8x1TCWK3LEYH5NdT2lsW2ls56E2lQoUZLebXY7SEIfS\nGHt2XTs7lBjaRmmHQqEwtvv1/43G53b+73fmwH/QpnfYg4+PnL+0cp7mPMp3+4PnxRnK3j70XwUm\nKi4vdadbn8GP3K7I24yF8+l88Kd8yx3/1Z3OVny/o//7EWzYlW955V7d6C79wWIKe78r9f0HXW24\nP5SUJCpWkqU0zD3Ib05C/X/2JtS/rRUiM9g+9F8Fljt3e4ZQ+SCymJ5L1xZ8Dvof/8LRXmWFyAx6\nFBZft75WiKriemfkKKKP/mKz7/dGe3/6qJU455G8pqqVtJf+YFFB3foSvX2PzfaHguKzBFlHxUq+\nbufOrg7upKpN/7rNueonpH6wlSIz8ErM4r5j3n9533dUECQfRBaldnah1rgxpDnZmWzPmdVv7Ymq\nth5fRWPr77fOwYlvn3uRVLXpV0iqWsm3z71o9fgqmvCD59nXtUee7/e+rj2s/n7n118t5bGYTLt5\n82ZmzJhhss3Ozo7Tp08bn3/00Ud8/PHHJCYmEhgYyNSpU6lXr16xjleUybQAFxOu8P4Py6l2MQG3\nFC3JLnbc8vNk3DNj8PMs3rEtydYvh6uI0lNTWLZ2FU53E2jRxPbWUUlPTSHm6x2kXL8ml6uXgfLQ\nH2w5vopi+J5fAVBmplHr0mFckxO45+bJNd+n0Tk4seYfgVaO0CA9NYU5i95n68ZPZTLtozp//jyd\nOnUySVZyn6ffvn07S5cuZc6cOdSvX59FixYxbNgw9uzZg0pVesPtfp71WNpzDkev/c7NlNt4u1Ql\npFYLVFYc4s/tiaD21F8baPLFFCofRKVK7exCwlPPATCmY1MrR2NO7ewip/zKUHnoD7YcX0WROxGZ\n8H1l7gPv2eD7rVBX4qZfW+BTi7b7WCQqFy5coHXr1lSrVi3P8qioKIYMGUK3bt0AWLBgAaGhoezb\nt48ePXqUamwqexWhdUNK9RglIV9MQgghCvPn3VSWx1zifpblr/55LOaoXLx4EV/fvK+xSUhI4MqV\nK4SEPEgWnJ2dadq0KTExMWUVohBCCFEuabQ6lsdcIlmT/5WLJVHhE5WbN2+SlJTEjz/+SLdu3ejQ\noQPjxo3j5s2bAMTFxQHg7e1tsp+Xl5exTAghhBDmMrU6Dl69VWpJClSAUz85E1fzolKp+PDDDwGw\nt7dn0aJF3Llzh4ULFzJ48GB27txJWloaAI4PLVevUqnIyMgo9PjLli1j+fLlJXwVQgghhO3Q6fWk\naLK4m57JnfRMkjIyuZOu4W56JnczMg3/pmtIySydS5JzK/eJire3N3v27MmzTKlUUr9+fY4cOYKH\nh4dxu5+fH+3bt+fgwYPUqlULAI1GY7KvRqPBycmp0OOHh4cTHh5usq2g5EkIIYQojgytjowsLVo9\n/HwtkYDq7qjsHv3ESEaWljsPJRwPHj9ITLQFXBSstlfi7uhAbbdKZOp0XLyTWpKXVqByn6g4ODjk\nO/8kR+4kBQyndapUqcKNGzcICgoC4NatW9StW9dYJz4+vtB2hRBCiLKQM1k1ZwQj6vgV3M7YMybI\nl/ruhhtAanV6kjUPEo676ZncydA8eJ6djKRl5T8KYqeAyo4O1K1cCXdHB9zVhp8qagcqO6qokv1c\nbf9gzRSNVsfE70+W2umfcp+oFGbDhg2sXr2a77//HgcHBwCuXbtGYmIi/v7+eHp6Uq9ePY4ePWpM\nWlJTUzl58iT9+vWzZuhCCCEEGq2OZTGXuPdQIpCsyWLeT+ep4aLmXkYWSRmZBd7nx9nBDg8nB9wd\nnY3Jh7vaITshMSQhLip7w609HoHKTsmYIF+Wx1wivRivrzAVPlF55plnWLRoEREREYwcOZK7d+8y\ne/ZsWrZsSdu2hiXqBw8ezLx586hbty7+/v4sXLgQLy8vnn32WStHL4RtyFlwKj+2suCUEOVVepaW\nhDSNyU9i9r83UtLzHQXJ0um5di8ND7UK3yrOVFGrciUfhgQk53FxThMVVX13Z+Z2bEr49psWb7vC\nJyp16tRh/fr1LFiwgJdeegkHBwc6derEhAkTjHX69+9PcnIyc+fOJTU1lcDAQKKiokp1sTchhBCP\nB71eT2qm1iQBuZ2WYZKMpOYzKdVOAY52BS9N38OvBi/41yiN0B+Jyk6JqpBYi6PCJyoALVq0YOPG\njQXWGTlyJCNHjiyjiIQoX0xXxjwJ2ObKmEJYg06vJzkjM8/RkJyfDG3eC6GplAo8nFTUq+yMp5MK\nDycVVZ1Uxsfuagd+uX6HqONX8j1+tUqO+ZZVBI9FoiKEEEIUl1an5066Jt9TM4npGrJ0ec8OcbK3\no1olRzyzkw/PXElIVScVLip7k1u65CWgujtuZ+zznKzqprInoLq7RV6nrZJERQghxGNNo9WZjYDk\nnJ5JTNNwJz3/SaquKnt8XJ3MEhFDMuJIJYeSnwrJPVk1d7LipjJc9VOac09sgSQqQgghKrQ04/yQ\njDxHRPK7rFYBVFE74FvFOVcC4mgcDfFwUpVZkpAzWXX8gRNo9TCgSe1ir6Niabkn26dnZFq8fUlU\nhBBClFv67BVUHz4tkzsZuZ/PFTN2CsP8kMZ5jIh4OKmoolZhr3y0S3VLk8pOiWP2+iWtankUUrvi\nkERFCCGEzdLp9dxNzzQ/NZOu4fb9DBLTM9HkM1HV0U6Jh5OKBk7OZnNDPJ1UuDk6PPKaIcJc7sn2\nsbGxdJ5p2fYlURFCCFEslljSPUun40569hUz93Odnkk3jIYkpuW/lHslBzu8nc0nquacnnFxsCt0\noqqwfZKoCCGEeGRFWdIdDMlM4kNzQxLuGxKRhDQNSQVMVK3saE+dyrlOy6gd8az0ICHJvYx7RZbX\ngou5t1X0BRclURFCCPFINFqd2RUoYFjS/f2fzvNkNTfjKElKPhNVlQpwd1Th7+GCR+7RELXK+NzB\nBiaKCuuTREUIIYSJLJ2O5Ox7xyRlZJKckcXdjEySs59fu5eW75UyGp2e328mYa9U4KFWUTv3RNVc\noyHujirsbGiiqi2r6CMmhZFERQghbJQl5oDk0Ov1pGVpScqVgCSlZ5KsySQpPXdSkmk8nZOfwtKL\nbg286d2opkxUFRYhiYoQQtigos4BydLpuacxJB05SUjOyIfh58G2zHxWT81Ryd4ON0d7ark6UdnR\nAbfsm9m5OdpT2dHB+HPyVhJrj/+Vbzs+rk6SpAiLkURFCCFsTEFzQOb/dB7fKs7c02SRlJGV7xyQ\nHHYKcHV0oKaLE5XVDxION2PiYW98XtTRmsDqVdh+5tpju6S7KFuSqAghhI3Q6/VcT0ln3+Wb+c4B\nydTpOZuQgpO9EjdHB2q6qHMlH6YjH5Ud7XFW2Vt8dONxX9JdlC1JVIQQwkr0ej1xqemcTUjhXMI9\nziemcK+QERKAF/yq07NhzTKIMH+2vKS7qFgkURFCiDKi1+u5mZrBucR7nMtOTnKPSLg7OtCqZhUc\n7JRE/52QbzvVndVlEW6hHtcl3UXZkkRFCCFKiV6v59b9DMOISeI9ziekcDfXTdsqO9oTUqMKjTxd\naeTpglclRxQKBRqtjj9uJskcECGQREUIISxGr9dzO03DuYTsEZPEe9xJf5CYuKrsCarhTiMPVxp5\nulLd2THPJd5lDogQD0iiIoQQJZCQlsG5hBTOZicniekaY5mLyp6W1XMSExdquKiLfO8ZmQMihIEk\nKkII8QgS0zQmc0xupz1ITJwd7Ajwrmw4lePhSk1XdYmuuJE5IEJIoiKEEAW6m64xnsY5l5BC/P0M\nY1klezuae1WmcfYck1qy0JkQFieJihBC5JKUkWkyx+Rm6oPExMleSTMvN+Mck9pukpgIUdoqVKKi\n0Wjo27cvQ4cOpWfPniZlH330ER9//DGJiYkEBgYydepU6tWrZyw/ceIEs2fP5syZM3h7e/P666/T\nq1evMn4FQoiylpyRyflEw2mcc4kp3EhJN5Y52ilpWs3NMGLi4UJtt0pyIz0hyliFSVRSUlL4z3/+\nw7lz58zKtm/fztKlS5kzZw7169dn0aJFDBs2jD179qBSqUhMTGTYsGG88MILzJ49m8OHDxMREUHV\nqlUJDQ21wqsRQpSWe5oszmcnJecS7nH9ocSkSVU3Gnm60MjTlbqSmAhhdRUiUTl8+DBTpkzBzc0t\nz/KoqCiGDBlCt27dAFiwYAGhoaHs27ePHj16sH37dlxcXIiIiECpVOLr68vp06dZt26dJCpClHOp\nmizDiEn2HJPYe2nGMpVSwROehtM4jT1dqFvZGXtJTISwKRUiUTlw4AC9evVixIgRPPXUUyZlCQkJ\nXLlyhZCQEOM2Z2dnmjZtSkxMDD169CAmJobg4GCUygeX/YWEhDB9+nT0en2RLycUQljf/czsxCR7\njklscho59wx2UCqMp3EaebpS370S9kq53FcIW1bsRGXx4sV07NiR5s2bWzKeYpk0aVK+ZXFxcQB4\ne3ubbPfy8jKWxcXF8eSTT5qVp6WlcefOHTw85LJAIWxVWqaWC3dSjBNgrybfNyYm9koF/tlJSWNP\nF+pXdsZB1iERolwpdqKycuVK1q1bR2RkJM8//7wlYzIRGxtL586d8yxTqVScOHGiwP3T0gzDvI6O\njmb7ZmQYZvOnp6ejUqnMysEwQbcgy5YtY/ny5QXWEUJYTnqWlguJKcY5Jn8lPUhM7BQK/Kq4GOeY\nNHB3lgXShCjnipyoXLp0CV9fX5NtOp2O//u//yMxMZEBAwbku29MTAz79+9n/Pjxjxygt7c3e/bs\nybNMWYQhW7XacPOuhxMOjUaDk5OTsU5e5YCxTn7Cw8MJDw832VZQciWEeDQZWVou3kk1XpVzJSkV\nXXZmYqeABlWcaeRhGDFpUMUFR0lMhKhQCk1UEhMTWbJkCd9//z0//vijSdm4cePYtWsXs2bN4tat\nW7z55pt5tnH9+nU++uijYiUqDg4OZgnSo6hRowYAt27dom7dusbt8fHxxnarV6/OrVu3TPaLj4+n\nUqVKuLq6FvvYQohHl6HVcflOivFGflfupqLNTkyUCqhX2dk4x8SvirNx5VYhRMVUYKJy4MABxo8f\nT5cuXfjyyy/Nyj09Pdm4cSNjxoxh1apVJCQkMGPGDJuafOrp6Um9evU4evQoQUFBAKSmpnLy5En6\n9esHQMuWLfnss89MJs7+/PPPBAYGFmnURghRfJlaHZfuphrnmPyZlEpW9pCJAqhbuVL2kvQu+Hu4\noJbExOqG7/m1wG1r/hFYluGICq7AROXatWtotVpatWqFu3vetxWvVKkSq1atYvz48Wzfvp3ExEQW\nLVpkNufDmgYPHsy8efOoW7cu/v7+LFy4EC8vL5599lkA+vbtS1RUFFOnTmXQoEEcPnyY3bt3s2bN\nGitHLkTFk6nV8WdSqmHEJOEel++aJiZ13CoZ55j4V3HByUESEyEeZwUmKmFhYdSpU4d58+bx5Zdf\nsm7dujzrOTg4sHDhQuMIy9ChQ/nggw9s5rRJ//79SU5OZu7cuaSmphIYGEhUVJQxmapatSpRUVHM\nmjWLXr16UbNmTSIjI2nTpo2VIxei/MvS6fjz7n3jHJNLd1LIzJWY+Lg5Ge8u3NDDhUoOFWLVhApN\nRkxEWSr0E6FDhw6EhoayZcuWQhuLiIigWrVqLFy4kAEDBrB27VqqVatmkUCLKq+VaQFGjhzJyJEj\n892vRYsW7Nixo7TCEqJCyNDqyMjSotXDz9cSCajubnZVTZZOz1+5Rkwu3UlBkzP7FfBxdTKMmHi4\n0tDDBWeVJCZCiPwV6RPCzs6uwKt6chsxYgRVq1ZlypQp9OvXj7Vr15YoQCGEbfjzbirLYy6RkqkF\nIOr4FdzO2PN6ywYoFArjHJOLd1LI0OqM+9V0URvnmDT0dMVVEhMhxCMo9idGfsvVA/zzn/+kSpUq\nvPXWW/Tv35/u3bsX9zBCCBug0epYHnOJZE2WyfZkTRbvHTlvsq2Gi9p4VU5DDxfcHB3KMlQhRAVT\n7ETl6NGjZGVl5VvesWNH1q1bx2uvvcbmzZuLexghhA34Le6uWZKSW2MPF9rVqUojT1cqS2IihLCg\nEl17a29fcJ4TEBDA5s2bzZavF0KUL7fSMgosb1TVlZCaHpKkCCEsrtRPFvv5+bFr1y7++OOP0j6U\nEKKUVHNyLFG5EEIUV5msZubq6krbtm3L4lBCiFJguLon74Uc3VT2BFTPe50lIYQoKYuMqDRu3LjQ\n1WgdHR3x9vamVatWDBs2jDp16lji0EKIMpCckYlWp0cB6HNtd1PZMybIV278Z2Gy8qsQD1gkURk9\nejT79+/n4sWLtGvXznhPnStXrhAdHU3Dhg1p1aoVV69e5bPPPuOrr75i06ZNPPHEE5Y4vBCilO08\ndx2tHgY9VYf/nr2GVg8DmtTOcx0VIYSwJIskKl5eXty5c4e9e/dSu3Ztk7K//vqLsLAw/Pz8GD9+\nPFeuXOFf//oXixcvZtWqVZY4vBCiFP15N5WjN+5Q160ST/t4svtiHACtanlYObKKS0ZMhHjAIn8K\nrV27lgEDBpglKQB169ZlwIABrF69GoB69erRr18/fv3VfGhTCGFb9Ho9289cA+ClJ2qhtKEbjgoh\nHg8WSVTi4uKws8v/xmF2dnbcuHHD+NzHxweNRmOJQwshStHvN5O4cCeFFt6VaeRpG/fuEkI8XiyS\nqPj5+bF161YSExPNyhISEti6dSt+fn7GbX///TdVq1a1xKGFEKUkS6dnx9lrKBXQp1Eta4cjhHhM\nWWSOyvjx4xk+fDjPPvssXbt2NV7Rc/XqVb755hs0Gg2zZs0CQKPRsHPnTkJDQy1xaCFEKfnx6i3i\n72fQsW41qruorR2OEOIxZZFEpVWrVmzZsoWlS5eyd+9e0tPTAcMlyW3atCE8PJwmTZoAoFKp+OGH\nHwo8VSSEsK77mVp2XbiB2l5JD7/q1g5HCPEYs9jKtE8++SQrV65Ep9ORkJAAgKenJ0ql+dklSVKE\nsG17LsWRkqnln41q4irL4gshrMjiS+grFApjIlLYInBCCNtz+34G+6/E46F2oHM9L2uHI4R4zFks\nUbl8+TKLFy8mOjqatLQ0AJycnGjXrh1jx46lQYMGljqUEKIU7Tx/nSydnt6NaspibkIIq7NIonL2\n7FkGDBhAZmYmzz77LPXr1wfgzz//5LvvviM6OprNmzfTuHFjSxxOCFFK/rybytHrd6jj5kRITVnQ\nTQhhfRZJVN5//33c3NzYtGkTtWqZXsZ4/fp1BgwYwIIFC1izZo0lDieEKAV6veFyZICXnvCRxd2E\nEDbBIonKr7/+ymuvvWaWpADUrFmTfv36yXL5Qti44/FJnE9MoblXZRrL4m7CRr28dVSB5dv+9WEZ\nRSLKSpmcgJZJtULYNpPF3RrXtHY4QghhZJERlYCAAD799FNefPFFqlc3XXMhLi6OTz/9lICAAEsc\nqkAajYa+ffsydOhQevbsadyemppKy5Yt0ev1JvXnzZtnrHfixAlmz57NmTNn8Pb25vXXX6dXr16l\nHrMQtuDQ1dvcTM3gmTpVqeHiZO1whMhX7hGT0bsiAFjRY7a1whFlwCKJyltvvcXAgQPp1q0bXbt2\npV69eoBhMu23336LUqlk3LhxljhUvlJSUvjPf/7DuXPnzMouXrwIwHfffYda/WCFTTc3NwASExMZ\nNmwYL7zwArNnz+bw4cNERERQtWpVWUFXVHj3M7V8eTF7cTf/GtYORwib1qlTJ5RKJbt27cLJyTSp\nDwsLo06dOsyePZtGjRoREBDAJ598YraeWKdOnejbty+vv/56vsfJyMhg1apVfPXVV1y7dg1nZ2cC\nAwMZPXo0TZs2BSA2NpbOnTsDsGfPHnx9fU3a0Gg0PP3009y7d4+DBw+aDCR8/fXXfPLJJ5w9exat\nVou/vz///ve/+cc//lGi96c0WOTUT5MmTdi+fTtt27bl22+/ZenSpSxdupTvvvuO0NBQtm3bxhNP\nPGGJQ+Xp8OHD9OrVy7jQ3MPOnz9PjRo1qF27NtWqVTP+ODo6ArB9+3ZcXFyIiIjA19eXsLAwXnzx\nRdatW1dqMQthK76+FEeKJovnG1THTRZ3E+VERpaG9KwMUjPTiP7rKJqssrvR7d9//83ChQsLrffb\nb7+xYcOGYh3j3Xff5euvvyYiIoKvv/6atWvX4uTkxMCBA7l06ZJJXQcHB77++muzNg4dOkRKSorZ\n9pkzZzJx4kTatWvHpk2b2L59O507d+btt99m9erVxYq3NFlsHRU/Pz9WrFiBTqcz3pzQw8Mjz5Vp\nLe3AgQP06tWLESNG8NRTT5mVX7hwocB1XGJiYggODjaJNSQkhOnTp6PX62WOjaiwEtI0fHslnipq\nB7rUl8Wb6SQoAAAgAElEQVTdRPlwMeEKkdEfcE+TCsDSn9ZTWe3K+NDX8fOsV+rHr127Nps2beL5\n558nMDCwwHqLFy+mc+fO1K5du8jtp6Sk8NVXX/HBBx/Qvn17AHx8fJg/fz5du3Zl27ZtTJw40Vi/\ndevW7Nu3j9GjR5u0s3fvXlq2bElMTIxx2/79+9m0aROrV6+mQ4cOxu0535FLlizJcxqHNVk8i1Aq\nlVStWpWqVauWSZICMGnSJMaMGYNKpcqz/MKFC9y/f5+wsDCefvpp+vXrx8GDB43lcXFxeHt7m+zj\n5eVFWload+7cKdXYhbCmz8/J4m6ifNFkaYiM/oCk9Hsm25PS7xEZ/UGZjKz07t2bgIAAIiIiyMjI\nyLfeiBEj8PLyIiIiwmyOZGGUSiXR0dFotVrjNjs7Oz7++GNGjBhhUrdbt26cO3eOK1euGLdpNBoO\nHDhgdipn69atNGnSxCRJyfHKK6+wfv16PDxsaw2lYo2oNG7c+JFHGRQKBadPn37kY+U+B/cwlUrF\niRMnCm3jwoULuLi4MGnSJKpUqcLu3bsZOXIk69evp02bNqSnp5slOTnPNZqCO/2yZctYvnx5EV+N\nELbjr6T7/HQ9kTpuTrSSxd2ElWz8/b/89PevRa6fnpVhHEl5WFL6PUbtehe1vWOR2mpdO5CwFn2K\nfOwcCoWC2bNn07NnT5YtW5bvHExHR0dmz55NWFgYW7ZsoX///kVq38XFhVdeeYWNGzeyb98+2rZt\nS3BwMG3btsXHx8esfr169WjUqBHffPONMYn58ccfqVmzJn5+fiZ1T506xXPPPZfncZ2dnQkODi5S\njGWpWInK6NGjy+x0iLe3N3v27MmzrKgjNt9++y2AceJTkyZNuHDhAh9//DFt2rRBrVabJSQ5zx+e\nLPWw8PBwwsPDTbYVlFwJYQv0ej3bzsQC0LexLO4myg+tXleickupX78+b7zxBgsXLqRbt27GCa4P\nCw4Opn///syfP59nnnmGGjWKNmF90qRJNGvWjB07drBnzx6++OILFAoFXbt2Zfbs2bi6mq511K1b\nN77++mtjorJnzx6ef/55s3aTkpLM9rV1xUpUHv5iLk0ODg5mM5kfVV7JRsOGDfnf//4HQPXq1bl1\n65ZJeXx8PJUqVSp3v1AhiiJncbdmXm48UVX6uLCesBZ9HmlUI/qvoyz9aX2+5cNa9iO0boglQivU\nkCFD2LdvHxMnTuSzzz7Lt964ceM4ePAgkydPJioqyqRs2LBhHDt2zPh8zZo1BAUFAfDiiy/y4osv\ncv/+fY4dO8bevXvZuXMnSqWSxYsXm7TTrVs3lixZQmxsLFWrVuX7778nPDyc+Ph4k3pVqlQhKSmp\npC+9TFX4k9K3b98mKCiIb775xmT7yZMnjUNiOZONcp9D/PnnnwkMDCyzeTZClJUsnZ7/Zi/u1rex\n+WrSQtiykFotqKzOO7murHYlpFaLMovFzs6OOXPm8Oeff7Jy5cp86zk7OzNz5kwOHTpkltDMnj2b\nzz//3PjTtGlTfv75ZyIjI411KlWqRLt27ZgzZw7Dhw83mWOZo0GDBjRs2JB9+/bx448/UrduXeN9\n93ILCAjg+PHjecaZkpLCoEGDOHr0aFHfgjJR4b+Fq1atSkBAAJGRkRw5coTLly8zb948fvvtN157\n7TUA+vbtS2JiIlOnTuXSpUts3LiR3bt3M2zYMCtHL4TlHfr7NnGpGbSrLYu7ifJHZa9ifOjrZslK\nzlU/Kvu8L6ooLf7+/owaNYpVq1Zx9erVfOu1bduWPn368N5775lcMuzt7U3dunWNP2q1mpSUFNat\nW8epU6fM2nF1dcXT0zPPY3Tr1o1vvvmGffv25bseSp8+fTh9+nSeyc6mTZv45Zdf8rwdjjVV+EQF\nYMGCBbRr14533nmHnj178uuvv7J+/Xr8/f0BQzITFRXF6dOn6dWrF5s2bSIyMpI2bdpYOXIhLOt+\nppYvL9zA0U7Ji7K4myin/DzrsaL7LFxVzlRycOKN1kNY0X1WmVyanJcRI0bg6+tLXFxcgfUmTpyI\no6NjoadeOnbsSHBwMCNHjmT79u389ddfnDt3ji1btrBq1Sqzy5BzdOvWjePHj3PgwIE856cAdOjQ\ngb59+zJ27FjWrl3LpUuXOHfuHIsWLWLp0qWMGzfO5hIVi62jYivyWpnWzc2NadOmMW3atHz3a9Gi\nBTt27CjFyISwvq8vGxZ369WwhizuJso1lb3KeHVPWc1JyY+DgwNz587lpZdeKrCeq6srM2bMMI7m\n50epVLJ69WqioqL46KOPmDVrFgqFgsaNGzNnzhy6du2a536+vr74+/ujUqkKXLdl1qxZNG/enG3b\ntvHhh4ZbEvj7+7NkyRKeffbZQl5t2atwiYoQIm8JaRq++zMed7UDXep7F76DEMLMgQMH8tz+5JNP\nmpyqyeuPZjCMluRXllulSpV44403eOONN/Kt4+PjY9bWrl27TJ63atXKrI5CoeDll1/m5ZdfLjQO\nWyCJihCPic/PXydTp6d3w5o4yuJuopx6eeuoArflvmmhqBjk00qIx8BfSff56Voitd2caF1LFncT\nQpQfMqIiRAWn1+vZnr2420uyuJso52TE5PEjIypCVHB/xCdzLjGFp6rJ4m5CiPJHEhUhKjCtTs+O\ns7EokMXdhBDlk5z6EaICy1ncrX3tqtR0Lf7ibsP3mN80Lve2Nf/I/1b3QghREpKoCFFBpeVe3K2h\nLO4mKob/9Sz4vkBtv/hvGUUiyookKkJUUF9fjuOeJoueDWtQuYSLu8mIiRDCWiRREaICSkzT8O2f\n8bg7OvCsLO4mKpDcIyYxww0rvAatyf+GgKL8k8m0QlRAOYu79Woki7sJIco3+QQTooLJWdzNx9WJ\nNrK4mxAW1alTJ7p06UJaWppZWVhYGBEREQA0atSIfv36odPp8mzjgw8+KPQ4uevodDo++ugjevTo\nwVNPPUVQUBCDBw/m8OHDJXxFtk8SFSEqEL1ez/azseiBl56oJYu7iQpLm5GBNj2drNRUbh08hDYj\no8yO/ffff7Nw4cJC6/32229s2LDBIsdcvHgxa9euZcyYMezZs4dNmzbh5+fHsGHDOHLkiEWOYask\nURGiAjlxK5lzCSk0rebGk1XdrB2OEKXi3oWLHBsxiqzke2hT73N+4WKOjRjFvQsXy+T4tWvXZtOm\nTfz6q/ll+w/XW7x4MX///XeJj7l161ZGjBjBc889R+3atWncuDGTJk0iMDCQzZs3l7h9WyaJihAV\nhGFxt2uyuJuo0LQZGZyZNYfMu0km2zPvJnFm1pwyGVnp3bs3AQEBREREkFHA8UaMGIGXlxcRERHo\n9foSHVOpVPLTTz+ZHW/BggVMnjy5RG3bOrnqR4gKIjr2NjdS0mlX25NaJVjcTYiy9Of6j0k4XPRT\nF9r0dLKS7+VZlnk3iZhhI7FTq4vUlufTbag/ZFCRj51DoVAwe/ZsevbsybJlyxg3blye9RwdHZk9\nezZhYWFs2bKF/v37P/KxcgwfPpzIyEhCQ0N5+umnCQ4O5umnn6ZBgwbFbrO8kBEVISqA9CwtX5w3\nLO7Ws2FNa4cjRKnRa7UlKreU+vXr88Ybb7Bu3TpOnjyZb73g4GD69+/P/PnzuXHjRrGP9+qrr7J6\n9WoCAwM5ePAgM2fO5Pnnn2fQoEHExcUVu93yQEZUhKgAvr5807C4m3/JF3cToizVHzLokUY1bh08\nxPmFi/Mt9x05gmod2lkitEINGTKEffv2MXHiRD777LN8640bN46DBw8yefJkoqKiTMqGDRvGsWPH\njM/XrFlDUFBQnu106NCBDh06oNFoOH78ON9++y1btmzhjTfeYNu2bZZ5UTZIEhUhyrnENA3fXr6Z\nvbibl7XDEaJUebQOwcG9stkcFQAH98p4tA4ps1js7OyYM2cOvXv3ZuXK/Bedc3Z2ZubMmbz66qtm\nCc3s2bNJT083Pvf2Nl+g8ezZs3z66adMmjQJBwcHVCoVwcHBBAcH4+vry5QpU0hMTMTDo2IuRyCn\nfoQo5744fx2NTk+vhjVxtLezdjhClCo7R0eemPQuDu6VTbY7uFfmiUnvYufoWKbx+Pv7M2rUKFat\nWsXVq1fzrde2bVv69OnDe++9R0pKinG7t7c3devWNf6o85lfs2XLFr7//nuz7a6urqjValxcXEr+\nYmyUjKgIUY5dTb7PkZzF3Xwq5l9TQjzM1d+Plqs/JGbYSPRaLb4jR+DROqTMk5QcI0aM4JtvvuHs\n2bMF1ps4cSKHDh0iPj7+kdpv3LgxPXr0YOLEiVy/fp127Qyntk6dOsX777/P8OHDUalUxY7f1lWI\nEZVTp04xePBggoKCCA0NJSIigrt37xrLtVotCxYsIDQ0lICAAN544w1u375t0kZ0dDQ9e/akWbNm\n9OjRg4MHD5b1yxDikej1enacuWZY3K2xLO4mHi92jo7YqdXYOztTrUM7qyUpAA4ODsydOxd7+4L/\n9nd1dWXGjBnFOsZ7771HeHg4X375JX379qV3796sX7+eMWPGMHr06GK1WV4o9CW9uNvKbt68SY8e\nPejatStDhgzh7t27TJs2DU9PTz766CPAsKLfjh07iIyMxN3dnenTp2NnZ8enn34KwMWLF+nduzev\nv/46Xbt2ZdeuXURFRbFz5078/f0fOabY2Fg6d+7M/v378fHxseTLFY+BCd8briB4r2PTAuudiE9i\nacwlmlZzY2ywX1mEJoTV/a9nnwLLc9+0UJS90vj+K/cjKnv37kWlUjF9+nR8fX1p2bIlU6dO5ciR\nI1y/fh2NRsOGDRt46623aNu2LU2aNGHhwoX8+uuvxlUFN2zYQIsWLRg1ahS+vr68+eabBAQEWGzp\nYyEsTavTs10WdxNCPAbK/RyVTp060bRpU+zsHkwiVGQPgScnJ3P79m1SU1MJCXkwE9zHx4datWoR\nExNDYGAgMTExPP/88ybttmrViq+++qpsXoQQj+h/sQmyuJt4LMmIyeOn3I+o1KlTx+ya8zVr1uDt\n7Y2/v79xIZyHL/ny8vIylsXFxRVYLoQtMSzudh2VnZIX/WVxNyFExWbzIyo557vyolKpOHHihMm2\n999/nx9++IEVK1ZgZ2dHWloaSqUSBwcHs31z7pmQnp5uNmM6d3lBli1bxvLlyx/lJQlRIvsu3yRZ\nk0UP/xq4q2VxNyFExWbziYq3tzd79uzJs0ypfDAgpNVqmTFjBlu3bmXatGnG5EatVqPT6cjKyjKZ\nka3RaHByMgyZOzo6kpmZadJ27vKChIeHEx4ebrKtoORKiJK4k67hm8s3qezowHOyuJsQ4jFg84mK\ng4MDvr6+BdbJyMhg7NixREdHM3/+fHr06GEsq1GjBgC3bt0yPgaIj483nu6pUaOG2XXtucuFsBVf\nnL+BRqenf8MasribEOKxYPOJSmF0Oh1jx47lp59+4sMPPzQuhJOjcePGODs7c/ToUXr27AkYRjyu\nXbtGcHAwAC1btuSXX34x2e/nn3/O934LQljD38n3ORybQC1XNU/7eFo7HCGsYvieXwssX/OPwDKK\nRJSVcp+ofPrpp3z//ffMmjWLxo0bc+vWLWOZu7s7KpWKV155hXnz5lGlShU8PT2ZPn06ISEhtGjR\nAoCBAwfSp08fli5dSvfu3dm9ezfHjx9n2rRpVnpVQpjS6/VsNy7u5iOLuwkhHhvlPlHZtWsXAJMm\nTTIr27x5M0FBQbz55ptkZWXx9ttvk5WVRbt27ZgyZYqxXqNGjVi+fDnz589nzZo1NGjQgJUrVxZ6\nykmIsnLqdjJnEu7RpKobTaq5WTscIawm94hJURdHFOVbuU9UtmzZUmgde3t7JkyYwIQJE/Kt88wz\nz/DMM89YMDIhLEOrM4ymyOJuQljfhAkTiIuLM658DnDw4EHWrFnDqVOn0Ov11K9fn759+/LKK68Y\n1/XKT0REBC1atOCll15iwoQJ7Ny501imUChQq9U0aNCAkSNH8txzz+XbTlhYGHXq1GH27NnFel0Z\nGRn07duXDz/80OZWVC/366gIUdEdvpbA9ZR02vp44uMmi7sJAZCh1ZGRpeV+ppafryWi0eqsEseh\nQ4cYPXo0Xbp0YceOHXz++ee8/PLLREZGsmLFigL3PXLkCH/88Qd9+jy4LUBQUBDR0dFER0dz6NAh\nPv/8c5o2bcrYsWP57bff8m1r2bJlTJw4sdivw9HRkeHDhzN58uRit1FaJFERwoblXtytZ8Mahe8g\nxGPgz7upvPv9SVIytaRlaYk6foWJ35/kz7upZR7Ltm3b6NixI4MHD8bX15d69erRv39/hg0bVuht\nWBYvXsy///1vk6U2HBwcqFatmvGnXr16TJ48GScnJ/bu3ZtvW+7u7ri4uJTotbzwwgtcuHCBI0eO\nlKgdS5NERQgbtu/yTZIysniuvhfu6op7G3chikqj1bE85hLJmiyT7cmaLJbHXCrzkRWlUsnp06fN\nlrgYPHgwW7duzXe/3377jVOnTvHss88WeoycW8TkLEwaFhbGlClT+Oc//0lwcDAHDhwgLCyMiIgI\n4z4xMTEMHDiQgIAAnn76aWbNmkVaWhpguPK1UaNGrFy5kjZt2vD888+j0WhQKpU899xzJqe1bEG5\nn6MiREX1YHE3e7o2kDV9RMW0/Uwsx+LuFrl+RpaWlExtnmXJmizGHzhR5DWGWlZ356UnSjYfY9Cg\nQQwaNIhOnToRHBxMSEgIrVu3pnnz5ri55T/x/cCBAzRv3hx3d/cC209KSmLFihWkp6fTtWtX4/bt\n27ezaNEi6tWrh4+PD+vXrzeWHT9+nMGDBxMWFsb06dOJjY1l2rRpxMbGsnLlSmO9r776ik2bNpms\nzt6hQwdGjx5Neno6arW6uG+LRUmiIoQNyTnvrtVD1O9X0Oj09GtYE7Us7iYEAFp9ycotLTAwkM8+\n+4x169bxww8/cPjwYcBwH7q5c+fmux7X8ePH8fPzM9t+9OhRAgICAMM6Yenp6dSoUYOZM2fSrFkz\nY71mzZrRrVu3PNtet24dTZs2Zfz48QD4+voybdo0RowYwYULF4yrrg8YMMDs6taGDRui0Wg4ffo0\ngYG2sSaNJCpC2Ig/76ayPOaS8a/F84kpKBVQy8U2/qoRojS89ITPI41q/HwtkajjV/ItH9CkNq1q\neVggsqLz9/dn7ty56PV6zp07x48//siGDRsYPnw43333HZ6e5gs0JiQk5JkINGvWjMjISMBwWsnZ\n2RkPD/PXU9CVORcuXKBDhw4m23ISpgsXLhgTntq1a5vtm3OshISEfNsvazJHRQgbkN95d50eVhy7\nbLUrGoSwNQHV3XFT5f03tpvKnoDqBZ9KsaTU1FRmzpzJ+fPnAcPlxI0bN2bEiBFs3LiR+/fvm616\nnkOhUKDTmf+/VqvV1K1bl7p161K7du08k5ScevnJq0yvNww15b7nnaOjo1k9rVZrjM9WSKIihA34\nLe6uWZKSI1mTxW+PcA5fiIpMZadkTJCvWbLiprJnTJAvKruy+1pzcnJi9+7dbNu2zawsZ35K1apV\n89y3WrVqJCYmlkpcvr6+ZpcyHzt2zFhWkJyYvLxs56ancupHCBtwKy2jROVCPE7quzszt2NTxh84\ngVZvON0TUN29TJMUMJya+b//+z/jSue9evWicuXKXL58mZUrV9KqVat856g0a9aMQ4cOlUpcw4cP\np3fv3kRGRvLSSy9x7do1pk+fTocOHfD19SU2Njbffc+cOYOTkxMNGzYsldiKQxIVIWxANSfzIdhH\nKRficaOyUxqv7inrOSm5vfzyy1StWpWPP/6YoUOHkpqaire3N927d+e1117Ld79OnToRFRVFUlIS\nlStXtmhMDRs2ZOXKlSxevJiNGzfi7u5O9+7defPNNwvd96effqJt27Y2c8UPgEKfc+JKWExsbCyd\nO3dm//79NrcUsbBNGq2Oid+fzPP0j5vKnrkdm5b5X4tC2Lryfq+fl156iZ49ezJw4EBrhwJAZmYm\n7du3Z9GiRbRu3bpYbZTG95+MqAhhA3LOuz88odYa592FsGXD9/xa4LbcNy20dWPHjmX27Nn079/f\nuKibNe3evRs/P79iJymlRT79hLAROefdXRzscLK3Y1jzeszt2JT67s7WDk0IUQpCQ0Np0aIFO3bs\nsHYoZGRkEBUVxZw5c6wdihkZURHChtjKeXchbFV5GjEpirlz51o7BMBwqfJXX31l7TDyJCMqQggh\nhLBZkqgIIYQQwmZJoiKEEEIImyWJihBCCCFsliQqQgghhLBZkqgIIYQQwmZJoiKEEEIIm1UhEpVT\np04xePBggoKCCA0NJSIigrt3H9xt9uLFizRq1MjsJyYmxlgnOjqanj170qxZM3r06MHBgwet8VKE\nEEIIkUu5T1Ru3rzJkCFD8PHxYevWrSxZsoQ//vjD5OZL58+fp0qVKkRHR5v8NG/eHDAkMqNGjaJb\nt27s3LmTzp07M3r0aC5cuGCtlyWEEEIIKkCisnfvXlQqFdOnT8fX15eWLVsydepUjhw5wvXr1wFD\nouLn50e1atVMfhwcHADYsGEDLVq0YNSoUfj6+vLmm28SEBDAhg0brPnShBBCiMdeuU9UOnXqxOLF\ni01u6KRQKABITk4G4MKFCzRo0CDfNmJiYggJCTHZ1qpVK5NTQ0IIIYQoe+X+Xj916tShTp06JtvW\nrFmDt7c3/v7+gCFRycjI4OWXX+batWv4+/vz1ltv0axZMwDi4uLw9vY2acPLy4u4uLhixaTVao3t\nCvGoUm/HA4bbpQshRHmS872X8z1oCTafqMTGxtK5c+c8y1QqFSdOnDDZ9v777/PDDz+wYsUK7Ozs\nSE9P5++//8bDw4N33nkHlUrFpk2bGDhwIDt37sTX15f09HRUKpVZ2xkZGYXGt2zZMpYvX55n2YAB\nA4r4KoUw9421AxBCiGK6desWdevWtUhbNp+oeHt7s2fPnjzLlMoHZ660Wi0zZsxg69atTJs2zZjc\nqNVqfvnlF1QqlTEZee+99zh16hSffPIJkydPxtHRkczMTJO2NRoNTk5OhcYXHh5OeHi4ybb09HSm\nT5/Oa6+9ZnJKqiAff/wxgwYNsljdguoUp6xz587s37+/SPFZw6O8f2XddnH2l/5QMtIf8q9bkvLy\n2B9Ksy9Yov3S7A8l/WworDyvMq1WS9euXWnatGmh8RWVzScqDg4O+Pr6FlgnIyODsWPHEh0dzfz5\n8+nRo4dJuYuLi8lzpVKJn58fN27cAKBGjRrEx8eb1ImPjzc7HVRUarWamjVrPlI26ebmho+Pj8Xq\nFlSnuGVFjc8aHuX9K+u2i7O/9IeSkf6Qf92SlJfH/lCafcES7ZdmfyjpZ0Nh5QWVqdXqQuMrqnI/\nmVan0zF27Fh++uknPvzwQ7Mk5eTJkwQGBnLy5EnjNq1Wy9mzZ41zWFq2bMkvv/xist/PP/9MUFBQ\nseN6eHKuJesXpW5BdYpbZstKM+6Stl2c/aU/lIz0h9IpL4/9obRjtuX+UNLPhsLKy6o/KPR6vb5M\njlRKNm/ezIwZM5g1axbPPPOMSZm7uzsKhYLevXvj4ODA1KlTqVSpEmvWrOGHH35g7969eHp6cu7c\nOfr06cOIESPo3r07u3fvZu3atcY5LAIaNWrEuXPnrB2GsBHSH0Ru0h9EbpbuDzZ/6qcwu3btAmDS\npElmZZs3byYoKIioqCjmzZvHa6+9RlpaGoGBgWzatAlPT0/A8KYuX76c+fPns2bNGho0aMDKlSsl\nSRFCCCGsrNwnKlu2bCm0jre3NwsWLCiwzjPPPGM2IiMeGDNmjLVDEDZE+oPITfqDyM3S/aHcn/oR\nQgghRMVV7ifTCiGEEKLikkRFCCGEEDZLEhUhhBBC2CxJVIQQQghhsyRREUIIIYTNkkRFCCGEEDar\n3K+jImzD1KlTOXDgAPHx8bJC5WPuxo0bTJgwgfj4eJRKJR06dODtt99GoVBYOzRhBQMHDiQ5ORm9\nXk/9+vWZM2eO2f3XxONn+vTpfPLJJ0X6vpARFWERL7zwAjt37rR2GMIG2NnZMW7cOPbu3cvOnTv5\n448/+Oabb6wdlrCSDz/8kC+//JJdu3ZRo0YNoqKirB2SsLKYmBju379f5PqSqDzG/vrrL6ZMmUKP\nHj144oknCAsLy7PexYsXGTRoEM2bNyc0NJQlS5ag1WpN6gQHB1O1atWyCFuUEkv1By8vL5566ikA\nVCoVjRo1Mt6pXJQPlvxscHV1BQw3kE1LS5ORtXLIkv1Bo9Hw/vvvM378+CIfX079PMYuXLjAwYMH\nad68OVlZWXnWSUpKYvDgwfj5+fHBBx9w9epVIiMj0el0/Oc//ynjiEVpKo3+cOfOHb777jvWrVtX\n2uELC7J0Xxg+fDgnTpzA39//kb6ghG2wZH9YsWIFffv2xcPDo+gB6MVjS6vVGh+Hh4frBw4caFZn\n5cqV+qCgIP29e/eM21avXq1v1qyZybYcDRs2LJ1gRamzdH/IyMjQDxw4UL927drSC1qUitL4bMjK\nytJHRkbqV69eXTpBi1Jjqf5w5swZ/aBBg/Q6nU6v1xf9+0JO/TzGlMrCf/0//vgjoaGhJpPfunfv\nTnp6OkePHi3N8EQZs2R/0Gq1jBs3jieffJJXX321VOIVpac0Phvs7Ozo3bs3X3zxhUVjFaXPUv3h\n119/5eLFi3Tu3JlOnToB0KlTJxITEws+fgliF4+By5cv06BBA5NtNWvWxMnJicuXL1spKmEtRe0P\nU6ZMwdnZmQkTJpR1iKKMFKUvJCUlcfv2bWP5vn378Pf3L9M4RdkoSn945ZVXiI6O5sCBAxw4cACA\nAwcOFHoaSOaoiAIlJycbJ8Pl5ubmRnJysvF5REQEhw4dAqB9+/a0a9eO2bNnl1mcomwUpT8cO3aM\nHTt20LBhQ3r16gVAnz59+Pe//12msYrSVZS+kJyczJtvvolGowGgQYMGTJ48uUzjFGWjqN8VxSGJ\nilf5bYAAAAxDSURBVLAISUpEjpYtW8paOgKA2rVr89///tfaYQgbVdTPCTn1Iwrk5uZGSkqK2fbk\n5GTc3NysEJGwJukPIof0BZFbafYHSVREgRo0aGA2F+XGjRukpaWZnY8UFZ/0B5FD+oLIrTT7gyQq\nokDt27cnOjraJFPes2cParWakJAQK0YmrEH6g8ghfUHkVpr9wW7atGnTShifKKfS0tLYv38/Fy9e\nJDo6mqSkJDw9Pbl48SK1atXCwcEBf39/tm7dys8//4yXlxeHDx9m4cKFDBo0iA4dOlj7JQgLkv4g\nckhfELlZuz8o9Hq93kKvRZQzsbGxdO7cOc+y/fv34+PjAxiWRZ4xYwa///47bm5u9O3bl/DwcOzs\n7MoyXFHKpD+IHNIXRG7W7g+SqAghhBDCZskcFSGEEELYLElUhBBCCGGzJFERQgghhM2SREUIIYQQ\nNksSFSGEEELYLElUhBBCCGGzJFERQgghhM2SREWIciosLIwmTZpw5syZPMuffPJJli1bVupxxMbG\n0qhRI7744otSP9aj2rhxI6GhoTRr1ozVq1dbO5xi6dSpExEREdYOQwirkURFiHIsKyuLd999l6ys\nLGuHYnPu37/P3Llzeeqpp1i7di0vvviitUMSQhSDJCpClGOurq6cPn2aNWvWWDsUm3Pv3j20Wi1d\nunQhODiY6tWrWzskIUQxSKIiRDnWtGlTunfvzgcffMClS5fyrZff6ZkJEybw7LPPGp936tSJDz74\ngJkzZxISEkLLli2ZMWMGaWlpREZG0qpVK1q1akVERAQZGRkmbcXFxTF06FCaNWtG586dWb9+vUm5\nTqdj5cqVdOnShaZNm9KtWze2b99uUicsLIzx48czevRomjdvzmuvvZbva/r9998ZMmQIwcHBBAcH\nM3bsWGJjYwH47LPPaN++PQDvvvsujRo1yred3bt38+KLL9KsWTPatGnDuHHjuHnzprH8/v37zJ8/\nn65du9K0aVMCAwMZOnQoZ8+eNXkfR44cyebNm+nYsSPNmzdn6NCh3Lp1ix07dtClSxcCAgIYPHiw\nMcac93vp0qXMnDmTli1b0rp1a6ZNm0ZaWlq+8aanpxMZGUn79u156qmn6NWrF/v37zepc/LkSQYN\nGkTLli2Nx/3999/zbVMIWyaJihDl3KRJk3B2dubdd99Fp9OVuL2oqCju3r3LkiVL6NevH5s3b6Z3\n797cuHGDBQsWEBYWxo4dO9i8ebPJfkuWLKFWrVqsWLGCLl268N5775mM9EybNo3ly5fTu3dvVq5c\nSceOHZk8eTIbN240aWf37t24u7uzcuVKBg0alGeM//vf/3jllVewt7cnMjKSKVOmcObMGfr168ft\n27d55pln+PDDDwEYNWoUW7duzbOdY8eO8c4779C1a1eioqKYMGECP/30E+PGjTPWeeedd/j8888Z\nOXIk69atY+LEiZw7d45x48aR+1Zpv/zyC//973+ZMmUKU6ZM4ejRo4SFhbFx40YmTJjAzJkzOX78\nOLNmzTKJYePGjZw+fZr58+czatQoPv/8c95+++0849Xr9YwZM4Zt27YxdOhQVqxYwRNPPMHo0aP5\n7rvvAEhJSWHYsGFUqVKFZcuWsWjRItLS0hg2bBgpKSl5tiuELbO3dgBCiJLx8PBg8uTJvPXWW3z8\n8ccMGTKkRO1VqVKF+fPno1QqadWqFVu3biUzM5P3338fe3t7QkND2bdvn9lf6B06dGDGjBkAtGvX\njvj4eKKionj11Ve5evUq27Zt45133uHVV18FIDQ0FK1Wy5IlS+jbty9OTk4AODo6MnXqVFQqVb4x\nLly4EF9fX1atWoVSafh7q2XLljz33HOsXbuW8ePH8+STTwJQp04dWrRokWc7x44dQ61WM2LECOPx\n3N3dOXHiBHq9Ho1GQ1paGpMnT6Zbt24AhISEkJKSwnvvvcedO3fw8PAAIDU1lSVLllC7dm0Avv32\nW77//nu+++4747bffvuN3bt3m8RgZ2dHVFQUzs7OxuczZ87kwoUL+Pv7m9Q9fPgwhw4dYunSpTz3\n3HMAtG/fnuTkZObPn0+XLl24ePEid+7c4d///jeBgYEANGjQgK1bt5KamoqLi0u+76sQtkhGVISo\nALp3706nTp1YsmQJV69eLVFbTz31lPHLX6lUUqVKFZo0aYK9/YO/a9zd3UlOTjbZL+eLPEfnzp25\ne/fu/7dzfyFN/X8cx58njbTNreVoaUpGCYYEzTYx/0QKRcT6c7GbAkGI6pCSRIWVEoIVxjJLDcMc\n1VKsGyGIlkgU7CKJ4YXUVRhBf0RXaKxcsmS/C9l+rv3qq99v/L5L3g/YxTnbOe/PObvYa5/Pe2N4\neJiBgQFCoRClpaV8//498igrK8Pv9zM0NBQ5bt26db8MKZOTk7x8+ZKdO3dGxgmQnp6OxWLh+fPn\nc75Wq9VKIBDAZrPR1NSE1+uluLiYqqoqFEVhyZIlOJ1OduzYwejoKAMDA9y9e5cnT54AEAwGI+dK\nTU2NBJLwtsFgiNq3bNky/H5/1BjKysoiIQVg+/btAHi93pjxPnv2jISEBLZs2RJzH9+8ecO7d+/I\nzs5m+fLlqKrK2bNn6e/vx2g0cvLkSUwm05zvjRDxQmZUhFgg6uvrsdls1NbW4nK5/vZ5Zn9ohi1d\nuvQvjzMajVHbqampwExT68TEBBAbZsLGxsbmXMvv9xMKhWLqhWt++PDhL8caZjab6ejo4NatW9y8\neZOOjg6MRiOqqlJeXg6Ax+PhwoULvH79Go1GQ05OTmSMs5d+/u59W7FiRdR2eIbmxyAIMDExwfT0\n9E9niMbGxsjIyKC7u5v29nbcbjf37t0jKSmJPXv2UFdX98sQKEQ8kqAixAJhMpmoqamhtraWnp6e\nqOcURQGI6WGZnJz8bfU/f/4cte3z+YCZ8JCSkgJAV1cXSUlJMcdmZGTMuY5Wq0VRFD5+/BjznM/n\nw2AwzGfYlJSUUFJSQiAQYGBgAJfLxblz5zCbzeh0OiorK9m2bRsdHR1kZGSgKArd3d14PJ551fmZ\ncIgL+/TpE/DfwDJbSkoKKSkpMY3KYWvWrAFmlnocDgfT09MMDQ1x//59enp6yMrKiiy9CfGnkKUf\nIRYQu91OUVERly5digol4b6EkZGRyL5gMBi15PJP/fjB/ejRI0wmE6tXr8ZisQAzYWbDhg2Rx8jI\nCC0tLb/8lcuPNBoNubm5PHz4MOoaR0ZGGBwcjPRlzIXD4cButxMKhUhOTqa0tJSamhpg5ldML168\nYGpqClVVyczMjAS+8LX+juZlj8cT9T84fX19KIpCQUFBzGutVit+v5/ExMSo+zg0NER7ezuKotDf\n309BQQE+n4+EhATMZjP19fXodLqo91+IP4XMqAixwDQ0NGCz2aKWJfR6PWazmdu3b5OZmYler8fl\ncvHt2zcWL178W+q63W5WrlxJfn4+fX19PH78mMbGRhRFIScnB5vNxpkzZ3j79i3r16/n1atXNDc3\nk5ubS3p6+rxqHTt2jIMHD6KqKvv27ePr16+0trai1WqpqKiY83kKCwtxOp2cOnWK3bt3EwwG6ezs\nxGAwkJ+fz/j4OImJiTgcDioqKpiamqK3t5enT58CzCtg/cz79++pqqpi//79DA8Pc+XKFex2e1Rv\nS9jWrVvJy8tDVVWOHDlCVlYWg4ODXLt2DZvNhkajIS8vj1AoRGVlJYcOHUKj0eB2u/ny5Uuk/0WI\nP4kEFSEWmFWrVnH8+HEaGhqi9jc2NtLQ0EBdXR1arRa73c6mTZvo7e39LXVPnz7NgwcP6OzsJC0t\njYsXL7J3796o+tevX6erq4vR0VGMRiN2u52jR4/Ou1ZxcTFOp5OWlhaqq6tJTk6msLCQEydOxPR8\n/EpRURGXL1+ms7Mz0kBrsVhwuVzodDp0Oh1NTU20tbWhqip6vZ6NGzdy584dysvL8Xq9rF27dt7j\nn23Xrl0kJSVRXV2NVqvlwIEDVFZW/s/XLlq0iBs3bnD16lXa2toYHx8nLS0NVVU5fPgwMLPU5nQ6\naW5upra2lkAgQHZ2Nq2trVit1n80ViH+DUpo9tcuIYQQ/zdlZWVs3ryZ8+fP/9tDESJuSY+KEEII\nIeKWBBUhhBBCxC1Z+hFCCCFE3JIZFSGEEELELQkqQgghhIhbElSEEEIIEbckqAghhBAibklQEUII\nIUTckqAihBBCiLj1H2OQGtj0pW2KAAAAAElFTkSuQmCC\n", 619 | "text/plain": [ 620 | "" 621 | ] 622 | }, 623 | "metadata": {}, 624 | "output_type": "display_data" 625 | } 626 | ], 627 | "source": [ 628 | "plt.figure(figsize=(8,3.6))\n", 629 | "\n", 630 | "def get_res(res):\n", 631 | " filtered = np.array(res)\n", 632 | " return np.nanmean(filtered, 0), np.nanstd(filtered, 0)\n", 633 | "\n", 634 | "tmp_m, tmp_s = get_res(dcsmc_results_Z) \n", 635 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2, color=sns.color_palette()[1])\n", 636 | "\n", 637 | "tmp_m, tmp_s = get_res(nnis_results_Z)\n", 638 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2, color=sns.color_palette()[2])\n", 639 | "\n", 640 | "tmp_m, tmp_s = get_res(lwis_results_Z) \n", 641 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2, zorder=0,color=sns.color_palette()[5])\n", 642 | "\n", 643 | "\n", 644 | "plt.legend(['NN-SMC', 'NN-IS', 'IS (Prior)'], loc='lower right')\n", 645 | "\n", 646 | "\n", 647 | "plt.semilogx();\n", 648 | "\n", 649 | "plt.xlim(sizes[0]-1, sizes[-1])\n", 650 | "plt.ylim([-250, 0])\n", 651 | "\n", 652 | "plt.ylabel(\"$\\log \\hat Z$\")\n", 653 | "plt.xlabel(\"Number of samples\")\n", 654 | "plt.tight_layout()" 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": 23, 660 | "metadata": { 661 | "collapsed": false 662 | }, 663 | "outputs": [ 664 | { 665 | "data": { 666 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAADmCAYAAAD/cH5oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVNX/P/DXvXdmYFgGRUEUVAREK3dRTE0Scs9WLTUV\nszAtTftov1zaDM0sP6VJabmUWn3Tj2WfMrRyCfWTSRRqbiiQCyqCCowss917f3/M4gwzAwPOBryf\njwfNzDnn3vsGJufNuWdhRFEUQQghhBDiRVhPB0AIIYQQUh0lKIQQQgjxOpSgEEIIIcTrUIJCCCGE\nEK9DCQohhBBCvA4lKIQQQgjxOpSgEEIIIcTrUIJCCCGEEK9DCQohhBBCvA4lKIQQQgjxOpSgEEII\nIcTrSDwdgLdQqVQ4ceIEQkJCwHGcp8MhhBBCGgye51FcXIwuXbrA19fXKeekBMXgxIkTeOqppzwd\nBiGEENJgffnll4iLi3PKuShBMQgJCQGg/+GGhYV5OBpCCCGk4SgsLMRTTz1l+ix1BkpQDIy3dcLC\nwhAREeHhaAghhJCGx5lDJGiQLCGEEEK8DiUohBBCCPE6lKAQQgghxOvQGJQ6Skn/q8b6dSN7uSkS\nQgghpPGiHhRCCCGEeB3qQakj6iEhhBBCXI96UAghhBDidShBIYQQQojXoVs8hBBCPOqJrTNqrN/2\n5Bo3RUK8CfWgEEIIIcTrUA8KIYQQjzLvIXnhh0UAgI9GL/VUOMRLUA8KIYSQJi0xMRGdOnUyfXXp\n0gVJSUlYvnw5ysvLrdpnZmbioYceQs+ePfHMM8/g2rVrNs9bVFSEpUuX4oEHHkC3bt2QmJiIN954\nw257I7VajQ8//BDDhg1Dly5dEB8fjxkzZuDEiROmNgUFBaZ48/LyrM6h0WgQFxeHTp06obCw0KJu\n9+7dmDx5Mvr27YvevXtj3LhxSE9Pd+RH5VaUoBBCCPEKap0GKp0aFdoqHLqQCY1O47Zrp6Sk4NCh\nQzh06BDS09Mxd+5c/Pjjj3j22Weh0dyOQ6lU4vnnn8eQIUOwfft2lJSUYOlS696e3NxcPProozhz\n5gwWL16M9PR0LF26FGfOnMGECRNQVFRkN5aFCxdi9+7dWLRoEXbv3o0NGzZALpdj4sSJVsmIVCrF\n7t27rc5x8OBBm8lVamoqFixYgPvuuw9ffPEF/vOf/yApKQkvv/wyPv3007r8yFyuyd/iOXLkCDIz\nM6FUKj0dCiGENFm5N85j+aGPcUtTAQD48PfPEOQbiFcGPo+YFpEuv76fnx9CQkJMr9u1a4f27dvj\n8ccfxzfffIPx48cD0Pdc3Lp1C0OGDEF0dDT69++PjIwMq/O9/PLLaNeuHTZu3AipVAoAiIiIQNeu\nXTFs2DB89NFHWLx4sdVx5eXl+PHHH/Hxxx9j0KBBpuPee+89DB06FNu2bcOCBQtM7fv164effvoJ\nL7zwgsV5du3ahd69eyMrK8tUtnfvXnzxxRf49NNPkZCQYCqPiooCAKxatQoPPfQQwsLC6vzzc4Um\n34MSHx+PWbNmITk52dOhEEJIk6TRabD80McoU92yKC9T3cLyQx+7tSfF3D333IPevXtb3P6IiYlB\naGgoPvjgA+Tk5OC7777D6NGjLY47fvw4Tp06hWnTppmSE6OAgAB88sknmD59ut3rsiyLQ4cOged5\nUxnHcdi0aROmTZtm0Xb48OHIycnB+fPnTWUajQb79u3DyJEjLdpu3boV99xzj0VyYjRhwgR89tln\nCA4Otv8DcbMm34NCCCHEubYc/Qa/X6p53zJzKp3a1HNSXZnqFmb8sBC+Eh+HztWvbS9M6vG4w9eu\nTWxsrEWCIpPJ8Oqrr+LFF1/E//73P8ycOdMqaTh58iQAoHv37jbP2aVLF7vXCwgIwIQJE7Blyxb8\n9NNPGDBgAPr06YMBAwYgIiLCqn1kZCQ6deqEn3/+2RTHgQMH0KZNG8TExFjFNWzYMJvX9ff3R58+\nfezG5QlNPkGhWzyEEOJZvCjcUb0rKRQKi7Ece/bsweuvv47Y2FicPXvWdHukvLwcAQEBAGD6PFEo\nFPW65quvvopu3bph+/btSE9Px3//+18wDIOhQ4di6dKlCAwMtGg/fPhw7N6925SgpKenY8SIEVbn\nLSsrszrWmzX5BCU+Ph7x8fEoKCjA5s2bPR0OIYQ0eJN6PF6nXoxDFzLx4e+f2a1/tvc4DGzf1xmh\n1VlFRYXpQ/3EiRN48cUXMXfuXDzzzDOYO3cu5s+fj8jISEyaNAnJycl4/vnn0bx5cwD6hKBFixb1\nuu5DDz2Ehx56CJWVlfjzzz+xa9cu7NixAyzLYuXKlRZthw8fjlWrVqGgoAAtW7bE/v37MWvWLKuB\nuM2bN0dZWVm94vGEJj8Gpb7UvIDfL9/EztyrOHL5JjS85zJ8QghpyPqG90CQr+2/7IN8A9E3vIeb\nI7rt5MmTuPvuuwEAP/zwAyIjI/HMM88AAJYuXYrIyEhMnDgRZWVlSExMBAD06KGP9+jRozbPuXHj\nRrz++us2644cOYLly5ebXvv5+eG+++7D22+/jZSUFJsDcqOiohAbG4uffvoJBw4cQPv27dGhQwer\ndj179sSxY8dsXre8vBzJycnIzMy096NwO0pQ6uGf0gos3H8CG46dx3/PXsX6Y+exYP8J/FNq+x4q\nIYQQ+2QSGV4Z+LxVkmKcxSOTyDwS15kzZ5CdnW0aBCuXy6FUKqHVagEAvr6+ePfdd1FWVobw8HDT\nmI/Y2Fh069YN69atg06nszhnSUkJNm7caDEA1lx5eTk2btxoGsdiLjAw0G6PzPDhw/Hzzz/jp59+\nshoca/T444/j1KlTNpOcL774An/88QfCw8Pt/DTcz+FbPEVFRcjOzoZWq4UoigAAQRBQVVWFrKws\nrFixwmVBulJdx6BoeAFpWXlQaizfdEqNDmlZeVg2uAtkHOV9hBBSFzEtIvHRqCWY8cNC8KKAZ3uP\nQ9/wHm5LTiorK1FcXAwAUKlUOH78OFasWIE+ffrgoYceAqD/gP/ss8+waNEiPPfcc7h58ybeffdd\nxMbG4tKlS5g3bx6WL18OHx8fpKamIjk5GVOnTsWMGTMQERGBvLw8vP/++/D19cVLL71kM47Bgwej\nT58+eO655zB79mz07dsXKpUK2dnZ+OSTT7Bo0SKbxw0fPhyrV6/G2bNnMWfOHJttEhISMGbMGMye\nPRuzZs3C/fffD51Oh/T0dKxbtw7z5s1reAnKzz//jLlz50Kr1YJhGACAKIqm58ZBQg1RXcegZBeW\nWiUnRkqNDtmFpYgP955pWoQQ0lDIJDLTbB13jzlZt24d1q1bB0A/oyU8PBxPPPEEpkyZAo7jAABt\n27bFxo0b8e677+Lhhx9GUFAQRowYgTlz5uDEiRN4++23UVpailatWqFz587Ytm0b1q5diwULFuDG\njRsIDQ3F/fffjxkzZqBly5Y242BZFp9++inWr1+Pzz//HEuWLAHDMOjcuTPefvttDB061OZx0dHR\n6NixI2QyGdq2bWv3+1yyZAm6d++Obdu2Yc0a/RYDHTt2xKpVqzBkyJA7+RE6HSMau0Nq8Nhjj0Eq\nleKNN97Al19+CZ7nTffC3n//fXzyyScYMGCAO+J1mYKCAiQlJWHv3r02p3IZ7cy9iv+evWq3/uHY\n1ngwprUrQiSEkEavIe/FY/6He1Pj6GdoXTjUg5KXl4d///vfuPvuuxEfH4+NGzciOjoa0dHRuH79\nOtauXdvgExRHhchrnotfWz0hhBBLT2ydUWOZ+WaC3qypJieu4tBgCZZlERQUBABo37498vPzIQj6\nWSv33XcfcnNzXRehl+kZ1gwKme28TiGToGdYMzdHRAghhDQ+DvWgdOjQAdnZ2ejTpw+ioqKg0Whw\n5swZ3H333aioqLDYSKmxk3EsZsZFWw2UlUv05TRAlhBC6qah9JAQ93IoQXniiSeQmpqKyspKzJkz\nB/369cOiRYswduxYbNmyBffcc4+r46zR4sWL8dVXXyEnJ6fOx9ZnJdkOzfyxbHAXZBeW4nxZBfac\nL0awXIbIIL86X58QQggh1hxKUMaNGweNRoPLly8D0G/XnJKSgrfeegvh4eFYuHChS4OsSVZWFior\nK+t9fH1XkpVxLOLDgxEfHgylWofMqyU4UaxE19CgesdCCCGEED2H10GZPHmy6Xnbtm2xa9culJSU\nIDg42DQexREXLlzAhg0bkJ2djdzcXMTFxWHLli1W7XJzc5GamoqjR48iMDAQY8eOxcyZM03TvQD9\njo0rVqzAxx9/jO+++87hGJxtRHQrZF4tQXpeISUohBBCiBM4lKAkJSXho48+QufOnU1lDMMgODgY\nx48fx7Rp0/D77787dMFz584hIyMD3bt3t1phz6isrAxTpkxBTEwMPv74Y1y8eBHLly+HIAgWi9t8\n9NFHGDNmjMe3h45Q+KFbqALHi5Q4e7McscEBHo2HEEIakpT0mnc+Xjeyl5siId7EboKyc+dOUwJx\n+fJl/PLLLzhz5oxVu8OHD9dpkGxiYiIeeOABAMCLL76IkpISqzZff/011Go10tLSEBAQgAEDBqC8\nvBxpaWlISUlBQEAAzpw5g2PHjtldMc/dRkaH4XiRErvyChEbHFP7AYQQQgixy26CcvLkSXz2mX53\nSYZh8NFHH9lsxzAMpk6d6vAFWbb2WS4HDhzAwIEDTVtXA8CoUaOwYsUKZGZmIjExEX/99Rdyc3OR\nlJRkapOYmIjt27d7pEclunkAOgUH4ESxEhfKKtGeBswSQohDzHtI5u8/AQB4Z3AXT4VDvITdBOVf\n//oXpkyZAlEUcf/992PNmjWmHR2NWJZFQEAA5HK5U4PKz89Hv379LMratGkDuVyO/Px8JCYmYsKE\nCZgwYYKpvlOnTti3b59T46irkdFhyLmZi115hZjeq+Eu/08IIU3J/PnzUVhYiM8//9xUlpGRgXXr\n1uHkyZMQRREdOnTAmDFjMGHChFoXZFu0aBF69OiBsWPHYv78+dixY4epjmEY+Pr6IioqCs899xyG\nDRtm9zyTJk1Cu3btsHRp/VbVVavVGDNmDNasWeO01V3dyW6CIpVK0apVKwDA3r17ERISApnMPZs2\nKZVKBAZab72tUCjqNB3YntWrVyMtLe2Oz1PdXS0D0T7ID38VlqKwXIWwAF+nX4MQQohrHTx4EC+8\n8ALmzZuHxYsXg+M4HD58GMuWLUNJSQlmzpxp99jDhw/j+PHjSE1NNZXFxcVh5cqVptcVFRXYuHEj\nZs+ejf/7v/9Dz549bZ5r9erVkEgcnstixcfHBykpKXjttddMd0QaEodWFQsPD4dGo8Enn3yCyZMn\nY8SIETh37hw2bNiAw4cPuzpGh9RlDZRZs2YhJyfH4mvv3r13HAPDMBgZHQYRwO78a3d8PkIIaUrU\nvAC1jkellseRyzeh4R2fIepM27Ztw+DBgzFlyhRER0cjMjIS48ePx7PPPlvrchQrV67E5MmTLYYz\nSKVShISEmL4iIyPx2muvQS6XY9euXXbP1axZM4uhDvXx4IMP4ty5c17zWV0XDiUo165dwyOPPGLa\n+fD8+fPQaDQ4evQoUlJSnP6NKxQKlJeXW5UrlUooFAqnXsvZerQKQusAX/x++QZuVDWdFXYJIeRO\n/FNagYX7T6Bcy6NKx2P9sfNYsP8E/imtcHssLMvi1KlTKCoqsiifMmUKtm7dave47OxsnDx50qFd\ngY1LZhjvTEyaNAmvv/46HnvsMfTp0wf79u3DpEmTsGjRItMxWVlZmDhxInr27In+/ftjyZIlqKqq\nAqDfrK9Tp05Yu3Yt7r33XowYMQIajQYsy2LYsGEWt68aCocSlGXLlkEmk2Hv3r3YuHEjjBsgr1q1\nCvfee6/dAbT1FRUVhfz8fIuyq1evoqqqClFR3j22g2UYjIhuBV4EfqZeFEIIqZWGF6y2DwEApUaH\ntKw8t/ekJCcno6ioCImJiXj66aexZs0aZGdnIyAgAB06dLB73L59+9C9e3c0a1bznmxlZWV45513\noFKpMHToUFP5f/7zH0ybNg1btmxB3759LY45duwYpkyZgq5du2L79u1YtmwZ9u7da7H0BgD8+OOP\n+OKLL7BixQpT8pOQkIDffvsNKpWqrj8Kj3Lo5tahQ4eQmpqKFi1agOd5UznLspg4cSLmzp3r1KAG\nDRqEDRs2oLy83NS9lZ6eDl9fX6tfmjfq0zoY/z17FQcvXceomDAofKSeDokQQtzmP6cL8GdhqcPt\n1Toe5VreZp1So8Mr+/6Gj4SzWV9d77BmGHvXnQ0I7dWrF7799lts3LgRv/76K3777TcAQLt27bBs\n2TLExcXZPO7YsWOIibFeZiIzM9M0zkQQBKhUKrRu3Rqpqano1q2bqV23bt0wfPhwm+feuHEjunTp\ngldeeQUAEB0djTfffBPTpk3DuXPnTJNVnnrqKURHR1scGxsbC41Gg1OnTqFXr4azpoxDCYogCPDx\n8bFZx/O8qUfFEVVVVcjIyACgv3VUXl6O3bt3A9BneXK5HOPGjcOWLVswa9YspKSk4NKlS0hLS8OU\nKVPu+H6cO0hYBsOiWuGrk5ew53wRHusU7umQCCHEa/G1fITUVu8KHTt2xLJlyyCKInJycnDgwAFs\n3rwZKSkp2LNnD1q0aGF1zI0bN2wmAN26dcPy5csB6P+w9/f3t7kcRk0zbc6dO4eEhASLMmOidO7c\nOVOi07ZtW6tjjde6ceOG3fN7I4cSlN69e+PTTz9F//79IZXqewOM06y2bdtWp4zsxo0bmD17tkWZ\n8fXevXsRERGBoKAgfP7553jrrbcwffp0KBQKJCcnY9asWQ5fx9MGRLTAznNX8euFYgyPagU/af1H\nYhNCSEMy9q6IOvViHLl8E+uPnbdb/9Q9bREf7p71rSoqKvD+++/jySefRGxsLBiGQefOndG5c2cM\nGTIEw4cPxx9//GGzp4NhGJtbv/j6+qJ9+/a1XtvX1/7MT1t1xs4B85k+tjoTjHc+apse7W0c+tSc\nN28eJkyYgKFDh6Jfv35gGAabNm1Cbm4u8vLy8OWXXzp8wYiICIdm3MTExNRp8z5vI+NYDOkQim9y\nrmD/Bf2tHkIIIdZ6hjWD4rTEagwKAChkEvQMq3lMhzPJ5XLs3LkTDMPg1VdftYzFMEmjZcuWNo8N\nCQnBzZs3XRJXdHQ0srOzLcr+/PNPU11NjDGFhoa6JDZXcWiQbKdOnfDNN9+gT58++O2338BxHDIy\nMhAeHo6vv/4a99xzj6vjbJAS2oXAT8Jh7/kiqD00XY4QQrydjGMxMy4aCpnl38wKmQQz46Ih4xz6\nqHIKlmUxd+5cfPHFF1iyZAlOnDiBS5cuISMjAzNnzkR8fLzdMSjdunXDqVOnXBJXSkoK/v77byxf\nvhz5+fk4ePAgFi9ejISEhFoTlNOnT0MulyM2NtYlsbmKw/cdIiMj8e9//9uVsTQ6cimHxMgQ7Mwt\nxKFL15EU2bCyV0IIcZcOzfyxbHAXvLLvb/Ci/rZOz7Bmbk1OjJ544gm0bNkSmzZtwjPPPIOKigq0\natUKo0aNwvTp0+0el5iYiPXr16OsrAxBQc7d2T42NhZr167FypUrsWXLFjRr1gyjRo1yaD+633//\nHQMGDKjxFpI3YkQHR7iKooj8/HzcunXL5j22hjQy2JaCggIkJSWZxsE4yy2NDvP3n0CAlMPS+++B\nxIG9iAghpKlq6HvxjB07Fg8//DAmTpzo6VAAAFqtFoMGDcIHH3xgtYWMM7niM9ShHpSTJ0/i+eef\nNy1aY8xpGIaBKIpgGAanT592SkCNTaBMgkFtW2LP+SIcuVyCAW2tR34TQkhTlpL+V41l5psJervZ\ns2dj6dKlGD9+vGkxNk/auXMnYmJiXJqcuIpDCUpqaio4jkNqaioiIiIc2pGY3DakQyj2XyjGrvxC\n3BsRDLaBjaQmhBDimIEDB6JHjx7Yvn07nnzySY/GolarsX79eqxdu9ajcdSXQwnK6dOn8d5771ms\neEccFyyXoX9EMA5euoG/CksR17q5p0MihBCv0ZB6SByxbNkyT4cAQD/l+Mcff/R0GPXmUFdIcHCw\naf0TUj/DolqBAZCeV1inhe0IIYSQpsihBGX8+PH45JNPbG7gRxzTyt8Xca2b45KyCieKlZ4OhxBC\nCPFqdm/xTJ061fRcFEUcP34cgwYNQmxsLPz8/CzaMgyDDRs2uC7KRmJkdCv8cbUE6XmF6Brq3Clo\nhBBCSGNiN0HRarUWr3v37m23jjgmQuGHbqEKHC9S4uzNcsQGe/++QoQQQogn2E1QtmzZ4s44moyR\n0WE4XqTErrxCxAZb73pJCCFNzf8efrzG+gH//cZNkRBv4tAYlMmTJyMvL89m3ZkzZ/Dwww87NajG\nLLp5ADoFB+BEsRIXyyo9HQ4hhBDilez2oGRlZZlmm2RmZuKPP/6wuQnS/v37ceHCBddF2AiNjA5D\nzs1cpOcVYnqvKE+HQwghHmXeQ5KVol9KPm5dw1y7gziP3QTlm2++wY4dO8AwDBiGweLFi63aGBOY\n0aNHuy7CRuiuloFor/DDX4WlKCxXISygYe2PQAghjcn8+fNRWFiIzz//3FSWkZGBdevW4eTJkxBF\nER06dMCYMWMwYcIEMLUstrlo0SL06NEDY8eOxfz587Fz505888036NSpk0W71atX4/vvv8cvv/wC\nAJg0aRJycnKwc+dOq52Hq8c4fvx4LFiwAN26dbvzH4CXspugLFq0CGPGjIEoipg4cSLeeustqx0T\nOY5DYGAgoqKoF6AuGIbByJhWWPPXP9idfw1TurX3dEiEEOJxvFoNXqWCyPMozjiI4H59wfn4uD2O\ngwcP4oUXXsC8efOwePFicByHw4cPY9myZSgpKcHMmTPtHnv48GEcP34cqamppjKtVouFCxdi27Zt\ntS5/X1ZWhsWLF+Ojjz6qsd28efOwYMEC7NixAzKZrG7fYANhN0EJCAgwzdzZvHkz7rnnHvj7+7st\nMHc5cuQIMjMzoVS6d22SHq2aIczfB79fvoHRHVujhbxxvsEIIcQRt87l4vSSt6FT3gIAnH1/JaTN\ngnDXqwsR2NG9Ewq2bduGwYMHY8qUKaayyMhIFBcXY/PmzTUmKCtXrsTkyZMttoRp3bo1zpw5gw0b\nNmDatGk1XjsiIgJ79uzBjz/+iFGjRtlt17t3b/j7++P777/HmDFjHP/mGhCHBsn27du3USYnABAf\nH49Zs2YhOTnZrddlGQYjosPAi8DP+dfcem1CCPEmvFqN00vehra0zKJcW1qG00veBq9WuzUelmVx\n6tQp0wa5RlOmTMHWrVvtHpednY2TJ09iyJAhFuWRkZF49tlnkZaWhvz8/Bqv3a9fPzz44INITU21\nOe7T3PDhwy1uSzU2Du3FQ1yjb5tgfH/uKg5duo5RMWFQ+NB2AoSQhu+fzzbhxm+HHW7Pq1SmnpPq\ntKVlyHr2OXC+jo3Va9H/XnR4+s7+4ExOTkZycjISExPRp08f9O3bF/369UP37t2hUCjsHrdv3z50\n794dzZo1s6p74YUXsGfPHixcuBBfffVVjZvuvvrqqxg1ahRSU1PxwQcf2G2XkJCA5cuX49KlS2jb\ntm3dvskGoMlvS3zkyBGsXr0amzZtcvu1JSyDYVGtoBFE7D1fVPsBhBDSCIk8f0f1ztarVy98++23\nGD16NM6cOYOVK1di3LhxGDZsGLKysuwed+zYMcTE2L4dJZPJ8Pbbb+PYsWPYvHlzjddv3rw5Xnvt\nNaSnp2PPnj1220VGRkIqleLo0aOOfWMNTJPvQYmPj0d8fDwKCgpqfdO4woCIFth57ir2XyjGsKgw\n+ElrHkBFCCHersPTyXXqxSjOOIiz76+0Wx/93DSEJNznjNAc1rFjRyxbtgyiKCInJwcHDhzA5s2b\nkZKSgj179qBFixZWx9y4cQO9etnfmbl79+5ITk7GypUrkZSUVOP1R4wYgfT0dLz55pvo06ePzTYc\nx6FZs2a4ceNG3b65BqLJ96B4moxjMaRDKKp0An69UOzpcAghxO2C+/WFtJnt/cmkzYIQ3K+v22Kp\nqKhAamoqzp49C0A/67Jz586YNm0atmzZgsrKSvzxxx82j2UYBoIg1Hj+OXPmIDQ0FIsWLap1Z/s3\n3ngDWq0Wy5Yts9uG5/lapz03VA71oIiiiG+//Ra//vorKisrrX6otFngnUloF4Jdedew53wRkjqE\nwoejvJEQ0nRwPj6469WFVgNljbN43DnVWC6XY+fOnWAYBq+++qpFnXH8ScuWLW0eGxISUuvAVl9f\nXyxduhSTJk3ClStXakwuWrZsiQULFuCVV15B27ZtERERYVEvCALKysqs1kxpLBxKUN5//32sW7cO\nERERCAsLa1TZmqemGZuTSzkMjgzBj7mFOHTpOpIiG+ebjRBC7AnsGIPen65B1rPPQeR5RD83zSPr\noLAsi7lz5+L1118HADzyyCMICgpCfn4+1q5di/j4eMTFxdk8tlu3bjh48GCt1+jTpw/Gjx+Pr776\nCu3ataux7SOPPIJdu3bh119/tUpQcnJywPN8o12szaEEZceOHXj66afxyiuvuDoet/P0GBSjpMhQ\n/PJPEX7Ov4aEdi0hqWGENyGENEacj49pto67x5yYe+KJJ9CyZUts2rQJzzzzDCoqKtCqVSuMGjUK\n06dPt3tcYmIi1q9fj7KyMgQF2b5lZTRv3jxkZGQ4FM9bb71lc02UI0eO4K677kJ4eLhD52loHEpQ\nysvLMXjwYFfH0qQFyiQY1LYl9pwvwpHLJRjQ1noAFiGEEOd75513rMoSExORmJhYp/N0794dd999\nN3744QdMnDjR7rkBwN/fH/v27bMo27Jli822rVq1sjl76LvvvsOkSZPqFGND4lCC0rNnT/z111/o\n29d9A5WaoiEdQrH/QjF25Rfi3ohgsI3oVhohhNjzv4cfr7HMfDNBbzd79mwsXboU48ePr3VZ+zuR\nmZmJqqoqPPzwwy67hqc5lKBMnz4dc+fOhU6nQ69eveBrY8GcmqZWEccEy2XoHxGMg5duILuwFL1b\nN/d0SISzYYOsAAAgAElEQVQQQupg4MCB6NGjB7Zv344nn3zSZdd5//338c4770AiabyrhTBibfOc\nAHTu3NnyILO/7EVRBMMwOH36tPOjc6OCggIkJSVh7969VgOR3OlahQqvZZxCW4Ucrw7o3KgGJBNC\nCGmcXPEZ6lDq5cnBozW5evUq5s+fj6KiIrAsi4SEBLz88st1+lD3hlk85lr5+yKudXP8cbUEJ68r\n0SWk5oFWhBBCSGPkUILirWNPOI7DvHnz0LVrV2g0GkydOhU///wzhg0b5vA5vGUWj7mR0a3wx9US\npOdeowSFEEJIk2Q3QVm7di0ee+wxhIaGYu3atTWehGEYPPfccw5f9MKFC9iwYQOys7ORm5uLuLg4\nm6OXc3NzkZqaiqNHjyIwMBBjx47FzJkzTQOPQkNDTQvUyGQydOrUCVevXnU4Dm8VofBD1xAF/i5W\n4uzNcsQGB3g6JEIIIcSt7CYoK1euRP/+/REaGoqVK+3vkQDUPUE5d+4cMjIy0L17d+h0OpttysrK\nMGXKFMTExODjjz/GxYsXsXz5cgiCgJdeesmqfUlJCfbs2YONGzc6HIc3GxkThr+LldiVV4jYYNub\nTxFCCCGNld0E5cyZMzafO0NiYiIeeOABAMCLL76IkpISqzZff/011Go10tLSEBAQgAEDBqC8vBxp\naWlISUlBQMDtXgWNRoMXX3wRycnJiI6OdmqsnhLTPACxwQE4UazExbJKtAvy83RIhBBCiNt4ZLlS\n1oFVUg8cOICBAwdaJCKjRo2CSqVCZmamqYznecybNw933303pk6d6pJ4PWVkdBgAID2v0MOREEII\nIe7ltROo8/Pz0a9fP4uyNm3aQC6XIz8/37TC3+uvvw5/f3/Mnz/f4XOvXr0aaWlpTo3XFe5uGYj2\nCj/8VViKwnIVwgKs158hhBBCGiOv3fBFqVQiMDDQqlyhUJimBP/555/Yvn07Tpw4gUceeQQPP/yw\nQzNxZs2ahZycHIuvvXv3Ov17uFMMw2BkTCuIAHbnX/N0OIQQQojbeG0PiiN69+6NnJwcT4fhUj1a\nNUOYvw9+v3wDozu2Rgu5zNMhEUIIIS7nUA9KVlYWtFqtq2OxoFAoUF5eblWuVCqhUCjcGosnsQyD\nEdFh4EXgl3+oF4UQQkjT4FCCMmPGDPz444+ujsVCVFQU8vPzLcquXr2KqqoqREVFuTUWT+vbJhgt\n5DIcvHgdSrV7E0VCCCHEExxKUAICAixm07jDoEGDcOjQIYtelPT0dPj6+nrtyrauImEZDOvQChpB\nxN7zRZ4OhxBCCHE5h8agzJgxA0uWLMH58+fRuXNn+PlZr8lRl92Mq6qqkJGRAQC4du0aysvLsXv3\nbgBAQkIC5HI5xo0bhy1btmDWrFlISUnBpUuXkJaWhilTprg9WfIGA9q2wA+5V7H/QjGGRYXBT+q6\nbbwJIYQQT/PIbsbGXQ9tMd8JMTc3F2+99RaOHj0KhUKBMWPGYNasWaal7p3JW3YzrsmuvEJ8m3MF\nj8a2wciYME+HQwghhABoRLsZR0REODT7JiYmxms28PMG97cLwa68a9hzvghJHULhw3ntLHFCCCHk\njjTo3YybGrmUQ2JkCH7MLcT/Ll1HYmSop0MihBBCXMLhP8Hz8vIwZ84c9O/fH127dsWgQYPwr3/9\nC7m5ua6Mj1STFBkKGcfip/xr0AmCp8MhhBBCXMKhHpScnByMHz8ecrkcSUlJaNGiBYqLi7F//37s\n378fX3/9NTp16uTqWAmAQJkEg9q2xJ7zRThypQQDIlp4OiRCCCHE6RxKUFasWIGoqChs3rzZYgZP\nZWUlpkyZgpUrV2LNmjUuC5JYGtIhFPsvFGN3XiHuDQ8GazZomRBCCGkMHF5Jdvr06VbTi/38/PDs\ns88iKyvLJcER24LlMtwbHozCCjWyC0s9HQ4hhBDidA4lKHK53G4dwzDged5pARHHDI9uBQZAel4h\nHJgpTgghhDQoDiUoPXr0wLp166BWqy3KVSoV1q9fj549e7okOGJfK39fxLVujovKKpy8rvR0OIQQ\nQohTOTQGZe7cuRgzZgySkpKQmJiIli1b4vr169i3bx8qKirw5ZdfujpOYsOI6Fb442oJ0nOvoUtI\nkKfDIYQQQpzGoQQlOjoaW7duRVpaGvbu3YuysjIoFAr06dMHL7zwAmJjY10dJ7GhrcIPXUMU+LtY\niXM3y9ExuOltAUAIIaRxcihBWb9+PZKSkvDhhx+6Oh5SRyNjwvB3sRK78grRMTjG0+EQQgghTuHQ\nGJTVq1fjwoULro6F1ENM8wDEBgfg72IlLiorPR0OIYQQ4hQOJSjR0dG4ePGiq2Mh9TQyWr9x4K68\nQg9HQgghhDiHQ7d4HnjgAfz73//GoUOH0LlzZ6v1UBiGwXPPPeeSAEnt7m4ZiHYKOf68WorCjiqE\nBfh6OiRCCCHkjjiUoBjHnhw4cAAHDhywqqcExbMYhsHI6DCszf4Hu/Ov4X8FN2psv25kLzdFRggh\nhNSPQwnKmTNnXB0HuUM9w5ohzN8Hv1+uOTkhhBBCGgKHEpRHH30Uc+bMQUJCgqvjcbsjR44gMzMT\nSmXDXuyMZRiMiA7DZ8cvICkyBOPubuvpkAghhJB6c2iQ7IULF+Dr2zjHNcTHx2PWrFlITk72dCh3\nrG+bYAT7ynDw4nXcUms9HQ4hhBBSbw4lKA8++CA+//xz3LhBtw+8mYRlMDyqFTSCiD3niz0dDiGE\nEFJvDt3iuXz5Mo4cOYKBAweiRYsW8Pf3t2rz008/OT04d2gst3iMBrRtgR9yr2L/hWIMi2oFPynn\n6ZAIIYSQOnMoQQkNDcXo0aNdHYtHxMfHIz4+HgUFBdi8ebOnw7ljMo7FkA6h+DbnCjIuFmOEYY0U\nQgghpCFxKEFZtmyZq+MgTnR/uxDsyruGX/4pQmJkKHw4h+7kEUIIIV7DoQTFqLCwEL///juKiorw\n6KOPori4GDExMZDJZK6Kj9SDXMphcPsQpOcV4ssTFxHq74MQuQ96hjWDjJIVl0tJ/6vGelqHhhBC\naudwgrJ8+XJs2bIFOp0ODMNgwIABeP/993Ht2jVs2rQJLVq0cGWcLtPYxqAYxQb7Iz0POHz5pqlM\ncVqCmXHR6NDMegwRIYQQ4k0YURTF2hp9+umn+PDDD/H//t//w+DBgzFkyBB888030Gg0ePHFF5GQ\nkIAlS5a4I16XKSgoQFJSEvbu3YuIiAhPh3NHNLyABftPQKnRWdUpZBIsG9yFelIIIYQ4jSs+Qx3q\nQdm6dStmzZqFyZMng+d5U3nPnj0xZ84crFq1yinBEOfILiy1mZwAgFKjw2sZJ9HcVwYZx8JHwuof\nORY+HHf7uaH8dh0LGceZ6ny42/Usw7j5OySEENLYOZSgFBUVoWvXrjbrwsPDUVpa6tSgyJ0prlLX\nWK/U6FCq1kKote/MMTKWMSQrXLWEx+y55HbyU/1RfwxneYwhCeIYBkwDTYDUvIDswlJcr1LTGCBC\nCKkjhxKUdu3a4eDBg+jfv79VXVZWFtq2pWXVvUmI3KfG+ild26Nvm+bQCSI0vAA1L5ge9c95qHW2\nyg2vdbzNOrVOwC21FmpegNZJ2Q/LwKL3RlatB8e856d6749Fz4/EOjlyZe/PP6UVSMvKs+jJojFA\nhBDiOIcSlOTkZLzxxhvQ6XRITEwEwzC4dOkS/vzzT2zYsAHz5s1zdZykDnqGNYPitMTuGJSeYc3A\nMAykHAMpx8IVH5eCKN5OaAzJiyn5qZbUWCZJvM06taHulkafADm190fCVUt4zBMhzsatLvMkybLn\nR8axYBhgdVYeblX7+Ss1OqRl5dEYIEIIcYBDCcoTTzyBkpISrFmzBl988QVEUcScOXMglUoxdepU\nPPXUU66O06azZ8/ilVdeQUVFBaKiorBixQoEBATU6RyNcRaPjGMxMy7a+i94mf4veHd8OLIMA18J\nB1+J81eyFUXRovfHIrmxSnjs9wppTIkTDw0v4pZai+tO7P2xRanR4beCG7i/fYjLrkEIIY2BQ7N4\njMrLy5GdnY3S0lIEBgaie/fuaN68uSvjq9H48eMxffp0JCQk4N1334VMJsOcOXPqda7GNIvHSGMY\nA1FMYyDqxNj7ozb14lTr9bFIbm7XG8suKitRWFHzOKAgHykiAuWIUMjR1vDYyt8XErZhjrchjQ+t\n50PqwmOzeIwCAgJw33333dEFL1y4gA0bNiA7Oxu5ubmIi4vDli1brNrl5uYiNTUVR48eRWBgIMaO\nHYuZM2eC4/R/kV+/fh0FBQVISEgAAIwZMwYzZ86sd4LSGMk4FvHhwZ4Oo8Gx6P2peTiPTUcu38T6\nY+ft1rdVyFGh4XHyuhInr9/uuZOwDNoE+CIiUI62Cj+EB8rRViFHgKxO/5sSQkij4PZ/+c6dO4eM\njAx0794dOp3tqbBlZWWYMmUKYmJi8PHHH+PixYtYvnw5BEHASy+9BEC/qm1Y2O19Ztq0aYOrV6+6\n5XsgpCa1jQGaf28nyDgWFVodCpRVKLhVhQJlFS7dqsKVW1W4qKwCzBbYa+Yr1feyGHtcFHKE+vmC\no94W4kLmPSTz958AALwzuIunwiFNkNsTlMTERDzwwAMAgBdffBElJSVWbb7++muo1WqkpaUhICAA\nAwYMQHl5OdLS0pCSkoKAgADU4c4UIW7l6Bggf6kEnVoEolOLQFMbXhBRVKkyJSzGx7+Llfi7+HZv\ni5Rl0CbgdsJiTF78pY79L03d94QQb+f2BIVlax8DceDAAQwcONBiwOuoUaOwYsUKZGZmIjExEWFh\nYSgsLDTVX7lyxaJHhRBP6tDMH8sGd6nzGCCOZdA6QI7WAXL0MSsv1+hu97QoK1FwqwqXy6twQVlp\ncXywrxQRpoTFD20D5Qj196HF9LxMQ0oQ1YalBXhRf/uSxrIRd/HKm9v5+fno16+fRVmbNm0gl8uR\nn5+PxMREhISEIDw8HBkZGUhISMD27dsxdOhQh86/evVqpKWluSJ0QkycOQYoQCZB5xaB6Fytt6Ww\nQnX7NtGtKlxSVuF4kRLHi273tshYBm0CLXta3kvsgtRDZ+zehtLwAn0IEdN6PuVa/Qri64+dp/V8\niNvUmKBotVocOHAAly5dQkxMDAYOHGjV5tq1a9ixYwemT5/utKCUSiUCAwOtyhUKhcV04DfffBPz\n58/H0qVL0aFDB6xYscKh88+aNQuzZs2yKDOOQG4M/vfw4zXWD/jvN26KhLgSxzIID5QjPFCOeLPy\nW2qtKVkxT1zOl1XaPZc5pUaH785ewd0tA8ExDDiW0T+aPZewtss5lgXHoMGu/usu60b2sr2Yn+E2\noCeIouXU/XK1Fquy8lCh5S3aKTU6rMzMxXO9OkAu4SBhGcMXa3pvSFjWVE69d/XTkHrZXMVugnLz\n5k08/fTTyMnJAaD/B+eee+7BypUrLaYQFRYWYtWqVU5NUBzVuXNnfPfdd2695hNbZ9RYv+3JNW6K\nhBDbAn2kuMtHirtaKkxlOkFEYbkKBbf0t4f+KixFcaXG7jl++acIv/xTVO8YOAbVEhcGHMMaHmFR\nLjGVM+BYVEuE2GrngVkiZEiUrJKkuiZVts/DujDR0vCCVXICOLaYnyiK0AiiaUVn41o+xmnvap6H\nyrAekMpwe8b8udpsCr3arEzDC3B0ZF+ljscHmbkOtWUZQGL8XbKWCQzHMJCyxt/B7aRGX27e3ljO\nmrW3dR4b9YbfqbHedD5KqO+YeRKlulns9PPbTVDee+893Lp1C9u2bUNUVBR++eUXvPvuu5gwYQI2\nb96MyMhIpwdjpFAoUF5eblWuVCqhUChsHEHMUQ8JqU7CMvqxKQo5AKBtoF+NU6HjWzdHuEIOXhDB\niyJ4QYTO8MhXe7RZbqyrVq4VBKgE63beOuTdbgJUPcGppd54DmOSdK1CVeOGnu8dPgs/GadPLMzW\n2zEmHnf689JvIaHfAkIu5dDMVwpfs/2yfCUcrtyqwj819LrFBvsjMshf/3vl9b9LnSBAZ/Z71wqC\n6X2g/9LXawUBKp1le0+9BxighoRGn0BLOMas3CxxYqolVmbnsOhZsmhvllgZz8dYJmISlsW/k7ri\nzYOnrVakBprObVi7Ccrhw4cxZ84cdOvWDQDw6KOPonfv3pg8eTKmTp2KrVu3IiTENathRkVFIT8/\n36Ls6tWrqKqqQlRUlEuu6SjzHpIXflgEAPho9FJPhUNIvdQ2FXpyt/Zu/cdPqJ7Q2Eh6qic7tZU7\nWm+ZbAngBdg5jwBe1I/9UQsCeJ2hThAMMTvv53HeMPjZuBeVD6dflyfIR2pKLIxbLPhIbu8y7mvc\nkVzCwtewFYOv5PYeVT6GrRkkbO2bcNa2ns+gtiFOXWfpdrIrmCU0hiTG8HvQCtXqzZIivlp7U6Jk\nlhjZbm/rGAGV2tvH8IbfvbdQanR4/YB+V3opq99iQ8oxpucyjjU8Z8zqDXWsfosTGWssY8zas6YE\nzBEfDeuBBftP2E2275TdBEWpVCI0NNSirF27dtiwYQMmTJiAlJQUfPnlly4JatCgQdiwYQPKy8tN\nM3nS09Ph6+uLvn37uuSadaXWaaDSqcGLAg5dyETf8B6QSWSeDssCr1bjxuEjUBcVwbdVKwT36wvO\npx4rj5FGxxu2QzDHMgxYjoHU+TsjuI1oSFJMSY0pcame7ADHrpXih9xCu+ea2KUt+oe3cCiRcBVH\n9vRyJo5lwIEBvLRXwDyJtpXQ2EuatMLt5Ld64sWb2lfrfRJEXL5VhWuV9lekLlXpcLNK65KeJ45B\ntSRGf+vMPJGRcSxKVRqXJSdADQmK8bZO9R2Mo6Oj8eGHH+LZZ5/F888/X+exJ1VVVcjIyACgH2Bb\nXl6O3bt3AwASEhIgl8sxbtw4bNmyBbNmzUJKSgouXbqEtLQ0TJkypc577Tjb/x5+HIXBEvyQEIRK\nuf5f0w9//wx+VTxGZ5Th8c+2ejQ+QD9OptUNLUZnlMJfdfvtW+HL4IeEZlj9/HoPRke8hXEq9Cv7\n/gYvAk/d05amkN4BhmEgYQAJGIADDP+xqXWALzIuXrf74X9veAtIPfx78LYk1tMsk2jXZ9K19WA9\n3U2/Kz0vitDw+gHOWkF/C1DLC9AIov6RF6AR9GVaXoTGos3tMsu2lue7pdaazufOjiS7CcozzzyD\nl156CVevXsUTTzyBxMREU118fDyWL1+Ol19+uc5Ly9+4cQOzZ8+2KDO+Nq7hHxQUhM8//xxvvfUW\npk+fDoVCgeTkZKuZN56g44AfEoKgkbLo9I8KigoeygAOuRE++D4hCA9q1fCReraXgtOJVskJAPir\n9OX8M2rqSSEAYLgdoP/HlrZFcJ+G8uFPSaznOLorvX5cC+Dnhu5H40atxsTljysl2HbmssuuZzdB\nGTFiBERRxKeffoqsrCyLBAUARo4cicDAQCxatKhOF4yIiDDNDKpJTEwMNm/eXKdzuwP3wSsI/PFT\njM64abN3InnHvxAo84efVA4/mRz+Uj+z53L4ywyvTc999W1k+jK5xNeiS1fkeQgaDXi1BoJGDUGt\ngaDRQFCrDeXGMv0jr1bj9UvRuKmyPaLaXyXin/WfoVnP7pD4+YHz8wPnJwcn94PE3w+sjw+NaG8C\nbE1hNC9rClMYPa2hfPhTEusZL/x01G6dUqPzyPuEYRj9WBeOhZ8USGgfgt3519w/BgXQJyEjR460\nW3/fffdhz549OH78uNMD81bXSgrt9k48vL8UhxNaQ8bqwKuLwavVgFYHDQ8IOhEqXkSZToSEFyHl\nRUh4QGJ4LTGVAxIekPIiOJ0ITnB+h9q1n3/BtZ9/sV3JsuDkckj8/cDJ5eD8/CDxkxsSGX2ZeWJz\n+7lZnb8fWJmMEh1CakEf/qQhM+8JVLng/He8kuyxY8ewZ88exMXFOSMer9cq7yYYle2kQa4RkfjL\nlXqdV5Cw4CUseI6FTgZUcAw0rAgNJ0LHMbe/JICOY6CVWJfpnzNoc51H3EnradpGhd3DEdg+EjKN\nAKmWB6fmwam1YNRaMCoNoFJDqFRBff0G+KoCQBDq/g2xrCF5MSY5hoTH30/fW2OW9EgMPTicv3kC\npK9viIlOQ1grZ8ontcw8G0lT1QnxpIbQi1nbYnJ36o4TlFOnTmHz5s1YsGCBM+LxehFaX9R0xy2w\n6z1oERcH1kcGViYD5+MDViYDW+2R8/HRt/HxASuVgrGzR5EgCKjUVaFSU4UKbRUqtVWo0FSi0ux5\nhVZfX6mtQoW2EmduXsFdeRVWvTyA/lbU9k4a8JJztXynLDhWATkXgkDGB4GCFAE8B3+Bg5xn4ccz\n8NUx8NECMq0AqUaARMuDU+vAqrVgVFowKjWEKhV0RcWorKoC6rHBI8NxFregjL06Fj07ph6fagmR\n2XNGKnVbojP7q1oWOHvSLWEQQkiD5pV78Xgz/9bhNda3HjIEIQn3Oe16LMsiQOaPAJnj+14cupCJ\nrdc+tTuL56GuI9CxRSQqtSqodCpUadWo0lXpH7UqVOn05ZVaFVRaFSp1KlzW3kIV1BAYQf+ukQBw\naJytBEAAWARCYUh0AgUJ/AUJ/HkWfjwHXx3gowN8tCJkGhESLQ+JWgdWo092oNJAV6WCtvAWBJWq\nfomORGJIbuSQ+PkbkhfrxMc8sdEnPv632/nJwcocn0ouQr8IlL3XnrRqQmiN9QPcFAchpOEy7+Up\nKChAUqpzz08JSh0F9+sLabMgaEvLrOqkzYIQ3M/z67T0De+BTeHB+OwhCWIK1FCU355pFBCgwON3\nj6jXmi36vTq0+gRGqzIlOLYTnSpU6dSo0qlQZUh0qnT68hKdGlXaMvBiXW4dyfRfYgB8eEAhyPS9\nOqIE/joWfoIEch7w1TLw0YmQaUVINQKkGh6cRgdGrYWg0kJQqaEpU0JU1e+OKSOR1Doup+34J1Gw\n4ztAZbmGAQOACfCHprQUUoXCbq+ZOxhvM/FqNX6dNBGMCHSd+SKtleMmNEiZkNpRglJHnI8P7np1\nIU4vedsiSZE2C8Jdry70in/cZRIZXhn4PJYf+hg5kbdM5UG+gXhl4PP1XlCOYRj4SGTwkcgA3zvb\nckAURWgFnamHxjzhMSY1+oRHZerVqTIlQvqvcp0KxVoVqnSV0AmOjiL30X+JgZBpRfhogUBRggBB\nigBeAn+BhZxnIdcBPjpG36ujFSDRCJBodODUOogqDXhVJVBSClFtfyElm993eQX+SH4GAMD6+Oh7\nbuS+4HwNj3I5OLkcrK/v7Tq5ZR3na+u5LxiubtMMb53LxanUt+Gr1ieKZ99fCUlQEO5+bSECO8bU\n6Vyk8THfdHS0sWzl7XraUoO4GiUodWRvp2BtaRmOz3vFa/6njWkRiY9GLcGMHxaCFwU823ucV612\nyzAMZJwUMk4KBax3rq4rLa9FlU5t6KmxTGqqzHpv9I8qUy+QsU2ZTo1CbRWqdCpoeW0tV+MAyAHI\nwQgipDrRkMiICBAkuPtsBWLzK+weXdlcDi5IAVbDg9VowVbcAm7eBKO2v3mfI1iZzH6SUy0BYiQS\nXPi/ryFWWfYk6crK8Pebb6Hnu+9AEuAPRirVj5GSSBrcYGVvVnZrHQD9LL7oS2qL9ZR4CQPA8wOp\nCfE0uwnK1KlTHTrBlSv1m7VCXE8mkcFXou/RGdje87eeXEnKSSHlpFD43PlKwzqBt53oGG9jGRKd\n6uN0jMdcrrxQY4Jy4C4pciJFACxMPToAIOqTHanxFpXhUaYTITW89tUBvgKrH6CsA3x4QKYDZFoR\nEq1+oLKkogxcyQ2wGh5MPcbriOUV+Ot560URjckKK5Uankv0s6wkt5+zUon+tcy8nZ3nMilYidlz\nQyKkP4/+nIy0+nNJo0iWZn9VhOsKDn5qAX7q27+jSh8GlT6sVwykXjUhtMYVqWmckns0hC1LeLUa\nN37PdPp57SYoWm1tf0XqhYSEuGzTQG9k3kNCmwU2ThKWQ4CPPwJ8HB+YbG7j9+NR4cvYnUXVY9ij\nGN/mLmh4DTS8FmqdBmpeY/u1TmtWp8EtnRY3eEO9RZ2N/19F/Vo7+mRHsEh87spXodNF+7eniptx\nKA2UgBNEcLwITgA4XoRE0IDjNZCoRHAVuF3Pi2DduAa2KOEACQdIJIBU/8hIJIBUAsaYxEilYKQS\nq+SIlcnAGZ5zMh/DowyczAcSmQycVD/LTiLVz8SznUjpnzMcV69kScfCKjkBYHgtQFVRDl9/z27r\nQStSe5axt97eYHtv6K03j/GG5s56gG2xm6Bs2bLF6RcjpCmIvqLBD/cFYejvSqu/PH/up8DbXYY7\n/VabIArQ8jpoTMmLIdnhNVAbnmsMz9W8Bpf27gEunrZ7vn96h4Pv1Rm8yIMXePCiAEEU9M8Nr3lR\ngCDw0Ik8BEGAwPMQdTpApwN0PBidAOh0YAzPOUGfMN1OamAou50AWSZDhjLB0Nbw/PY5BHCCGpxG\nBKeC/jyG492VLIkABI4Bz7EQOAaChIXIsRA4FqKEhSDhIHIsIOEgSjiIEhbgJEC4Ai0uKW2e008t\nYu/i+Qjo0AEcy4JjObAMC44xPufAsRwkDAuW5fRtGBYsU0OyZKe8puTqzSsdcb2GFanz1nyKwNiO\nAMOA4Vj9uVgWDMPqH1lGPxCcYfWPFmWMWdnt16Yy47mszmF+PFf7tc3P1QB73XgG4Kq9lxlDOa/2\njgTRVozOQmNQGiFbC4WZl3nDQmGNWcK33yD8xnmsaJeGkNwbpllUxTEtMO/+mS4ZB8QyrGkAsyMj\neg5Bior9Z+z28vQZMQYDOzqvE18URX2CY0hqeGOyIwqWSZBg+Zo3e208Xl/Hgxf0SZNO4CGIPHSC\nALWhnNdpIei04DVaiFoNRK0OgkYDUaeDoNUCWh0EnQ7Q6hMqUcsbEit9QgUdD2h5MLz+OavjwfAC\nGJ0AVieA4QWwpi8RLC/oe5F4HpxWp18J2pBM1edjsVnOVSDnKgBAMHy5bs/Y+ine/yuK9//q6TAc\nZ4SKKBsAABVjSURBVJ6oVEuCbic3lomNKTnibCdRt5Mmzux8dUm0zBK0aonW+dZSRF61fSeDE4FT\ny5YjoG1biKKoX3pBBAARoiAC0JeZ14mioG9jXl6tvWPtAIgCRFHf0xpSyrvsV0YJCiEuENMiEh8+\n/DZm/LAQl7xwkHLfDn3w+pA2SPzlilUvz74hbfBWhz5OvR7DMOAYDhw4gJM69dzeyJiQ6XgdeJ0W\nWo0aOsMXr9bg5C/fwy/9d7vH3xjYGc26dYNO0EEr6KATeOgEHXiBh5bXgTeWiTx0vM5UZ2ynE/Tl\nWpEHb2wvVvsgsfNXrzGhirimQdzpKrsxZsfKcTVECkaE4UsEA0Pvlah/ZETRrN7BNgAYQd/OWMYa\nPjxZUz3AoPq5b5+LNZ3LXhvBUM9bnpsHGJ1ZTLB9bhhisozRobeGwyJrqVdmH4My+5hzL1pHrh7c\nQQlKI0Q9JN7BmwcpyyQyPPvov7Ciuft6eZoSU0LGcoDUB5BbjicJnjgDh/b/AXmV9V+fVXIOQ59f\n4PQxKMbeJi2vhZbXQiPobj/ntdAaXmsMX5n5maj4Z7/dXrabQ3qga+tYAKLhj299O1E0PbN8bXgU\njX/dm57rH82fm46udgxfwzGmKEXx9utarmNsA8DqGP33Zf7cGJvl92v5PQqAoO9tYEQREPRjisAb\neyhEfXJl6p0QwAiG40SAEfQ9E4woIiivCHF/ldj9fWb1bI7SDi0AhtFfmzH8zAy3skRDpikyuN0G\ngMgwpiz0dhvLY2yd0/SaMTwB0OzYPxj8p/0JAXeKEpQ6otsnpLEw9vKsX/gMlH4sho2b4VW9PI3Z\nnxOSUaHgIAqM9SweKVwyQJZlWMg4FjKHe7BEbE3ItjuL58lOg7wu8W5Mxn85HXedsj/YvvdTTzv1\nNmx9bPzF/oQAZ/Cufb0JIW4lk8gQfVmDnudUGNi+LyUnbtRSyUOmFXGlpQQFoVJcaSmBTCuipdJ1\n9/Trom94D6jCg/HZQy2xu78Cv3Xzx+7+Cnz2UEuowoPRN7yHp0Ns1KZ/U4yf+ylQ4Ws5isk42L6v\nk2/D1sfElZuwb0gbqxidpcn3oBw5cgSZmZlQKm2PqK+OekgIIXeqISxX4KoVqYljJDxw7/EKbB3a\nHG2u60y3Ya+0lGDkIaVX/PzNbxX7ZV8G8p17/iafoMTHxyM+Ph4FBQXYvHmzp8MhjQDdBiSNga33\nMQCUqW5h4Z7l9D52MWMSO1qnQeblo7hWfh2tAlrqb8M+4/nkxMh4qzg94hf8+B/rPabuRJNPUAgh\nhFib/VVRzQ28YLXbpkAmkXn9WB+ZRIa48O5OP2+TT1DqeouHkNrQX5aEEHLnmnyCQrd4CCHu1hBu\nA5qPk5m//wQA4J3BXTwVDmmCaBYPIYQQQrxOk+9BIaQpMm7yBQABNsq8YSOyxswbekhqk5JuPeDR\nvGzdyF7uDIc0QU0+QaExKIQQQoj3afIJCo1BIU0R9ZCQ2lAPCfE0GoNCCCGEEK9DCQohhBBCvA4l\nKIQQQgjxOpSgEEIIIcTrNPlBssZZPKWlpQCAwsJCD0dECCGENCzGz06ed95u3IwoiqLTztaAZWVl\n4amnnvJ0GIQQQkiD9eWXXyIuLs4p52ryPShGXbp0wWOPPYbp06eD4ziHj9u0aROSk5Od1ra2NjXV\n26tLSkrC3r17HYrRU+ryc/Tk+etzHnqPOAe9RxxvS+8R7z5/Y3yP8DyPoUOHoksX522HQAmKga+v\nL9q0aYP27dvX6TiFQoGIiAinta2tTU31NdU5GqOn1OXn6Mnz1+c89B5xDnqPON6W3iPeff7G/B7x\n9fV1KEZH0CBZM3371n1L67oc40jb2trUVF+f+L2Fq2N31vnpPeI59B5xvC29R7z7/PQecQyNQWkC\nOnXqhJycHE+HQbwYvUdIbeg9Qmrj7PcI9aAQQgghxOtwb7755pueDoK4Xnx8vKdDIF6O3iOkNvQe\nIbVx5nuEbvEQQgghxOvQLR5CCCGEeB1KUAghhBDidShBIYQQQojXoQSFEEIIIV6HEhRCCCGEeB1K\nUAghhBDidWgvHoI33ngD+/btQ1FREa0USaxcvXoV8+fPR1FREViWRUJCAl5++WUwDOPp0IgXmThx\nIpRKJURRRIcOHfD2228jICDA02ERL7N48WJ89dVXDn3WUA8KwYMPPogdO3Z4OgzipTiOw7x587Br\n1y7s2LEDx48fx88//+zpsIiXWbNmDb7//nv88MMPaN26NdavX+/pkIiXycrKQmVlpcPtKUFpoC5c\nuIDXX38do0ePxl133YVJkybZbJebm4vk5GR0794dAwcOxKpVq8DzvEWbPn36oGXLlu4Im7jR/2/v\nzoOqqt8Hjr9ZVAi4AjKuUIbihqCsbkCyuEyS6XT/MEcExQVDMwsFQYgRLRhcAkQJQRNkDFNHZxiN\nUbTCjEzN1LRGdBqjSLFY5Yos9/eHw/16QxGT5eLvec0ww9k+n+ccnrnnuefzuZf2ypG+ffvi4OAA\nQM+ePRk+fDilpaWdcg6iY7Xn64iZmRkATU1NqFQqecL2AmjP/Hjw4AGbNm0iPDy8zf3LEE83df36\ndb7++mvGjBlDQ0PDY/eprKwkKCiIoUOHsn37dm7dukVCQgJNTU2sWrWqkyMWna0jcqS8vJwTJ06w\na9eujg5fdIL2zpHFixdz+fJl7OzsnulGJHRTe+ZHamoqSqUSS0vLtgegFt1SY2Oj5vcVK1ao582b\n12KftLQ0taurq7q6ulqzLj09Xe3o6Ki1rtmwYcM6JljRJdo7R+rq6tTz5s1TZ2ZmdlzQolN1xOtI\nQ0ODOiEhQZ2ent4xQYtO0175ce3aNXVgYKC6qalJrVa3/V4jQzzdlL7+0/9033zzDR4eHloT1WbM\nmMH9+/c5e/ZsR4YndEB75khjYyNhYWGMGjWKhQsXdki8ovN1xOuIgYEBs2fP5siRI+0aq+h87ZUf\nFy5coLi4GF9fX3x8fADw8fHhn3/+ab3/54hd6LibN29ia2urtW7gwIEYGxtz8+bNLopK6JK25khM\nTAwmJiZERER0doiii7UlRyorK7l7965me35+PnZ2dp0ap+gabcmPuXPncvr0aU6ePMnJkycBOHny\n5FOHe2QOygusqqpKM3HtUQqFgqqqKs1yVFQUhYWFAHh5eeHp6cnGjRs7LU7RddqSI+fPn+fAgQMM\nGzaMWbNmAfDWW28xf/78To1VdI225EhVVRXvvfceDx48AMDW1pbo6OhOjVN0jbbeZ/4LKVCEFCOi\nVS4uLvL9OKJVNjY2HDx4sKvDEN1EW19PZIjnBaZQKKipqWmxvqqqCoVC0QURCV0jOSKeRnJEtKYj\n80MKlBeYra1ti7kmpaWlqFSqFmOG4v8nyRHxNJIjojUdmR9SoLzAvLy8OH36tFZ1e/ToUYyMjHB3\nd+/CyISukBwRTyM5IlrTkflhEBsbG/uc8YkuoFKpKCgooLi4mNOnT1NZWUmfPn0oLi5m0KBB9OjR\nAzs7O3Jzc/n+++/p27cvZ86cYcuWLQQGBvLaa6919SmIDiY5Ip5GckS0pqvzQ0+tVqvb6VxEJyop\nKcHX1/ex2woKCrC2tgYefgXx+vXruXjxIgqFAqVSyYoVKzAwMOjMcEUXkBwRTyM5IlrT1fkhBYoQ\nQgghdI7MQRFCCCGEzpECRQghhBA6RwoUIYQQQugcKVCEEEIIoXOkQBFCCCGEzpECRQghhBA6RwoU\nIYQQQugcKVCE0DEBAQHY29tz7dq1x24fNWoUKSkpHR5HSUkJw4cP58iRIx3e17PKzs7Gw8MDR0dH\n0tPTuzqc/8THx4eoqKiuDkMInSUFihA6qKGhgcjISBoaGro6FJ1TW1vLxx9/jIODA5mZmcycObOr\nQxJCdAApUITQQWZmZly9epWdO3d2dSg6p7q6msbGRvz8/HBzc6N///5dHZIQogNIgSKEDho9ejQz\nZsxg+/bt3Lhx44n7PWkYJiIigilTpmiWfXx82L59O3Fxcbi7u+Pi4sL69etRqVQkJCQwbtw4xo0b\nR1RUFHV1dVpt/fXXXwQHB+Po6Iivry+7d+/W2t7U1ERaWhp+fn6MHj2a6dOn88UXX2jtExAQQHh4\nOKGhoYwZM4aQkJAnntPFixdZsGABbm5uuLm5sXLlSkpKSgA4dOgQXl5eAERGRjJ8+PAntpOXl8fM\nmTNxdHRkwoQJhIWFcfv2bc322tpaEhMTmTp1KqNHj8bZ2Zng4GB++eUXreu4dOlScnJy8Pb2ZsyY\nMQQHB1NWVsaBAwfw8/PDycmJoKAgTYzN1zs5OZm4uDhcXFwYP348sbGxqFSqJ8Z7//59EhIS8PLy\nwsHBgVmzZlFQUKC1z5UrVwgMDMTFxUXT78WLF5/YphDdmRQoQuiodevWYWJiQmRkJE1NTc/dXkZG\nBhUVFSQlJTFnzhxycnKYPXs2paWlbN68mYCAAA4cOEBOTo7WcUlJSQwaNIjU1FT8/PyIj4/XerIT\nGxvLtm3bmD17NmlpaXh7exMdHU12drZWO3l5eZibm5OWlkZgYOBjY/z222+ZO3cuhoaGJCQkEBMT\nw7Vr15gzZw53795l8uTJ7NixA4Bly5aRm5v72HbOnz/PmjVrmDp1KhkZGURERFBUVERYWJhmnzVr\n1nD48GGWLl3Krl27WLt2Lb/++ithYWE8+i/KfvjhBw4ePEhMTAwxMTGcPXuWgIAAsrOziYiIIC4u\njp9++okNGzZoxZCdnc3Vq1dJTExk2bJlHD58mNWrVz82XrVazfLly9m/fz/BwcGkpqYycuRIQkND\nOXHiBAA1NTUsWrQICwsLUlJS2Lp1KyqVikWLFmn9q3shXhSGXR2AEOLxLC0tiY6O5v3332fPnj0s\nWLDgudqzsLAgMTERfX19xo0bR25uLvX19WzatAlDQ0M8PDzIz89v8Y78tddeY/369QB4enpy584d\nMjIyWLhwIbdu3WL//v2sWbOGhQsXAuDh4UFjYyNJSUkolUqMjY0B6NWrFx9++CE9e/Z8Yoxbtmxh\nyJAhfPrpp+jrP3z/5OLiwrRp08jMzCQ8PJxRo0YB8PLLLzN27NjHtnP+/HmMjIxYsmSJpj9zc3Mu\nX76MWq3mwYMHqFQqoqOjmT59OgDu7u7U1NQQHx9PeXk5lpaWANy7d4+kpCRsbGwAOH78OKdOneLE\niROadT/++CN5eXlaMRgYGJCRkYGJiYlmOS4ujuvXr2NnZ6e175kzZygsLCQ5OZlp06YB4OXlRVVV\nFYmJifj5+VFcXEx5eTnz58/H2dkZAFtbW3Jzc7l37x6mpqZPvK5CdEfyBEUIHTZjxgx8fHxISkri\n1q1bz9WWg4OD5qavr6+PhYUF9vb2GBr+732Kubk5VVVVWsc138Cb+fr6UlFRwY0bNygqKkKtVuPt\n7U1DQ4Pmx8fHh+rqai5duqQ5bujQoa0WJ7W1tfz888+8/vrrmjgBBg4ciKurK2fPnm3zubq5uaFS\nqfD392fz5s2cO3cODw8Pli9fjp6eHr169SIzM5Pp06dz+/ZtioqK+Pzzzzl16hQA9fX1mrb69Omj\nKUSaly0sLLTWmZubU11drRWDj4+PpjgBmDp1KgDnzp1rEe93332HgYEBXl5eLa7jb7/9RklJCXZ2\ndlhaWhISEkJMTAzHjx/HysqK1atX069fvzZfGyG6C3mCIoSOi42Nxd/fn6ioKLKysv5zO4/eLJu9\n9NJLTz3OyspKa7lPnz7Aw8mqFRUVQMsiptmdO3fa3Fd1dTVqtbpFf819/vnnn0+NtZmTkxPp6el8\n9tln7N69m/T0dKysrAgJCSEgIACAwsJCPvroI27evImJiQkjRozQxPjoEM9/vW59+/bVWm5+IvPv\nAhCgoqKCxsbGJz4RunPnDtbW1uTk5LBjxw6OHTtGbm4uRkZGvPnmm6xbt67V4k+I7kgKFCF0XL9+\n/QgPDycqKop9+/ZpbdPT0wNoMUeltra23fqvrKzUWi4rKwMeFg1mZmYA7N27FyMjoxbHWltbt7kf\nU1NT9PT0uHv3bottZWVlWFhYPEvYeHp64unpiUqloqioiKysLDZs2ICTkxMKhYLQ0FCmTJlCeno6\n1tbW6OnpkZOTQ2Fh4TP18yTNxVuzv//+G/hfofIoMzMzzMzMWkxAbvbqq68CD4d0EhMTaWxs5NKl\nSxw5coR9+/YxePBgzRCbEC8KGeIRohtQKpVMmjSJTZs2aRUjzfMOSktLNevq6+u1hlae179v2F9+\n+SX9+vXjlVdewdXVFXhYxDg4OGh+SktLSU5ObvVTK/9mYmKCvb09R48e1TrH0tJSLly4oJl30RaJ\niYkolUrUajXGxsZ4e3sTHh4OPPxU0pUrV6irqyMkJAQbGxtNodd8ru0xKbmwsFDre2zy8/PR09Nj\n/PjxLfZ1c3OjuroaQ0NDret46dIlduzYgZ6eHsePH2f8+PGUlZVhYGCAk5MTsbGxKBQKrb+/EC8K\neYIiRDcRFxeHv7+/1vBD7969cXJyYs+ePdjY2NC7d2+ysrK4f/8+PXr0aJd+jx07Rv/+/XF3dyc/\nP5+CggLi4+PR09NjxIgR+Pv7ExkZye+//87IkSO5fv06W7duxd7enoEDBz5TX6tWrWLx4sWEhITw\n9ttvc+/ePVJSUjA1NSUoKKjN7UycOJHMzEwiIiKYOXMm9fX1ZGRkYGFhgbu7O+Xl5RgaGpKYmEhQ\nUBB1dXUcOnSIr776CuCZCqsn+eOPP1i+fDlz587lxo0bfPLJJyiVSq25K80mT56Ms7MzISEhvPPO\nOwwePJgLFy6QmpqKv78/JiYmODs7o1arCQ0NZcmSJZiYmHDs2DFqamo081uEeJFIgSJENzFo0CA+\n+OAD4uLitNbHx8cTFxfHunXrMDU1RalU4uLiwqFDh9ql37Vr15KXl0dGRgYDBgwgISGBWbNmafWf\nlpbG3r17uX37NlZWViiVSt59991n7svDw4PMzEySk5NZuXIlxsbGTJw4kbCwsBZzOlozadIktmzZ\nQkZGhmZirKurK1lZWSgUChQKBZs3b2bbtm2EhITQu3dvxo4dS3Z2NgEBAZw7d44hQ4Y8c/yPeuON\nNzAyMmLlypWYmpoSHBxMaGjoY/fV19dn586dJCUlsW3bNsrLyxkwYAAhISEsXboUeDiklpmZydat\nW4mKikKlUmFnZ0dKSgpubm7PFasQukhP/ejbMSGEEM/Nx8eHCRMmsHHjxq4ORYhuS+agCCGEEELn\nSIEihBBCCJ0jQzxCCCGE0DnyBEUIIYQQOkcKFCGEEELoHClQhBBCCKFzpEARQgghhM6RAkUIIYQQ\nOkcKFCGEEELonP8DIk9xV8RPq2sAAAAASUVORK5CYII=\n", 667 | "text/plain": [ 668 | "" 669 | ] 670 | }, 671 | "metadata": {}, 672 | "output_type": "display_data" 673 | } 674 | ], 675 | "source": [ 676 | "plt.figure(figsize=(8,3.5))\n", 677 | "\n", 678 | "tmp_m, tmp_s = dcsmc_results_L2.mean(0), dcsmc_results_L2.std(0)\n", 679 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2, color=sns.color_palette()[1])\n", 680 | "\n", 681 | "tmp_m, tmp_s = lwis_results_L2.mean(0), lwis_results_L2.std(0)\n", 682 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2,color=sns.color_palette()[5])\n", 683 | "\n", 684 | "tmp_m, tmp_s = nnis_results_L2.mean(0), nnis_results_L2.std(0)\n", 685 | "plt.errorbar(sizes, tmp_m, 2*tmp_s,marker='.', capsize=4, markeredgewidth=2, color=sns.color_palette()[2])\n", 686 | "\n", 687 | "plt.legend(['D&C SMC', 'IS (Prior)', 'IS (NN)'], loc='upper right')\n", 688 | "plt.loglog();\n", 689 | "\n", 690 | "plt.xlim(sizes[0]-1, sizes[-1])\n", 691 | "\n", 692 | "plt.ylabel(\"L2 error in theta\")\n", 693 | "plt.xlabel(\"Number of samples\")\n", 694 | "plt.tight_layout();" 695 | ] 696 | }, 697 | { 698 | "cell_type": "markdown", 699 | "metadata": {}, 700 | "source": [ 701 | "Note that the L2 error above is relative to the MCMC run, which is not authoratative." 702 | ] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "execution_count": null, 707 | "metadata": { 708 | "collapsed": true 709 | }, 710 | "outputs": [], 711 | "source": [] 712 | } 713 | ], 714 | "metadata": { 715 | "kernelspec": { 716 | "display_name": "Python 2", 717 | "language": "python", 718 | "name": "python2" 719 | }, 720 | "language_info": { 721 | "codemirror_mode": { 722 | "name": "ipython", 723 | "version": 2 724 | }, 725 | "file_extension": ".py", 726 | "mimetype": "text/x-python", 727 | "name": "python", 728 | "nbconvert_exporter": "python", 729 | "pygments_lexer": "ipython2", 730 | "version": "2.7.11" 731 | } 732 | }, 733 | "nbformat": 4, 734 | "nbformat_minor": 0 735 | } 736 | -------------------------------------------------------------------------------- /saved/trained_hmm_params.rar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tbrx/compiled-inference/93ee7f4282d44b5c62907c6a531b303340abaf55/saved/trained_hmm_params.rar -------------------------------------------------------------------------------- /saved/trained_poisson_params.rar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tbrx/compiled-inference/93ee7f4282d44b5c62907c6a531b303340abaf55/saved/trained_poisson_params.rar -------------------------------------------------------------------------------- /saved/trained_poisson_theta.rar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tbrx/compiled-inference/93ee7f4282d44b5c62907c6a531b303340abaf55/saved/trained_poisson_theta.rar --------------------------------------------------------------------------------