├── .gitignore ├── LICENSE ├── README.md ├── data └── water │ └── data_0 │ ├── set.000 │ ├── box.npy │ ├── coord.npy │ ├── energy.npy │ └── force.npy │ ├── type.raw │ └── type_map.raw ├── deepmd └── tutorial │ └── model_descriptor_inference.ipynb ├── model └── se_e2_a │ └── water-14000.pb └── tutorial ├── machine_learning ├── 1.1_introduction.ipynb └── mtters_needing_attention.ipynb └── others ├── figs ├── optimization-1.png ├── optimization-2.png ├── optimization-3.png ├── optimization-4.png └── optimization-5.png └── optimization.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # deepmd-kit-tutorial -------------------------------------------------------------------------------- /data/water/data_0/set.000/box.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/data/water/data_0/set.000/box.npy -------------------------------------------------------------------------------- /data/water/data_0/set.000/coord.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/data/water/data_0/set.000/coord.npy -------------------------------------------------------------------------------- /data/water/data_0/set.000/energy.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/data/water/data_0/set.000/energy.npy -------------------------------------------------------------------------------- /data/water/data_0/set.000/force.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/data/water/data_0/set.000/force.npy -------------------------------------------------------------------------------- /data/water/data_0/type.raw: -------------------------------------------------------------------------------- 1 | 0 2 | 0 3 | 0 4 | 0 5 | 0 6 | 0 7 | 0 8 | 0 9 | 0 10 | 0 11 | 0 12 | 0 13 | 0 14 | 0 15 | 0 16 | 0 17 | 0 18 | 0 19 | 0 20 | 0 21 | 0 22 | 0 23 | 0 24 | 0 25 | 0 26 | 0 27 | 0 28 | 0 29 | 0 30 | 0 31 | 0 32 | 0 33 | 0 34 | 0 35 | 0 36 | 0 37 | 0 38 | 0 39 | 0 40 | 0 41 | 0 42 | 0 43 | 0 44 | 0 45 | 0 46 | 0 47 | 0 48 | 0 49 | 0 50 | 0 51 | 0 52 | 0 53 | 0 54 | 0 55 | 0 56 | 0 57 | 0 58 | 0 59 | 0 60 | 0 61 | 0 62 | 0 63 | 0 64 | 0 65 | 1 66 | 1 67 | 1 68 | 1 69 | 1 70 | 1 71 | 1 72 | 1 73 | 1 74 | 1 75 | 1 76 | 1 77 | 1 78 | 1 79 | 1 80 | 1 81 | 1 82 | 1 83 | 1 84 | 1 85 | 1 86 | 1 87 | 1 88 | 1 89 | 1 90 | 1 91 | 1 92 | 1 93 | 1 94 | 1 95 | 1 96 | 1 97 | 1 98 | 1 99 | 1 100 | 1 101 | 1 102 | 1 103 | 1 104 | 1 105 | 1 106 | 1 107 | 1 108 | 1 109 | 1 110 | 1 111 | 1 112 | 1 113 | 1 114 | 1 115 | 1 116 | 1 117 | 1 118 | 1 119 | 1 120 | 1 121 | 1 122 | 1 123 | 1 124 | 1 125 | 1 126 | 1 127 | 1 128 | 1 129 | 1 130 | 1 131 | 1 132 | 1 133 | 1 134 | 1 135 | 1 136 | 1 137 | 1 138 | 1 139 | 1 140 | 1 141 | 1 142 | 1 143 | 1 144 | 1 145 | 1 146 | 1 147 | 1 148 | 1 149 | 1 150 | 1 151 | 1 152 | 1 153 | 1 154 | 1 155 | 1 156 | 1 157 | 1 158 | 1 159 | 1 160 | 1 161 | 1 162 | 1 163 | 1 164 | 1 165 | 1 166 | 1 167 | 1 168 | 1 169 | 1 170 | 1 171 | 1 172 | 1 173 | 1 174 | 1 175 | 1 176 | 1 177 | 1 178 | 1 179 | 1 180 | 1 181 | 1 182 | 1 183 | 1 184 | 1 185 | 1 186 | 1 187 | 1 188 | 1 189 | 1 190 | 1 191 | 1 192 | 1 193 | -------------------------------------------------------------------------------- /data/water/data_0/type_map.raw: -------------------------------------------------------------------------------- 1 | O 2 | H 3 | -------------------------------------------------------------------------------- /deepmd/tutorial/model_descriptor_inference.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "本篇教程将介绍如何使用deepmd和训练好的模型，得到数据集上descriptor或模型的输出结果，descriptor的输出结果可以作为特征在下游任务中使用。 \n", 8 | "运行本教程需要安装`deepmd-kit`和`dpdata` \n", 9 | "- `deepmd-kit`安装介绍：https://github.com/deepmodeling/deepmd-kit \n", 10 | "- `dpdata`支持从vasp, lammps, deepmd, Amber, cp2k等软件读取输入数据，安装介绍可参考：https://github.com/deepmodeling/dpdata \n", 11 | "\n", 12 | "本篇教程的内容可以参考代码注释：https://github.com/deepmodeling/deepmd-kit/blob/v2.1.5/deepmd/infer/deep_pot.py" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "import dpdata\n", 22 | "from deepmd.infer import DeepPot\n", 23 | "\n", 24 | "import numpy as np" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "# 训练好的模型路径\n", 34 | "model_path = '../../model/se_e2_a/water-14000.pb'\n", 35 | "# 数据集的路径，数据集可以是deepmd的标准格式或者VASP\n", 36 | "system_path = '../../data/water/data_0'\n", 37 | "max_nframe_each = 10\n", 38 | "type_map = [\"O\", \"H\"]" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "## 获取descriptor的输出" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "# 加载模型\n", 55 | "dp = DeepPot(model_path)\n", 56 | "\n", 57 | "# 使用dpdata读取数据\n", 58 | "ms = dpdata.System(type_map=type_map)\n", 59 | "ms.from_deepmd_npy(system_path, labeled=False)\n", 60 | "\n", 61 | "# 获取当前system的frame数量\n", 62 | "nframe = ms.get_nframes()\n", 63 | " \n", 64 | "# nframes x natoms x 3\n", 65 | "coord = ms['coords']\n", 66 | "# nframes x 9\n", 67 | "cell = ms['cells'].reshape([nframe, -1])\n", 68 | "# List[int]\n", 69 | "# len(atype) == natoms, 且第i个原子的元素种类为type_map[atype[i]]\n", 70 | "atype = ms['atom_types'].tolist()\n", 71 | "\n", 72 | "start_idx = 0\n", 73 | "end_idx = start_idx + max_nframe_each\n", 74 | "\n", 75 | "descriptor_out = []\n", 76 | "# 限制每次inference时的frame数量，防止内存溢出(Out Of Memory, 简称OOM)\n", 77 | "while start_idx < nframe:\n", 78 | " out = \\\n", 79 | " dp.eval_descriptor(coord[start_idx: end_idx, ...], \n", 80 | " cell[start_idx: end_idx, :],\n", 81 | " atype)\n", 82 | " descriptor_out.append(out)\n", 83 | " start_idx += max_nframe_each\n", 84 | " end_idx += max_nframe_each\n", 85 | "\n", 86 | "# nframes x natoms x feature_num\n", 87 | "descriptor_out = np.concatenate(descriptor_out, axis=0)\n", 88 | "descriptor_out.shape" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "descriptor的输出结果为nframes x natoms x feature_num, 其中feature_num的大小跟input.json中的参数有关，在这个例子中为1600" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "## 获取模型的输出" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": {}, 109 | "outputs": [], 110 | "source": [ 111 | "# 加载模型\n", 112 | "dp = DeepPot(model_path)\n", 113 | "\n", 114 | "# 使用dpdata读取数据\n", 115 | "ms = dpdata.System(type_map=type_map)\n", 116 | "ms.from_deepmd_npy(system_path, labeled=False)\n", 117 | "\n", 118 | "nframe = ms.get_nframes()\n", 119 | "# nframes x natoms x 3\n", 120 | "coord = ms['coords']\n", 121 | "# nframes x 9\n", 122 | "cell = ms['cells'].reshape([nframe, -1])\n", 123 | "# List[int]\n", 124 | "# len(atype) == natoms, 且第i个原子的元素种类为type_map[atype[i]]\n", 125 | "atype = ms['atom_types'].tolist()\n", 126 | "\n", 127 | "start_idx = 0\n", 128 | "end_idx = start_idx + max_nframe_each\n", 129 | "\n", 130 | "energy = []\n", 131 | "force = []\n", 132 | "virial = []\n", 133 | "while start_idx < nframe:\n", 134 | " e, f, v = \\\n", 135 | " dp.eval(coord[start_idx: end_idx, :], \n", 136 | " cell[start_idx: end_idx, :],\n", 137 | " atype)\n", 138 | " \n", 139 | " energy.append(e)\n", 140 | " force.append(f)\n", 141 | " virial.append(v)\n", 142 | "\n", 143 | " start_idx += max_nframe_each\n", 144 | " end_idx += max_nframe_each\n", 145 | "\n", 146 | "# [nframe]\n", 147 | "energy = np.concatenate(energy, axis=0).squeeze()\n", 148 | "# nframe x -1\n", 149 | "force = np.concatenate(force, axis=0).reshape([nframe, -1])\n", 150 | "# nframe x 9\n", 151 | "virial = np.concatenate(virial, axis=0).reshape([nframe, -1])" 152 | ] 153 | }, 154 | { 155 | "cell_type": "markdown", 156 | "metadata": {}, 157 | "source": [ 158 | "模型将输出energy(shape 为[ nframe ]), force(shape为[nframe, -1]), virial force(shape为[nframe, -9]). 其中shape为-1的意思是该维度的值可能为任意值，将在计算其它维度的大小之后再确定。例如shape[a, b, c]的矩阵reshape([nframe, -1])之后shape将变成[nframe, a * b * c / nframe]. \n", 159 | "输出的结果可以保存标准的npy格式，并重新被dpdata读取：" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": null, 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "from pathlib import Path\n", 169 | "save_path = Path('./system')\n", 170 | "# 将不含energy, force, virial force的data保存成标准deepmd格式\n", 171 | "ms.to('deepmd/npy', save_path)\n", 172 | "# 找到coord.npy的路径\n", 173 | "coord_path = list(save_path.glob('**/coord.npy'))[0]\n", 174 | "\n", 175 | "# 将energy, force, virial force跟coord.npy保存在同一路径下，组成标准的deepmd格式\n", 176 | "with open(coord_path.parent / 'energy.npy', 'wb') as f:\n", 177 | " np.save(f, energy)\n", 178 | "with open(coord_path.parent / 'force.npy', 'wb') as f:\n", 179 | " np.save(f, force)\n", 180 | "with open(coord_path.parent / 'virial.npy', 'wb') as f:\n", 181 | " np.save(f, virial)" 182 | ] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "execution_count": null, 187 | "metadata": {}, 188 | "outputs": [], 189 | "source": [ 190 | "# 读取包含energy, force, virial force的数据\n", 191 | "ls = dpdata.LabeledSystem()\n", 192 | "ls.from_deepmd_npy('./system/')" 193 | ] 194 | } 195 | ], 196 | "metadata": { 197 | "kernelspec": { 198 | "display_name": "Python 3.8.13 ('dp2.1.5')", 199 | "language": "python", 200 | "name": "python3" 201 | }, 202 | "language_info": { 203 | "codemirror_mode": { 204 | "name": "ipython", 205 | "version": 3 206 | }, 207 | "file_extension": ".py", 208 | "mimetype": "text/x-python", 209 | "name": "python", 210 | "nbconvert_exporter": "python", 211 | "pygments_lexer": "ipython3", 212 | "version": "3.8.13" 213 | }, 214 | "orig_nbformat": 4, 215 | "vscode": { 216 | "interpreter": { 217 | "hash": "8ba00acb958a154a84efb1320fed20ca107ed539226fcb2e3aa961180e579300" 218 | } 219 | } 220 | }, 221 | "nbformat": 4, 222 | "nbformat_minor": 2 223 | } 224 | -------------------------------------------------------------------------------- /model/se_e2_a/water-14000.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/model/se_e2_a/water-14000.pb -------------------------------------------------------------------------------- /tutorial/machine_learning/1.1_introduction.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "本教程面向非计算机专业的同学，将介绍一些使用deepmd-kit（分子动力学计算）、deepks（量化计算）等软件时可能用到的深度学习知识。在这个教程中你将学到如何训练神经网络来拟合一个简单的函数。神经网络模型的训练无需使用显卡，在个人笔记本电脑上花费几十秒就能完成训练（支持windows/linux/mac os系统）。\n", 8 | "\n", 9 | "运行教程中的代码需要安装pytorch(cpu版本)，matplotlib（用于画图）和jupyter notebook，建议安装Anaconda来运行本教程，安装方法网上可以搜到。安装好相关环境之后，可以使用下面的命令安装pytorch、matplotlib和jupyter notebook\n", 10 | "\n", 11 | "```\n", 12 | "# conda安装\n", 13 | "conda install pytorch -c pytorch\n", 14 | "conda install matplotlib jupyter notebook\n", 15 | "\n", 16 | "# 或者pip安装（与conda安装只能二选一）\n", 17 | "pip install pytorch\n", 18 | "pip install matplotlib jupyter notebook\n", 19 | "```" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": {}, 25 | "source": [ 26 | "理论上神经网络可以拟合任意函数，这里将给出一个神经网络拟合`cos`函数的例子,首先用python写一个`cos`函数：" 27 | ] 28 | }, 29 | { 30 | "cell_type": "code", 31 | "execution_count": 78, 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "import math\n", 36 | "def func(x):\n", 37 | " return math.cos(x)" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "上面的`func`函数接受一个输入x，输出cos(x)。接下利用`func`函数生成一批数据" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 79, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "# 先定义x的区间为0-4和6-8，然后在这两个区间上随机生成500个数据\n", 54 | "intervals = [[0, 4], [6, 8]]\n", 55 | "data_num = 500" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": 80, 61 | "metadata": {}, 62 | "outputs": [ 63 | { 64 | "name": "stdout", 65 | "output_type": "stream", 66 | "text": [ 67 | "data_num: 499\n" 68 | ] 69 | } 70 | ], 71 | "source": [ 72 | "import random\n", 73 | "from copy import deepcopy\n", 74 | "\n", 75 | "# 给定x的区间和数据量，根据func生成相应的数据\n", 76 | "def random_data(intervals, data_num):\n", 77 | " total_len = 0\n", 78 | " for interval in intervals:\n", 79 | " total_len += (interval[1] - interval[0])\n", 80 | " data_density = data_num / total_len\n", 81 | "\n", 82 | " data_pairs = []\n", 83 | " for interval in intervals:\n", 84 | " # 求出该区间上需要产生的数据有多少个\n", 85 | " _data_num = int((interval[1] - interval[0]) * data_density)\n", 86 | " for i in range(_data_num):\n", 87 | " x = random.uniform(interval[0], interval[1])\n", 88 | " y = func(x)\n", 89 | " data_pairs.append((x, y))\n", 90 | " print('data_num:', len(data_pairs))\n", 91 | " return data_pairs\n", 92 | "\n", 93 | "def data_pairs_to_list(data_pairs, shuffle=False):\n", 94 | " x_list = []\n", 95 | " y_list = []\n", 96 | " if shuffle:\n", 97 | " data_pairs = deepcopy(data_pairs)\n", 98 | " random.shuffle(data_pairs)\n", 99 | "\n", 100 | " for d in data_pairs:\n", 101 | " x_list.append([d[0]])\n", 102 | " y_list.append([d[1]])\n", 103 | " return x_list, y_list \n", 104 | "\n", 105 | "data_pairs = random_data(intervals, data_num)" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "因为有取整操作，只产生了499个数据，但这对后面的操作没有有影响。 \n", 113 | "接下来将会用到一个三层的神经网络模型，每层网络都由一个线性层（nn.Linear, 即$y=a*x+b$）和一个激活函数（nn.ReLU, 即activation(x) = max(x, 0))组成，计算方式为： \n", 114 | "$$y = max(a * x + b, 0)$$\n", 115 | "其中x为当前层的输入，a和b可以理解为神经元，在模型训练的过程中它们的值会根据输入的数据发生改变，完成对数据的学习。激活函数activation(x)的计算方式为max(x, 0)，即当x大于0时，保持不变，当x小于0时输出0。很多个这样的函数堆叠之后就形成了神经网络： \n", 116 | "$y1 = max(a1 * x1 + b1, 0)$ \n", 117 | "$y2 = max(a2 * y1 + b2, 0)$ \n", 118 | "$y3 = max(a3 * y2 + b3, 0)$ \n", 119 | "... \n", 120 | "使用Deepmd和Deepks等软件时，神经网络都已经写好，只需要调节相关参数，准备数据并训练就行。下面给出的这个神经网络有三个参数可以调节：\n", 121 | "- `width`: 代表每一层网络神经单元的数量，`width`值必须大于等于2；\n", 122 | "- `depth`: 代表有几层神经网络，`depth`值必须大于等于2，且最好小于10；\n", 123 | "- `activation`：激活函数种类，只能选择`relu`或`tanh`；\n", 124 | "\n", 125 | "Deepmd-kit中用到的神经网络，以及该网络可以调节的参数跟下面这个神经网络模型几乎完全一样：" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": 81, 131 | "metadata": {}, 132 | "outputs": [], 133 | "source": [ 134 | "import torch\n", 135 | "import torch.nn as nn\n", 136 | "\n", 137 | "class FittingNet(nn.Module):\n", 138 | " def __init__(self, width: int, depth: int, activation: str) -> None:\n", 139 | " super().__init__()\n", 140 | " assert width >= 2, 'width should be greater than 2'\n", 141 | " assert activation in {\"relu\", \"tanh\"}, \"activation should be relu or tanh\"\n", 142 | " \n", 143 | " self.width = width\n", 144 | " self.depth = depth\n", 145 | " self.activation = nn.ReLU() if activation == 'relu' else nn.Tanh()\n", 146 | " self.layer1 = nn.Linear(1, width)\n", 147 | " layers = []\n", 148 | " for i in range(depth - 2):\n", 149 | " layers.append(nn.Linear(width, width))\n", 150 | " layers.append(self.activation)\n", 151 | " self.layers = nn.Sequential(*layers)\n", 152 | " self.layer3 = nn.Linear(width, 1)\n", 153 | " \n", 154 | " def forward(self, inputs):\n", 155 | " inputs1 = self.activation(self.layer1(inputs))\n", 156 | " if self.depth > 2:\n", 157 | " inputsn = self.layers(inputs1)\n", 158 | " ret = self.layer3(inputsn)\n", 159 | " return ret" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "接下还要实现一套训练代码。" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 88, 172 | "metadata": {}, 173 | "outputs": [ 174 | { 175 | "name": "stdout", 176 | "output_type": "stream", 177 | "text": [ 178 | "steps: 200, loss: 0.0849, lr: 0.0080\n", 179 | "steps: 400, loss: 0.0505, lr: 0.0064\n", 180 | "steps: 600, loss: 0.0342, lr: 0.0051\n", 181 | "steps: 800, loss: 0.0258, lr: 0.0041\n", 182 | "steps: 1000, loss: 0.0207, lr: 0.0033\n", 183 | "steps: 1200, loss: 0.0173, lr: 0.0026\n", 184 | "steps: 1400, loss: 0.0149, lr: 0.0021\n", 185 | "steps: 1600, loss: 0.0131, lr: 0.0017\n", 186 | "steps: 1800, loss: 0.0117, lr: 0.0013\n" 187 | ] 188 | } 189 | ], 190 | "source": [ 191 | "batch_size = 10\n", 192 | "x_list, y_list = data_pairs_to_list(data_pairs)\n", 193 | "x_list = torch.tensor(x_list)\n", 194 | "y_list = torch.tensor(y_list)\n", 195 | "\n", 196 | "model = FittingNet(width=10, depth=3, activation='relu')\n", 197 | "opt = torch.optim.SGD(model.parameters(), lr=0.01)\n", 198 | "scheduler = torch.optim.lr_scheduler.StepLR(opt, 200, 0.8)\n", 199 | "model.train()\n", 200 | "\n", 201 | "loss_list = []\n", 202 | "start_idx = 0\n", 203 | "end_idx = batch_size\n", 204 | "numb_steps = 2000\n", 205 | "for numb_steps in range(numb_steps):\n", 206 | " while end_idx < len(x_list):\n", 207 | " xs = x_list[start_idx: end_idx, :]\n", 208 | " ys_label = y_list[start_idx: end_idx, :]\n", 209 | " \n", 210 | " ys = model(xs)\n", 211 | " \n", 212 | " loss = ((ys - ys_label)**2).sum() / batch_size\n", 213 | " loss_list.append(loss.item())\n", 214 | " \n", 215 | " opt.zero_grad()\n", 216 | " loss.backward()\n", 217 | " opt.step()\n", 218 | " \n", 219 | " start_idx += batch_size\n", 220 | " end_idx += batch_size\n", 221 | " \n", 222 | " start_idx = 0\n", 223 | " end_idx = batch_size\n", 224 | " scheduler.step()\n", 225 | " if numb_steps > 0 and numb_steps % 200 == 0:\n", 226 | " print('steps: {}, loss: {:.4f}, lr: {:.4f}'.format(numb_steps, sum(loss_list) / len(loss_list), opt.param_groups[0][\"lr\"]))\n", 227 | " " 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "训练过程简单来说就是有n个标注好的训练数据，即n个`x`值和n个`y_label`值，每个`x`, `y_label`都有`y_label = func(x)`. 将n个`x`输入模型，得到模型输出的n个`y_model_output`，然后求`y_label`和`y_model_output`的平方差，就能得到误差loss，即$loss = (y\\_label - y\\_model\\_output)^2$。利用求导公式可以将误差传递回模型中让模型学习，从而让模型的输出越来越接近`y_label`。\n", 235 | "\n", 236 | "训练大概十几秒就能完成。训练过程中应该观察到`loss`值快速下降。 接下来对训练的结果进行可视化，首先来看看训练数据的样子" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": 89, 242 | "metadata": {}, 243 | "outputs": [ 244 | { 245 | "data": { 246 | "text/plain": [ 247 | "" 248 | ] 249 | }, 250 | "execution_count": 89, 251 | "metadata": {}, 252 | "output_type": "execute_result" 253 | }, 254 | { 255 | "data": { 256 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAjlklEQVR4nO3df5CU9Z0n8PebYdgdWHeROHja6mJcylvM4LTXBXJUuRqDP6O0RIMsZr2rLThrdStowgXUirgZIgkbQnYrlysw3pliFvEHjuRgReKulw0rxEGGX2EJYAgycMxskGiAkmHmc3/0M9gM3c/TPf3086Of96uqa7qf7/N0f8SZ/jzf3zQziIhIcg0JOwAREQmXEoGISMIpEYiIJJwSgYhIwikRiIgk3NCwAxiMiy66yMaMGRN2GCIisbJly5Z/N7PGgcdjmQjGjBmD9vb2sMMQEYkVkr8udFxNQyIiCadEICKScEoEIiIJp0QgIpJwSgQiIgnny6ghks8B+DyALjP7TIFyAvgegDsAnATwX8zsXafsNqesDsCzZrbIj5gGerJtB1o3HcTAJfYmXzUKrbMmVeMjRaQCU5a8hb1dJ1zP0d+vP+jH6qMkbwDwOwA/KpII7gDw18glgokAvmdmE0nWAfglgCkADgF4B8AMM/uF2+dlMhkrZ/jok207sGLTQc/zLr5gGDY/MaXk9xURf7Vt7cT81dtxqqev7GvHjh6BDY/d6H9QNYTkFjPLDDzuS9OQmf0UwDGXU6YilyTMzDYBGEnyEgATAOwzs/fM7DSAF5xzfbVy8/slnXf0o9MYM28tZi5/2+8QRMRF29ZOXPP11zFnVcegkgAA7O06ob/fQQqqjyAFIP/b+JBzrNjx85CcTbKdZHt3d3dZH95bZq1n4/5j+JPH16Fta2dZ14lIeWYufxtj5q3FnFUdOHG615f33Lj/GMbMW4sn23b48n5JEFQiYIFj5nL8/INmy8wsY2aZxsbzZki7qmOhj3F3ps8wZ1WHfqFEqmT8U69j4363hoTKrNh0ULWDEgWVCA4BuDzv9WUADrsc99WMiZd7n+RixaaDmLLkLX+CERHMXP42PvzYnxqAm437jykZlCCoRLAGwF8w53oAvzWzI8h1Do8leSXJYQDud871VUu2CQ9cf0XB6kep9nadUDIQ8Uk1awKFPks1e3e+JAKSKwG8DeBqkodI/iXJh0g+5JyyDsB7APYBWA7grwDAzM4AeATAegC7AbxoZrv8iGmglmwTfrXoThxYdOegk8LerhOYuHCD77GJSHEE8MD1V3zytzvIO7oVmw7q77cIX4aPBq3c4aNuSh1amk/D1EQqM2beWtfyhvoheGbaeGTTBceOnNW2tROPruoo3LFYRJLnHlR1+GictWSbzt5plGpv1wmMf+r1KkYlUtsmXzWqaNnS6c3Y/Y3bPZMAAGTTKfyqzL/fIJul4iLxiaBfS7YJS6c3Y1hdafXODz/uVTIQGaTWWZPOSwaTrxqFA4vuLCkBDNR/Q+eWYPI1P/2GhofnSXzTUCFtWzvx1Ze24Uyf979NkquZIlE0ceEGHP3otOd5Qwgs+WLzoBJPXKlpqAzZdAr7vnkHhpRQOdi4/5hGI4hEyOYnppRUM+gz4GuvbA8gouhTInCx5IvNJY0uai2zs1lEqqt11iQcWHQnfm+o+1fcx2f61EQEJQJX2XQK353e7HmeAZq0IhJB3/rCeDTU17meM3/1jsQnAyUCD9l0CktLSAYb9x9T57FIxGTTKTwzrcn1nFM9vXji1WQ37yoRlCCbTpU0PO3Dj3s1+1gkYrLplGefwYnTvYmu1SsRlKh/eKlXn4HXRhoiErxCw1UHSvLADyWCMpTaZ6BagUj0tM6a5NnMu2LTwUT2FygRlCmbTqHe419tb9eJxN5ZiERZNp3yrNUncUipEsEgLL6v2fOcf9isIaUiUTTTo7/v4zN9iavVKxEMQikjiUqYlCwiIWjJNnn2F+ztOpGozmMlgkEqZSSRmodEoql11iQ0eLTxbtx/LDH9BUoEFWjJNrn+A67YdFDJQCSinpk23vOcx1cno79AiaBCS6Y3u/4jrtz8fmCxiEjpSplfcLKnL6BowuXXDmW3kdxDch/JeQXK55LscB47SfaSHOWUHSC5wymr3pKiVZJNp7DEpb+g1ywx1UuRuGmdNQleK88nYcWAihMByToA3wdwO4BxAGaQHJd/jpktNrNmM2sGMB/A/zWz/N0hbnLKz1seNQ6y6RTqXPbPm7OqI1EdTyJx8p0vNruWf/hxL/5kvvuOanHnR41gAoB9ZvaemZ0G8AKAqS7nzwCw0ofPjZQZEy93LU/yrEWRKMumU56rlJ6x2h784UciSAHIbwg/5Bw7D8nhAG4D8EreYQPwBsktJGcX+xCSs0m2k2zv7u72IWx/tWSbPEcRlbs3sogE41tfGI86jw1Iarm/z49EUOhfr9go+rsAbBzQLDTZzK5DrmnpYZI3FLrQzJaZWcbMMo2NjZVFXCUt2SakRja4nlPLdxUicZVNp/Cd+651Pac3hrs5lsqPRHAIQH67yGUADhc5934MaBYys8POzy4AryLX1BRbc2+92rVcm9iIRFMpW1bWal+fH4ngHQBjSV5JchhyX/ZrBp5E8o8A/BmA1/KOjSB5Qf9zALcA2OlDTKHx+mUyqFYgElVezbu12tdXcSIwszMAHgGwHsBuAC+a2S6SD5F8KO/UewC8YWb56zRfDOBnJLcB+DmAtWYW+7FapfQVaEipSPSU0tdXi7V6WgzbvTKZjLW3R3vKwafnrYXbVJQLh9dj69dvCSweESnd5EX/hM7jp4qWL53eXFJTUtSQ3FJomL5mFleJ2yQzAPjgZE8wgYhI2bz6+mptqWolgirJplMYMcx902w1D4lEk9e+Ix+f6aupvgIlgipaeI/7ptmL1+8JKBIRKZfXviO1NC9IiaCKvJaq7jx+SrUCkYjKplOeS1XXSq1AiaDKWrJNuHB4fdHyOas6lAxEIsprqepamW2sRBCAp+66Bg31xfsL5r7UEVwwIlIyr6Wqa2W2sRJBALLpFJ6ZVry/ICFLnovEUuusSUU3vHdbdThOlAgC4jXmuFbaGkVqUbEN771WHY4LJYKIqKURCCK1pn/GcX8NoI7EA9dfgZas+8jAuFAiCJDXtnjqNBaJrpZsE/Y/cwcOLLoTMyZejpWb38eYeWtx1fx1sa/RKxEEqHXWJNfyWputKFKLnmzbgRWbDp7tKO41w4pNB2O9MqkSQcDcZht/fKYv1r9MIklQbMjoxv3HYlurVyIImNds4zj/MokkgduQ0a++tC3ASPyjRBCwUmYrPr5aTUQiUeU2ZPRMn8WyVq9EEAKv2Yone/pUKxCJKK8hoxv3H3MtjyIlghB4zVYEgCdejfcoBJFaVcqQ0bjdyPmSCEjeRnIPyX0k5xUov5Hkb0l2OI+vl3ptrfIaQXTidG9AkYhIuUY2FF8/DIhf827FiYBkHYDvA7gdwDgAM0iOK3Dqv5hZs/P4mzKvrUlui9GJSHQtuPsa1/KTMVs3xo8awQQA+8zsPTM7DeAFAFMDuDb2nrrL/Zcp7pNURGpVNp3CcI9BH3FqHvIjEaQA5A+sPeQcG2gSyW0k/5Fk/zdgqdeC5GyS7STbu7u7fQg7fF59BSs2HVQyEImob3oM+nhsVUcwgfjAj0RQaCzVwIG27wL4YzO7FsDfA2gr49rcQbNlZpYxs0xjY+NgY42c1lmTXDevqZX1zkVqjdeNXB/iUyvwIxEcApA/nuoyAIfzTzCzD83sd87zdQDqSV5UyrVJ4DYKoVbWOxepRV6DPuIy+s+PRPAOgLEkryQ5DMD9ANbkn0DyP5C5WRgkJzif+5tSrk2KWlnXXEQ+ceJ0byxqBRUnAjM7A+ARAOsB7AbwopntIvkQyYec0+4FsJPkNgB/B+B+yyl4baUxxZHbJJX/+MS6ACMRkXJ4zQl6+sfR/0qjxbDpIZPJWHt7e9hh+G7MvLVFy8aOHoENj90YXDAiUrJPz1sLtwGjBxbdGVgsbkhuMbPMwOOaWRwTe7tOhB2CiBSxZHqza3nUm4eUCGIk6r9MIknltZjkgjXRbh5SIoiQWmhrFEkqt8Ukj5/qifSNnBJBhHgNRfvgZE9AkYhIubLplOuyMVGeYKZEEDFLPdoaRSS63JaNifIEMyWCiPFqa9SSEyLRlU0XXCHnrKg27yoRRJBbW+OKTQcje1chIu4+OBnNvgIlggjyuquI+ggEkSTzGvSxeP2egCIpnRJBRKVGNhQtO35KncYiUdU6axLGjh5RtLzz+KkAoymNEkFEzb316rBDEJFBitsqAEoEEZVNpzBiWF3R8ii2M4pIaaI26EOJIMIW3lN8eeootjOKyCfcVhSO2qAPJYIIc+s07jx+KlK/SCJyLrcVhYFo3cwpEUScW6fx3Je2KRmIRJTbhlNAtDqNlQgibu6tV6OhvnBfQU+faSipSIS5LTkRpc2ofEkEJG8juYfkPpLzCpTPJLndefwryWvzyg6Q3EGyg2TtbTJQoWw6hWemFb+z0FBSkehyW3IiStvQVpwISNYB+D6A2wGMAzCD5LgBp/0KwJ+Z2XgA3wCwbED5TWbWXGjDBPGeYKbmIZFoyqZTGNlQvFYwZclbwQXjwo8awQQA+8zsPTM7DeAFAFPzTzCzfzWzD5yXm5DbpF58EqVOJxE514K7i9cK9nadiMSNnB+JIAXg/bzXh5xjxfwlgH/Me20A3iC5heTsYheRnE2ynWR7d3d3RQHHkVtzYpQ6nUTkXHFYMsaPRFDoK6pg4xfJm5BLBF/LOzzZzK5DrmnpYZI3FLrWzJaZWcbMMo2NjZXGHDszJ17hWj5x4YaAIhERP0Whn8+PRHAIQP6A2csAHB54EsnxAJ4FMNXMftN/3MwOOz+7ALyKXFOTDNCSbXJdzOroR6cDjEZEyuG1EF3Y/EgE7wAYS/JKksMA3A9gTf4JJK8AsBrAl8zsl3nHR5C8oP85gFsA7PQhpprktYOZiEST199u2EtOVJwIzOwMgEcArAewG8CLZraL5EMkH3JO+zqATwH4HwOGiV4M4GcktwH4OYC1ZvZ6pTGJiESN2+ihsJecoEVoLGupMpmMtbcnc8rBmHlri5Ytnd7s2TElIuFo29qJOS77Fo9sqEfHU7dUNQaSWwoN09fM4phx29N4/uodkRiKJiLn89rcPsxOYyWCmMmmU1g6vbng9PRTPb14fPX2EKISkVK4zTQOkxJBDGXTqaLT00/29IXe8SQihXk13Yb1t6tEEFNuC1at3Px+0TIRCZdb81BYncZKBDHltmBVlBazEpFzeTUPfe2V4Jt3lQhiym2fAkAL0YlElVen8cdn+gKMJkeJIKa8Nrd/+sfhr18iIoV51QqCvpFTIogpr7uKD072qFYgElFRW4hOiSDGnrrrmoIr/vXT8tQi0eW2/tDxU8HeyCkRxFg2ncLM64uvSqrlqUWiq3XWJNS7fAMHWStQIoi5lmyT6xomah4Sia7F9zUXLQuyVqBEUAPcdkB64lVNLhOJKq++vqCad5UIaoBbx9OJ072qFYhEmNsIoqCad5UIEmC+1h8SiSyvWkEQN3JKBDXCrZ/gVE/wE1REpHRutYIgOo2VCGqEWz8BoE5jkShza94NYnlqXxIBydtI7iG5j+S8AuUk+XdO+XaS15V6rZQmm065zinQTGOR+Kr2jVzFiYBkHYDvA7gdwDgAM0iOG3Da7QDGOo/ZAH5QxrVSIrc5BR+cDG/TCxHx5tZPUO3mIT9qBBMA7DOz98zsNIAXAEwdcM5UAD+ynE0ARpK8pMRrpUQt2SbXcu1TIBJdbv0E1W4e8iMRpADkL4B/yDlWyjmlXAsAIDmbZDvJ9u7u7oqDTqIVmw6GHYKIFBHmfuN+JIJCTdMDF8Qvdk4p1+YOmi0zs4yZZRobG8sMMTncRg8B6jQWiatq/u36kQgOAbg87/VlAA6XeE4p10oZvEYPBb2qoYiULqx+Aj8SwTsAxpK8kuQwAPcDWDPgnDUA/sIZPXQ9gN+a2ZESr5UyZNMp/N7Q4v9bg17VUERK59VPUK2/3YoTgZmdAfAIgPUAdgN40cx2kXyI5EPOaesAvAdgH4DlAP7K7dpKY0q6b31hvGu5hpKKRJPXLONq/e0O9eNNzGwdcl/2+cf+Z95zA/BwqddKZbLpFNp/faxo57CGkopE11N3XYM5qzoKlvVvOOV3x7JmFtcor6Gkah4SiSavWsHjVVg7TImghrmNIFLzkEh0ufUVnOzp831OkBJBDXMbQaTmIZHoyqZTrjdyfs8JUiKoYWFOUBGRygS5kKQSQY0rdlfhNfFMRMLldSPnZ/OuEkGNW3D3Nagfcu4E7voh9LzbEJFo87N5V4mgxmXTKSy+71qkRjaAAFIjG7D4vmvVbCQSA5OvGuVa7lfzEHND/OMlk8lYe3t72GHETtvWTjz9411n7yRGNtRjwd3XKCmIRNin561FsT0GUyMbsHHeZ0t+L5JbzCwz8LhqBAnRtrUTc1/edk518vipHsx9aZvmFIhE2JLpzUXLDvu0ub0SQUIsXr8HPb3n1/56+gyL1+8JISIRKYXbUNJLRzb48hlKBAnhdufQ6dNdhYhUx4K7r0FDfd05xxrq6zD31qt9eX8lgoTwunNQ85BIdGXTKTwzremcQR/PTGvyrX9PncUJ0d9HUKh5CAAa6odg9zduDzgqEQmSOosTLptOYfG91xYtP9XTp1qBSEIpESRIkDMVRSQ+lAgSxm15Wy1EJ5JMFSUCkqNIbiC51/l5YYFzLif5zyR3k9xF8st5ZQtIdpLscB53VBKPeHNb3hYAZi5/O6BIRCQqKq0RzAPwppmNBfCm83qgMwC+YmZ/CuB6AA+THJdX/l0za3Ye2qmsyrLpFIbXF//fvnH/sQCjEZEoqDQRTAXwvPP8eQDZgSeY2REze9d5/hFyexNrTYMQfXOa+57GIpIslSaCi83sCJD7wgcw2u1kkmMApAFszjv8CMntJJ8r1LSUd+1sku0k27u7uysMO9m8Oo01ekgkWTwTAcmfkNxZ4DG1nA8i+QcAXgEwx8w+dA7/AMBVAJoBHAHwnWLXm9kyM8uYWaaxsbGcj5YyFds4W0Rqk2ciMLPPmdlnCjxeA3CU5CUA4PzsKvQeJOuRSwKtZrY6772PmlmvmfUBWA5ggh//UeLNa3lbdRqLJEelTUNrADzoPH8QwGsDTyBJAD8EsNvMlgwouyTv5T0AdlYYj5SoddYk13J1GoskR6WJYBGAKST3ApjivAbJS0n2jwCaDOBLAD5bYJjot0nuILkdwE0AHq0wHinDgI3LzqO+ApFkGFrJxWb2GwA3Fzh+GMAdzvOfASj4lWNmX6rk86Uyfz7xCqzYdLBo+YI1u7RpjUgCaGZxgrVkm3DxBcOKlh8/pZnGIkmgRJBwm5+Y4lqu5iGR2qdEIK7mvtQRdggiUmVKBFJ0GzwA6OnTUFKRWqdEIFhwt/tCdBpKKlLblAgE2XQKD1x/RdhhiEhIlAgEQG4EkZsn23YEFImIBE2JQEqyYtNBjSASqVFKBHKW1/pDC9ZoK0uRWqREIGd5rT+kCWYitUmJQM7hNpRURGqTEoGcw2soqYjUHiUCOUc2ncKIYXUFy4ZQS06I1CIlAjnPwnuaUF93/oKxfZbbvUxDSUVqixKBnCebTmHxvdcW3a9AQ0lFaktFiYDkKJIbSO51fhbcfJ7kAWcDmg6S7eVeL8HLplPos+Llj2lfY5GaUWmNYB6AN81sLIA3ndfF3GRmzWaWGeT1EiF9YQcgIr6pNBFMBfC88/x5ANmAr5cq8hpKquYhkdpQaSK42MyOAIDzc3SR8wzAGyS3kJw9iOtBcjbJdpLt3d3dFYYtpfAaSjp/9faAIhGRavJMBCR/QnJngcfUMj5nspldB+B2AA+TvKHcQM1smZllzCzT2NhY7uUyCF77FZ/q6VOtQKQGeCYCM/ucmX2mwOM1AEdJXgIAzs+uIu9x2PnZBeBVABOcopKul/B4LU+9eP2egCIRkWqptGloDYAHnecPAnht4AkkR5C8oP85gFsA7Cz1eglXS7YJwwrMKejXefxUgNGISDVUmggWAZhCci+AKc5rkLyU5DrnnIsB/IzkNgA/B7DWzF53u16i5dv3XutarglmIvFGM5fB4hGVyWSsvb3d+0TxzZNtO7Bi08Gi5QcW3RlgNCIyGCS3DBjCD0Azi6VE2sFMpHYpEYgvVm5+P+wQRGSQlAikZMVWJQWA3hg2MYpIjhKBlGzhPcWbh+pYfGSRiESbEoGULJtOFd3XeMbEywOORkT8okQgZWmdNQkPXH/FOTUAIrc09VXz16nTWCSGlAikbC3ZJux/5o6zs477ewd6zbBi00ElA5GYUSKQQSs2Uqh1c/H5BiISPUPDDkDiq9hIIbPcEtVei9bViratnVi8fg8OHz+FS0c2YO6tVyfmv11qg2oEMmhuI4W+8mJHcIGEqG1rJ+a+vA2dx0/BkFt7ae7L27Qqq8SKEoEMmttIoV5Lxmzjp3+8Cz2959aMenoNT/94V0gRiZRPiUAGzWvZCbe1iWrFByd7yjouEkVKBFIRr3lkU5a8FUgcYajl/zZJFiUCqcjMie4b1+ztOoGZy98OKJrgzFz+NvZ2nSha7rXfs0iUKBFIRVqyTRg6xL1asHH/sYCiCY7Xf5PXfs8iUaJEIBX72/vcN65JmguH12v4qMRKRYmA5CiSG0judX5eWOCcq0l25D0+JDnHKVtAsjOv7I5K4pFwZNMpz72Na6k93aup66m7VBuQeKm0RjAPwJtmNhbAm87rc5jZHjNrNrNmAP8JwEnkNrDv993+cjNbN/B6iYeWbBPGjh5RtHxv14maGFv/ZNsO12ahsaNHqDYgsVNpIpgK4Hnn+fMAsh7n3wxgv5n9usLPlQja8NiNruWPr94eTCBV0ra103NIrNe/gUgUVZoILjazIwDg/Bztcf79AFYOOPYIye0knyvUtNSP5GyS7STbu7u7K4taQnGypy/Wk8weXdXhWq49GSSuPBMByZ+Q3FngMbWcDyI5DMDdAF7KO/wDAFcBaAZwBMB3il1vZsvMLGNmmcbGxnI+WgJUbL+Cfis2HYxlE9GTbTvgtQeb9mSQuPJMBGb2OTP7TIHHawCOkrwEAJyfXS5vdTuAd83saN57HzWzXjPrA7AcwITK/nMkbK2zJqHO48Y4jssveDUJDauj50xrkaiqtGloDYAHnecPAnjN5dwZGNAs1J9EHPcA2FlhPBIB3/lis2t53JZfKGVC3Lfv1RBaia9KE8EiAFNI7gUwxXkNkpeSPDsCiORwp3z1gOu/TXIHye0AbgLwaIXxSASUMpw0Ts1DXpPHNFJI4q6i/QjM7DfIjQQaePwwgDvyXp8E8KkC532pks+X6GrJNuGVLYdwqqevYPn81blO46h/gXrNfxhCjRSS+NPMYqmaZ6aNR32R5SdO9fTiKy9Ge93+8U+97rqeEAAs8WgGE4kD7VAmVdN/tz+nyLDLXrOzZVGrGUxcuAEfftzreo6ahKRWqEYgVZVNp5Aa2eB6zmMe4/ODNnP52zj60WnXcyZfNUpNQlIzlAik6ubeejUa6uuKlvchWruZlbJaauusSQFEIhIMJQKpumw6hWemee9mFoX+glIWx3NbU0kkjpQIJBCltKU/+mJHqMlg4sINnp3DgEYJSe1RIpDAeC0/YQbMfTmckURTlrzl2S8AAEunN1c/GJGAKRFIYFpnTfJsVunpNTzxarD9BW1bO0uqCUy+apRGCUlNUiKQQG147EaMGFa84xgATpzuDXSf46++tM3znLGjR6iDWGqWEoEEbuE9Taj3WJlu4/5jmLhwQ9Vjmbn8bZzp81pXVP0CUtuUCCRw2XQKi++9Fl6r9x/96HRVt7icufztkoaKevVtiMSdEoGEIptO4bsldLzu7TqB9N+84WsHctvWTlw5b23JSUBNQlLrlAgkNKWsUgrklq2es6rDl9rBzOVvY86qDs9NZgDggeuvUBKQRFAikFC1ZJtKbnrZ23UCn563dlC1g3JqAUCuc1gbzUhS0KyUe6NoyWQy1t7eHnYY4qOJCzeUNI4/39jRIzw7cdu2dmL+6u1Fl8MuRM1BUqtIbjGzzHnHlQgkKqYseauk8fwDDUwIT7bt8NxashglAallVUkEJO8DsADAnwKYYGYFv51J3gbgewDqADxrZv07mY0CsArAGAAHAHzRzD7w+lwlgtrVtrWz6LLV1XbxBcOw+YkpoXy2SBCKJYJK+wh2ApgG4KcuH1wH4PvIbV4/DsAMkuOc4nkA3jSzsQDedF5LgpXagey3saNHKAlIYlW6VeVuACBdR4RPALDPzN5zzn0BwFQAv3B+3uic9zyAtwB8rZKYJP76O2kH27xTDgL47vRmLR0hiRbEqKEUgPfzXh9yjgHAxWZ2BACcn6OLvQnJ2STbSbZ3d3dXLViJhpZsEw4surOqk7nGjh6BXy26U0lAEs8zEZD8CcmdBR5TS/yMQtWFsjsmzGyZmWXMLNPY2Fju5RJTrbMm+d5UNGJYHZZOb9ayESIOz6YhM/tchZ9xCMDlea8vA3DYeX6U5CVmdoTkJQC6KvwsqUEt2Sa0ZJvwZNsOrNz8PnoHOcBheP0QfHPaeNUARAYIYvP6dwCMJXklgE4A9wP4c6dsDYAHASxyfr4WQDwSU/0JoV/b1k4sXr8HncdPFb3mwuH1eOqua/TlL+Ki0uGj9wD4ewCNAI4D6DCzW0leitww0Tuc8+4AsBS54aPPmdlC5/inALwI4AoABwHcZ2aeUz81fFREpHyaUCYiknDVmkcgIiIxp0QgIpJwSgQiIgmnRCAiknCx7Cwm2Q3g14O8/CIA/+5jOH5RXOVRXOVRXOWLamyVxPXHZnbejNxYJoJKkGwv1GseNsVVHsVVHsVVvqjGVo241DQkIpJwSgQiIgmXxESwLOwAilBc5VFc5VFc5YtqbL7Hlbg+AhEROVcSawQiIpJHiUBEJOESkwhI3kZyD8l9JCOzNzLJ50h2kdwZdiz9SF5O8p9J7ia5i+SXw46pH8nfJ/lzktuc2J4OO6Z+JOtIbiX5f8KOJR/JAyR3kOwgGZnVGkmOJPkyyX9zftcmRSCmq51/p/7HhyTnhB0XAJB81Pmd30lyJcnf9+29k9BHQLIOwC8BTEFuo5x3AMwws1+EGhgAkjcA+B2AH5nZZ8KOBwCcTYIuMbN3SV4AYAuAbET+vQhghJn9jmQ9gJ8B+LKZbQo5NJB8DEAGwB+a2efDjqcfyQMAMmYWqclRJJ8H8C9m9izJYQCGm9nxkMM6y/ne6AQw0cwGO4HVr1hSyP2ujzOzUyRfBLDOzP63H++flBrBBAD7zOw9MzsN4AUApW61WVVm9lMAnnswBMnMjpjZu87zjwDsxif7TIfKcn7nvKx3HqHfzZC8DMCdAJ4NO5Y4IPmHAG4A8EMAMLPTUUoCjpsB7A87CeQZCqCB5FAAw/HJTo8VS0oiSAF4P+/1IUTkiy3qSI4BkAawOeRQznKaYDqQ29p0g5lFIbalAP47gL6Q4yjEALxBcgvJ2WEH4/g0gG4A/8tpTnuW5IiwgxrgfgArww4CAMysE8DfIreB1xEAvzWzN/x6/6QkAhY4FvpdZNSR/AMArwCYY2Yfhh1PPzPrNbNm5Pa/nkAy1CY1kp8H0GVmW8KMw8VkM7sOwO0AHnaaI8M2FMB1AH5gZmkAJwBEqe9uGIC7AbwUdiwAQPJC5FoxrgRwKYARJB/w6/2TkggOAbg87/Vl8LFaVYuc9vdXALSa2eqw4ynEaUp4C8Bt4UaCyQDudtriXwDwWZIrwg3pE2Z22PnZBeBV5JpKw3YIwKG82tzLyCWGqLgdwLtmdjTsQByfA/ArM+s2sx4AqwH8Z7/ePCmJ4B0AY0le6WT6+wGsCTmmyHI6ZH8IYLeZLQk7nnwkG0mOdJ43IPcH8m9hxmRm883sMjMbg9zv1j+ZmW93a5UgOcLp8IfT9HILgNBHqJnZ/wPwPsmrnUM3Awh9MEKeGYhIs5DjIIDrSQ53/j5vRq7vzhdD/XqjKDOzMyQfAbAeQB2A58xsV8hhAQBIrgRwI4CLSB4C8JSZ/TDcqDAZwJcA7HDa4gHgcTNbF15IZ10C4HlnRMcQAC+aWaSGa0bMxQBezX13YCiAfzCz18MN6ay/BtDq3Jy9B+C/hhwPAIDkcORGGP63sGPpZ2abSb4M4F0AZwBshY9LTSRi+KiIiBSXlKYhEREpQolARCThlAhERBJOiUBEJOGUCEREEk6JQEQk4ZQIREQS7v8DLZjZa13UkIgAAAAASUVORK5CYII=", 257 | "text/plain": [ 258 | "
" 259 | ] 260 | }, 261 | "metadata": { 262 | "needs_background": "light" 263 | }, 264 | "output_type": "display_data" 265 | } 266 | ], 267 | "source": [ 268 | "import matplotlib.pyplot as plt\n", 269 | "\n", 270 | "plt.scatter(x_list, y_list)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": {}, 276 | "source": [ 277 | "上图就是训练数据，可以看到我们实现的`func`，也就是`cos`在指定区间[0,4]和[6,8]上的样子。 \n", 278 | "接下来再次在[0, 8]范围内随机生成100个x值作为神经网络的输入，看看神经网络的输出跟函数`func`的输出有什么区别。" 279 | ] 280 | }, 281 | { 282 | "cell_type": "code", 283 | "execution_count": 90, 284 | "metadata": {}, 285 | "outputs": [ 286 | { 287 | "name": "stdout", 288 | "output_type": "stream", 289 | "text": [ 290 | "data_num: 100\n" 291 | ] 292 | } 293 | ], 294 | "source": [ 295 | "intervals = [[0, 8]]\n", 296 | "data_pairs_new = random_data(intervals, 100)\n", 297 | "x_list_new, y_list_new = data_pairs_to_list(data_pairs_new)\n", 298 | "x_list_new = torch.tensor(x_list_new)\n", 299 | "\n", 300 | "model.eval()\n", 301 | "with torch.no_grad():\n", 302 | " y_model_output = model(x_list_new)\n" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 91, 308 | "metadata": {}, 309 | "outputs": [ 310 | { 311 | "data": { 312 | "text/plain": [ 313 | "" 314 | ] 315 | }, 316 | "execution_count": 91, 317 | "metadata": {}, 318 | "output_type": "execute_result" 319 | }, 320 | { 321 | "data": { 322 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAvJklEQVR4nO3df3icZZno8e89Cb+KJekPfgnNhN11DaxCSgopugKLri2rFyi77qqBdVuxCxQyM3XPOR7qodNziNeec7Qzk7bgKVBXaNT1KB65VmkVV0Svi1SSNpQfjS5Ck1bAttCEYos0ee/zx/POZCa/mjSTvPPj/lxXruZ932nmaTpzz/Pez/Pcj6gqxhhjSl8o6AYYY4yZGRbwjTGmTFjAN8aYMmEB3xhjyoQFfGOMKROVQTdgPPPnz9fa2tqgm2GMMUWjs7PzoKqeOdq1gg74tbW1dHR0BN0MY4wpGiLSM9Y1S+kYY0yZsIBvjDFlwgK+McaUCQv4xhhTJizgG2MKy/D6XlbvK28s4BtjCkc8DtHoUJBXdcfxeHBtKiEW8I0xhUEVHn0UWltdkPc8iETc8dat7thMiRRyeeRFixapzcM3pkxcdRU8/zwcPDjyWnOz+3POHOvtH4eIdKrqotGuWQ/fGBM8z4M33nDBfv783Gvz57vrra1w6JDl9KfAAr4xJnihEHR2wiWXjOzhHzwIGzYM9fLXrp359pWI0gv4NsJvTHESgSuvHPu6quvl9/XB4OCMNauUlFbAj8chFssd4Y/FLOdnTDHwPHjggbGvr1/vevlf+QosWgRXXz1jTSsVeQn4IrJZRPaLyLNjXBcRaRWRF0Rkl4hcmo/nzaHqPvlTKTZXxQiJsrkqBqmUO289fWMKl+e5IH7kCPuZT5LbR31Y9/P+47q6oL/fevqTlK8e/r8AS8e5fi3wLv9rBXBvnp53iAhtixJsrIyw/HAKjxDLD6dYH2pm/oPrCFUItbXQtkUt+BtTaEIhfv27KnZSz9m8Sh/zSHIH+8kdwK17bAN0dbF3Xj10dMDnP2938JOQl4Cvqk8Ar4/zkOuBB9VpB6pF5Nx8PHe21V8Ubh9I5Jy7zGvni4diqCo9PcrBm6K8dM4V9iIxJkBtbVBb68Zqa2vd8UX7H+dSOoEK1rIGEM7iIEkiCMdy/n7Na0+x8dTP2x38JM1UPfzzgL1Zx/v8c68Mf6CIrMDdBVBTUzOpJ+ntURLEcs4t5pcs5peZ4witsB+62xupU3UDRcaYGdPWBis+pxw56t57PT3ueHBQGOqDCn3MIUmEGF9hB7nTypWTYAA2VkaoXpSgyd7HE6OqefkCaoFnx7j2A+DPs45/AjQc72c2NDTohHmePjA7ogqaIKLgaYJm9RM4OV8JmjVc4038Zxtj8iZRtSbzHnVvSU8TRHQNa0Z5uw7oDupVQXdQr3As5wEwoOFw0P+iwgJ06BgxdaZm6ewDFmQdnw+8nNdnEGHRh6rZWBkhRgIQYiRJ0jzioTGS9O4Vm8JpzExTRfr7iJLy78bdXXmUFNX0AcPfgxX043L7l9JBgn/KuZrg8/T26KgpIjOKsT4JJvvF+D38jwCPAgIsBn45kZ85qR6+b8tDnobDqiKq8+Z6muCOUXv4iTPu0t1LIhqu8VRENVzj6e4lEdU1ayb9nMaYiQvXuB597nvSvRdvvXXUm3KFgczfcXcHQ8f3nhrRWad5OY+fNUt1y5aA/6EBYZwefr6C/Tdx+fhjuN78Z4FbgFv86wJsBH4DPAMsmsjPPZGAn+F5qs1DKZ0EzTkpnt559cPSPxFVcEHfs3SPMdNlyxbVWad5ORF91mneuAH61ltV44xMBW2ojOj/PG1N1jnNXCvXVM+0B/zp+ppSwFd1vfXGRt29xOXsBU8feEez7v/jRk2ccdeYvQxjzDTy/LvprPfeRDta2Xfw4bA7Hu2DID0mkHlcGfX2yzfgq7oXUfYLyT8WcS+M3AEgd94YM008TzUSce+5SGT040n+vJGTNXKPwX1A3Hprvv8xhWm8gD9T0zKDM3y6ln9cs0CJ9uZO4UwQI7nADfgaY6aBCFRXuzr3iYQ7TvhrZ6qrJz9NWoRT7kmwcRlEB1JESQGQpDkzeQNcx/bee93399yTn39KURrrk6AQvvLSwx9N1i3laDn8LQ8OjrhtNMbk0fCe/BTHzbY8lHu3nqB5zBRPqad3KIBpmYVFhLrF1XQviZCsSSAiJGsSdC+JMKeni77lq+jp8VDFX50b4weL1gTdamNKxxh33idElaaO3Lv1KK0kiAJe1rTPQ6h6rF594k9V9Mb6JCiEr2nr4acN71UMDmbygW6Rx2Cm57+Den36Y3dNb3uMMZMzyhjA7g+PXHDpZundoU/SqHHWBNzo6YX18McwvFcRCnHz4XXspJ6FdKFUECWVOe74Sb8tzjKmkIwyJlD3aIK9cy4e9kAlynoWs50qDlEb1rJcnGV72g5TW4tL51CRcz5JhFUk8NQGdI0pOKpDHThVtwl6a+uIh7nB3CQgzJoFmzZBU9NMNnT62Z62k9Byt5Jk1YjzMdZRE7Zgb0xaQZUzyA72sZgL9s2jlVUZmrlz5IiWXT7fAn42f/An4qdxsnVJAy3/wwumXcYUmLY2WLHCVbp0kxvcceBpknSKp7l51PRrun5PuobPsp74DDcwWBbws4lAVRXUu5z95tkRQgzy3En1XKJd/GHlKkKiQxupGFOmVq+GI0dyzx05QmH0mNesccF+/Xp33NzMc5WXAEOzdxJEiZJiwey+shqXK/2FV5O1dq3786qrWJ5IsFyEXR+/jqe/D72Hq1CEnh6lb1mMXQ9Xc/HD8UCba0wQensnd35GicDcudDY6L6SSboWKQN/fymX8DRRXG5/Y2WE6nsSZbUnhvXwR7N27dAqQFU6ftLPJdpFNf2kbwVXDqToeKyvrHoHxqSNtTfRJPcsmj7xODz5JCSTIELTTSGefXBHzkOeuznB6i9KYYxBzBAL+GNJf+qLcPPhBEkiREmhhIiSIkmEmw+XV+/AmLSWFpg1K/fcrFnufMEQyRnMberMnYzxZ/dF6enRoTGIz5X+VE0L+BNQExZ/dH/IiFk71tM3JS57Vs7q1fCZz0A47GJqOFzAUxzTM3dSKTdf3/PYeXIjKwfTq3HdIG7L0RgHVsaDbes0s4A/AS13Kxsqc5du7+RS3nzDc7eDYaV7adQ2Rjcl67bb4KabcmflfP3rrkfvebBnT4EGexi5OAt44u1GABazHSBTfkH6+0q78zbWEtxC+Jr20goTkbV0+4HZERUG9VU52y+3cIlffsEt5T7wx422eYopOVu2qF9OfPjXsE1GCv21n9U+t+tWbgmGUtkPAyutMAVZvYPl/Qk8T/jB6X8HwEKe9ssvuFH/R/Y3BthQY6bH6tUjO71riJMgRm+PfyGdNinku9ys8baWLwmrT0vmXF59WoKWL5X2mJwF/ImIx4dm7Yjw2TdHbo6epJnPHk7aIK4pOSOnWirVuI3I758dy82R9/UVRUqk6dNK55W5adrOK2M0fbrw2z4VFvAnKiuQV4zxWxvrvDHFbORUSzeJIUWE5YdTbhQ3PSCa7hgVMv8Dqm7b0CAukYg7jsWK4gPrRFmImixVvuxFM2mctCitfNmLDr1YSvhFY8pDelZOT89o5euFX92SO3OtKII9jL3rViRyYrtuFREL+CfgqpPdyH6SZgSPdi53509qdw8ohnymMeNoa4Nly1ywh9z+SzgMDz2o3HNKbkqkqHrH2WlaGAr6Jf6etYA/WSJUfGQpGyvSZVahncUAnF/jv3iKLJ9pzHCRCBw7NvL8vHmw5yV/h6lUbkqEVJGlRPK561aRsFo6J+Dih+M8s0UJf1Ho7YXkgiQXV8E1z7S6fCbQvSRCXbHc4hozzGuvjXZWee01ya1ImY+NyM2MsQ1Q8qCtzS3L/v3RoRum00/z2HSfFO5iFGPGMTxmr2EN1fQTI4GquF59LAZz5gylQbI3ITGBsQ1QptnqO92y7GwtR2OsvrNwP0yNGc+8eUPfr2EN1/MIUVLce2rMBfuGBrfJSHba0oJ9wbOAP1WqxHpjmYJqgpcptBbrLaJ8pjFZUik4+WRwc+77WUgXXdRzy1spqKiAri6or4d16yzQFxHL4U+VCFpVTbI/ktk+LVNorara3gymKKVTkatXC6t6EpwxGzfnPltnZ2bMyhQHy+HnQTqHf+ToUHCfdZpaDt8Ut+ycvOe5nn22YlloVWYshz/Nmppg032SWyrWgr0pZvE4RKN+jTQ/Z5+tvr74pmEaS+nkS1NTAZeHNWYyVGHrVti+faiX39Xlrl12GVxxhRuwra93e0BbD79oWMA3xozU2OgCfnoj8LQrrhhK41RVDe0BbYqCBXxjTC4RtxcsuJ58WnNzZo9Yy90XJ8vhG1PGsrctnNRG3hbsi5L18I0pU21tsGIFHDnijnt63DGqND0Vze3dw9Bxupdvio718I0pU6tXDwX7tCNH3Hm2u4qwNDe7WTrN/oY/6fOmKFkP35gyNXInK//8XoFlS93Abbo3n87pz5ljvfsiZgHfmDJVUzNU7374eeLx3IVX6aBvwb6oWUrHmDLV0uJWhGebdZrS0uIflGG9+FJnAd+YMpOemfPCjXG+ojHmzVW3QrzGbezd9B/xoJtY8E54dlPA8hLwRWSpiPxKRF4QkS+Mcv1qEekXkS7/6658PK8xZnLSM3N6epQq+rjlrRRr34jx0IPKno/7G3vbTm3jGvodul9TenZTUQR9VZ3SF1AB/Ab4I+Bk4GngomGPuRr4t8n+7IaGBi0lWx7yNBxWFVENh92xMTMpHFZ1YUoVPE0QyT6hGomoeva6HI/7HXo5vzZw7+1CAHToGDE1Hz38y4EXVPVFVX0b+BZwfR5+bknZdUOcvmUxenrU7xUofcti7LohHnDLTDnJnZmTVco7zVbQHteynjgJYkD6LkhJEGNZTzzAVk1MPgL+ecDerON9/rnhrhCRp0XkURH5s7F+mIisEJEOEek4cOBAHppXAFTpeKyPlQMp/4XikSDGyoEUHY/1uXnOxsyAuXNheKDKYdUvx6fKgtl9REllgn4CtwHSgtl9hf+7G6vrP9Ev4BPA/VnHNwHrhz3mDOAd/vd/BfzHRH52KaV0ZJTb5wQRhUHdvSSiumZNwC00pe7WW1XXsCbzuku/HndQr09//C6XzrG0znFtecjTDZWRnPfyhspIwaRomeaUzj5gQdbx+cDLwz5U3lDVN/3vfwicJCLz8/DcRaMmLMRYl3MuxjoSrKJuW4ru9r7C7x2YotXWBl+9V6nmkN87XUUfVezkEhbSxcU1/W67wkgEqqstrTOOphuF6q/lpsKqv5ag6cbC/53lY+HVU8C7ROQC4LfAJ4FPZz9ARM4BfqeqKiKX41JJr+XhuYtGy91K37JVMDB0TnE7CCWJkNydYI+9ycw0Wb0a7iIOQJJmogxtV9hOI4vTuXvL4R+fKk0duamwpo4YNBX+727KPXxVHQBuB7YBu4Fvq+pzInKLiNziP+xvgGdF5GmgFfikf+tRHvwXyMqB9EbngzmXY6yjZ4xl7sbkw7KeNVzPI0RpHXHt+dmNQwcFHrACp+rGOVIpdzfkee7PItn9Ky+lFfw0zQ+Hnftq1vcbgA35eK6iJOJukyMRkg+v48m978u5nCCGALtumMPFD8eDaKEpZaqcfUofC//QxU7qRwT9911R2EGqoGS9l8m+K4KiSIXZJuYzyfPovjZG3Y/cGy7JHYBk3oCbZzezvD9Z8C8aU1x23RDnZ98/xKCnRMndwerf33MH1zy73jYkn6zsOkOjHQdovE3MrXjaTAqFqLtiDskfuVKz2T2tdhq5+XCS5QXyojElwp8SfIfX6ncwcl1zjcA1zUXROy0oRVpnyHr4AagNKz29kD1JSvAIh4U9e4JrlylNIVHWERnRu99JPQvpyt260BS98Xr4VjwtAC0tsKEyd5R/Q2WMlrsL98PXFK+aGoDcYJ6k2QX7+nrr3edRoRdVs4A/07Jm7GyeHSGEx+bZEVYOpNzUrgK+4zLFqaUF3ie5O1WdVAHdH26G666DtWsDallpaWuDFZ/T3KJqn9OCCvoW8Gda1ij/8v4EngrL+xO24MXkV7pch9/BuFy3s3l281AHY7CVujrcRicmLw6sjNNyNLfGTsvRGAdWxgNsVS4btA3CaLsJ2QwJky9XXw39/dDZ6XILVVUwfz7LL+pi+c8ENAExrIORT6pIfx8Rf0FbjESmxk6yP1Iws3gs4AelSEf5TYHzPBfsu7rYWdHApXSyg0dYyEF44w13PRSyDka+iZCoSaC9ECWVWcmcJEKMdfx6pXDPPQG3EUvpGFNaQiFua+zMzMBRKliIW3B1W6Pf4wcL9tOg5UtCP1U559L1ss6+d01B5PJtWqYxJaayEgYHvUytJgBhkIqKEAMD4/xFMzWex86KBjf7yZf+4N1JPT874zqi/dM/QG7TMo0pI4ODHjtoyDm3gwYGB23fhWmjCqtWsZAu9jNUCDh9vJAu5I3+wGfhWcA3phSkA4nngn26VykMZnqZO2iwzXamiz/7rntJhHN4NefSWRwkSYRETfDjJjZoa0yxi8fh0CG3WjYU4vRzzmD/q/P5PtcBIX/gtoHTz60ayuGb/IvHqfM8HqtfBc/kXlp96jo2fSn4cRP73zemmKnC1q3Q2grRKKjyp39bz1kcZCnbAKWiIsR9/9jJn778eMCNLXF+WueaZ1K8XlOfc+nJYw3cdKPnVt9uCS6tYwHfmGLX6Nezb211PfhWV5RvcXMj6sHAANzzVXurTzsRt+ahvp65vV0QibDrY/+N/czn4sEu1rGKnh6P1/4+yq4b4oE00VI6xhQzEZfKgUygB6wgWlDSZSquugrWrePtk9/HWRxkJ5fQRxUJYjRrKx2PNIKumfH/Hwv4xpSA7l9B3TjHZgatXZsZRP/FYCOL2M5CnmYhT2ce4s7PPLvPM6aYqdJ9bZS6bbm7WNVta6X72mjg0wDLlgiIECNJkuacS0maiZEM5O7LAr4xRe7Nx1wlzCTNCF4mwKTPm+DMmzu589PNUjrGFDMR/m1wKb+g0fUaEf9P6BucwyLL4QdHlV8sima2NE2L0srSRYAmZ7yXbz18Y4rcWuKZYO+4oL+WeHCNMgDU9bu7rKHS1M0552ea9fCNKXLz5sFrrw3vKQrz5gXSHJMmAkuXQmMjy5P+ftWahCgwZ04gOXwL+MYUuVQKli2DY8eGzp10kjtvAjba3hcBTpe1lE4RaNuiuftkBrhSzxSepib42tcgHHZxJBx2x01NQbfMAAW194X18AvcrhvivPb/DtGjSUDo6VG3Uu/hOVz8cDzYxpmC0dRkAd4cn/XwC5kqbz+ylWZtJUEUUBJEadZW3n5kq82xNqZYDX/vztB72QJ+gfvFoKuTEqUVJUSU1pzzxpgiE4/TfW2M2rC6NG1Y6b42NiMbylvAL2QFuFLPGDMFqnS391G3LUW0N4aqEu2NUbctRXd737T39C2HX+DmzQVeH+O8KS/Zsz1GOzaFT4Slu12Cdvhm58ndCfZM8/+n7WlbyFTpXjpypR7AV09pZvZ9CZpuspu0shCPQ18fJPxdk1QhFoPq6hlJBZj8CYVAVdGsBIvgISJ52ZDM9rQtYukVeR0VjSRpzqR36v+wnb5lscDqapsZpOqCfSrlgnw62KdS7nwBd9rMSDULlASxnHMJYtQsmP7/Rwv4hSy9Uq+5mV2nNWYGbNNBf+VgKx2P9dkbvtSJuJ59JOKCfCjk/oxEhnr8pjiosvXCGFFSJIn4xe4iREmx9cLYtL+XLaVTDNSN5q8jlsn5gcv7rSKBp/aGLwuquXvSep4F+2IUj9Pd3sfS3Ql69wo1C9yHQN3i6ryk58ZL6VjALxK1tdDTMzLvFw4Le/YE1iwzU9RN3avbNvSB370kQt2j1sMvStM4AG85/BLQcreyoTI377ehMkbL3YX7gW3yJCvYZ6cB6ral3PztAu60mTEEVG7BpmUWA1WaOmIwkGLz7Ag3H05w/+wYKw+noANosl5eyfJ7flvbq9lKhBjrcOWPE+56ezV19n9vJshSOsXCpuWVn6z/81CFoOqRYBV9VPu17jVvU/lM6RgvpWM9/GIxWplVm6FRulTh0CFodTOzahYkiPbGiNJKkgiggFBTE2grTZHJSw5fRJaKyK9E5AUR+cIo10VEWv3ru0Tk0nw8b9kpoDKrZpqtXev+bG6GVIo9va6OUjuNfjpHmDULWloCbaUpMlMO+CJSAWwErgUuAj4lIhcNe9i1wLv8rxXAvVN9XmNK1rDefbbn39GI4Greb9pkJZFLzjRX0cxHD/9y4AVVfVFV3wa+BVw/7DHXAw+q0w5Ui8i5eXhuY0pPund/xx0jgv7y5W76/Z49FuxLTjw+tJIahsbp8jhGl4+Afx6wN+t4n39uso8BQERWiEiHiHQcOHAgD80zpoikyyi0tsLPf55z6bmT6t35mE3FLDkzVD4jH4O2oyWSh7duIo9xJ1U3AZvAzdKZWtOMKTLpwXjVEb37Hx/7AI9XXMkHequ52MZvSov//97dDXWpVGZD4u4lEeryODkjHz38fcCCrOPzgZdP4DHGmDEkaSbKeo4NwnWda4JujpkGbd8QGp5I5JxreCJB2zfy9+Gej4D/FPAuEblARE4GPgk8MuwxjwB/78/WWQz0q+oreXhuY0rSgR9sH3EuSTN9zKF3r/XuS9HqO5WWo7mr6VuOxlh9Z/4SHVMO+Ko6ANwObAN2A99W1edE5BYRucV/2A+BF4EXgPuA26b6vMaUJD93e+ZvtpOkOauaokvvrGWNzb0vRarEekevohnrzd+YTV4WXqnqD3FBPfvcV7O+V2BlPp7LmJImwq7eav6dSGa+fbqMQh/VgNjc+1IkglZVk+wf+f9OVXX+CqtZaQVjCku6MmruXAd3PG8eHDwYTLvM9GprgxWfU44cHfp/n3Wasuk+mdQUXKuWaUyhy+p49faO9gAXBFKp0a6ZUtDUBJvuE8Jh16EPh5l0sD8e6+EbE7RhhfFqw0q0N5ZVJM2x3r2ZCOvhG1OoRllwk94Cr5o+0stVZs2y3r2ZOquWaUyQ0gutwEX0VIo63IKb5O4EstdVxGxpsVIKZuospWNMIbD9ak2eWErHmEKWrpuSzerlmGlgAd+YIGUVydo8O0IIj82zI7lFtIzJE8vhGxMkf6HVzysj3H7YLbj57OEERyuxImkm7yzgGxOw63bE6RnIXmgl3D6QILxD2BNgu0zpsZSOMQFzC62G9+RljAVYxpw4C/jGBGysYmhWJM3kmwX8UjfNe2SaqWtpcQurstkG5WY6WMAvZfE43dfGqA0roRDUhpXua/O7R6aZuqYmtyF5Tg0V26DcTAMbtC1VqnS391G3LUUUiJEg2hujrjdFNxHqVG1hTwFparIAb6afBfxSJcLS3QmiQJQUUVwhliRuyf4eC/bBGf5hax++ZoZYaYUSFgqBqqJZmTvBQ0TwvAAbVs6GVcbMLLyqrrZUm8kLK61QpmoWKAlyl+wniFGzoHA/5EvaKJUx06ts6euzAXUz7SylU6r8Mrt1vW6PzBgJEriyu0svBDRhaYSZNkplTAAikaEevzHTyHr4pUqEusXVrsxuTQIRIVmToHtJhLrF1RZcgpId9NMs2JsZYj38UhaPU6eaNUAr1rMP2liVMS3omxlgPfxSd7wgYnnjmWOVMU3ALOCXE1uIFSy/MubGygifPZxA/cqYGysj7Oqtth6+mXaW0ikXthCrIFhlTBMkC/jlwhZiBSfrw3T0CphWGdPMDEvplJHevUKM3BkiMRL07rVgP23i8Zz8fHptxBriOQ+zyphmJljALyO2EGuGjbLQauuFbi1ENX2A+71bZUwzUyzgl4usYJMkguCRJEKUFFsvtBki0yI95z7iz8QJhajblspZG2GVMc1Mslo65SQep7u9j6W7XRqnZoG/Gndxtc3UmU6qrrBRmufZALmZNuPV0rFB23JiC7Fmni20MgXEUjrlZniQGX6HZ2U080N12EKrZltoZQJnPfxydvXVvP5SP5dqJ737QtSc77FDGph7QRU8/njQrSteWSWQd/VW8/OKZo4dhrtYy2cPJzhaCR/oreZi6+GbGWY9/HLlebz+Uj9ze7v43t4GVD2+t7eBub1dvP5Sv/X0T9SwmTnXda7h2CBEafVn5sDtAwmu2xEPspWmTFkPv1yFQvzp4U5+TAML6UKpAGAn9fzl4U4OhqwvcEKGlUDek7XAza2BGG8BljHTy2bplDGXUfAywR5AGARCll6eqmEzcwSPoXIKbqPyPXtmvlmm9NmOV2YMHjtoyDnjjj0bUJwK9YvSZXEL3myhlQmWBfxy5Xk8XeHSOa9wDknuYCf1LKSLHTTQvTRic/NPhB/s67aNXOCWIEa4Rm2hlQmM5fDLVSjE2e+qoqu7nsf5AFHWk+QOAM7kAOf/aD3dYlU0J02Ere3VbM3K2WfqF1VVs6fHfpcmOFPK4YvIXOBfgVpgD/C3qnpolMftAQ4Dg8DAWPml4SyHP/3aHvL4zD8IX/ZimQqa4FfRrElYgJoM/8MxFIKh91X696eIiE1+MtNuvBz+VAP+/wJeV9V/FpEvAHNU9b+M8rg9wCJVPTiZn28Bf2akA5SSO8hoAWoSsube114g9PS4QnV9VLPWr4xpA7VmJkznoO31wNf9778OfGyKP88EwKpoTtGwufctdysbKnOrYtpArSkIqnrCX0DfsONDYzzuJWAH0AmsmOjPb2hoUDPNPE93L4mogiaIKHiawB3vXhJR9byAG1gkPE81EkkXVFAFfWB2RAVPw2HVLVuCbqApF0CHjhFTjztoKyKPAeeMcmn1JD5X3q+qL4vIWcCPRaRbVZ8Y4/lWACsAamxXiOknQt3iarr9na9kr5A8fx1LL8JV0RTJ2bHJjEGEtkUJmrLGQU65J4F3o/3eTOGYag7/V8DVqvqKiJwLPK6q7z7O34kDb6rql4/38y2HP4PSQT1dQvn5da6+jpVQnpC2LUrfshgrB4YC/sbKCNVfS9BkQd/MoOnM4T8CfMb//jPA90d58tNFZHb6e+DDwLNTfF6Tb35PPrPR+d5VqKrb6Hxbiu72PluMNRZV/nCbC/bZc+9XDqT4w21WFdMUjqnOw/9n4Nsi8lmgF/gEgIi8E7hfVf8KOBv4nriUQCXwDVXdOsXnNdPBNjo/MSLsPVydUy8nPfe+/3C1pcNMwbBaOiaHTdE8MbW10NOjZNfLASUcFpuKaWaU1dIxE2ZTNCchq7PU0gKzTsu9PGuW2FRMU1As4JshttH5xMXjObtWNX1a6bwyRqIqjgi2ObkpSFZLxwwZbYrmggRLL8yaomlyF1qBq38fc4PbdZEI0YRNYzWFyXL4ZqTh8+5tHv5IWVUx07qXRKh71DYnN8GyHL6ZnOEBywLYCG3fEBqeSOSca3giQds37HdlCpcFfGMmS5XVdyotR3MHt1uOxlh9Z+HeMRtjOXwzIW1blNVfFHp7oaYGWu7W8lxBGo/DoUPEeiFCK0maAVjMdqKkkF5ALa1jCpP18M1x7bohTt+yGD09iqqbb963LMauG+IBt2yGpQdrW1tpZHsm2EdppZ1GkjSjVdUW7E3Bsh6+GZ8qHY/1sXIgxTEgRoIErozA5sciXFxOA7oitDWso68SVg6kWMx2gMwK21mnwaaNZfK7MEXJevhmfCLcfDiRmY+vhDLz9G8+XGapi3icwytWcfvAupzTfVRRUSFsuk9s3r0paBbwzXHVhLP2ZfXFSFATLqNg7xeWu+WtFDtoyLl0PY/gDXoW7E3Bs4Bvjiu9g1O2DZVuZ6eyIcLS59exk3oW0pU5nT6+f/YqW4lsCp4FfDM+VZo6/Jz97AghPDbPdqV/mzrKpNyC/2/s3Rfi+1yXc+lSOkkSYdGHqssrvWWKkg3amvGJQHU1RCIsTyRYLuKmHcZw50s9yMXjdLcfYunuJKr4e9QOSbCKu+esI/qw9Z1M4bOAb44vHs8tryDi6seUerBX5cBDW6l7cTtR/1SUVgDaaaSdRqKkWHo5NvfeFAUL+GZiyrHcQjzOyy+9xZkMBfq0dhpdYbmLxArLmaJhAd9MXSkWW1Ole3s/l+jT7OQSFvJ05tJOLiFGAu0NWc/eFBVLPJqpicfpvjZGbVgJhaA27KpIFv2G5/52j0mac4I9wM+4iszOVhbsTRGxgG9OXPam572x0tn0PD0rp3f0y1FaufeUaPH++0zZsoBvTlymFzxyFe7S3UWa6ojH6V4apTasKMrf8a85l3dyCQB/c/72IFpnzJRYDt9MSe9etwo3ytBGIDHcbllFR5WDW7ZS95vtRHEbkp/L7wBo5zLauYIoreydewkLmpYU5weaKWsW8M2U1CxwaZxsCaIkFyTJ5LmLaBD3O/sauYXtRFmfc76dK4Zm5TRWwdq1AbXQmBNnAd+cOH/T87reVKZi5JN+L3jphYAm3eNiMbdIq9AHckW49Q9J3iJ3GmaSZmIk0V6xWTmmqFkO35y49KbnSyIkaxII8Pw7GgE4+tPthELK5qqY2+y7rw88L9Dm5oUFe1PErIdvpiYep06VPX4gbNuSZOM/wMq3W/GogMOwsTJCdcM6mlatKuyevir3nhLllj/kLrKK0sqpp+DuWCzgmyJmPXwzdVlBcPUXhdsHc0sp3z6wjoOfWeV6+ocOFdZ0xmFt+evz2gGXxhG8zK5Wf22zckwJsB6+yaveHiXBqpxzSgUopGjmvT+Fa9auLYxefjzuUk1ZdYHOPFM4KJeTPJZE9grJBUmWXgh1i+dY794UPevhm/xR5f7ZMX8ufnOmd5y5DFzzTGthLMpK70+bSrG5KkZI/PGG7duZ/5HF7Nnjhhz29Ah1jyYL4wPKmCmygG/yR4RFH6pmY2VkxA5Z4HLhSZqDX5TleW5/2kUJNlZGWH44hUeI5YdTbKyM0HZZMrd91rM3JcICvsmrix+OU715HUlZlQnww41VsmBGXHUVNDSA57nxhoEv51y+fSDB6i9agDelyQK+ybumm0L8xceqSY0S7BeznZqaABoFcNddvLXzeejqYmdFAz09A/ye2TkPSRCjt6eABpWNySML+GZaXPzdNbz3vek0TiQz42Ux29l6YQBbI6rS/cs3OPXwQfYzn4V0oZzELN7iCKciHMvUBLp/dpls3WjKjs3SMdNDhGtumEP3OyMkdyeyZrwEt2HI0t0JopBT9wfgdA4DlcRIcFIlfMD2pzUlSrSAezKLFi3Sjo6OoJthpqIQNkeJx+lu7+PCbQnAQ4f1c3ZSz6V0Eg6HaLlbabrRgr0pXiLSqaqLRrtmPXwzvY63NaLnQSg09vFUZdXsTzDIp/lWzuV0emcHDSx8sTO/z21MgbGAb4Jz9dXsfbafC17vZFCFClFemtvAgvdUwU9/mp87Ab9mfxQdVhDtdiBElFb2M5/Tzz3Dgr0peZbSMcHwPPae1cCC17p4hXP4Vz7BVfychXTxtNRzygf/nLr3z3MLntKv0Ql+ALS1werVbvpnTQ309AAomjVHQXCF3JLE+JNLz+Ajnf89v/8+YwIyXkrHujQmGKEQF7zeyU7qOZdXibKehbjgf5b+lrrHNtD95CGX4olGYfHi3NWuY3RUPvQhuPFGF+RV3Z+CkmB4zf4Y4RqIeAkL9qZsWMA3gRnUEJfSmXPuXF7lXA4AsPVHkKyIQWsr/PKX8PrrLoqruhr7d92V+Xu33QYiHj/5ieKKOKR5rCOa2XrRTQ910y+3Xpj7IWBMqZtSwBeRT4jIcyLiiciotxD+45aKyK9E5AUR+cJUntOUjgrx2EHDmNejtObk3b+6SQiFyNTYP/y/76VSBhGBe+8d4GXOo58zcJMvBwElQZS/4bu00+hq9ouQrEnQvSQS2PRQY4IypRy+iFwIeMD/Af5JVUck3EWkAvg18JfAPuAp4FOq+vzxfr7l8EtYVg5/J/X8jA+M2FYwW5LmnOC/n/mcxUF2Uk8/s7mcp5jFWznX3+YkzucVADa/o5nlbySHAnwRbbtozGRM27RMVd3tP8F4D7sceEFVX/Qf+y3geuC4Ad+UsFCIBe+pYu+z9Tzxmgv2Se7gKn7On/AfzOb3w/5CbsfkbF5lB4tYSFfm3DEqOIlBAM7iYOb8Tuo55Z6EFUQzZW8mcvjnAXuzjvf550YlIitEpENEOg4cODDtjTMBevxxFuzvJLJmrtsmcUGSn/HnmWDfTmOm+Nrw3n+Cz3MpuXd/6WA/3H/6i06abrLhKmOO+y4QkcdE5NlRvq6f4HOM1pUaM4+kqptUdZGqLjrzzDMn+BSmaIVCbpvERxPs6Q1xzcfnsd0P9FfwJDES7KQecB8A2YOuv+OcCT3FY++12jjGwAQCvqp+SFXfM8rX9yf4HPuABVnH5wMvn0hjTQnzUywXPxznhYee5O65SVxfIcSjldexPuQ+AECI8ZWcHL5wjCOcOvbPbm11Uzst6JsyNxP3uU8B7xKRC0TkZOCTwCMz8LymSDXdKBx8TTIzMO88tpa5X08SDgsiEA5XoO++iL3z6rmMp9jBZcziLbz0zeSpfvCfP9/9ec45bvN0y9ubMjelQVsR+TiwHjgT+IGIdKnqEhF5J3C/qv6Vqg6IyO3ANqAC2Kyqz0255aasNN0oNN2YfeZn4HkMhEJwdRX01xP66EfhjTfgK1+Byy6DM86A+noX7NeuDabhxhQQK61gSkO66Fp6uuXwY2PKhJVWMKUvXfgsHdyHHxtjLOAbY0y5sIBvjDFlwgK+McaUCQv4xhhTJgp6lo6IHAB6JvnX5kNWIZXCYm2bvEJtF1jbTpS1bfIm066wqo5apqCgA/6JEJGOsaYkBc3aNnmF2i6wtp0oa9vk5atdltIxxpgyYQHfGGPKRCkG/E1BN2Ac1rbJK9R2gbXtRFnbJi8v7Sq5HL4xxpjRlWIP3xhjzCgs4BtjTJkomYAvIktF5Fci8oKIfCHo9mQTkc0isl9Eng26LdlEZIGI/FREdovIcyISCbpNaSJyqoj8UkSe9ttWUPWNRaRCRHaKyL8F3ZbhRGSPiDwjIl0iUjDlZkWkWkS+IyLd/mvuiqDbBCAi7/Z/V+mvN0QkGnS70kQk5r8HnhWRb4rIOLv9HOdnlUIOX0QqgF8Df4nbYesp4FOqWhAbpYvIlcCbwIOq+p6g25MmIucC56rqDhGZDXQCHyuE35uICHC6qr4pIicBvwAiqtoecNMAEJFVwCLgDFX9aNDtySYie4BFqlpQC4hE5OvAz1X1fn8zpFmq2hdws3L4seS3QKOqTnbR53S05zzca/8iVT0qIt8Gfqiq/3IiP69UeviXAy+o6ouq+jbwLWCie+5OO1V9Ang96HYMp6qvqOoO//vDwG7G2WB+Jqnzpn94kv9VEL0TETkf+Ahwf9BtKRYicgZwJfAAgKq+XWjB3vdB4DeFEOyzVAKniUglMIspbBFbKgH/PGBv1vE+CiRwFQsRqQUWAtsDbkqGnzbpAvYDP1bVQmlbEvjPgBdwO8aiwI9EpFNEVgTdGN8fAQeAr/mpsPtF5PSgGzWKTwLfDLoRaar6W+DLQC/wCtCvqj860Z9XKgF/tF0uCqI3WAxE5B3Ad4Goqr4RdHvSVHVQVetxG99fLiKBp8NE5KPAflXtDLot43i/ql4KXAus9FOKQasELgXuVdWFwO+BQhtrOxm4Dvi/QbclTUTm4LIVFwDvBE4XkRvH/1tjK5WAvw9YkHV8PlO47Sknfn78u0Cbqj4cdHtG49/6Pw4sDbYlALwfuM7Pk38LuEZEtgTbpFyq+rL/537ge7iUZ9D2Afuy7tK+g/sAKCTXAjtU9XdBNyTLh4CXVPWAqh4DHgbed6I/rFQC/lPAu0TkAv9T+pPAIwG3qeD5A6MPALtVdV3Q7ckmImeKSLX//Wm4F353oI0CVPW/qur5qlqLe539u6qecI8r30TkdH8AHj9l8mEg8NlhqvoqsFdE3u2f+iAQ+OSAYT5FAaVzfL3AYhGZ5b9fP4gbazshlXlrVoBUdUBEbge2ARXAZlV9LuBmZYjIN4Grgfkisg9Yo6oPBNsqwPVWbwKe8XPlAHeq6g+Da1LGucDX/VkTIeDbqlpwUyAL0NnA91xsoBL4hqpuDbZJGXcAbX6n7EVgWcDtyRCRWbhZfv8YdFuyqep2EfkOsAMYAHYyhTILJTEt0xhjzPGVSkrHGGPMcVjAN8aYMmEB3xhjyoQFfGOMKRMW8I0xpkxYwDfGmDJhAd8YY8rE/wcBZAd4m3f9uQAAAABJRU5ErkJggg==", 323 | "text/plain": [ 324 | "
" 325 | ] 326 | }, 327 | "metadata": { 328 | "needs_background": "light" 329 | }, 330 | "output_type": "display_data" 331 | } 332 | ], 333 | "source": [ 334 | "plt.scatter(x_list_new, y_list_new, c='b')\n", 335 | "plt.scatter(x_list_new, y_model_output, c='r', marker='x')" 336 | ] 337 | }, 338 | { 339 | "cell_type": "markdown", 340 | "metadata": {}, 341 | "source": [ 342 | "上图中蓝色的圆点是我们定义的函数`func`也就是`cos`的值，而红色的x点为模型的输出。可以看到在[0, 4]和[6, 8]区间上，模型的输出结果和`func`的输出非常吻合，而在[4, 6]区间上，尽管红色和蓝色的值非常接近，但偏差明显比[0, 4]和[6, 8]区间大。\n", 343 | "\n", 344 | "可以尝试将`func`中的函数替换成其它任意函数来测试这个模型，比如可以替换成分段函数：" 345 | ] 346 | }, 347 | { 348 | "cell_type": "code", 349 | "execution_count": 66, 350 | "metadata": {}, 351 | "outputs": [], 352 | "source": [ 353 | "def func(x):\n", 354 | " if x <= 0:\n", 355 | " return -1\n", 356 | " elif 0 < x <= 1:\n", 357 | " return 1\n", 358 | " elif 1 < x <= 2:\n", 359 | " return 3\n", 360 | " elif 2 < x <= 3:\n", 361 | " return 1\n", 362 | " else:\n", 363 | " return 0" 364 | ] 365 | }, 366 | { 367 | "cell_type": "markdown", 368 | "metadata": {}, 369 | "source": [ 370 | "替换成上面的分段函数后，选择在区间[0, 2.5]上生成500个训练数据，在区间[0, 4]上画图比较结果，就能得到下面这张图: \n", 371 | "![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAW4AAAD0CAYAAAC7KMweAAAKp2lDQ1BJQ0MgUHJvZmlsZQAASImVlwdUU9kWhs+96SGhJfQWepPeAkgJPYD0LiohCRBKCIEAYkcGR3BEURFBRdGRpuCoFBkLIIptUKxYB2QQUEaxYEPlXWARZuat9956e619z7f+7LPPPmfdc9cOAGRplkCQCksDkMbPEoZ4u9GiomNouOeACCAggzw1WexMASMoyB8gNj/+3T7cQ2IRu20yk+vff/+vJsPhZrIBgIIQjudkstMQPoX4C7ZAmAUA6gCia+dkCWa4G2GqECkQ4f4ZTpzj8RmOn2U0mI0JC3FHmAoAnsRiCRMBINEQnZbNTkTykFwRNudzeHyEBQg7p6WlcxA+jrABEoNopJn89Pi/5En8W854cU4WK1HMc3uZNbwHL1OQylr5fx7H/7a0VNH8GnqIk5KEPiHIKIucWX9Kup+Y+fEBgfPM48zGz3KSyCd8ntmZ7jHzzGF5+Innpgb4z3MCz4spzpPFDJtnbqZn6DwL00PEayUI3RnzzBIurCtKCRfrSVymOH9eUljkPGfzIgLmOTMl1G8hxl2sC0Uh4vq5fG+3hXW9xHtPy/zLfnlM8dyspDAf8d5ZC/Vz+YyFnJlR4to4XA/PhZhwcbwgy028liA1SBzPTfUW65nZoeK5WcgLuTA3SHyGySzfoHkGQcAH+AMasAR2wApEIE+Qxc3NmtmIe7pgpZCXmJRFYyA3jEtj8tmmi2iW5paWAMzc17nX4V3/7D2E5PEL2pppABwUEdBY0IJfA9C8H7l6kQua3ggA0r4AdLaxRcLsOW32LmGQb4AUoAIloA60gQEwQeqzBY7AFXgCXxAIwkA0WA7YIAmkASHIAavBBlAIisE2sAtUgCpwCNSCY+AEaAVnQCe4BK6Bm+AueAQGwDB4CSbABzAFQRAOIkMUSAnSgHQhY8gSokPOkCfkD4VA0VAclAjxIRG0GtoIFUOlUAV0EKqDfoFOQ53QFagPegANQmPQW+gLjIJJMBVWg/VgM5gOM2A/OAxeBifCGXAeXABvhcvhavgo3AJ3wtfgu/AA/BKeRAGUBEoepYkyQdFR7qhAVAwqASVErUUVocpQ1ahGVDuqB3UbNYAaR31GY9EUNA1tgnZE+6DD0Wx0Bnotegu6Al2LbkF3o2+jB9ET6O8YMkYVY4xxwDAxUZhETA6mEFOGOYJpxlzE3MUMYz5gsVh5rD7WDuuDjcYmY1dht2D3YZuwHdg+7BB2EofDKeGMcU64QBwLl4UrxO3BHcWdx93CDeM+4SXwGnhLvBc+Bs/H5+PL8PX4c/hb+BH8FEGaoEtwIAQSOISVhBLCYUI74QZhmDBFlCHqE52IYcRk4gZiObGReJH4mPhOQkJCS8JeIliCJ7FeolziuMRliUGJzyRZkhHJnRRLEpG2kmpIHaQHpHdkMlmP7EqOIWeRt5LryBfIT8mfJCmSppJMSY7kOslKyRbJW5KvpAhSulIMqeVSeVJlUielbkiNSxOk9aTdpVnSa6UrpU9L35eelKHIWMgEyqTJbJGpl7kiMyqLk9WT9ZTlyBbIHpK9IDtEQVG0Ke4UNmUj5TDlImWYiqXqU5nUZGox9Ri1lzohJytnLRchlytXKXdWbkAeJa8nz5RPlS+RPyF/T/6LgpoCQ4GrsFmhUeGWwkdFFUVXRa5ikWKT4l3FL0o0JU+lFKXtSq1KT5TRykbKwco5yvuVLyqPq1BVHFXYKkUqJ1QeqsKqRqohqqtUD6leV51UU1fzVhOo7VG7oDauLq/uqp6svlP9nPqYBkXDWYOnsVPjvMYLmhyNQUulldO6aROaqpo+miLNg5q9mlNa+lrhWvlaTVpPtInadO0E7Z3aXdoTOho6S3RW6zToPNQl6NJ1k3R36/boftTT14vU26TXqjeqr6jP1M/Tb9B/bEA2cDHIMKg2uGOINaQbphjuM7xpBBvZGCUZVRrdMIaNbY15xvuM+xZhFtkv4i+qXnTfhGTCMMk2aTAZNJU39TfNN201fWWmYxZjtt2sx+y7uY15qvlh80cWsha+FvkW7RZvLY0s2ZaVlnesyFZeVuus2qzeWBtbc633W/fbUGyW2Gyy6bL5ZmtnK7RttB2z07GLs9trd59OpQfRt9Av22Ps3ezX2Z+x/+xg65DlcMLhtaOJY4pjvePoYv3F3MWHFw85aTmxnA46DTjTnOOcDzgPuGi6sFyqXZ65artyXI+4jjAMGcmMo4xXbuZuQrdmt4/uDu5r3Ds8UB7eHkUevZ6ynuGeFZ5PvbS8Er0avCa8bbxXeXf4YHz8fLb73GeqMdnMOuaEr53vGt9uP5JfqF+F3zN/I3+hf/sSeInvkh1LHgfoBvADWgNBIDNwR+CTIP2gjKBfg7HBQcGVwc9DLEJWh/SEUkJXhNaHfghzCysJexRuEC4K74qQioiNqIv4GOkRWRo5EGUWtSbqWrRyNC+6LQYXExFzJGZyqefSXUuHY21iC2PvLdNflrvsynLl5anLz66QWsFacTIOExcZVx/3lRXIqmZNxjPj98ZPsN3Zu9kvOa6cnZwxrhO3lDuS4JRQmjCa6JS4I3EsySWpLGmc586r4L1J9kmuSv6YEphSkzKdGpnalIZPi0s7zZflp/C709XTc9P7BMaCQsFAhkPGrowJoZ/wSCaUuSyzLYuKNEbXRQaiH0SD2c7ZldmfciJyTubK5PJzr680Wrl55UieV97Pq9Cr2Ku6Vmuu3rB6cA1jzcG10Nr4tV3rtNcVrBte772+dgNxQ8qG3/LN80vz32+M3NheoFawvmDoB+8fGgolC4WF9zc5bqr6Ef0j78fezVab92z+XsQpulpsXlxW/HULe8vVnyx+Kv9pemvC1t4S25L927Db+NvubXfZXlsqU5pXOrRjyY6WnbSdRTvf71qx60qZdVnVbuJu0e6Bcv/ytj06e7bt+VqRVHG30q2yaa/q3s17P+7j7Lu133V/Y5VaVXHVlwO8A/0HvQ+2VOtVlx3CHso+9PxwxOGen+k/1x1RPlJ85FsNv2agNqS2u86urq5etb6kAW4QNYwdjT1685jHsbZGk8aDTfJNxcfBcdHxF7/E/XLvhN+JrpP0k42ndE/tbaY0F7VALStbJlqTWgfaotv6Tvue7mp3bG/+1fTXmjOaZyrPyp0tOUc8V3Bu+nze+ckOQcd4Z2LnUNeKrkcXoi7c6Q7u7r3od/HyJa9LF3oYPecvO10+c8Xhyumr9Kut12yvtVy3ud78m81vzb22vS037G603bS/2d63uO/cLZdbnbc9bl+6w7xz7W7A3b574ff678feH+jn9I8+SH3w5mH2w6lH6x9jHhc9kX5S9lT1afXvhr83DdgOnB30GLz+LPTZoyH20Ms/Mv/4OlzwnPy8bERjpG7UcvTMmNfYzRdLXwy/FLycGi/8U+bPva8MXp167fr6+kTUxPAb4Zvpt1veKb2reW/9vmsyaPLph7QPUx+LPil9qv1M/9zzJfLLyFTOV9zX8m+G39q/+31/PJ02PS1gCVmzrQAKcTghAYC3NQCQowGg3ASAuHSun541aO4/wCyB/8RzPfes2QLQuB6AoA4APBGvQVgPcSlXREM8zBXAVlZin+99Z/v0GWOQkF4lAoDYkkebsOCfNtfD/6Xuf45gJqs1+Of4L9f6BLK5BwHtAAAAVmVYSWZNTQAqAAAACAABh2kABAAAAAEAAAAaAAAAAAADkoYABwAAABIAAABEoAIABAAAAAEAAAFuoAMABAAAAAEAAAD0AAAAAEFTQ0lJAAAAU2NyZWVuc2hvdJyaAukAAAHWaVRYdFhNTDpjb20uYWRvYmUueG1wAAAAAAA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5zOm1ldGEvIiB4OnhtcHRrPSJYTVAgQ29yZSA2LjAuMCI+CiAgIDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+CiAgICAgIDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSIiCiAgICAgICAgICAgIHhtbG5zOmV4aWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20vZXhpZi8xLjAvIj4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI0NDwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4zNjY8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpVc2VyQ29tbWVudD5TY3JlZW5zaG90PC9leGlmOlVzZXJDb21tZW50PgogICAgICA8L3JkZjpEZXNjcmlwdGlvbj4KICAgPC9yZGY6UkRGPgo8L3g6eG1wbWV0YT4K22VXRwAAMDdJREFUeAHtnQt8VNW97/+TSQLEZBKeXmJiLcYiAgEDCPYAgi0qESmCeuCDUouKQS22Vq0YPT2gPIrlXL3Hiif2c+gH9Hz43IutCAaueiD2SEFEUAvae+IDSCJIosQkYt5z13/NXjt779nzJMnsmfmtT2b2Xu//+q6d31qz9svlFY7gQAAEQAAE4oZAStxYCkNBAARAAAQkAQg3DgQQAAEQiDMCEO446zCYCwIgAAKpvYlg0KBBdNFFF/VmlagLBEAABOKewPHjx6m2tlZvR68KN4v2wYMH9cqxAwIgAAIgEJrA+PHjTYmwVGLCAQ8IgAAIOJ8AhNv5fQQLQQAEQMBEAMJtwgEPCIAACDifAITb+X0EC0EABEDARADCbcKRYB6+KdZ4Y6zyqzDl7+z0NVz5jfFGJCodh3Ea9hvTqvzGPNgHARDodgK9elVJt1uPAgMSaFz7BHX8dTeljCwiz+rfyXQNjz4owl4nV/Ygcl85nTr27fHlb28j9+QZ5G1soM6jh3xh6X0o5QejyLPqKSKXixrXrKSO/3qd3FOukX4u21t/mlw5Qyjnlf8kLrvzo8MyL5ed9cjjMp0Ud5Gfy5COxZ33w9lyBmM+Xwld3xynBhPeV+Vy2SnanEQNJka/sdyu0rAHAnFDAMIdN10VgaFCrLzf1JP39DHqEJ+GR315O/b8Se54TzdRx192kffMSb3Q9p1biNqadb9rwAXUsftlaigl8jy5Toq29+sauU0ZPUGWzYm5jjPTRhO1ntXzdvzl/1KDqL+z8ijRd98SpaaR+4dXU9avHxPlPUSd/31EDgquLI8cLLj8hsceJuXneGptsR10uBIeeHhA4vTtu/63rJcHEDUY8YCSOvMfpZDzAEM8MIkBhwcTOcCIwUnaw4MLHAjEIQFXbz5kiq9FxHXcvXSUCPGWM2xNrFWt7ulz5a4ScRVu3HIanmmzmLJ4K8dizuIdtkvrqw8G7mk3+oRU2KPKCbXleoLam9pHiHKLbo4r53+IXwGnpN/VP5dSCq+gjrde8fmF7dJf8Wfpl23kXyJqRq+Xgh0QcB4Bq3baCndzczNNnTqVWlpaqL29nW666SZasWKFqTX8UMH777+fysvLKSMjg/74xz9SUVGRKY3VY63cGg9/NxMQfXTmymGmQvvv+0z6reHGRDKNtuxgTNf/r5/SmR9ebEwa8b776nlyBm8dFFRBeryYmVsHFyniok0dmviqPNatGhCs4coP0VYksI0XAlbttD052adPH9q9ezd98MEH9P7779OuXbto//79pjbu3LmTKisr5aesrIyWLl1qiocnxgSEwL170y/9jOAwu3BjwndvFvnE2vG7Nz9gDKYjk/7B5I/Gs/2yNXL9WW5tCtDjR671i90uwraP+q1fuDVg+YVvWoNMfi7npf/ooAuHtZIrrYVcfcVHbNn/0kvtprTwgIATCdgKt0vMtjIzM6W9bW1txB8OM7pt27bRokWLZPikSZOovr6eTp7sWjM1psV+LxPQRLugZpuseEP1LOIPOw7jz4dNg6RffdW1pqldKqjeJkW6oPoVke8GGvDexyL9ELog5ZTcbqgu1tPyzodNg03+YJ4vSn9FS0uaibd2TsVXLTcPGpy26pEH6MQj/oORtZz5b19lDTL5uexb7+ygqipxEpNdh2/D/p/d0wHx9uHAt4MJBDw52dHRQePGjaNPPvmE7r33Xpo4caKpGTU1NZSfn6+H5eXlEYcNHTpUD8NOjAiIQfaNwx6q73sBvdtwOZV+6buqhK35yeA36YuWgfTm19OotbNChHgpPaWdyut+TNmpDTTBw1eVuKilsw+92nSDyCuuKhGuvG6G+H5D27roYEMu5feto6rmQbKOwszX6GxHCmW4tUsLRWoeDAalt3F2MQBcL75dtDRvO3349jtUOPS0HAQKM83bO4dq8Rec1vL5BpyleTvobvExurrWVFF+1wz5w6aBVJj5lfhwmYNpb/0EUV+5zMIDj8//Gt19wQ7qFJpt5KLKbRPnWJc/3kkLF6oQbEHAeQQCCrfb7ZbLJDyTvvHGG+nIkSM0atQovQV2L86xzso5MS+j8Ied8elWMgBfPUagtFpcDuIVH4NjoSr9sitg3VfqqgqeeWq/qAzxXSmJOO26rx7T0/n2ffkeHviknNF/0+6hHw+okELO5Y3OPEqe1m/FAJFK37T31/Nz+IbqiSLMI8S0Qdi0jlad/7Du5/iDDWm2gw7bxAMPD0ic/2e5W6VNVc08GE0Xde0RA8pXYoARly0KG3iA8Q1MM2QbOGyC57DIm8NF2bqqam0mbhuLQBCIPQHbk5NWs/jE5HnnnUcPPvigHnX33XfTtGnTaMGCBTJs+PDhVFFREXTGbV1g1wvDTrcT4PVafSkgmtLdIpO2hBBN9q48hkFBBYYqO1S8Kkdulcgal/KsdVr9pgL8PPn5LjrxWbpfOAJAIFYErNppu8bNM2OeabP77rvv6M0336RLL73UZPPs2bNp06ZN4j4KrzxxmZ2dHVS0TZnh6XECa54QXdu1bN1VH//GsgvvSkEuoVlLF7soLcMQGPWuUVBF1aLMYGWreFsb2Xa/I5bLN9cR0h+EAdcv2UXdXmQEgZ4n4PdvwFXyScbp06dTYWEhTZgwgWbMmEGzZs2i559/Xn44TXFxMQ0bNowKCgrorrvuoueee46D4RxCYOHCVHrxBTflDOgSNd5/8Q9uGc6zSum6oqWX02wuc4v+TKeNz7nFeQyRQPzxduldLt3P6dLPszRWHU1qK6I5jbRBK4PLNJXNRfAMWziuQ8UHtP3fzW3y5ez65rqC2cnxfgws9TM7OBBwMoGwlkq6qwHW6X53lYtyQAAEQCCRCVi10zA3SuRmo20gAAIgkDgEINyJ05doCQiAQJIQgHAnSUejmSAAAolDAMKdOH2JloAACCQJAQh3knQ0mgkCIJA4BCDcidOXaAkIgECSEIBwJ0lHo5kgAAKJQwDCnTh9iZaAAAgkCQEId5J0NJoJAiCQOAQg3InTl2gJCIBAkhCAcCdJR6OZIAACiUMAwp04fYmWgAAIJAkBCHeSdDSaCQIgkDgEINyJ05doCQiAQJIQgHAnSUejmSAAAolDAMKdOH2JloAACCQJAQh3knQ0mgkCIJA4BCDcidOXaAkIgECSEIBwJ0lHo5kgAAKJQwDCnTh9iZaAAAgkCQEId5J0NJoJAiCQOAQg3InTl2gJCIBAkhCAcCdJR6OZIAACiUMAwp04fYmWgAAIJAkBW+Guqqqi6dOn04gRI2jkyJH0zDPP+OGoqKig7OxsGjt2rPysXLnSLw0CQMCxBLxes2lWvzkWPhBwFIFUO2tSU1Np/fr1VFRURI2NjTRu3DiaMWMGXXbZZabkU6ZMoR07dpjC4AEBpxNoXPsEeRu+Ic+qp4hcLiIh2g2lD5HLk01ZjzzudPNhHwiQ7Yx76NChUrSZT1ZWlpx519TUABcIxD8BIdIs2h27X5ZirUSb/RzOfjgQcDoBW+E2Gn3s2DE6fPgwTZw40Rgs9/ft20djxoyhmTNn0tGjR/3iOaCsrIzGjx8vP7W1tbZpEAgCvUZAzLB5pu2+ep4U7zNXDpNb9usz8F4zBhWBQHQEXF7hAmVtamqiq666ikpLS2nu3LmmZA0NDZSSkkKZmZlUXl5O999/P1VWVprSWD0s4AcPHrQGww8CvU9AHPYs2sr13/eZb9lEBWALAg4iYNXOgDPutrY2mjdvHi1cuNBPtLk9Ho9HijbvFxcXE6evq6tjLxwIOJuAEG1e0zY66Q88hzEmxT4IxJyArXDzJPyOO+6Qa9sPPPCArZGnTp0Sy4G+yfqBAweos7OTBg4caJsWgSDgGAKaaPOaNi+P8ExbLZtAvB3TSzAkBAHbq0r27t1LmzdvptGjR8tL/biM1atX04kTJ2RxJSUltHXrVtqwYQPxFSj9+vWjLVu2iBP04gw9HAg4mYA4RvnqEeOaNq9tN5SKlRIRLq8ycbL9sA0EBIGga9zdTci6TtPd5aM8EAibAP9aNE40rP6wC0JCEOh5AlbttF0q6XkzUAMIxJiAUbTZFKs/xuahehAIRgDCHYwO4kAABEDAgQQg3A7sFJgEAiAAAsEIQLiD0UEcCIAACDiQAITbgZ0Ck0AABEAgGAEIdzA6iAMBEAABBxKAcDuwU2ASCIAACAQjAOEORgdxIAACIOBAAhBuB3YKTAIBEACBYAQg3MHoIA4EQAAEHEgAwu3AToFJIAACIBCMAIQ7GB3EgQAIgIADCUC4HdgpMAkEQAAEghGAcAejgzgQAAEQcCABCLcDOwUmgQAIgEAwAhDuYHQQBwIgAAIOJADhdmCnwCQQAAEQCEYAwh2MDuJAAARAwIEEINwO7BSYBAIgAALBCEC4g9FBXOIT4HdNGp3Vb4zDPgg4hACE2yEdATN6n0Dj2ifE290fIlJiLbbs53A4EHAyAQi3k3sHtvUcASHS3oZvqGP3y7p4s2izn8N1Me85C1AyCERNIDXqnMgIAvFMQLzV3bPqKSHaJMX6zJUvy9a4r54nw/HW93ju3MS33XbGXVVVRdOnT6cRI0bQyJEj6ZlnnvEj4RUzlmXLllFBQQEVFhbSoUOH/NIgAAQcTUATb6ONLOYQbSMR7DuRgK1wp6am0vr16+njjz+m/fv30+9//3v66KOPTPbv3LmTKisr5aesrIyWLl1qiocHBBxPQFvTNtppWvM2RmAfBBxEwFa4hw4dSkVFRdLMrKwsOfOuqakxmb1t2zZatGiRmJy4aNKkSVRfX08nT540pYEHBBxLQBNtXtPm5ZH++z6TW+Oat2Nth2FJT8BWuI1Ujh07RocPH6aJEycag4mFPD8/Xw/Ly8uTYXqAtsOz8fHjx8tPbW2tNRp+EIgNATHhcHmypVir5RHesohzOJZLYtMtqDU8AkFPTjY1NdG8efPo6aefJo/HYyqR17itjmffVrdkyRLiDzsWcDgQcAqBrEce9109oo5bsVUi7hQbYQcI2BEIOONua2uTor1w4UKaO3euX16eYfNJTOWqq6spNzdXebEFgfggoERbWWv1q3BsQcBBBGyFm2fTd9xxh1zbfuCBB2zNnT17Nm3atElc7uqVJzCzs7OJ18bhQCDuCFh/PVr9cdcgGJzoBGyXSvbu3UubN2+m0aNH09ixYyWD1atX04kTJ+R+SUkJFRcXU3l5ubwcMCMjgzZu3JjorNC+BCTAd0nyDTf6EokQbb6yhNe55VJKArYZTYp/ArbCPXnyZDmTDtY8Xs/mywThQCBuCQiR7rp7krQbcnx3T/JJSnn3JJZO4rZ7E9lwW+FO5AajbSCgExCizDNt3D2pE8FOnBCwXeOOE9thJgicOwFNvI0F6csmxkDsg4CDCEC4HdQZMCUGBLQ1bWPNuHvSSAP7TiQA4XZir8Cm3iGgiTbunuwd3Kil+whgjbv7WKKkeCMglkns7p7kNW/cPRlvnZlc9kK4k6u/0VoLAdw9aQECb1wQwFJJXHQTjOxRAtZL/qz+Hq0chYNA5AQg3JEzc14OsVZrcla/KRIeEACBeCcA4Y7zHsR7E+O8A2E+CERBAMIdBTTHZBEz6647/3wvvcV7Ex3TOzAEBHqMAE5O9hjaXihYu3kEd/71AmtUAQIOIoAZt4M6IypTNPE25sWdf0Ya2AeBxCMA4Y73PtVuIjE2A3f+GWlgHwQSjwCEO577VBNt3PkXz50I20EgcgJY446cmXNyiGUS3PnXTd3Bl1Aar9+2+rupGhQDAt1BAMLdHRRjWAbu/Dt3+HiZwrkzRAm9SwBLJb3Lu2dqM84UuQarv2dqTYxSxcwal1QmRlcmUysw406m3kZb/QmIQQ4vU/DHghBnE8CM29n9A+t6g4Am3saqcEmlkQb2nUYAwu20HoE9vU9AuzrHWDEuqTTSwL7TCEC4ndYjsKd3CWiijUsqexc7ajs3AljjPjd+yB3vBMQyCS6pjPdOTD77IdzJ1+dosYUALqm0AIHX8QSwVOL4LoKBvULAegml1d8rRqASEAiPgK1wL168mIYMGUKjRo2yLaWiooKys7Np7Nix8rNy5UrbdAgEARAAARDofgK2SyW333473XfffbRo0aKANU6ZMoV27NgRMB4RIAACIAACPUPAdsY9depUGjBgQM/UiFJBAARAAATOiYCtcIdT4r59+2jMmDE0c+ZMOnr0aMAsZWVlNH78ePmpra0NmA4RIAACIAAC4RGwXSoJlbWoqIiOHz9OmZmZVF5eTnPmzKHKykrbbEuWLCH+sGMBhwMBEAABEDg3AlHNuD0ejxRtrrq4uJja2tqorq7u3CxBbhAAARAAgbAIRCXcp06dIi8/r1i4AwcOUGdnJw0cODCsCpEIBEAABEDg3AjYLpUsWLCA+JI/nkXn5eXRihUr5KyaqyopKaGtW7fShg0bKDU1lfr160dbtmwRTxJ1nZslyA0CIAACIBAWAZeYOfumzmElP7dEvMZ98ODBcysEuUEABEAgyQhYtTOqpZIkY4bmggAIgICjCEC4HdUdMAYEQAAEQhOAcIdmhBQgAAIg4CgCEG5HdQeMAQEQAIHQBCDcoRkhBQiAAAg4igCE21HdAWNAAARAIDQBCHdoRkgBAiAAAo4iAOF2VHfAGBAAARAITQDCHZoRUoAACICAowhAuB3VHTAGBEAABEITgHCHZoQUIAACIOAoAhBuR3UHjAEBEACB0AQg3KEZIQUIgAAIOIoAhNtR3QFjQAAEQCA0AQh3aEZIAQIgAAKOIgDhdlR3wBgQAAEQCE0Awh2aEVKAAAiAgKMIQLgd1R0wBgRAAARCE4Bwh2bUsymMb44TL10Wb17uqq+jg8Rbmbv8vG/0c4zV35UaeyAAAglKwPZlwY5qqxIm3qaIcYaFjV9MrPy8VX71wmKVhxvCYZyH89qltWusNR37VZi1TFW2KkelC7UV6RvXPkHehm/Is+opqr95JnlPV5HrvBxyT71W1te+c4uw202pM/+Rsn79GNX/5GpZS8623XqbG0ofIpcnm7IeeVxZgC0IgECCE3C0cLOwdfx1N3m/qSVXRja5p1xDHW+/QZSaRt760+QanEcpw0dT538foZQfjJIC2PjbJ0X6euo8ekh2nfvK6dTxX6+Te/IM8jY1kivLQ97GhoBip8RUpfM8uY7OTB8jhLSTXNmDpWDm/PlNqr/hKvJ+J8rrlyXtylr+T1JsWUiVPcYyGh572Fy3EGIW7Y7dL1PDcjGwfNtI1HqWvOLT8ZddwtYzRG3Nsg3eM19Tw6MPCmE/Jv2871n9O+K6OL/76nnmgUWmwhcIgECiEnDuW97FjJUFqmPPn7rYi9kndYrlA+XS+kpxcw24gLxf15B7+lwZY8qjpSFrWiF2PNOVM3JVHtepiaEq09V/KHnPnFQpfFtVpmYPp83Z8RdicWYh1fNqdln9LLSyblGaqs9cgb3Prn16WerXhn1WhEZLgH85sVN8rX5fLL5BoEcJWN/y7lzhZgx24m3BI4VLzIqVaFqibb1Bxc4g3raZLYFKlFVwKHv86hb1nblymMoedNt/32cy3phehilRCZobkZESUL/4UkYWyV84nJ8nE/xrzv3Dq33LUyzkRv5Wf6SVIj0I2BCwCrdY+PV3ixcvpiFDhtCoUaP8I0WIVxycy5Yto4KCAiosLKRDh3zLEraJzyVQ/ENsH7k2aAnbL1sj16/lNmjKrkiZ1vjP1hUl/wkjKWv5hW8ac1Moe0x1C47v3vyAKX8wz7s3/5LevemXpiSc/6UX2+jCYa3kSm+R25deajelgSc4AebV/3zBL61FfnifmfKSGy9P8S+41fnL5If3OYzjGteslP134fdFPmYvttwfLPh2zq+fRB3sjOF9PM2yLGlL3xZaWuJbLuN099wjbBRhMk7Up+z12/ZpoT45mk3iuAh1PFjbz3klD1EHb9W+KlPVx/5QZbPdcN1PwFa4b7/9dtq1a1fA2nbu3EmVlZXyU1ZWRkuXLg2Y9lwi+ICuXh5c2L4o/ZU8uHkbruO0XLad4/BIypr/9lWmYkLZo9etiXZB9Su0ofp6+rBJrJ9r7sOmQVTXmqa8Ir5YfGZRQfU2KqjZJvcHvPd3sb1BhL0iGP2Kqqr4BCrJ7c/u6cA/lE4v+A4Lz21LOqj+awFPc7x/65JO+nX9avq3mlkydGneDuIPuzIR9qqYMPz9vTOSf0nbQ5I9b7k//n7wa57dyLTq66+3/rM8roz9xMfCwStm6OGvDium1y+8ilYN+RU9PPBJog4v8fHFaVi0N7wgzoWolUJZvvBzp0tn2Irg1m99UVxfsOOB23/rXeb2c17JQxTJW7WvytQqlHXcuhjHmuLRm9uASyXHjh2jWbNm0ZEjR/zsufvuu2natGm0YMECGTd8+HCqqKigoUOH+qU1Blin+8Y4v31xYP7rJffTrYO261HfdbqoX4o6QEmK26D0NiF6Q6gw87QUNE6s/sF4nwWQ06itSvti3Wz6eeXTfj9z//WSX4g6X9XLZBEtzKzjonSnylL2cJnT/t9btOr8h0Xd2w15fXapOtVW1b364t+I/7B6Kv3yKXr14mK6JKOavmzNpvK6a0RdXroz9/9Qq9dNm07eROu+eoxev+RH0oZrKv9Tt2XV+Q/RN+3ZMl4PFDv5+S468Vm6MQj7NgT4l4oUU5s4EqdUWCi/HnepKZYHTebL4sxizX2uHA+mz6c9RSc+76OCZDp1XHE89zf3m/FY4cH7H3LelccxZzT6+bhpbD+PslK/lcfZqxdfL4+VAWmt1NKZQhe+/zeqGD6VBqV9S26XlyrP5tLsT8tFKcJGzQU6HoK2X2UOsQ1UdohsiI6AgFU7o7qqpKamRhy4+Xq1eXl5xGF2ws0zcv6wq62t1fOE3BFLGTVN2XQw/QLK7fMV1bVlCUGbQcWD3qTWzlTK71snDtAL6G+nR9HozKPiQJ8o/yF4psKz0wkeXr5x0ZtfTxN53hB5f0zZqY1C5Dy0t76BGsTWtDbJBsk6PbSh+QY9XemX6+jIqMvlP0RV80BZ5jWVb4h/lGmUk9pE9e2Z0i6ui/8h2Sl7VF1cBou68qu6S6tL9QmT7x9NDUq+fzgWa7EOJMvkL6Ngq0BVp/KrbVW1KkuFYGtHICgnIdqrzn/QLxuHlVb9jrucSr1PmYRb9keXXvryiuNq2fF11NjslWmV0PtE3HdsqDBV2dK81+Sub1KwRxxv06WoVwy/SoR30PnprTI+zd1JtUWXieOzq1I3VYv0U+QxzscVDxINZ7NFev8lHPv287HTVZ6sKMiXfRlBMiDqnAlENeO+/vrrafny5TR58mRpwI9+9CNat24djRs3LqhB1lEjaGIRaZ4NqIPJuhUJtZlR4PJUnq4UgWYJ5jq70vsU1ngwqzLV1pA2hD2q7sB1GcqKclfVEWX2pMkWrA9WC4Eu0ZZHeDLATv2ae7HuBnqqfR2VtPt+ZSlgtjNuEanqMc7eeeaujl1juCqLtwPe+1ikEcddp5cqfnCVPiM3pjHuq1+DHMaiv7d+grD5NVK/8qyTldXD/okm9N1N7zZcLiYevxNLNE/IiQ5Pjpo7+9Lfmi6TxX/TniN+1T2qTUB4nycVvmNfHmufiqU9w+Ahl4qMflkKvqIlYNXOqGbcPMOuqqrSbaiurqbc3Fzd3107a55IkeuPXjm5UKJp3qZlEN250EV/eMlLbWcD1azy+OI5D5dt5zic1wT9yzKX0TUjMYeHssdYd+C67CwTNYqVD7m0aVye1/5ffIxCt8++5OQNNR9jBg6C68WXDKD3vryADmiipmKv8BymcZP608vNy8Wa9nbxC8+8/DEjT4zc3n8xCdmalS6xlm2eva/LfYg+m7mOhu18WBXtt+UZ9pbJb5FL3EA27YW3xLLNCL80xgBeFlSOlw8LM1+jP5y8gXJX8S8E87HKB9OMyxvEeZMaGu+pEdm8puUaPu+iBqovWvrSsvz/oAwxw/cNYp3yV+egtAZKy8kXl7UWdl1eK25Kk/ctqBvD7G5SYyON9vCBbfSrRmBrS8BevWyTdgXOnj2bNm3aJPrdS/v376fs7GzbZZKuHNHtLVyYSpvL3JQzwHLAacXxSL/xOTc991y63LKff+Fxej0Pz37ZaVuVh8u2cxzOZcqyDPnSzxMeCy1Os/Quly+tqFeVbbLHUIaKV3Wb6hL5ZR1iqxwLtWyHVjazePEFzTYtjP0cLu3Vwth+VYcqC1t7AszJeowxc+Z6y54V9OWje+ipViF6muN9Dvvhiyvo0vED6JO8OXJNm487XttmP4ebREj8n9zw0XK6c+h2OfPlmTbPgNm/5sSPfeG1s+QMWdXjO2HtO0fCaX7/v1Lob5dPV9ERbXNXraeFt4qRyOqEUE7Y+j/pkwt+ImN4Zs5ir1xhZtfSZm6fZinavjifaHNaDh/S91vfjWTiHgi+0qZ+1lTp5xvMGlevkH4Ol06w4DuA5V3ALNbah+9nkGlY5NlxOO/zVvnVvi8kqb9tl0r4pCOfbKyrq6Pzzz+fVqxYQW1tvpG8pKRECvZ9990nrzzJyMigjRs3Ek/lQznrdD9UesSDgOMJsJgYZ4pWv9YAdUeuftOXSMdipe6yVY89oLNNlFJ4Bbly+lPWw6VS9Kiv+InYLO6qFTeZ8X0DfBOat/5UFxqXmFGIO3sDOb97B6wJhS3GewOs0cH87mk3+u7i1W4+U2nZzpztb/nuMNbsljepiTarG+RcQy4ivkaenQxLz5CPfKCMTHFjnfiZzZ+0dOK7n3kQMF0/ryqy21r7wOq3y+PwMKt22gp3T7XBWnlP1YNyQcCRBKwCovxqy0bzLJMHAjUYaMsM9TddRyzqLH71txSLa8lP+B6J4HLLRzHIu3tlPvHTstN3HT+LJw8CHRV/lo9F0AcNIxxRt98dysb4EPv6DWCinEDi73eTms0dzq7+ueIO5S98tak7k7W6jXF89zA/7kHnY7HPdoAUN025snO6nufDvPkjlp/klrnxA93cvMSlDcRqayk/Vl6rdtqvF8TKOtQLAolMQImxaqPyqy2Hs5gYnebP2Sruq9BEXN/XxIWF3CVm3TzDlc/q4dnphweIZ66eNevFzD5FPpvHT+xEfqNo8+yZ8/HMnp1VMDveFpcYas/PUSY2iHsI5HNzxIw7kOPB5swPL9ajpfAK35kr/6SH6aLNIZY6VFwo0WbR1Z//U0pyzZ2XZPiGKfm4COYl3JlphfIXSup1t/ieX5SZRfxAN9eQC+Wzj9Qzhmwf3qYx9xN4FS5r6PkvzLh7njFqAIGeJ6CJuqyIRYQ/ahAIIirW2/qlXzyUjR/kRul9KOWSkbLIjr1i4BAPQGPnGvw9MUNt9xN49bgHXuNW4i/Ta8/skZnFl90zd1RcsK0+uw+WSLTV7vk/SvSNA5V6fhE/gVM+A8nw7CH57CP1TCFtYFWzeSXsnid+Sw2P/5pcQvjVA+zUw+b0QZLZs1ODs9Xviw35bZ1xQ7hDIkMCEEhwAlYxMS7XaHHqscPuK68hz1pxxYwIlwL97TdCyMWSDD+d0/DMIF4ekU/zFIOAWpu3rnFHKuBKfHURDNQtwjbjsg3nU+vqnIX9ne/v71qasSnH77yAKFMNCPrSj1rSMTzATn8Kqbiihh0/3TTos25s6rYLsgo3lkrsKCEMBJKJgJoNqjarmTr7tTh9eYb92oeFWO2rpQNeXlAzb57xN4oi5GOVxSOZ2c/LJPU/OaRqkoIql2g+eKdLSJUgaqnUkg2Lb8OjYgkkyBo328ECG8zJ/CzuhuUba3q/8wGizRzWIJZg+Amg0qklHd4Km3mA4sdOy9k6r+OLOuSzbsRSDdvNTg0g/Kwbjld8fbHhf2PGHT4rpAQBEAiHgFWQjMs4nJ/jhVPr8SyIcolm3x6idnH1Wj9x7W00V5WIctWsWM2YTUsjstYoZ9xaXrbdOJtXwcatqpvDAtUfdPAxFqbtW2fcEG4bSAgCARDoJQJGked9/vCMX+2rGT772bE/iFPr0HLGLNIp4eRLD/nNUcovi1Az+zDXuNkmNTAEMYFMa/Eij1XoTfHBCjLEWYXbcgrbkBK7IAACINDTBIxCzPtqmUbtq3jeqv0gNvEr/PRlDpGeLwPkNW31uj850+0jrhMX14ynzpwvl3VSZ90mlzpcQy+Wfl6b51kzL/vodRpEm9e4pWPht3FyqYYHGs4jLkW0OhnG8efgMOM+B3jICgIgEAcEWCSNoq+Jqj6z57gwruNWs3m+qoTX7XktW78yRZu9qxOXPFiwcPM19OykX2zVGjf7I1kusc64cXJSYsUXCIBAwhIwijY3kv0qTG355hsVZ9z6QuW3fCG3NgjwSVf57lrtHbb6pYGGd9rKYrS7Q6VIiwA+Scl3gPIvAd0GWXpkX5hxR8YLqUEABEDAR0DN5ANtORXHsVMDhNXviw35jRl3SERIAAIgAAJhEFBiHGjLRag4VZzVr8Ij3OLkZITAkBwEQAAEYk0Awh3rHkD9IAACIBAhAQh3hMCQHARAAARiTQDCHeseQP0gAAIgECEBCHeEwJAcBEAABGJNAMId6x5A/SAAAiAQIQEId4TAkBwEQAAEYk0Awh3rHkD9IAACIBAhAQh3hMCQHARAAARiTQDCHeseQP0gAAIgECEBCHeEwJAcBEAABGJNAMId6x5A/SAAAiAQIYGAwr1r1y4aPnw4FRQU0Nq1a/2KraiooOzsbBo7dqz8rFy50i8NAkAABEAABLqfgO3zuDvEQ8XvvfdeeuONNygvL48mTJhAs2fPpssuu8xkwZQpU2jHjh2mMHhAAARAAAR6loDtjPvAgQNypj1s2DBKT0+n+fPn07Zt23rWEpQOAiAAAiAQFgFb4a6pqaH8/Hy9AJ51c5jV7du3j8aMGUMzZ86ko0ePWqPhBwEQAAEQ6AECtkslXvWWBkOFLssDwIuKiuj48eOUmZlJ5eXlNGfOHKqsrDTk8O2WlZURf9jV1tb6AvENAiAAAiAQNQHbGTfPsKuqqvRCq6urKTc3V/fzjsfjkaLN+8XFxdTW1kZ1dXXsNbklS5bQwYMH5Wfw4MGmOHhAAARAAAQiJ2Ar3HwykmfPn3/+ObW2ttKWLVvkyUlj8adOnRKvU/O9T43XxDs7O2ngwIHGJNgHARAAARDoAQK2SyWpqan07LPP0rXXXiveWt9BixcvppEjR9Lzzz8vTSgpKaGtW7fShg0biNP269dPirt1OaUH7EWRIAACIJD0BPCW96Q/BAAABEDA6QSsb3m3XSpxeiNgHwiAAAgkMwEIdzL3PtoOAiAQlwQg3HHZbTAaBEAgmQlAuJO599F2EACBuCQA4Y7LboPRIAACyUwAwp3MvY+2gwAIxCUBCHdcdhuMBgEQSGYCEO5k7n20HQRAIC4JQLjjsttgNAiAQDITgHAnc++j7SAAAnFJAMIdl90Go0EABJKZAIQ7mXsfbQcBEIhLAhDuuOw2GA0CIJDMBCDcydz7aDsIgEBcEoBwx2W3wWgQAIFkJgDhTubeR9tBAATikgCEOy67DUaDAAgkMwEIdzL3PtoOAiAQlwQg3HHZbTAaBEAgmQlAuJO599F2EACBuCQA4Y7LboPRIAACyUwAwp3MvY+2gwAIxCUBCHdcdhuMBgEQSGYCEO5k7n20HQRAIC4JBBTuXbt20fDhw6mgoIDWrl3r1ziv10vLli2T8YWFhXTo0CG/NAgAARBIDgIvvdROFw5rJVd6C/U/v1V+eJ/D7rmnVY9jP6fV06e1kKuv+Igt5+uT49tnP+eXWy2O84Tj9LKD1M82qXpVHcqvbLSrSy+b7evTZZ+y1ZqX03O7VB28H2477OrXw4QA+7n29nbvsGHDvJ9++qm3paXFK4TZe/ToUVO61157zXvdddd5Ozs7vfv27fNeccUVpng7z7hx4+yCEQYCIBDHBF58sc2b5mn2UmqYn37NXldGmGkNZXIeriuYC8uWvqHr5vZY6wqrbGGvysvpSbTVyiWcdljbaNVO2xn3gQMH5ExaiDelp6fT/Pnzadu2bbrY8w77Fy1aRC6XiyZNmkT19fV08uRJUxp4QAAEEp/A8sc7qe1sBO1sI/K2RpBeS8p5uK5gLixbwpi4c3usdYVVtjBO5ZX5RVutLpx2WPNY/bbCXVNTQ/n5+XravLw84jCjCycNpy8rK6Px48fLT21trbEI7IMACCQAgapqb6+1IlRdoeIjMdRaltUfrCxOGyx9sLhg5ao4W+EW03QVr295Zm104aTh9EuWLKGDBw/Kz+DBg41FYB8EQCABCOTnmbWhJ5sUqq5Q8ZHYZi3L6g9WFqcNlj5YXLByVZytcPMMu6qqSqWh6upqys3N1f28E04aUwZ4QAAEEpLAmidSKC0jgqalkTjxGEF6LSnn4bqCubBsSQ1Wgi+O22OtK6yyRXaVV+YXbbW6cNphzWP121KYMGECVVZW0ueff06tra20ZcsWmj17tikv+zdt2kQ8896/fz9lZ2fT0KFDTWngAQEQSHwCCxem0sbn3GJ5Vcy8xV/OAJf88D6HLb1LzD61ON6++IKbNpdp6RmP28eI86WfZ+BlmMhzHOfhuoI5qy229f/BLW1S9erlaXZwHm6PtS5T2ZzJqJ6arca8nJ7byrYrF247VPpAWxefvbSLLC8vp1/84hfU0dFBixcvptLSUnr++edl0pKSEinY9913H/FlgxkZGbRx40a5jm1XlgrjtW5eNoEDARAAARAIn4BVOwMKd/hFhp/SWnn4OZESBEAABJKXgFU7jZP95KWCloMACIBAHBGAcMdRZ8FUEAABEGACEG4cByAAAiAQZwQg3HHWYTAXBEAABHr15CTfgPO9730vbOp1dXU0aNCgsNP3ZkLYFh1tcAO36AhElytRjrfjx4+T8c7zXhXuSNFbz6RGmr8n08O26OiCG7hFRyC6XIl6vGGpJLrjAblAAARAIGYEINwxQ4+KQQAEQCA6Au5/Fi66rL2TSzyHtncqiqIW2BYFNJEF3MAtOgLR5UrE483Ra9zRdRNygQAIgEBiE8BSSWL3L1oHAiCQgAQg3AnYqWgSCIBAYhOIuXA7+aXEoWyrqKiQj7MdO3Ys8WflypW9drTwExuHDBlCo0aNsq2TH/oYq5c5h7ItVtz4GfPTp0+nESNG0MiRI+mZZ57xYxcrbuHYFituzc3NJN4pS2PGjJHcfvOb3ziGWzi2xYqbgsRPWL388stp1qxZKkjfRn28iYwxcz31UuLuaFA4tu3Zs8d7/fXXd0d1EZfx1ltved977z2vECDbvNG8zNm2oCgCQ9kWK25ffPGFZMZNamho8F5yySXd8hLsKBD5ZQnHtlhx4xeCNzY2SpvF8/nli8H5BeFGF6vjLRzbYsVN8Vm/fr13wYIFtloRLbeYzrid/FLicGzTh80Y7EydOpUGDBgQsOZYvsw5lG0Bje7hCH7RR1FRkawlKytLzryt71KNFbdwbOthPAGL59cWZmZmyvi2tjbij/VVhrHiFo5tARvWCxH89jAhznTnnXfa1hYtt5gKdzgvHA4njS2RcwwMt14x85A/IWfOnElHjx49x1q7L3u49ndfjZGVFGtux44do8OHD9PEiRNNhjuBWyDb2NBYceOf+7wcyMtzM2bMcBS3ULbFkhu/jGbdunWUkmIvtdEeb/almQ7lnvOInxJ+hVtH8nDS+BXSDQHh1MuzN36GwAcffEA///nPac6cOd1Qc/cUEY793VNT5KXEmltTUxPNmzePnn76afJ4PKYGxJpbMNtiyc3tdtP7778v3z/Lv0aPHDniGG6hbIsVtx07dsiBLth15NEebzEV7nBeOBxOGtMR1E2ecOrlf3r1E7K4uFj+hOSH2jjBhWN/rOyMJTf+mc+ivXDhQpo7d64fglhyC2VbLLkpUDk5OTRt2jT5ykIVxttYclN2BLItVtz27t1Lr776Kl100UU0f/582r17N916663KXLmNmptQ/Jg5caB6v//973s/++wzb0tLi7ewsNArRnKTPWLU8l533XVePgnBJ0TEi4xN8T3lCce2kydPSrvYhnfeecebn5+v+3vKLmO54mXOAU9Oxoqbsi+YbbHixsfQbbfd5r3//vuVmX7bWHELx7ZYcTt9+rT3zJkzktXZs2e9kydP9m7fvt3ELlbcwrEtVtyMgAKdII2WW/BXJpvGhu73pKam0rPPPkvXXnut/lJivkzL+FJinsnyi4sLCgr0lxJ3vyX+JYZj29atW2nDhg3Eafv160dbtmzxO2njX3L3hIiz1MSXOfEMn0ftFStWyBk/l84vc44VN64/lG2x4sYzoM2bN9Po0aPlei3bunr1ajpx4gTvxpRbOLbFipsQPvrpT38q/0fFAEO33HKLvLTNCf+n4dgWK27yoLL56g5uuOXdBiyCQAAEQMDJBGK6xu1kMLANBEAABJxKAMLt1J6BXSAAAiAQgACEOwAYBIMACICAUwlAuJ3aM7ALBEAABAIQgHAHAINgEAABEHAqgf8Pjp4KdwBIWEYAAAAASUVORK5CYII=)\n", 372 | "\n", 373 | "在这张图上可以看到，当输入x的范围在[0, 2.5]时，模型跟`func`的结果吻合地很好，但在x超过2.5之后开始出现明显的偏差，而当x>3时，模型的预测结果完全错误。这是因为训练数据是在0 < x < 2.5的范围上生成的，模型从未见过x > 3的数据，而恰好x > 3时`func`的输出发生了巨大的变化，因而模型无法预测。换而言之，**模型无法预测没有学过的东西！** " 374 | ] 375 | }, 376 | { 377 | "cell_type": "markdown", 378 | "metadata": {}, 379 | "source": [ 380 | "教程看到这里，建议尝试将`func`中的函数替换成其它函数，并改变训练的参数，如生成的数据量，模型的`width`和`depth`，训练的步数等来观察训练的情况，目标是使得loss尽可能地低。在训练时有这么几个问题需要注意：\n", 381 | "1. `batch_size`代表每次迭代用多少个数据来训练，一般来说这个值越大，训练出来的模型效果越好。但在实践中`batch_size`越大，消耗的内存/显存越多，甚至会导致内存溢出错误（OOM，out of memory）。\n", 382 | "2. 一般来说`func`越复杂，需要的神经网络越大（`width`和`depth`都需要加大），需要的数据也越多。自己训练时，`width`设置为2~40，`depth`设置为2~4已经足够用了。\n", 383 | "3. 如果loss降不下去，一般首先考虑的是增加训练步数和数据量，调节loss策略，再考虑加大模型；\n", 384 | "4. 实践中会将数据集分成训练集+测试集（上面的代码中没有实现），当训练集和测试集上的loss都不再下降时，认为模型训练已经完成；在这个教程中可以认为当训练集的loss不再降低时，训练完成；" 385 | ] 386 | }, 387 | { 388 | "cell_type": "markdown", 389 | "metadata": {}, 390 | "source": [ 391 | "理论上神经网络可以逼近任意函数，例如图片分类（输入一张图片，输出图片类别），物体检测（输入一张图片，输出图片物体的类别和位置），自然语言处理（输入一句话，输出针对这句话的回答）等等。只需要将这些输入转换为矩阵，输入模型得到结果，再转换为人类可以理解的输出即可。\n", 392 | "\n", 393 | "在量子化学计算和分子动力学模拟中，很自然就可以想到用神经网络来逼近复杂的势能面，代替第一性原理计算。Deepmd-kit的输入为原子的坐标和元素种类，输出为体系的energy，每个原子的受力force和体系的virial force。其计算速度是第一性原理计算的1000倍以上，更重要的是Deepmd-kit的计算时间跟体系原子数成正比，而传统的第一性原理计算如DFT，其消耗的时间与体系原子数的三次方成正比，因此使用Deepmd-kit可以完成上亿原子的动力学模拟，而这么大的体系使用DFT甚至连算单点能都做不到。 \n", 394 | "此外，在上面的实验中可以观察到一个现象：模型在数据集的覆盖范围[0, 4]和[6, 8]区间上表现很好，而在没有覆盖到的区间如[4, 6]上表现就会下降。因此在训练模型时，一个重要的问题是如何产生一个分布足够均匀的数据集，使得模型在待研究的区间上都能保持良好的精度。Deepmd-kit有一个配套的生产数据的工具叫dpgen，可以自动地寻找模型没有覆盖到的区间来生成标注数据。之后的教程中将会介绍这些工具的原理和使用方法： \n", 395 | "deepmd-kit项目: https://github.com/deepmodeling/deepmd-kit \n", 396 | "dpgen项目: https://github.com/deepmodeling/dpgen" 397 | ] 398 | } 399 | ], 400 | "metadata": { 401 | "kernelspec": { 402 | "display_name": "Python 3.8.10 64-bit", 403 | "language": "python", 404 | "name": "python3" 405 | }, 406 | "language_info": { 407 | "codemirror_mode": { 408 | "name": "ipython", 409 | "version": 3 410 | }, 411 | "file_extension": ".py", 412 | "mimetype": "text/x-python", 413 | "name": "python", 414 | "nbconvert_exporter": "python", 415 | "pygments_lexer": "ipython3", 416 | "version": "3.8.10" 417 | }, 418 | "orig_nbformat": 4, 419 | "vscode": { 420 | "interpreter": { 421 | "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" 422 | } 423 | } 424 | }, 425 | "nbformat": 4, 426 | "nbformat_minor": 2 427 | } 428 | -------------------------------------------------------------------------------- /tutorial/machine_learning/mtters_needing_attention.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "不同数据集比例" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [] 14 | } 15 | ], 16 | "metadata": { 17 | "language_info": { 18 | "name": "python" 19 | }, 20 | "orig_nbformat": 4 21 | }, 22 | "nbformat": 4, 23 | "nbformat_minor": 2 24 | } 25 | -------------------------------------------------------------------------------- /tutorial/others/figs/optimization-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/tutorial/others/figs/optimization-1.png -------------------------------------------------------------------------------- /tutorial/others/figs/optimization-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/tutorial/others/figs/optimization-2.png -------------------------------------------------------------------------------- /tutorial/others/figs/optimization-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/tutorial/others/figs/optimization-3.png -------------------------------------------------------------------------------- /tutorial/others/figs/optimization-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/tutorial/others/figs/optimization-4.png -------------------------------------------------------------------------------- /tutorial/others/figs/optimization-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MabinogiX/dp-tutorial/8a2315ae1c26ba5a4022b2c8229839b524ee225e/tutorial/others/figs/optimization-5.png -------------------------------------------------------------------------------- /tutorial/others/optimization.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "在机器学习和第一性原理计算的原子/离子relaxation中都涉及到了优化问题，对于机器学习，目标是寻找一组参数使得模型能够完成给定的目标，如给出原子坐标和元素种类后，能得到体系的能量和原子受力。对于原子/离子的relaxation问题，则是优化给定体系中原子/例子的坐标，使得每个原子/离子的受力最小（此时体系能量最低）。\n", 8 | "\n", 9 | "前面这两个例子中都用到了梯度下降法，这篇文章将围绕梯度下降法讲述一些基础概念，这些概念对于机器学习，原子/离子relaxation都有帮助。 \n", 10 | "\n", 11 | "首先看一个最简单的例子：二次函数优化，给定函数$y = x^2$，求函数最小值。这个例子比较简单，可以令函数的导数为0得到答案。但对于复杂的优化问题，我们没有函数的公式，无法求解析解，只能用数值计算求数值解，梯度下降法就是一种常用的数值求解方法。\n", 12 | "\n", 13 | "对于$y = x^2$，可以得到每一点的梯度（导数）公式为：$y = 2x$，然后随便猜一个初值，比如假设x=3这个点y值最小。然后计算得到x=3处的梯度grad = 2 \\* 3 = 6，接下来用下面的公式来更新x的位置：\n", 14 | "$$x_n = x_{n-1} - lr * grad$$\n", 15 | "`lr`可以对应机器学习中的学习率(learning rate),当`lr`设置得到并且n足够大时，我们就能找到$y = x^2$的最小值。下面我们来实现一下这个算法：" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": 12, 21 | "metadata": {}, 22 | "outputs": [ 23 | { 24 | "data": { 25 | "text/plain": [ 26 | "" 27 | ] 28 | }, 29 | "execution_count": 12, 30 | "metadata": {}, 31 | "output_type": "execute_result" 32 | }, 33 | { 34 | "data": { 35 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAamklEQVR4nO3df3BVZ5kH8O+T8CvQQtCmuya0DTBSakuXmGzHLR11iNuibVMEtytWcQwFu6sWWE0LuqZpnEpdVErXHSuQuDLL0jLCxrS4UKV2lKrVhGCxpbCFWktSbawELIRf5dk/7g/Ovbk/zs295573Pef7memU3NyE96b3/fbN8z7nPaKqICIic5X4PQAiIsqMQU1EZDgGNRGR4RjURESGY1ATERluhBff9JJLLtHq6movvjURUSB1d3f/SVUrUn3Ok6Curq5GV1eXF9+aiCiQROSVdJ9j6YOIyHAMaiIiwzGoiYgMx6AmIjIcg5qIKF/JZyYV+AwlT7o+hqOjpxerdx5A38AgKsvL0HTTlZhbU+X3sIiIMmtpwaGDR7BwxgL0HTuFygljsHHfZkydNgloaSnIX2HEirqjpxcrt+1D78AgFEDvwCBWbtuHjp5ev4dGRJSeKg4dPIKpm9uwaOvDUFUs2vowpm5uw6GDRwq2sjZiRb165wEMnn0r4bHBs29h9c4DXFUTkblEsHDGAiw62I/G7k40dncCANprG9A2YwGeESnIX2PEirpvYDCnx4mITNF37BRa6xcnPNZavxh9x04V7O8wIqgry8tyepyIyBSVE8agedf6hMead61H5YQxBfs7jAjqppuuRNnI0oTHykaWoummK30aERGRC6rYuG8zGrs70V7bgOp7Hkd7bQMauzuxcd/mYNWoY3Vodn0QkVVEMHXaJBxasAhtMxZAjp1C2/y78b5pFZGujwLVqMWLeybW1dUpD2UiotBQTQzl5I9dEJFuVa1L9TkjVtTJ2FNNRKbLmFMFWknHGBfUsZ7qWLterKcaAMOaiIxQ7JwyYjPRKVNPNRGRCYqdU8YFNXuqich0xc4p44KaPdVEZLpi55RxQc2eaiIyXbFzyrjNRPZUE5Hpip1T7KMmIjKAdX3UTuypJiJT+JVHRgc1e6qJyBR+5pFxm4lO7KkmIlP4mUdGBzV7qonIFH7mkdFBzZ5qIjKFn3lkdFCzp5qITOFnHrnaTBSR5QDuBKAA9gH4lKoW7j4zabCnmohM4WceZe2jFpEqALsBvEtVB0VkC4Afqup/pvsar/qo2apHRMVWrNwpRB/1CABlInIWwFgAfYUanFts1SOiYjMld7LWqFW1F8DXAfwewGsAjqnqk8nPE5ElItIlIl39/f0FHyhb9Yio2EzJnaxBLSITAdwGYDKASgDjROTjyc9T1XWqWqeqdRUVFQUfKFv1iKjYTMkdN10fHwDwsqr2q+pZANsAXO/tsIZiqx4RFZspueMmqH8P4D0iMlZEBEA9gP3eDmsotuoRUbGZkjtZNxNV9VkR+T6APQDOAegBsM7rgSVjqx4RFZspucNjTomIDGD1MafpsKeaiLxiWr5YGdSm9DYSUfCYmC9Gn/WRjim9jUQUPCbmi5VBbUpvIxEFj4n5YmVQm9LbSETBY2K+WBnUpvQ2ElHwmJgvVm4mmtLbSETBY2K+BKKP2rRWGiKyj985Esg+6hgTW2mIyC6m54iVNWonE1tpiMgupueI9UFtYisNEdnF9ByxPqhNbKUhIruYniPWB7WJrTREZBfTc8T6zUQTW2mIyC6m50gg2vOc/G6xISJ7mJQXgW7PczK9xYaIzGFTXlhfo3YyvcWGiMxhU14EKqhNb7EhInPYlBeBCmrTW2yIyBw25UWggtr0FhsiModNeRGozUTTW2yIyBw25UXg2vOcTGq9ISIzmJoLoWnPc7Kp9YaIisPWXAhUjdrJptYbIioOW3MhsEFtU+sNERWHrbkQ2KC2qfWGiIrD1lwIbFDb1HpDRMVhay4EdjPRptYbIioOW3Mh0O15Tqa25BCR92yY/6Fsz3OytSWHiPIXhPkf2Bq1k60tOUSUvyDM/1AEta0tOUSUvyDM/1AEta0tOUSUvyDM/1AEta0tOUSUvyDM/1BsJtrakkNE+QvC/HfVnici5QA2ALgGgAJoVNVfpHu+ie15Tja06hDR8Nk4xwvRnrcWwA5V/YiIjAIwtmCjK7IgtOoQUXpBnONZa9QiMh7AewG0AYCqnlHVAY/H5ZkgtOoQUXpBnONuNhOnAOgH8F0R6RGRDSIyLvlJIrJERLpEpKu/v7/gAy2UILTqEFF6QZzjboJ6BIB3A/i2qtYAOAFgRfKTVHWdqtapal1FRUWBh1k4QWjVIaI0VBPnsmMPzuY57iaojwA4oqrPRj/+PiLBbaUgtOoQUQotLcDy5Wi6cVpkjquiedd6LNu9yfo5njWoVfUPAF4VkdirrAfwgqej8tDcmiqsmjcDVeVlEABV5WWYX1uF1TsPYPKK7Zj14FPo6On1e5hElAtVHDp4BFi7Fn9e8hmMLhU88NM2NHZ3olJPY9WHr7F2IxFw3543E5H2vFEADgP4lKoeTfd809vznJJ3iIHICnvVvBlW/4clCpOOnl6s3PocmnY8gsbuzvjjhxYswtRN6wERH0fnTqb2PFdXJqrq3mj9+VpVnZsppG0TxB1iorBZvfMABs+dR2v94oTHF85YYEVIZxOKS8gzCeIOMVHY9A0MxmvSTou2PpywoWir0Ac1u0CI7Fc5YQyad61HY3cn2msbUH3P42ivbYiUQZYvtz6sQx/U7AIhsl/TnOk4OfZitNc2RMofIlg95y4cWrAIKC+3vvwRmltxZeI8F2BC2UiIAAMnz1pzRgBRWA2Zu1AMDJ67MHdnVloT0qG/FVc2c2uqMLemKpBnBBAFVfJ8HRg8i7KRpVjzjzMDN19DX/pwYgcIkT3CNF8Z1A7sACGyR5jmK4PagR0gRPYI03xlUDuwA4TIHmGar9xMdEi+ZU+sA2T5Y3uxeucBdoAQGSC502PMyJLAd2kxqJOwA4TIXGHq9HBi6SONMO0oE9kirPOSQZ1GmHaUiWwR1nnJoE4jTDvKRLYI67xkUKcRph1lIluEdV5yMzENdoAQmSOMnR5ODOoM2AFC5L+wdno4sfThQlh3molMwPnHoHYlrDvNRCbg/GNQuxLWnWYiE3D+MahdCetOM5EJOP+4megKO0CIii/snR5ODGqX2AFCVDzs9EjE0keOuANN5D3Os0QM6hxxB5rIe5xniRjUOeIONJH3OM8SMahzxB1oIu9xniXiZmKO2AFC5B12eqTGoB4GdoAQFR47PdJj6SMP3JkmKhzOp/QY1HngzjRR4XA+pcegzgN3pokKh/MpPQZ1HlLtTAsitepZDz6Fjp5efwZGZJGOnl7MevAp9A4MQpI+F+ZODyduJubB2QESe5Np9HPcWCTKLnkDUYH4PKoKeaeHE1fUeZpbU4VnVsxGVXlZPKRjuBFClFmqDcRYSD+zYjZDOsp1UItIqYj0iMgTXg7IVtwIIcod5407uayolwLY79VAbMeNEKLccd644yqoRWQSgJsBbPB2OPbiJa9EueO8ccftZuJDAO4BcHG6J4jIEgBLAODyyy/Pe2C24aXlRO7xUvHciGryFljSE0RuAfAhVf1nEXk/gC+o6i2Zvqaurk67uroKNkjbJO9kA5FVwqp5M/gGpNDj/EhNRLpVtS7V59yUPmYBaBCR3wF4FMBsEfmvAo4vcHgpLFF6nB+5yxrUqrpSVSepajWAjwJ4SlU/7vnILMadbKIUor+9x+dB0m/znB/psY/aA9zJJkrS0gIsXw6oRuaBKpp3rcey3ZviT+H8SC+noFbVp7PVp4mXlhMlUAUGBoC1a7Hlho+g9+hJNO9aj8buTow/dQJQZadHFryE3AO8tJzIQQQdC7+A47/4HRb+fBtu//k2AEB7bQNa6xejauJYdnpkwdKHR3hpOdEFq588iOb335nwWCykeal4dgxqj3FjkQjoi5Y7nJp3rUff0ZM+jcguLH14rLK8DL0pQpkbJxQaqvja7u/i9u7OeLkjVqO+aPQIQG8GJPmAU3Liitpj3FikMOvo6cWsr/0EfTI6HtIQQWv9Ymy87jZce80VDGkXuKL2GDcWKaycVyA+dMMdgCpEJHKM6cSxGP/tb2H6uyf5PUwrcEVdBNxYpDAacgViLKRjZ00zpF1jUBcRNxYpTPh+LxwGdRHxikUKE77fC4dBXUTcWKQw4M1qC4+biUXEjUUKOt6s1htcURcZNxYpyHizWm8wqH3CjRYKIr6vvcGg9km6DRUFWK8m68Tq0unuF8UNxPwwqH2SamMxJlavZliTDWJ16VRHJQDcQCwEBrVP5tZUYdW8GahKs9JgvZpskaouHVNVXhb6eyEWArs+fDS3pgpza6owecX2lL8ysq5HNkj3PhUAz6yYXdzBBBRX1AbghQFkM75/vcegNgAvhCEb8cKW4mHpwwC8EIZswwtbiosrakPwQhiyCS9sKS4GtWF4wQDZgO/T4mLpwzDpbt1VIoLJK7ajkr9Wkk86enqxeucB9A0MokQEb+nQXiVuIHqDK2rDpLsQ5i1VKHgxDPnDeVGLAilDmhuI3mFQG8Z5IYwAKE1xPznWrKnY0l3UUioCAS9s8RpLHwaKXQgDAJNXbE/5HNYCqZjSvd/Oq+LlB28u8mjChytqw/HwJvJTx54jiYctJZU8WJMuDga14Xh4E/nlxU//C47/02fRe/Rk5AFVNO9aj2W7NwFgTbqYGNSG4+FN5AtVPPfbV7DwVz9A86718ZBu7O7E+FMnUDVhDGvSRcQatQV4eBMVnQjuveFTePP0OTR2d6KxuxMA0F7bgK/UL8bLK+t9HmC4cEVtER5+Q8VUOXEsWusXJzzWWr8YlRPH+jSi8GJQW4SHN1ExxA9bOnoyUvZwaH16A5punObTyMKLQW2R5Hp1qsObGNaUj/iFLdGQbuzuRHttA6rveRxbrp+Hhb/6AeZu/PqQ7g/yFoPaMjy8ibwUv7BFBMfHjEN7bQNa6xejauJY3L77+8DSpUB5OZDiQizyDjcTLZVuAzFWBuF5IJSL2DkeznNmHrrhjsjKWSTyfhMB1qxhSPsg64paRC4TkZ+IyH4ReV5ElhZjYJRZpg1ElkEoFxlvThsN5fj7jSHtCzelj3MAPq+qVwF4D4DPiMi7vB0WZZPpQhiAZRByL9PNaQFe2GKCrKUPVX0NwGvRP/9FRPYDqALwgsdjowyS7wqTCvuryY1M7xPercUMOW0mikg1gBoAz6b43BIR6RKRrv7+/gINjzJxbiymwvNAKJNYG166/g3ercUcroNaRC4CsBXAMlU9nvx5VV2nqnWqWldRUVHIMVIWPA+EcpWxLg2WO0zjKqhFZCQiIb1JVbd5OyTKFc8DoVxlqkvzbGnzZK1Ri4gAaAOwX1W/6f2QaDh4HgjlIt37QQA8s2J2cQdDWblZUc8C8AkAs0Vkb/SfD3k8Lhomnl9NmWSrS/PcGDO56frYjcj/aMkCTTddiZXb9qX8tTZWrwbAX2tDKFaXTlfyYF3aXLyEPGBYr6Z0WJe2F4M6gGJte+l+DeJpeyGieuE0vIHBlIcpxerSDGlz8ayPAKssL0vbfsUySAi0tODQwSNYOWU+Bs+dj9+l5fiYcZFzPKJYlzYfV9QBxsvMQ0wVGBjA1M1taNrxyJBbacVW1qxL24Er6gBzc5k5T9sLpo69fVj9V7diUe3LQ26l1Vq/GBDh5eEWEfXgAPC6ujrt6uoq+Pel4YvXKNMoG1nKzaSASOjuUMXv/u3W+Oeq73k8HtLslzaLiHSral2qz7H0ERIsg4RHvLsjWu5wat61HmUjSljusAyDOiSyte0B7AaxXXJ3R/KttNprG9DY3YknDm/F3JmVfg+XcsCgDpFsp+0BPMTJVkMOWUq6lRZE0Db/bmDpUkydNok3ALAMNxNDKNPVi8CFMgjr1fZIdTGL81ZaZSNL0TRnOjBzNkPaQlxRhxDLIMGRUO5IJbpxGN8oZkhbiSvqkIqdtpdpkvOiGLNlO7sDALs7AoIr6pBjN4i9eK/D8GBQhxzLIJZxcXYHwEOWgoalD2IZxBYuz+5guSN4uKKmOJZBDMazO0KNK2qK49kg5uLZHeHGsz4oJZ4NYg6e3REOPOuDcuamDLLssb3cZPRQbNNw2WN7eXZHyDGoKSU33SAALzn3ypBLwnl2R6ix9EFZZSuDxLBGmgfVSB165wH0DQyiBEByh/Sy3Zsw/tSJCzXpCWPwzB8fB8rLgZaW4o+ZCipT6YObiZRVtrNBYtjCN0wp2u6+lKLtjmd3hBdLH5SV2zIIEKldf37LbzB5xXbWr13o2HMEW360L2vbXRzP7ggllj4oJ27Ol3Bid0h68Z/lmXPxcI5xtt3F8GcZbJlKHwxqyllHT2/GXutUWL++IOXPL03bXakIzquikj+/wGONmgoqdsl5Lqvr0Nevo7Vl5yo6vlpO03a3es5dWDX/2nD+vCgBg5qGzXklY9/AIEpE8FaG39Bivderdx4I1+owulm4cMYC9B47lXhGx6yPJbTdtdYvjn/8vmkVmDpzjt+jJwNwM5HyEru918sP3oxv3P43GS+SiQlV77UqDh08gqmb27Bo68NDNwuBhFtmlY0agbet+w/eMosSsEZNBZVr/TpwtetoiQOI/ix2vIjegcHsm4WqqJo49sLPwvF9KBy4mUhFl0v9WgAoAhDaSSUOUcWXHSWOVJuFALs5KIJnfVDR5dJ7HVsq9A4MYvlje1FtQw+2Y4HT0dOLWV/9Mdqf6EkocXzZUeJo/vG6hC9v3rU+sormAf/kAlfU5Llce69jjF1ptrQAAwPAmjXo2NuHlVufQ9OOR3B89DiMP31iSIkDqmjc8/iQzcJDCxZh6qb1LHEQAK6oyWe5rK6dYl0iM+9/EjWtTxb/akfnIkYVHXuOYNaqXWh/ogdYuxabrp+HZY/2oGnHI5GV8+kTaJ19Z8K3aK1fjONlFyXUpNvm3x0JaW4WkktcUVNRDXd17eRJTTu2eRf79333AceOoeMTn8cb9/4r5OhRQCRSb75+AbZ/bxmufv1w/MvbaxvQOvtOND+1IfWmIYCyUSMu/IbAzUJKws1EMoqzMyQWusMV+/ryspEQAQZOns18Fd/589EvvNBtgfvuw6GX+vD0H89Ajh/Hwx/8NB7d8DlM/8NhPH/pFJwpGYGaPxwEkFjKcKpu6oyHdHKJo722AW3z70bTnOnmlXHIGHkHtYjMAbAWQCmADar6YKbnM6jJrUKGtlOqAN+65Yu4uu//8JfSUXj14gq8cNlVuHH/z/C2EwMoBXByxCiMPXcGb5SNx9sHj8f/ncrzl04ZsqI+Pnosxp8+ia/UL4ZGjyHduG9zpMTBY0gpi7yCWkRKARwE8PcAjgD4NYAFqvpCuq9hUNNwDOcMEdfOn8f27y3F1a+/XLBvmXLlPO9zaPrgVRdWzixxkEv5BvXfAWhR1ZuiH68EAFVdle5rGNSUj0LUsVMqYFg/f+kU3PzJh4CSEkAVrU9vwHUzp2D6d76Z/zgplPLt+qgC8Krj4yPRx5L/kiUi0iUiXf39/cMbKRESu0QEkfLFxLEjAURKGsNWUoKbP7k2py95/tIpaK+5Jf5xe20D2t99K65+/TAe+Fk7JHpF4fhvf4shTZ5xcyhTqrkxZBmuqusArAMiK+o8x0UhFzuhL1leNe3oijobZ4366tcP40zpCLTX3IKLykbh+muuwPRHvgEsX447ystxR8stWb8fUb7cBPURAJc5Pp4EoM+b4RBl5gzwWGj3DQxiQnTT8OjJs6kD3GXZ440xF+Ptg8fx4l9PwY7Jf4tKOYtrr7kCjY98I/KEWL15zRrWnqlo3AT1rwG8U0QmA+gF8FEAH/N0VEQuZFt1OwN84ORZnB53MU6PLhvS9VF+8jiOXjQRI664HG+fdytw7BimT5iA6fffn34zkCFNReS2Pe9DAB5CpD2vXVUfyPR8biaSsVL1UatGNgVjj7NTg3yQ9x1eVPWHAH5Y0FER+aEkaf+cq2WyAM/6ICIyHIOaiMhwDGoiIsMxqImIDOfJ6Xki0g/glWF++SUA/lTA4fgpKK8lKK8D4GsxUVBeB5Dfa7lCVStSfcKToM6HiHSla1GxTVBeS1BeB8DXYqKgvA7Au9fC0gcRkeEY1EREhjMxqNdlf4o1gvJagvI6AL4WEwXldQAevRbjatRERJTIxBU1ERE5MKiJiAxnZFCLyFdE5DkR2SsiT4pIpd9jGg4RWS0iL0Zfy/+ISLnfYxouEfkHEXleRM6LiHWtVCIyR0QOiMhLIrLC7/HkQ0TaReR1Efmt32PJh4hcJiI/EZH90fdW9rs6GEpExojIr0TkN9HXcn9Bv7+JNWoRGa+qx6N/vhvAu1T1Lp+HlTMRuRHAU6p6TkS+BgCqeq/PwxoWEbkKwHkA3wHwBVW15hzb4dyg2WQi8l4AbwLYqKrX+D2e4RKRdwB4h6ruEZGLAXQDmGvjfxcREQDjVPVNERkJYDeApar6y0J8fyNX1LGQjhqHHO+4ZApVfVJVz0U//CUid8exkqruV9UDfo9jmK4D8JKqHlbVMwAeBXCbz2MaNlX9KYA/+z2OfKnqa6q6J/rnvwDYjxT3Y7WBRrwZ/XBk9J+C5ZaRQQ0AIvKAiLwK4A4AzX6PpwAaAfyv34MIKVc3aCb/iEg1gBoAz/o8lGETkVIR2QvgdQA/UtWCvRbfglpEfiwiv03xz20AoKpfUtXLAGwC8Fm/xplNttcRfc6XAJxD5LUYy81rsZSrGzSTP0TkIgBbASxL+m3aKqr6lqrOROQ35+tEpGBlKVd3ePGCqn7A5VP/G8B2APd5OJxhy/Y6ROSTAG4BUK8mbgg45PDfxDa8QbOhovXcrQA2qeo2v8dTCKo6ICJPA5gDoCAbvkaWPkTknY4PGwC86NdY8iEicwDcC6BBVU/6PZ4Qi9+gWURGIXKD5k6fxxR60Q24NgD7VfWbfo8nHyJSEevqEpEyAB9AAXPL1K6PrQCuRKTL4BUAd6lqr7+jyp2IvARgNIA3og/90sbuFQAQkQ8D+HcAFQAGAOxV1Zt8HVQOcr1Bs8lEZDOA9yNypOYfAdynqm2+DmoYROQGAD8DsA+RuQ4AX4zeo9UqInItgO8h8v4qAbBFVVsL9v1NDGoiIrrAyNIHERFdwKAmIjIcg5qIyHAMaiIiwzGoiYgMx6AmIjIcg5qIyHD/D5UuUTXiL4jnAAAAAElFTkSuQmCC", 36 | "text/plain": [ 37 | "
" 38 | ] 39 | }, 40 | "metadata": { 41 | "needs_background": "light" 42 | }, 43 | "output_type": "display_data" 44 | } 45 | ], 46 | "source": [ 47 | "import matplotlib.pyplot as plt\n", 48 | "import numpy as np\n", 49 | "\n", 50 | "def func(x):\n", 51 | " return x * x\n", 52 | "def grad(x):\n", 53 | " return 2 * x\n", 54 | "\n", 55 | "x_guess = 3\n", 56 | "lr = 0.1\n", 57 | "threshold = 0.01\n", 58 | "\n", 59 | "def optimization(x_guess, lr, threshold):\n", 60 | " x = x_guess\n", 61 | " diff = 99999\n", 62 | " y = func(x_guess)\n", 63 | "\n", 64 | " xs = [x_guess]\n", 65 | " ys = [y]\n", 66 | " max_iter = 1000\n", 67 | " while diff > threshold and max_iter > 0:\n", 68 | " x = x - lr * grad(x)\n", 69 | " xs.append(x)\n", 70 | " max_iter -= 1\n", 71 | " if len(xs) > 1:\n", 72 | " # 当当前的x值与上一步x值的距离小于一定值时，认为已经找到了最小值\n", 73 | " diff = abs(xs[-1] - xs[-2])\n", 74 | "\n", 75 | " # 以下代码仅用于画图\n", 76 | " ys.append(func(x))\n", 77 | " return xs, ys\n", 78 | " \n", 79 | "xs, ys = optimization(x_guess, lr, threshold)\n", 80 | "# 以下代码仅用于迭代过程可视化\n", 81 | "x_arr = np.linspace(-3, 3, 100)\n", 82 | "y_arr = [func(x) for x in x_arr]\n", 83 | "plt.scatter(x_arr, y_arr)\n", 84 | "plt.scatter(xs, ys, c='r', marker='x')" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 26, 90 | "metadata": {}, 91 | "outputs": [ 92 | { 93 | "name": "stdout", 94 | "output_type": "stream", 95 | "text": [ 96 | "iteration number: 21\n" 97 | ] 98 | }, 99 | { 100 | "data": { 101 | "text/plain": [ 102 | "[0.007130534626283792,\n", 103 | " 0.004563542160821628,\n", 104 | " 0.0029206669829258416,\n", 105 | " 0.0018692268690725384,\n", 106 | " 0.0011963051962064245]" 107 | ] 108 | }, 109 | "execution_count": 26, 110 | "metadata": {}, 111 | "output_type": "execute_result" 112 | } 113 | ], 114 | "source": [ 115 | "print(\"iteration number: {}\".format(len(xs)))\n", 116 | "ys[-5:]" 117 | ] 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "metadata": {}, 122 | "source": [ 123 | "上面的代码中使用了$\\left|x_n - x_{n-1}\\right| < threshold$来作为迭代终止的条件(max_iter=1000作为辅助判断条件，防止死循环)，可以看到一共迭代了21步，最后5步的值已经非常接近0了，几乎就是正确答案。神经网络训练中最常用的SGD算法，VASP中IBRION=3(steepest descent algorithm, POTIM可以认为是lr)算法归根结底都是上面的梯度下降法。\n", 124 | "\n", 125 | "在神经网络的训练中，当loss不再降低时，认为迭代次数已经足够，模型训练完成；在原子/离子relaxation问题中，当当前步和上一步中energy或者原子受力force差值的绝对值小于一定值时，认为迭代次数已经足够，结构优化完成。" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "## 优化过程中的几个重要问题" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "### 初猜值问题\n", 140 | "\n", 141 | "数值优化方法（梯度下降法是其中一种方法）都需要一个初始猜测值，并且计算的结果和迭代的次数都依赖初猜值。在$y = x^2$这个问题中，x_guess = 3需要迭代21次，而x_guess = 1只需要迭代16次，好的初始猜测值能明显加速模型收敛。\n", 142 | "\n", 143 | "在神经网络中，模型优化对于初始猜测值不算太敏感，一般使用随机初始化即可（框架可自动完成）。而第一性原理计算对初始猜测值就比较敏感了，细心的同学应该能发现，vasp做结构优化，第一个离子步中电子步的迭代一般都需要几十步，之后电子步迭代次数会明显降低。这就是因为第一个离子步中电子结构初始猜测比较差，需要较多步数才能收敛（注意：电子步的优化方法不是梯度下降，数值优化方法都存在初猜的问题），后面每一个离子步中的电子结构初猜，都使用前一个离子步中已经收敛的电子结构，因此只需少量迭代就能收敛。Guassian的一大优势就是电子初猜很不错，导致需要的迭代次数少于其它同类型软件。\n", 144 | "\n", 145 | "对于原子/离子relaxation同样存在这个问题，不借助工具随手画一个分子结构，将会需要很多离子步才能得到最终构型。但如果先用分子力学MM优化一下构型，作为一个好的初始猜测，再跑第一性原理构型优化，所需的离子步将会显著减少。" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "### 局部最小值 vs 全局最小值\n", 153 | "数值优化方法（包括梯度下降法）一般只能找到局部最小值点，而很难或者几乎不可能找到全局最小值点，这导致不同的初猜值甚至可以得到完全不同的结果。\n", 154 | "考虑函数:\n", 155 | "$$y = x^4+x^3-7x^2-x+6$$\n", 156 | "其图像大致如下：\n", 157 | "\n", 158 | "![](figs/optimization-1.png)\n", 159 | "\n", 160 | "使用x_guess=-3.5和x_guess=0.5将会得到完全不同的结果，并且前者找到的是全局最小值，而后者只能找到局部最小值。\n", 161 | "\n", 162 | "对于神经网络来说，从实践来看，随机给一个初始猜测，最后训练的模型都差不多，没有显著区别。一种解释认为神经网络参数量太大（上亿参数），里面的极小值点效果都差不多，不一定非要找到全局最小值。\n", 163 | "\n", 164 | "对于第一性原理结构优化来说，不同的初始构型将会得到不同的结果。简单的问题如下图中的乙醇，O上的H原子可以旋转180°作为另一个初猜来做结构优化，最终可以得到两种不同构型的乙醇分子，并且两种结构的energy不一样。\n", 165 | "\n", 166 | "![](figs/optimization-2.png)\n", 167 | "\n", 168 | "复杂的问题例如表面催化反应，一般认为反应发生在缺陷处，计算时首先要构建不同的缺陷位点，然后找到表面能最低的那个构型再去计算化学反应。但缺陷位点构型不计其数，基本不可能找到最稳定的构型，只能靠人的经验判断，加上一些简化和假设来做计算。" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "### lr值设置\n", 176 | "lr全称learning rate，是个经验性很强的参数。\n", 177 | "\n", 178 | "考虑函数$y = \\left|0.01*x\\right|$，它的导数$y^{\\prime} = 0.01$非常小，如果初猜x_guess=10，那需要迭代1000次才能找到正确结果，增大learning rate能加速收敛。\n", 179 | "\n", 180 | "当learning rate过大时，比如设置为0.9，此时跑$y = \\left|x\\right|$的例子，就会发现迭代进入死循环，数值求解的结果会在正确结果附近反复震荡，永不收敛。" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": null, 186 | "metadata": {}, 187 | "outputs": [], 188 | "source": [ 189 | "def func(x):\n", 190 | " if x > 0: return x\n", 191 | " else: return -x\n", 192 | "def grad(x):\n", 193 | " if x > 0: return 1\n", 194 | " else: return -1\n", 195 | "lr = 0.9" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "![](figs/optimization-3.png)\n", 203 | "\n", 204 | "在神经网络的训练中，对于lr设置有大量技巧，一般在训练开始时需要warm-up,即把learning rate从0缓慢增加到指定值，然后在训练过程中逐渐减小learning rate帮助模型收敛。把learning rate的值在整个训练过程中的变化画出来大概是这样子： \n", 205 | "![](figs/optimization-4.png) \n", 206 | "设lr=0.1,在训练开始的0-3000步，lr缓慢增长到0.1，然后采用指数(exp)或者sin下降法不断降低lr值，直到训练结束。Deepmd的模型训练就采用了exp下降法，不过没有实现warm-up步骤。 \n", 207 | "另外一种值得一提的lr调节方法是余弦退火法，在训练过程中不断增大lr又再次降低，在一定程度上可以帮助模型跳出局部最小值区域，找到更好的极小值点： \n", 208 | "![](figs/optimization-5.png)\n", 209 | "\n", 210 | "总结一下，在神经网络模型训练初期，此时的初猜离真正的结果很远，需要较大的lr值来加速收敛（减小迭代次数），当模型接近真正的结果时，需要降低lr来减少震荡，帮助找到真正的结果。\n", 211 | "\n", 212 | "在VAPS的结构优化中，可以设置`IBRION=3`来启用梯度下降法，然后设置`POTIM`值来调节lr（在vasp中也称为步长），但`POTIM`是固定的，不能逐渐降低。官方文档推荐在构型比较糟糕的情况下，刚开始可以用梯度下降法快速迭代，之后需要换成`IBRION=1`准牛顿法（收敛速度最快）或者`IBRION=2`共轭梯度法（通用性最强）来找到精确解。 \n", 213 | "实际上，梯度下降法是一种收敛速度比较慢的算法，牛顿迭代法/准牛顿迭代法收敛速度远快于梯度下降法，但这两种方法代价很大，特别是牛顿迭代法需要求出体系的Hessian矩阵，对于结构优化只能退而求其次使用准牛顿法，而神经网络模型参数量经常几千万上亿，只能使用梯度下降法来优化。" 214 | ] 215 | } 216 | ], 217 | "metadata": { 218 | "kernelspec": { 219 | "display_name": "Python 3.9.12 ('base')", 220 | "language": "python", 221 | "name": "python3" 222 | }, 223 | "language_info": { 224 | "codemirror_mode": { 225 | "name": "ipython", 226 | "version": 3 227 | }, 228 | "file_extension": ".py", 229 | "mimetype": "text/x-python", 230 | "name": "python", 231 | "nbconvert_exporter": "python", 232 | "pygments_lexer": "ipython3", 233 | "version": "3.9.12" 234 | }, 235 | "orig_nbformat": 4, 236 | "vscode": { 237 | "interpreter": { 238 | "hash": "306f9674966356e6d2017fd9a046da687f90cc683eac906171c3efa96e5922a4" 239 | } 240 | } 241 | }, 242 | "nbformat": 4, 243 | "nbformat_minor": 2 244 | } 245 | --------------------------------------------------------------------------------