├── # B-AMA_protocol_TS_forecasting_ML.ipynb ├── LICENSE.md ├── LSTM_module.py ├── README.md ├── configuration_settings.txt ├── ddm_run.py ├── dependencies.txt ├── environment.yml ├── input ├── Nord │ └── ddm_input_Nord1_.csv ├── Rhine │ └── Rhine.csv ├── USA_GW │ └── groundwater.csv ├── USA_GW_deep │ └── GW_confined.csv └── USA_GW_north │ └── USA_GW_north.csv ├── output ├── Nord │ ├── Nord_0svm.csv │ ├── Nord_0svm.png │ ├── Nord_hyper_param.png │ ├── Nord_ivs_forward.png │ ├── Nord_scatter_svm.png │ └── results.pkl ├── Rhine │ ├── Rhine_0svm.csv │ ├── Rhine_0svm.png │ ├── Rhine_hyper_param.png │ ├── Rhine_ivs_forward.png │ ├── Rhine_scatter_svm.png │ └── results.pkl ├── USA_GW │ ├── USA_GW_0svm.csv │ ├── USA_GW_0svm.png │ ├── USA_GW_hyper_param.png │ ├── USA_GW_ivs_forward.png │ ├── USA_GW_scatter_svm.png │ └── results.pkl ├── USA_GW_deep │ ├── USA_GW_deep_0svm.csv │ ├── USA_GW_deep_0svm.png │ ├── USA_GW_deep_hyper_param.png │ ├── USA_GW_deep_ivs_forward.png │ ├── USA_GW_deep_scatter_svm.png │ └── results.pkl ├── USA_GW_north │ ├── USA_GW_north_0svm.csv │ ├── USA_GW_north_0svm.png │ ├── USA_GW_north_hyper_param.png │ ├── USA_GW_north_ivs_forward.png │ ├── USA_GW_north_scatter_svm.png │ └── results.pkl └── readme.txt └── protocol ├── LSTM_module_config.txt ├── __pycache__ ├── data_division.cpython-38.pyc ├── data_transformation.cpython-38.pyc ├── ivs.cpython-38.pyc ├── model_testing.cpython-38.pyc ├── model_training.cpython-38.pyc ├── postprocess.cpython-38.pyc └── utils.cpython-38.pyc ├── advanced_configurations.txt ├── data_division.py ├── data_transformation.py ├── ivs.py ├── model_testing.py ├── model_training.py ├── postprocess.py └── utils.py /LICENSE.md: -------------------------------------------------------------------------------- 1 | GNU General Public License 2 | ========================== 3 | 4 | _Version 3, 29 June 2007_ 5 | _Copyright © 2007 Free Software Foundation, Inc. <>_ 6 | 7 | Everyone is permitted to copy and distribute verbatim copies of this license 8 | document, but changing it is not allowed. 9 | 10 | ## Preamble 11 | 12 | The GNU General Public License is a free, copyleft license for software and other 13 | kinds of works. 14 | 15 | The licenses for most software and other practical works are designed to take away 16 | your freedom to share and change the works. By contrast, the GNU General Public 17 | License is intended to guarantee your freedom to share and change all versions of a 18 | program--to make sure it remains free software for all its users. We, the Free 19 | Software Foundation, use the GNU General Public License for most of our software; it 20 | applies also to any other work released this way by its authors. You can apply it to 21 | your programs, too. 22 | 23 | When we speak of free software, we are referring to freedom, not price. Our General 24 | Public Licenses are designed to make sure that you have the freedom to distribute 25 | copies of free software (and charge for them if you wish), that you receive source 26 | code or can get it if you want it, that you can change the software or use pieces of 27 | it in new free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you these rights or 30 | asking you to surrender the rights. Therefore, you have certain responsibilities if 31 | you distribute copies of the software, or if you modify it: responsibilities to 32 | respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether gratis or for a fee, 35 | you must pass on to the recipients the same freedoms that you received. You must make 36 | sure that they, too, receive or can get the source code. And you must show them these 37 | terms so they know their rights. 38 | 39 | Developers that use the GNU GPL protect your rights with two steps: **(1)** assert 40 | copyright on the software, and **(2)** offer you this License giving you legal permission 41 | to copy, distribute and/or modify it. 42 | 43 | For the developers' and authors' protection, the GPL clearly explains that there is 44 | no warranty for this free software. For both users' and authors' sake, the GPL 45 | requires that modified versions be marked as changed, so that their problems will not 46 | be attributed erroneously to authors of previous versions. 47 | 48 | Some devices are designed to deny users access to install or run modified versions of 49 | the software inside them, although the manufacturer can do so. This is fundamentally 50 | incompatible with the aim of protecting users' freedom to change the software. The 51 | systematic pattern of such abuse occurs in the area of products for individuals to 52 | use, which is precisely where it is most unacceptable. Therefore, we have designed 53 | this version of the GPL to prohibit the practice for those products. If such problems 54 | arise substantially in other domains, we stand ready to extend this provision to 55 | those domains in future versions of the GPL, as needed to protect the freedom of 56 | users. 57 | 58 | Finally, every program is threatened constantly by software patents. States should 59 | not allow patents to restrict development and use of software on general-purpose 60 | computers, but in those that do, we wish to avoid the special danger that patents 61 | applied to a free program could make it effectively proprietary. To prevent this, the 62 | GPL assures that patents cannot be used to render the program non-free. 63 | 64 | The precise terms and conditions for copying, distribution and modification follow. 65 | 66 | ## TERMS AND CONDITIONS 67 | 68 | ### 0. Definitions 69 | 70 | “This License” refers to version 3 of the GNU General Public License. 71 | 72 | “Copyright” also means copyright-like laws that apply to other kinds of 73 | works, such as semiconductor masks. 74 | 75 | “The Program” refers to any copyrightable work licensed under this 76 | License. Each licensee is addressed as “you”. “Licensees” and 77 | “recipients” may be individuals or organizations. 78 | 79 | To “modify” a work means to copy from or adapt all or part of the work in 80 | a fashion requiring copyright permission, other than the making of an exact copy. The 81 | resulting work is called a “modified version” of the earlier work or a 82 | work “based on” the earlier work. 83 | 84 | A “covered work” means either the unmodified Program or a work based on 85 | the Program. 86 | 87 | To “propagate” a work means to do anything with it that, without 88 | permission, would make you directly or secondarily liable for infringement under 89 | applicable copyright law, except executing it on a computer or modifying a private 90 | copy. Propagation includes copying, distribution (with or without modification), 91 | making available to the public, and in some countries other activities as well. 92 | 93 | To “convey” a work means any kind of propagation that enables other 94 | parties to make or receive copies. Mere interaction with a user through a computer 95 | network, with no transfer of a copy, is not conveying. 96 | 97 | An interactive user interface displays “Appropriate Legal Notices” to the 98 | extent that it includes a convenient and prominently visible feature that **(1)** 99 | displays an appropriate copyright notice, and **(2)** tells the user that there is no 100 | warranty for the work (except to the extent that warranties are provided), that 101 | licensees may convey the work under this License, and how to view a copy of this 102 | License. If the interface presents a list of user commands or options, such as a 103 | menu, a prominent item in the list meets this criterion. 104 | 105 | ### 1. Source Code 106 | 107 | The “source code” for a work means the preferred form of the work for 108 | making modifications to it. “Object code” means any non-source form of a 109 | work. 110 | 111 | A “Standard Interface” means an interface that either is an official 112 | standard defined by a recognized standards body, or, in the case of interfaces 113 | specified for a particular programming language, one that is widely used among 114 | developers working in that language. 115 | 116 | The “System Libraries” of an executable work include anything, other than 117 | the work as a whole, that **(a)** is included in the normal form of packaging a Major 118 | Component, but which is not part of that Major Component, and **(b)** serves only to 119 | enable use of the work with that Major Component, or to implement a Standard 120 | Interface for which an implementation is available to the public in source code form. 121 | A “Major Component”, in this context, means a major essential component 122 | (kernel, window system, and so on) of the specific operating system (if any) on which 123 | the executable work runs, or a compiler used to produce the work, or an object code 124 | interpreter used to run it. 125 | 126 | The “Corresponding Source” for a work in object code form means all the 127 | source code needed to generate, install, and (for an executable work) run the object 128 | code and to modify the work, including scripts to control those activities. However, 129 | it does not include the work's System Libraries, or general-purpose tools or 130 | generally available free programs which are used unmodified in performing those 131 | activities but which are not part of the work. For example, Corresponding Source 132 | includes interface definition files associated with source files for the work, and 133 | the source code for shared libraries and dynamically linked subprograms that the work 134 | is specifically designed to require, such as by intimate data communication or 135 | control flow between those subprograms and other parts of the work. 136 | 137 | The Corresponding Source need not include anything that users can regenerate 138 | automatically from other parts of the Corresponding Source. 139 | 140 | The Corresponding Source for a work in source code form is that same work. 141 | 142 | ### 2. Basic Permissions 143 | 144 | All rights granted under this License are granted for the term of copyright on the 145 | Program, and are irrevocable provided the stated conditions are met. This License 146 | explicitly affirms your unlimited permission to run the unmodified Program. The 147 | output from running a covered work is covered by this License only if the output, 148 | given its content, constitutes a covered work. This License acknowledges your rights 149 | of fair use or other equivalent, as provided by copyright law. 150 | 151 | You may make, run and propagate covered works that you do not convey, without 152 | conditions so long as your license otherwise remains in force. You may convey covered 153 | works to others for the sole purpose of having them make modifications exclusively 154 | for you, or provide you with facilities for running those works, provided that you 155 | comply with the terms of this License in conveying all material for which you do not 156 | control copyright. Those thus making or running the covered works for you must do so 157 | exclusively on your behalf, under your direction and control, on terms that prohibit 158 | them from making any copies of your copyrighted material outside their relationship 159 | with you. 160 | 161 | Conveying under any other circumstances is permitted solely under the conditions 162 | stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 163 | 164 | ### 3. Protecting Users' Legal Rights From Anti-Circumvention Law 165 | 166 | No covered work shall be deemed part of an effective technological measure under any 167 | applicable law fulfilling obligations under article 11 of the WIPO copyright treaty 168 | adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention 169 | of such measures. 170 | 171 | When you convey a covered work, you waive any legal power to forbid circumvention of 172 | technological measures to the extent such circumvention is effected by exercising 173 | rights under this License with respect to the covered work, and you disclaim any 174 | intention to limit operation or modification of the work as a means of enforcing, 175 | against the work's users, your or third parties' legal rights to forbid circumvention 176 | of technological measures. 177 | 178 | ### 4. Conveying Verbatim Copies 179 | 180 | You may convey verbatim copies of the Program's source code as you receive it, in any 181 | medium, provided that you conspicuously and appropriately publish on each copy an 182 | appropriate copyright notice; keep intact all notices stating that this License and 183 | any non-permissive terms added in accord with section 7 apply to the code; keep 184 | intact all notices of the absence of any warranty; and give all recipients a copy of 185 | this License along with the Program. 186 | 187 | You may charge any price or no price for each copy that you convey, and you may offer 188 | support or warranty protection for a fee. 189 | 190 | ### 5. Conveying Modified Source Versions 191 | 192 | You may convey a work based on the Program, or the modifications to produce it from 193 | the Program, in the form of source code under the terms of section 4, provided that 194 | you also meet all of these conditions: 195 | 196 | * **a)** The work must carry prominent notices stating that you modified it, and giving a 197 | relevant date. 198 | * **b)** The work must carry prominent notices stating that it is released under this 199 | License and any conditions added under section 7. This requirement modifies the 200 | requirement in section 4 to “keep intact all notices”. 201 | * **c)** You must license the entire work, as a whole, under this License to anyone who 202 | comes into possession of a copy. This License will therefore apply, along with any 203 | applicable section 7 additional terms, to the whole of the work, and all its parts, 204 | regardless of how they are packaged. This License gives no permission to license the 205 | work in any other way, but it does not invalidate such permission if you have 206 | separately received it. 207 | * **d)** If the work has interactive user interfaces, each must display Appropriate Legal 208 | Notices; however, if the Program has interactive interfaces that do not display 209 | Appropriate Legal Notices, your work need not make them do so. 210 | 211 | A compilation of a covered work with other separate and independent works, which are 212 | not by their nature extensions of the covered work, and which are not combined with 213 | it such as to form a larger program, in or on a volume of a storage or distribution 214 | medium, is called an “aggregate” if the compilation and its resulting 215 | copyright are not used to limit the access or legal rights of the compilation's users 216 | beyond what the individual works permit. Inclusion of a covered work in an aggregate 217 | does not cause this License to apply to the other parts of the aggregate. 218 | 219 | ### 6. Conveying Non-Source Forms 220 | 221 | You may convey a covered work in object code form under the terms of sections 4 and 222 | 5, provided that you also convey the machine-readable Corresponding Source under the 223 | terms of this License, in one of these ways: 224 | 225 | * **a)** Convey the object code in, or embodied in, a physical product (including a 226 | physical distribution medium), accompanied by the Corresponding Source fixed on a 227 | durable physical medium customarily used for software interchange. 228 | * **b)** Convey the object code in, or embodied in, a physical product (including a 229 | physical distribution medium), accompanied by a written offer, valid for at least 230 | three years and valid for as long as you offer spare parts or customer support for 231 | that product model, to give anyone who possesses the object code either **(1)** a copy of 232 | the Corresponding Source for all the software in the product that is covered by this 233 | License, on a durable physical medium customarily used for software interchange, for 234 | a price no more than your reasonable cost of physically performing this conveying of 235 | source, or **(2)** access to copy the Corresponding Source from a network server at no 236 | charge. 237 | * **c)** Convey individual copies of the object code with a copy of the written offer to 238 | provide the Corresponding Source. This alternative is allowed only occasionally and 239 | noncommercially, and only if you received the object code with such an offer, in 240 | accord with subsection 6b. 241 | * **d)** Convey the object code by offering access from a designated place (gratis or for 242 | a charge), and offer equivalent access to the Corresponding Source in the same way 243 | through the same place at no further charge. You need not require recipients to copy 244 | the Corresponding Source along with the object code. If the place to copy the object 245 | code is a network server, the Corresponding Source may be on a different server 246 | (operated by you or a third party) that supports equivalent copying facilities, 247 | provided you maintain clear directions next to the object code saying where to find 248 | the Corresponding Source. Regardless of what server hosts the Corresponding Source, 249 | you remain obligated to ensure that it is available for as long as needed to satisfy 250 | these requirements. 251 | * **e)** Convey the object code using peer-to-peer transmission, provided you inform 252 | other peers where the object code and Corresponding Source of the work are being 253 | offered to the general public at no charge under subsection 6d. 254 | 255 | A separable portion of the object code, whose source code is excluded from the 256 | Corresponding Source as a System Library, need not be included in conveying the 257 | object code work. 258 | 259 | A “User Product” is either **(1)** a “consumer product”, which 260 | means any tangible personal property which is normally used for personal, family, or 261 | household purposes, or **(2)** anything designed or sold for incorporation into a 262 | dwelling. In determining whether a product is a consumer product, doubtful cases 263 | shall be resolved in favor of coverage. For a particular product received by a 264 | particular user, “normally used” refers to a typical or common use of 265 | that class of product, regardless of the status of the particular user or of the way 266 | in which the particular user actually uses, or expects or is expected to use, the 267 | product. A product is a consumer product regardless of whether the product has 268 | substantial commercial, industrial or non-consumer uses, unless such uses represent 269 | the only significant mode of use of the product. 270 | 271 | “Installation Information” for a User Product means any methods, 272 | procedures, authorization keys, or other information required to install and execute 273 | modified versions of a covered work in that User Product from a modified version of 274 | its Corresponding Source. The information must suffice to ensure that the continued 275 | functioning of the modified object code is in no case prevented or interfered with 276 | solely because modification has been made. 277 | 278 | If you convey an object code work under this section in, or with, or specifically for 279 | use in, a User Product, and the conveying occurs as part of a transaction in which 280 | the right of possession and use of the User Product is transferred to the recipient 281 | in perpetuity or for a fixed term (regardless of how the transaction is 282 | characterized), the Corresponding Source conveyed under this section must be 283 | accompanied by the Installation Information. But this requirement does not apply if 284 | neither you nor any third party retains the ability to install modified object code 285 | on the User Product (for example, the work has been installed in ROM). 286 | 287 | The requirement to provide Installation Information does not include a requirement to 288 | continue to provide support service, warranty, or updates for a work that has been 289 | modified or installed by the recipient, or for the User Product in which it has been 290 | modified or installed. Access to a network may be denied when the modification itself 291 | materially and adversely affects the operation of the network or violates the rules 292 | and protocols for communication across the network. 293 | 294 | Corresponding Source conveyed, and Installation Information provided, in accord with 295 | this section must be in a format that is publicly documented (and with an 296 | implementation available to the public in source code form), and must require no 297 | special password or key for unpacking, reading or copying. 298 | 299 | ### 7. Additional Terms 300 | 301 | “Additional permissions” are terms that supplement the terms of this 302 | License by making exceptions from one or more of its conditions. Additional 303 | permissions that are applicable to the entire Program shall be treated as though they 304 | were included in this License, to the extent that they are valid under applicable 305 | law. If additional permissions apply only to part of the Program, that part may be 306 | used separately under those permissions, but the entire Program remains governed by 307 | this License without regard to the additional permissions. 308 | 309 | When you convey a copy of a covered work, you may at your option remove any 310 | additional permissions from that copy, or from any part of it. (Additional 311 | permissions may be written to require their own removal in certain cases when you 312 | modify the work.) You may place additional permissions on material, added by you to a 313 | covered work, for which you have or can give appropriate copyright permission. 314 | 315 | Notwithstanding any other provision of this License, for material you add to a 316 | covered work, you may (if authorized by the copyright holders of that material) 317 | supplement the terms of this License with terms: 318 | 319 | * **a)** Disclaiming warranty or limiting liability differently from the terms of 320 | sections 15 and 16 of this License; or 321 | * **b)** Requiring preservation of specified reasonable legal notices or author 322 | attributions in that material or in the Appropriate Legal Notices displayed by works 323 | containing it; or 324 | * **c)** Prohibiting misrepresentation of the origin of that material, or requiring that 325 | modified versions of such material be marked in reasonable ways as different from the 326 | original version; or 327 | * **d)** Limiting the use for publicity purposes of names of licensors or authors of the 328 | material; or 329 | * **e)** Declining to grant rights under trademark law for use of some trade names, 330 | trademarks, or service marks; or 331 | * **f)** Requiring indemnification of licensors and authors of that material by anyone 332 | who conveys the material (or modified versions of it) with contractual assumptions of 333 | liability to the recipient, for any liability that these contractual assumptions 334 | directly impose on those licensors and authors. 335 | 336 | All other non-permissive additional terms are considered “further 337 | restrictions” within the meaning of section 10. If the Program as you received 338 | it, or any part of it, contains a notice stating that it is governed by this License 339 | along with a term that is a further restriction, you may remove that term. If a 340 | license document contains a further restriction but permits relicensing or conveying 341 | under this License, you may add to a covered work material governed by the terms of 342 | that license document, provided that the further restriction does not survive such 343 | relicensing or conveying. 344 | 345 | If you add terms to a covered work in accord with this section, you must place, in 346 | the relevant source files, a statement of the additional terms that apply to those 347 | files, or a notice indicating where to find the applicable terms. 348 | 349 | Additional terms, permissive or non-permissive, may be stated in the form of a 350 | separately written license, or stated as exceptions; the above requirements apply 351 | either way. 352 | 353 | ### 8. Termination 354 | 355 | You may not propagate or modify a covered work except as expressly provided under 356 | this License. Any attempt otherwise to propagate or modify it is void, and will 357 | automatically terminate your rights under this License (including any patent licenses 358 | granted under the third paragraph of section 11). 359 | 360 | However, if you cease all violation of this License, then your license from a 361 | particular copyright holder is reinstated **(a)** provisionally, unless and until the 362 | copyright holder explicitly and finally terminates your license, and **(b)** permanently, 363 | if the copyright holder fails to notify you of the violation by some reasonable means 364 | prior to 60 days after the cessation. 365 | 366 | Moreover, your license from a particular copyright holder is reinstated permanently 367 | if the copyright holder notifies you of the violation by some reasonable means, this 368 | is the first time you have received notice of violation of this License (for any 369 | work) from that copyright holder, and you cure the violation prior to 30 days after 370 | your receipt of the notice. 371 | 372 | Termination of your rights under this section does not terminate the licenses of 373 | parties who have received copies or rights from you under this License. If your 374 | rights have been terminated and not permanently reinstated, you do not qualify to 375 | receive new licenses for the same material under section 10. 376 | 377 | ### 9. Acceptance Not Required for Having Copies 378 | 379 | You are not required to accept this License in order to receive or run a copy of the 380 | Program. Ancillary propagation of a covered work occurring solely as a consequence of 381 | using peer-to-peer transmission to receive a copy likewise does not require 382 | acceptance. However, nothing other than this License grants you permission to 383 | propagate or modify any covered work. These actions infringe copyright if you do not 384 | accept this License. Therefore, by modifying or propagating a covered work, you 385 | indicate your acceptance of this License to do so. 386 | 387 | ### 10. Automatic Licensing of Downstream Recipients 388 | 389 | Each time you convey a covered work, the recipient automatically receives a license 390 | from the original licensors, to run, modify and propagate that work, subject to this 391 | License. You are not responsible for enforcing compliance by third parties with this 392 | License. 393 | 394 | An “entity transaction” is a transaction transferring control of an 395 | organization, or substantially all assets of one, or subdividing an organization, or 396 | merging organizations. If propagation of a covered work results from an entity 397 | transaction, each party to that transaction who receives a copy of the work also 398 | receives whatever licenses to the work the party's predecessor in interest had or 399 | could give under the previous paragraph, plus a right to possession of the 400 | Corresponding Source of the work from the predecessor in interest, if the predecessor 401 | has it or can get it with reasonable efforts. 402 | 403 | You may not impose any further restrictions on the exercise of the rights granted or 404 | affirmed under this License. For example, you may not impose a license fee, royalty, 405 | or other charge for exercise of rights granted under this License, and you may not 406 | initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging 407 | that any patent claim is infringed by making, using, selling, offering for sale, or 408 | importing the Program or any portion of it. 409 | 410 | ### 11. Patents 411 | 412 | A “contributor” is a copyright holder who authorizes use under this 413 | License of the Program or a work on which the Program is based. The work thus 414 | licensed is called the contributor's “contributor version”. 415 | 416 | A contributor's “essential patent claims” are all patent claims owned or 417 | controlled by the contributor, whether already acquired or hereafter acquired, that 418 | would be infringed by some manner, permitted by this License, of making, using, or 419 | selling its contributor version, but do not include claims that would be infringed 420 | only as a consequence of further modification of the contributor version. For 421 | purposes of this definition, “control” includes the right to grant patent 422 | sublicenses in a manner consistent with the requirements of this License. 423 | 424 | Each contributor grants you a non-exclusive, worldwide, royalty-free patent license 425 | under the contributor's essential patent claims, to make, use, sell, offer for sale, 426 | import and otherwise run, modify and propagate the contents of its contributor 427 | version. 428 | 429 | In the following three paragraphs, a “patent license” is any express 430 | agreement or commitment, however denominated, not to enforce a patent (such as an 431 | express permission to practice a patent or covenant not to sue for patent 432 | infringement). To “grant” such a patent license to a party means to make 433 | such an agreement or commitment not to enforce a patent against the party. 434 | 435 | If you convey a covered work, knowingly relying on a patent license, and the 436 | Corresponding Source of the work is not available for anyone to copy, free of charge 437 | and under the terms of this License, through a publicly available network server or 438 | other readily accessible means, then you must either **(1)** cause the Corresponding 439 | Source to be so available, or **(2)** arrange to deprive yourself of the benefit of the 440 | patent license for this particular work, or **(3)** arrange, in a manner consistent with 441 | the requirements of this License, to extend the patent license to downstream 442 | recipients. “Knowingly relying” means you have actual knowledge that, but 443 | for the patent license, your conveying the covered work in a country, or your 444 | recipient's use of the covered work in a country, would infringe one or more 445 | identifiable patents in that country that you have reason to believe are valid. 446 | 447 | If, pursuant to or in connection with a single transaction or arrangement, you 448 | convey, or propagate by procuring conveyance of, a covered work, and grant a patent 449 | license to some of the parties receiving the covered work authorizing them to use, 450 | propagate, modify or convey a specific copy of the covered work, then the patent 451 | license you grant is automatically extended to all recipients of the covered work and 452 | works based on it. 453 | 454 | A patent license is “discriminatory” if it does not include within the 455 | scope of its coverage, prohibits the exercise of, or is conditioned on the 456 | non-exercise of one or more of the rights that are specifically granted under this 457 | License. You may not convey a covered work if you are a party to an arrangement with 458 | a third party that is in the business of distributing software, under which you make 459 | payment to the third party based on the extent of your activity of conveying the 460 | work, and under which the third party grants, to any of the parties who would receive 461 | the covered work from you, a discriminatory patent license **(a)** in connection with 462 | copies of the covered work conveyed by you (or copies made from those copies), or **(b)** 463 | primarily for and in connection with specific products or compilations that contain 464 | the covered work, unless you entered into that arrangement, or that patent license 465 | was granted, prior to 28 March 2007. 466 | 467 | Nothing in this License shall be construed as excluding or limiting any implied 468 | license or other defenses to infringement that may otherwise be available to you 469 | under applicable patent law. 470 | 471 | ### 12. No Surrender of Others' Freedom 472 | 473 | If conditions are imposed on you (whether by court order, agreement or otherwise) 474 | that contradict the conditions of this License, they do not excuse you from the 475 | conditions of this License. If you cannot convey a covered work so as to satisfy 476 | simultaneously your obligations under this License and any other pertinent 477 | obligations, then as a consequence you may not convey it at all. For example, if you 478 | agree to terms that obligate you to collect a royalty for further conveying from 479 | those to whom you convey the Program, the only way you could satisfy both those terms 480 | and this License would be to refrain entirely from conveying the Program. 481 | 482 | ### 13. Use with the GNU Affero General Public License 483 | 484 | Notwithstanding any other provision of this License, you have permission to link or 485 | combine any covered work with a work licensed under version 3 of the GNU Affero 486 | General Public License into a single combined work, and to convey the resulting work. 487 | The terms of this License will continue to apply to the part which is the covered 488 | work, but the special requirements of the GNU Affero General Public License, section 489 | 13, concerning interaction through a network will apply to the combination as such. 490 | 491 | ### 14. Revised Versions of this License 492 | 493 | The Free Software Foundation may publish revised and/or new versions of the GNU 494 | General Public License from time to time. Such new versions will be similar in spirit 495 | to the present version, but may differ in detail to address new problems or concerns. 496 | 497 | Each version is given a distinguishing version number. If the Program specifies that 498 | a certain numbered version of the GNU General Public License “or any later 499 | version” applies to it, you have the option of following the terms and 500 | conditions either of that numbered version or of any later version published by the 501 | Free Software Foundation. If the Program does not specify a version number of the GNU 502 | General Public License, you may choose any version ever published by the Free 503 | Software Foundation. 504 | 505 | If the Program specifies that a proxy can decide which future versions of the GNU 506 | General Public License can be used, that proxy's public statement of acceptance of a 507 | version permanently authorizes you to choose that version for the Program. 508 | 509 | Later license versions may give you additional or different permissions. However, no 510 | additional obligations are imposed on any author or copyright holder as a result of 511 | your choosing to follow a later version. 512 | 513 | ### 15. Disclaimer of Warranty 514 | 515 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. 516 | EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 517 | PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER 518 | EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 519 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE 520 | QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE 521 | DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 522 | 523 | ### 16. Limitation of Liability 524 | 525 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY 526 | COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS 527 | PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, 528 | INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE 529 | PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE 530 | OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE 531 | WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 532 | POSSIBILITY OF SUCH DAMAGES. 533 | 534 | ### 17. Interpretation of Sections 15 and 16 535 | 536 | If the disclaimer of warranty and limitation of liability provided above cannot be 537 | given local legal effect according to their terms, reviewing courts shall apply local 538 | law that most closely approximates an absolute waiver of all civil liability in 539 | connection with the Program, unless a warranty or assumption of liability accompanies 540 | a copy of the Program in return for a fee. 541 | 542 | _END OF TERMS AND CONDITIONS_ 543 | 544 | ## How to Apply These Terms to Your New Programs 545 | 546 | If you develop a new program, and you want it to be of the greatest possible use to 547 | the public, the best way to achieve this is to make it free software which everyone 548 | can redistribute and change under these terms. 549 | 550 | To do so, attach the following notices to the program. It is safest to attach them 551 | to the start of each source file to most effectively state the exclusion of warranty; 552 | and each file should have at least the “copyright” line and a pointer to 553 | where the full notice is found. 554 | 555 | 556 | Copyright (C) 557 | 558 | This program is free software: you can redistribute it and/or modify 559 | it under the terms of the GNU General Public License as published by 560 | the Free Software Foundation, either version 3 of the License, or 561 | (at your option) any later version. 562 | 563 | This program is distributed in the hope that it will be useful, 564 | but WITHOUT ANY WARRANTY; without even the implied warranty of 565 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 566 | GNU General Public License for more details. 567 | 568 | You should have received a copy of the GNU General Public License 569 | along with this program. If not, see . 570 | 571 | Also add information on how to contact you by electronic and paper mail. 572 | 573 | If the program does terminal interaction, make it output a short notice like this 574 | when it starts in an interactive mode: 575 | 576 | Copyright (C) 577 | This program comes with ABSOLUTELY NO WARRANTY; for details type 'show w'. 578 | This is free software, and you are welcome to redistribute it 579 | under certain conditions; type 'show c' for details. 580 | 581 | The hypothetical commands `show w` and `show c` should show the appropriate parts of 582 | the General Public License. Of course, your program's commands might be different; 583 | for a GUI interface, you would use an “about box”. 584 | 585 | You should also get your employer (if you work as a programmer) or school, if any, to 586 | sign a “copyright disclaimer” for the program, if necessary. For more 587 | information on this, and how to apply and follow the GNU GPL, see 588 | <>. 589 | 590 | The GNU General Public License does not permit incorporating your program into 591 | proprietary programs. If your program is a subroutine library, you may consider it 592 | more useful to permit linking proprietary applications with the library. If this is 593 | what you want to do, use the GNU Lesser General Public License instead of this 594 | License. But first, please read 595 | <>. -------------------------------------------------------------------------------- /LSTM_module.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Fri Oct 21 16:42:20 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | # Import the required libraries 9 | from keras.models import Sequential 10 | from keras.layers import Dense 11 | from keras.layers import LSTM 12 | import numpy as np 13 | 14 | 15 | def train_module(x_tr, x_cv, y_tr, hp): 16 | 17 | """ 18 | train_module(x_tr, x_cv, y_tr, hp) 19 | 20 | training function for an additional model module in the B-AMA protocol 21 | 22 | Input: 23 | - x_tr = normalized training set input [n_training_instances, n_features] 24 | - x_cv = normalized cross validation set input [n_cv_instances, n_features] 25 | - y_tr = dependent variable in the training set [n_training_instances] 26 | - hp = hyper-parameters values [problem dimensionality] 27 | 28 | 29 | Returns: 30 | - new_model = the model 31 | - yh = prediction on the cross-validation set [n_cv_instances] 32 | """ 33 | 34 | # Add parameters for dataset shape 35 | n_vars = 6 # Update according to case study 36 | n_lags = 3 # Update according to case study 37 | 38 | 39 | # Reshaping the data in the LSTM input format (n_instances, 1, n_features) 40 | x_tr = np.reshape(x_tr, newshape = (-1, n_lags, n_vars), order = 'F') 41 | x_cv = np.reshape(x_cv, newshape = (-1, n_lags, n_vars), order = 'F') 42 | 43 | # Define the new model 44 | new_model = Sequential() 45 | new_model.add(LSTM(int(hp[0]), input_shape=(x_tr.shape[1], x_tr.shape[2]))) 46 | new_model.add(Dense(1)) 47 | new_model.compile(loss='mae', optimizer='adam') 48 | # Fit the model 49 | new_model.fit(x_tr, y_tr, epochs=int(hp[1])) 50 | 51 | # Predict 52 | yh = new_model.predict(x_cv) 53 | return(new_model, yh) 54 | 55 | def test_module(m, x): 56 | 57 | """ 58 | test_module(m, x): 59 | 60 | test function for an additional model module in the B-AMA protocol 61 | 62 | Input: 63 | - m = the model 64 | - x = normalized test set input [n_test_instances, n_features] 65 | 66 | 67 | Returns: 68 | - y = normalized test set output [n_test_instances, ] 69 | """ 70 | # Add parameters for dataset shape 71 | n_vars = 6 # Update according to case study 72 | n_lags = 3 # Update according to case study 73 | 74 | # Reshaping the data in the LSTM input format (n_instances, 1, n_features) 75 | x = np.reshape(x, newshape = (-1, n_lags, n_vars), order = 'F') 76 | 77 | # Perform the forecast 78 | y = m.predict(x) 79 | 80 | # Reshape the data in (n_instances, ) 81 | y = y.reshape((y.shape[0], )) 82 | 83 | return(y) 84 | 85 | 86 | 87 | 88 | 89 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # B-AMA 2 | 3 | B-AMA (Basic dAta-driven Models for All) is an easy, flexible, and fully coded Python protocol for applying data-driven models (DDM) in hydrological problems. 4 | 5 | ## Overview 6 | 7 | B-AMA simplifies the process of applying data-driven models to hydrological problems, regardless of the input data type or case study. The protocol consists of the following straightforward steps: 8 | 9 | 1. **Installation**: Start by installing all the required libraries using the provided `environment.yml` file. Run the following command in your terminal: 10 | ``` 11 | conda env create -f environment.yml 12 | conda activate B_AMA 13 | ``` 14 | This will create a Conda environment and activate it. 15 | 16 | 2. **Loading Data**: Place your input data in the `input/case_study` folder. Ensure that the dependent variable is in the last column of your CSV file. 17 | 18 | 3. **Define Settings**: Specify the settings in the configuration file. Required information includes the case study name, the periodicity of the variable to be forecasted, the initial and final years with observations, and the modeling techniques to be used. You can specify your own configurations in protocol/advanced_configurations.txt 19 | 20 | 4. **Run the Protocol**: Open the Anaconda Prompt terminal, navigate to the B-AMA folder, and run the following command: 21 | ``` 22 | python ddm_run.py 23 | ``` 24 | Alternatively, you can run the `ddm_run.py` script directly from Spyder. 25 | 26 | ### The Jupyter Notebook 27 | 28 | To run the Jupyter Notebook, follow steps one and two described above. You can then simply open B-AMA_protocol_TS_forecasting_ML.ipynb and execute the protocol step by step. 29 | 30 | 31 | ## Additional Information 32 | - **Manuscript**: You can find more information about the methodology, validation, and applications of the software in the following publication: 33 | - [B-AMA: A Python-coded protocol to enhance the application of data-driven models in hydrology](https://www.sciencedirect.com/science/article/pii/S1364815222003097) 34 | - **Input Data Requirements**: Ensure that your input data is formatted correctly and meets the requirements specified in the documentation. 35 | - **Configuration Settings**: Consult the documentation for guidance on configuring the settings in the configuration file. 36 | - **Examples and Use Cases**: Explore examples and use cases to see how B-AMA can be applied in various scenarios. 37 | - **Contributing and Support**: We welcome contributions from the community. If you encounter any issues or have suggestions for improvement, please open an issue on GitHub. 38 | 39 | ## License 40 | 41 | This project is licensed under the [GNU General Public License version 3](LICENSE). 42 | -------------------------------------------------------------------------------- /configuration_settings.txt: -------------------------------------------------------------------------------- 1 | [Case_study] 2 | name = Rhine 3 | 4 | [Data_prop] 5 | period = 365 6 | start = 2008 7 | end = 2014 8 | 9 | [ddm] 10 | model = svm 11 | 12 | -------------------------------------------------------------------------------- /ddm_run.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon Oct 18 10:29:34 2021 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import configparser 10 | import dill 11 | import os 12 | import protocol.ivs as ivs 13 | import protocol.data_division as dd 14 | import protocol.data_transformation as dt 15 | import protocol.model_training as mt 16 | import protocol.model_testing as mte 17 | import protocol.utils as ut 18 | import protocol.postprocess as pp 19 | 20 | class B_AMA(): 21 | 22 | """ 23 | B_AMA employes all the fundamental blocks to develop data driven models for 24 | hydrological time-series. 25 | In particular: 26 | - Data division and transformation; 27 | - Input variable selection 28 | - Model calibration and k-fold cross-validation, 29 | with optimization of model architecture and hyper-parameters 30 | - Model testing and performance assessment 31 | - Visual analytics for results presentation 32 | """ 33 | 34 | 35 | def __init__(self): 36 | 37 | 38 | # Import configuration settings file 39 | Config = configparser.ConfigParser() 40 | 41 | Config.read('configuration_settings.txt') 42 | 43 | # Case study 44 | self.case_study = str(Config.get('Case_study','name')) 45 | 46 | # Input periodicity 47 | self.period = int(Config.get('Data_prop','period')) 48 | self.start = int(Config.get('Data_prop','start')) 49 | self.end = int(Config.get('Data_prop','end')) 50 | 51 | # Model type 52 | self.model = str(Config.get('ddm','model')) 53 | 54 | 55 | def protocol_run(self): 56 | 57 | """ 58 | protocol_run(self) 59 | 60 | Develops the step for implementing the DDM. 61 | 62 | Input: 63 | self: 64 | - case_study: name of the input folder where data are stored 65 | - period: system periodicity 66 | - start: initial year 67 | - end: final year 68 | - model: ddm to be trained 69 | 70 | Returns: 71 | results: 72 | - eps_c = Nash-Sutcliffe efficiency index in the training set 73 | - eps_v = Nash-Sutcliffe efficiency index in the test set 74 | - columns_selected = input variables selected in the final model 75 | architecture 76 | - ms = the models 77 | """ 78 | 79 | # Initialize the 'result' class 80 | class Object(object): 81 | pass 82 | 83 | results = Object() 84 | 85 | # Import data 86 | X = ut.read_model_data(self.case_study, self.period) 87 | 88 | # Allocate memory for the results 89 | eps_v = np.empty(X.shape[0]) # validation error 90 | eps_c = np.empty(X.shape[0]) # calibration error 91 | column_index = np.ndarray(shape = (X.shape[0], ), dtype = 'object') # variable selected 92 | ms = np.ndarray(shape = (X.shape[0], ), dtype = 'object') # model optimal architectures 93 | 94 | # Protocol start: iterate along the number of time series for which the forecast is necessary 95 | for i in range(0, X.shape[0]): 96 | 97 | xi = X[i, 0] 98 | 99 | # Data division 100 | si, n_v = dd.data_division().optimal_split(xi, self.period, i) 101 | c, v = dd.data_division().split(xi, si, self.period, n_v) 102 | 103 | # Data normalization 104 | cn, vn, mn, mX = dt.data_transformation().transform_data(c, v, 105 | self.period) 106 | 107 | # Input variable selection 108 | column_index[i] = ivs.input_variable_selection().select_input(cn, self.case_study) 109 | 110 | # Training and testing 111 | ms[i] = mt.model_training().train_model(cn, column_index[i], 112 | self.model, self.case_study) 113 | 114 | yc_rec, yv_rec, eps_c[i], eps_v[i], res = mte.model_testing().test_model(ms[i], 115 | cn, 116 | vn, 117 | column_index[i], 118 | mn, 119 | mX, 120 | self.model, 121 | self.period) 122 | 123 | # Save the forecasts and plot the results 124 | yr, yo, vs, cs = pp.postprocess().save_forecasts(yc_rec, yv_rec, c, v, 125 | si, i, n_v, self.period, 126 | self.model, self.case_study) 127 | 128 | pp.postprocess().plot_forecasts(yr, yo, vs, cs, i, self.period, self.model, 129 | self.case_study, 130 | self.start, 131 | self.end) 132 | 133 | # Fill the 'results' class 134 | results.calibration_error = eps_c 135 | results.validation_error = eps_v 136 | results.columns_selected = column_index 137 | results.models = ms 138 | 139 | try: 140 | # Save as pkl element 141 | fpt = os.path.join('output', self.case_study, 'results.pkl') 142 | with open(fpt, 'wb') as f: 143 | dill.dump(results, f) 144 | except: 145 | print('The selected module does not allow to save as pkl') 146 | 147 | # Return 148 | return(results) 149 | 150 | results = B_AMA().protocol_run() 151 | 152 | 153 | 154 | -------------------------------------------------------------------------------- /dependencies.txt: -------------------------------------------------------------------------------- 1 | dependencies: 2 | 3 | dill >= 0.3.4 4 | hydroeval >=0.1.0 5 | matplotlib >= 3.5.1 6 | numpy >= 1.20.3 7 | pandas >= 1.4.2 8 | scipy >= 1.8.0 9 | seaborn >= 0.11.2 10 | Sklearn >= 1.0.2 11 | -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: B_AMA 2 | channels: 3 | - defaults 4 | dependencies: 5 | - python=3.8 6 | - pip 7 | - jupyter 8 | - ipykernel 9 | - nbconvert 10 | - nbformat 11 | - notebook 12 | - jupyterlab 13 | - dill 14 | - matplotlib 15 | - numpy 16 | - pandas 17 | - scipy 18 | - seaborn 19 | - scikit-learn 20 | - keras 21 | - tensorflow 22 | - pip: 23 | - hydroeval 24 | -------------------------------------------------------------------------------- /input/Nord/ddm_input_Nord1_.csv: -------------------------------------------------------------------------------- 1 | 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,out 2 | 0.0,1.0,0.08362369337979095,0.5675087108013938,0.058801775147928996,1.7614644970414202,2.402644230769231,3.8185096153846154,5.414314516129032,5.766129032258065,2.3502976190476192,3.720833333333333,4.853365384615385,11.995192307692308,2.5271745365148313,4.604939582751856,2369.593 3 | 0.49999999999999994,0.8660254037844387,0.23192508710801393,0.08362369337979095,0.11020710059171597,0.058801775147928996,1.3882211538461537,2.402644230769231,4.219254032258065,5.414314516129032,1.7988095238095239,2.3502976190476192,3.3834134615384617,4.853365384615385,1.8553050598586556,2.5271745365148313,1376.543 4 | 0.8660254037844386,0.5000000000000001,23.581881533101047,0.23192508710801393,7.412352071005917,0.11020710059171597,9.555889423076923,1.3882211538461537,40.73689516129032,4.219254032258065,23.717559523809523,1.7988095238095239,54.60456730769231,3.3834134615384617,26.601524169996008,1.8553050598586556,1501.676 5 | 1.0,6.123233995736766e-17,53.34494773519164,23.581881533101047,29.326183431952664,7.412352071005917,58.54266826923077,9.555889423076923,105.47857862903226,40.73689516129032,71.78065476190476,23.717559523809523,120.79927884615384,54.60456730769231,73.21205194557764,26.601524169996008,2507.669 6 | 0.8660254037844387,-0.4999999999999998,111.13044425087108,53.34494773519164,106.42603550295858,29.326183431952664,123.90384615384616,58.54266826923077,81.40625,105.47857862903226,67.35803571428572,71.78065476190476,150.95673076923077,120.79927884615384,106.86355706519873,73.21205194557764,5104.599 7 | 0.49999999999999994,-0.8660254037844387,24.544425087108014,111.13044425087108,82.14090236686391,106.42603550295858,51.35757211538461,123.90384615384616,39.98185483870968,81.40625,49.7327380952381,67.35803571428572,94.41947115384616,150.95673076923077,57.02949394285841,106.86355706519873,5315.85 8 | 1.2246467991473532e-16,-1.0,1.0119773519163764,24.544425087108014,5.044748520710059,82.14090236686391,2.5522836538461537,51.35757211538461,34.99369959677419,39.98185483870968,8.590178571428572,49.7327380952381,0.19591346153846154,94.41947115384616,8.73146685936897,57.02949394285841,4076.267 9 | -0.4999999999999997,-0.8660254037844388,0.8412456445993032,1.0119773519163764,1.356139053254438,5.044748520710059,7.117788461538462,2.5522836538461537,33.28654233870968,34.99369959677419,29.031845238095237,8.590178571428572,40.10817307692308,0.19591346153846154,18.6236223021867,8.73146685936897,3450.045 10 | -0.8660254037844385,-0.5000000000000004,3.7504355400696863,0.8412456445993032,16.30103550295858,1.356139053254438,58.41646634615385,7.117788461538462,12.14289314516129,33.28654233870968,16.785416666666666,29.031845238095237,28.731971153846153,40.10817307692308,22.688036392476036,18.6236223021867,3338.644 11 | -1.0,-1.8369701987210297e-16,13.403745644599303,3.7504355400696863,32.78698224852071,16.30103550295858,90.71754807692308,58.41646634615385,51.18245967741935,12.14289314516129,30.663690476190474,16.785416666666666,62.18990384615385,28.731971153846153,46.8240549949678,22.688036392476036,3248.723 12 | -0.8660254037844386,0.5000000000000001,6.932926829268292,13.403745644599303,21.93380177514793,32.78698224852071,16.653245192307693,90.71754807692308,9.46320564516129,51.18245967741935,17.79434523809524,30.663690476190474,20.614182692307693,62.18990384615385,15.565284562048022,46.8240549949678,1874.929 13 | -0.5000000000000004,0.8660254037844384,0.2815766550522648,6.932926829268292,4.4733727810650885,21.93380177514793,3.6334134615384617,16.653245192307693,0.9329637096774194,9.46320564516129,1.9657738095238095,17.79434523809524,1.1947115384615385,20.614182692307693,2.0803019925530974,15.565284562048022,1141.105 14 | 0.0,1.0,0.0195993031358885,0.2815766550522648,0.03106508875739645,4.4733727810650885,1.3780048076923077,3.6334134615384617,0.19884072580645162,0.9329637096774194,0.31339285714285714,1.9657738095238095,0.06370192307692307,1.1947115384615385,0.3341007842686374,2.0803019925530974,1247.513 15 | 0.49999999999999994,0.8660254037844387,1.0886324041811846,0.0195993031358885,2.3221153846153846,0.03106508875739645,9.393629807692308,1.3780048076923077,10.767137096774194,0.19884072580645162,7.6068452380952385,0.31339285714285714,8.679086538461538,0.06370192307692307,6.642907744969975,0.3341007842686374,1206.871 16 | 0.8660254037844386,0.5000000000000001,8.835801393728223,1.0886324041811846,3.4212278106508878,2.3221153846153846,5.843149038461538,9.393629807692308,7.08820564516129,10.767137096774194,3.969940476190476,7.6068452380952385,3.527644230769231,8.679086538461538,5.447661432493607,6.642907744969975,1396.829 17 | 1.0,6.123233995736766e-17,42.94163763066202,8.835801393728223,24.273668639053255,3.4212278106508878,58.81129807692308,5.843149038461538,56.611139112903224,7.08820564516129,44.149702380952384,3.969940476190476,62.5625,3.527644230769231,48.224990973415665,5.447661432493607,2846.671 18 | 0.8660254037844387,-0.4999999999999998,86.147212543554,42.94163763066202,74.54474852071006,24.273668639053255,116.48677884615384,58.81129807692308,125.70791330645162,56.611139112903224,112.47321428571429,44.149702380952384,169.12379807692307,62.5625,114.08061092991782,48.224990973415665,3316.569 19 | 0.49999999999999994,-0.8660254037844387,58.792682926829265,86.147212543554,80.61205621301775,74.54474852071006,98.44651442307692,116.48677884615384,28.734627016129032,125.70791330645162,33.15625,112.47321428571429,41.78725961538461,169.12379807692307,56.92156503240627,114.08061092991782,6051.007 20 | 1.2246467991473532e-16,-1.0,2.960801393728223,58.792682926829265,18.22448224852071,80.61205621301775,47.48497596153846,98.44651442307692,33.26965725806452,28.734627016129032,24.038690476190474,33.15625,6.0048076923076925,41.78725961538461,21.997235838391678,56.92156503240627,4541.5 21 | -0.4999999999999997,-0.8660254037844388,9.325566202090592,2.960801393728223,11.79326923076923,18.22448224852071,8.77704326923077,47.48497596153846,15.194556451612904,33.26965725806452,4.1363095238095235,24.038690476190474,0.16826923076923078,6.0048076923076925,8.232502318047041,21.997235838391678,3659.365 22 | -0.8660254037844385,-0.5000000000000004,0.21994773519163763,9.325566202090592,0.823594674556213,11.79326923076923,0.4074519230769231,8.77704326923077,6.738911290322581,15.194556451612904,3.2669642857142858,4.1363095238095235,0.4230769230769231,0.16826923076923078,1.9799911386564275,8.232502318047041,2307.476 23 | -1.0,-1.8369701987210297e-16,0.8741289198606271,0.21994773519163763,11.32655325443787,0.823594674556213,29.623197115384617,0.4074519230769231,0.8127520161290323,6.738911290322581,1.2532738095238096,3.2669642857142858,2.574519230769231,0.4230769230769231,7.744070724350865,1.9799911386564275,1939.106 24 | -0.8660254037844386,0.5000000000000001,1.2489111498257839,0.8741289198606271,6.282174556213017,11.32655325443787,18.745192307692307,29.623197115384617,15.549899193548388,0.8127520161290323,3.392261904761905,1.2532738095238096,67.15264423076923,2.574519230769231,18.728513890468438,7.744070724350865,2310.024 25 | -0.5000000000000004,0.8660254037844384,0.09864982578397212,1.2489111498257839,0.08431952662721894,6.282174556213017,6.197115384615385,18.745192307692307,3.579133064516129,15.549899193548388,1.0833333333333333,3.392261904761905,23.810096153846153,67.15264423076923,5.808774548120366,18.728513890468438,1523.27 26 | 0.0,1.0,0.0006533101045296168,0.09864982578397212,0.0,0.08431952662721894,0.019831730769230768,6.197115384615385,0.20337701612903225,3.579133064516129,0.22142857142857142,1.0833333333333333,0.10336538461538461,23.810096153846153,0.09144266884112477,5.808774548120366,1311.241 27 | 0.49999999999999994,0.8660254037844387,0.4769163763066202,0.0006533101045296168,1.676775147928994,0.0,7.845552884615385,0.019831730769230768,4.540070564516129,0.20337701612903225,4.214880952380953,0.22142857142857142,5.117788461538462,0.10336538461538461,3.9786640645477562,0.09144266884112477,966.462 28 | 0.8660254037844386,0.5000000000000001,16.021341463414632,0.4769163763066202,14.988165680473372,1.676775147928994,35.93689903846154,7.845552884615385,58.78528225806452,4.540070564516129,44.475,4.214880952380953,93.56610576923077,5.117788461538462,43.96213236827415,3.9786640645477562,1362.048 29 | 1.0,6.123233995736766e-17,3.6077961672473866,16.021341463414632,7.2670118343195265,14.988165680473372,29.745192307692307,35.93689903846154,21.186491935483872,58.78528225806452,13.692857142857143,44.475,29.716346153846153,93.56610576923077,17.53594925690773,43.96213236827415,2096.605 30 | 0.8660254037844387,-0.4999999999999998,47.31946864111498,3.6077961672473866,76.60318047337279,7.2670118343195265,130.72536057692307,29.745192307692307,83.4765625,21.186491935483872,72.66964285714286,13.692857142857143,164.0841346153846,29.716346153846153,95.81305827732307,17.53594925690773,3715.413 31 | 0.49999999999999994,-0.8660254037844387,19.18466898954704,47.31946864111498,71.0525147928994,76.60318047337279,108.34194711538461,130.72536057692307,95.27671370967742,83.4765625,69.60922619047619,72.66964285714286,110.09975961538461,164.0841346153846,78.92747173556154,95.81305827732307,5145.394 32 | 1.2246467991473532e-16,-1.0,7.435540069686411,19.18466898954704,0.5225591715976331,71.0525147928994,1.4158653846153846,108.34194711538461,9.813004032258064,95.27671370967742,5.036607142857143,69.60922619047619,0.25240384615384615,110.09975961538461,4.0793299411947475,78.92747173556154,4056.158 33 | -0.4999999999999997,-0.8660254037844388,7.395252613240418,7.435540069686411,2.444896449704142,0.5225591715976331,8.63641826923077,1.4158653846153846,22.120211693548388,9.813004032258064,16.831845238095237,5.036607142857143,0.421875,0.25240384615384615,9.64174987730316,4.0793299411947475,3343.051 34 | -0.8660254037844385,-0.5000000000000004,43.70383275261324,7.395252613240418,15.988165680473372,2.444896449704142,39.26021634615385,8.63641826923077,0.9198588709677419,22.120211693548388,1.494345238095238,16.831845238095237,0.29086538461538464,0.421875,16.942880712153137,9.64174987730316,2987.733 35 | -1.0,-1.8369701987210297e-16,3.150261324041812,43.70383275261324,4.231508875739645,15.988165680473372,3.3545673076923075,39.26021634615385,0.16683467741935484,0.9198588709677419,0.4895833333333333,1.494345238095238,0.015625,0.29086538461538464,1.9013967530377418,16.942880712153137,1631.942 36 | -0.8660254037844386,0.5000000000000001,1.9409843205574913,3.150261324041812,0.7511094674556213,4.231508875739645,2.215144230769231,3.3545673076923075,0.3785282258064516,0.16683467741935484,0.3145833333333333,0.4895833333333333,0.09975961538461539,0.015625,0.9500181988844573,1.9013967530377418,1395.302 37 | -0.5000000000000004,0.8660254037844384,0.03397212543554007,1.9409843205574913,0.0,0.7511094674556213,3.120793269230769,2.215144230769231,0.24596774193548387,0.3785282258064516,0.13273809523809524,0.3145833333333333,0.04447115384615385,0.09975961538461539,0.5963237309476738,0.9500181988844573,1306.68 38 | 0.0,1.0,0.07970383275261324,0.03397212543554007,0.46597633136094674,0.0,10.485576923076923,3.120793269230769,20.606602822580644,0.24596774193548387,13.032142857142857,0.13273809523809524,27.25721153846154,0.04447115384615385,11.987869050895918,0.5963237309476738,1443.012 39 | 0.49999999999999994,0.8660254037844387,0.003048780487804878,0.07970383275261324,0.0,0.46597633136094674,1.1105769230769231,10.485576923076923,0.4558971774193548,20.606602822580644,0.3705357142857143,13.032142857142857,0.3605769230769231,27.25721153846154,0.38343925305778664,11.987869050895918,1169.639 40 | 0.8660254037844386,0.5000000000000001,1.4104965156794425,0.003048780487804878,2.555103550295858,0.0,15.270432692307692,1.1105769230769231,12.09274193548387,0.4558971774193548,9.176488095238096,0.3705357142857143,12.792067307692308,0.3605769230769231,8.882888349449544,0.38343925305778664,1450.439 41 | 1.0,6.123233995736766e-17,128.73867595818814,1.4104965156794425,138.99519230769232,2.555103550295858,249.20793269230768,15.270432692307692,219.12726814516128,12.09274193548387,204.87291666666667,9.176488095238096,230.9375,12.792067307692308,195.31324762833603,8.882888349449544,3730.459 42 | 0.8660254037844387,-0.4999999999999998,45.497168989547035,128.73867595818814,71.67899408284023,138.99519230769232,77.84134615384616,249.20793269230768,145.75579637096774,219.12726814516128,145.6779761904762,204.87291666666667,222.10096153846155,230.9375,118.09204055435647,195.31324762833603,6373.905 43 | 0.49999999999999994,-0.8660254037844387,6.741724738675958,45.497168989547035,24.13424556213018,71.67899408284023,31.353365384615383,77.84134615384616,24.421622983870968,145.75579637096774,46.607440476190476,145.6779761904762,65.78004807692308,222.10096153846155,33.17307453706767,118.09204055435647,5707.595 44 | 1.2246467991473532e-16,-1.0,4.180095818815331,6.741724738675958,15.808801775147929,24.13424556213018,41.51502403846154,31.353365384615383,21.322832661290324,24.421622983870968,21.40327380952381,46.607440476190476,4.961538461538462,65.78004807692308,18.1985944274629,33.17307453706767,4268.991 45 | -0.4999999999999997,-0.8660254037844388,1.824259581881533,4.180095818815331,25.44859467455621,15.808801775147929,48.57271634615385,41.51502403846154,17.311743951612904,21.322832661290324,5.096130952380952,21.40327380952381,0.7379807692307693,4.961538461538462,16.49857104596937,18.1985944274629,3569.001 46 | -0.8660254037844385,-0.5000000000000004,0.31903310104529614,1.824259581881533,0.3805473372781065,25.44859467455621,0.28305288461538464,48.57271634615385,4.389364919354839,17.311743951612904,1.2342261904761904,5.096130952380952,0.15745192307692307,0.7379807692307693,1.1272793926411235,16.49857104596937,2559.936 47 | -1.0,-1.8369701987210297e-16,11.062717770034844,0.31903310104529614,79.34578402366864,0.3805473372781065,134.55168269230768,0.28305288461538464,36.035786290322584,4.389364919354839,6.9035714285714285,1.2342261904761904,124.33052884615384,0.15745192307692307,65.37167850850983,1.1272793926411235,2241.658 48 | -0.8660254037844386,0.5000000000000001,0.9455574912891986,11.062717770034844,8.584689349112425,79.34578402366864,24.920072115384617,134.55168269230768,35.23160282258065,36.035786290322584,19.366964285714285,6.9035714285714285,54.96153846153846,124.33052884615384,24.001737420936607,65.37167850850983,3392.373 49 | -0.5000000000000004,0.8660254037844384,0.1914198606271777,0.9455574912891986,0.33986686390532544,8.584689349112425,0.6364182692307693,24.920072115384617,4.0703125,35.23160282258065,3.3443452380952383,19.366964285714285,0.30288461538461536,54.96153846153846,1.480874557873854,24.001737420936607,2115.618 50 | 0.0,1.0,0.013066202090592335,0.1914198606271777,0.0003698224852071006,0.33986686390532544,0.28064903846153844,0.6364182692307693,0.19909274193548387,4.0703125,0.3711309523809524,3.3443452380952383,0.06490384615384616,0.30288461538461536,0.15486876725127005,1.480874557873854,1136.502 51 | 0.49999999999999994,0.8660254037844387,2.8449477351916377,0.013066202090592335,7.916050295857988,0.0003698224852071006,21.87139423076923,0.28064903846153844,11.064516129032258,0.19909274193548387,13.496130952380952,0.3711309523809524,11.138221153846153,0.06490384615384616,11.388543416179703,0.15486876725127005,1345.274 52 | 0.8660254037844386,0.5000000000000001,15.96058362369338,2.8449477351916377,1.3369082840236686,7.916050295857988,8.685697115384615,21.87139423076923,3.7406754032258065,11.064516129032258,7.479166666666667,13.496130952380952,4.405048076923077,11.138221153846153,6.93467986165287,11.388543416179703,1285.847 53 | 1.0,6.123233995736766e-17,46.95492160278746,15.96058362369338,42.27773668639053,1.3369082840236686,108.76742788461539,8.685697115384615,39.770161290322584,3.7406754032258065,32.41517857142857,7.479166666666667,69.61778846153847,4.405048076923077,56.63386908284716,6.93467986165287,2434.775 54 | 0.8660254037844387,-0.4999999999999998,60.69773519163763,46.95492160278746,66.67159763313609,42.27773668639053,136.30889423076923,108.76742788461539,70.14692540322581,39.770161290322584,65.01130952380953,32.41517857142857,110.109375,69.61778846153847,84.82430616376304,56.63386908284716,4041.861 55 | 0.49999999999999994,-0.8660254037844387,75.52025261324042,60.69773519163763,136.5569526627219,66.67159763313609,91.4501201923077,136.30889423076923,114.66028225806451,70.14692540322581,171.4622023809524,65.01130952380953,249.91947115384616,110.109375,139.9282135435222,84.82430616376304,6262.292 56 | 1.2246467991473532e-16,-1.0,0.8288327526132404,75.52025261324042,2.676775147928994,136.5569526627219,10.946514423076923,91.4501201923077,14.788306451612904,114.66028225806451,8.60952380952381,171.4622023809524,9.301682692307692,249.91947115384616,7.858605879510594,139.9282135435222,5187.456 57 | -0.4999999999999997,-0.8660254037844388,7.285932055749129,0.8288327526132404,2.496671597633136,2.676775147928994,9.291466346153847,10.946514423076923,11.42741935483871,14.788306451612904,3.0672619047619047,8.60952380952381,0.4495192307692308,9.301682692307692,5.66971174831766,7.858605879510594,3894.826 58 | -0.8660254037844385,-0.5000000000000004,5.55095818815331,7.285932055749129,8.678254437869823,2.496671597633136,23.41346153846154,9.291466346153847,1.1232358870967742,11.42741935483871,0.26785714285714285,3.0672619047619047,0.17307692307692307,0.4495192307692308,6.534474019585919,5.66971174831766,2747.761 59 | -1.0,-1.8369701987210297e-16,22.73649825783972,5.55095818815331,24.75998520710059,8.678254437869823,77.83713942307692,23.41346153846154,31.110887096774192,1.1232358870967742,21.927678571428572,0.26785714285714285,112.0829326923077,0.17307692307692307,48.40918687475462,6.534474019585919,2614.677 60 | -0.8660254037844386,0.5000000000000001,8.054224738675957,22.73649825783972,9.69896449704142,24.75998520710059,40.09194711538461,77.83713942307692,33.114163306451616,31.110887096774192,17.05029761904762,21.927678571428572,55.41346153846154,112.0829326923077,27.23717646917713,48.40918687475462,3593.629 61 | -0.5000000000000004,0.8660254037844384,0.2077526132404181,8.054224738675957,1.6460798816568047,9.69896449704142,16.342548076923077,40.09194711538461,14.999495967741936,33.114163306451616,7.883333333333334,17.05029761904762,15.84375,55.41346153846154,9.487159978815928,27.23717646917713,2829.915 62 | -------------------------------------------------------------------------------- /output/Nord/Nord_0svm.csv: -------------------------------------------------------------------------------- 1 | Observed,Predicted 2 | 2369.593,1363.4156235633704 3 | 1376.543,1430.475657014815 4 | 1501.676,1893.755251196636 5 | 2507.669,3167.5489858876936 6 | 5104.599,4818.457886731527 7 | 5315.85,5300.635467846365 8 | 4076.267,4201.8868324594805 9 | 3450.045,3373.3788567506244 10 | 3338.644,3144.8842661171857 11 | 3248.723,2868.3640762464456 12 | 1874.929,2514.877249671992 13 | 1141.105,1620.5996006495295 14 | 1247.513,1314.7571943159394 15 | 1206.871,1428.9572933469726 16 | 1396.829,1817.0942195351217 17 | 2846.671,2605.789552397117 18 | 3316.569,4459.258922244113 19 | 6051.007,5414.672776154217 20 | 4541.5,4314.222794979783 21 | 3659.365,3496.357098884629 22 | 2307.476,2828.616621392669 23 | 1939.106,2242.6423413891152 24 | 2310.024,1924.4818978265093 25 | 1523.27,1691.5885381381754 26 | 1311.241,1365.86121130835 27 | 966.462,1409.0404383403747 28 | 1362.048,2059.1226049491656 29 | 2096.605,2972.4419196947542 30 | 3715.413,3768.7387730732203 31 | 5145.394,5328.796708502039 32 | 4056.158,4509.199392311457 33 | 3343.051,3234.5795269243094 34 | 2987.733,2960.5093574049783 35 | 1631.942,2426.8141259263393 36 | 1395.302,1719.2378183602373 37 | 1306.68,1400.6173895037678 38 | 1443.012,1365.5706246819695 39 | 1169.639,1557.4805272800527 40 | 1450.439,1748.5156953446624 41 | 3730.459,4033.352353603908 42 | 6373.905,6757.955925170088 43 | 5707.595,5252.364841556218 44 | 4268.991,3903.0669809695028 45 | 3569.001,3502.419320517995 46 | 2559.936,2947.4491769198867 47 | 2241.658,2685.1198703937557 48 | 3392.373,2889.33959102348 49 | 2115.618,1742.9253316505956 50 | 1136.502,1305.3169520841236 51 | 1345.274,1456.5696503674924 52 | 1285.847,1897.2451737991169 53 | 2434.775,2700.5001080482575 54 | 4041.861,4326.784979031445 55 | 6262.292,5700.187649443511 56 | 5187.456,5481.695841373541 57 | 3894.826,3261.661419940138 58 | 2747.761,2822.897077553943 59 | 2614.677,2624.3163262775224 60 | 3593.629,2632.5437503048724 61 | 2829.915,1845.3058846599 62 | -------------------------------------------------------------------------------- /output/Nord/Nord_0svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Nord/Nord_0svm.png -------------------------------------------------------------------------------- /output/Nord/Nord_hyper_param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Nord/Nord_hyper_param.png -------------------------------------------------------------------------------- /output/Nord/Nord_ivs_forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Nord/Nord_ivs_forward.png -------------------------------------------------------------------------------- /output/Nord/Nord_scatter_svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Nord/Nord_scatter_svm.png -------------------------------------------------------------------------------- /output/Nord/results.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Nord/results.pkl -------------------------------------------------------------------------------- /output/Rhine/Rhine_0svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Rhine/Rhine_0svm.png -------------------------------------------------------------------------------- /output/Rhine/Rhine_hyper_param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Rhine/Rhine_hyper_param.png -------------------------------------------------------------------------------- /output/Rhine/Rhine_ivs_forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Rhine/Rhine_ivs_forward.png -------------------------------------------------------------------------------- /output/Rhine/Rhine_scatter_svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Rhine/Rhine_scatter_svm.png -------------------------------------------------------------------------------- /output/Rhine/results.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/Rhine/results.pkl -------------------------------------------------------------------------------- /output/USA_GW/USA_GW_0svm.csv: -------------------------------------------------------------------------------- 1 | Observed,Predicted 2 | 16.9245524821948,17.166312122871236 3 | 16.7743843795854,16.756970036703468 4 | 16.5089786297493,16.356068667552023 5 | 16.2980858064037,16.24692171032498 6 | 16.2131371028489,16.12430115754042 7 | 15.8,16.015767799980456 8 | 16.2,15.703314051665945 9 | 16.9,16.219071527291526 10 | 16.6,17.022754768729715 11 | 17.1,16.787780961136903 12 | 18.2,17.292442860502995 13 | 17.7647698504784,18.204274961056832 14 | 18.2,17.693734949423515 15 | 17.9898045772165,18.002772080030837 16 | 18.0059532209848,17.657784231033602 17 | 18.024109929357,17.62097772212484 18 | 17.9,17.683192003116172 19 | 18.2410265355806,17.733168286149184 20 | 18.4730568969137,18.199023323074055 21 | 18.45,18.50242009202396 22 | 18.8,18.49193933707034 23 | 19.0,18.889955184842556 24 | 18.9193580429279,19.02671907688982 25 | 19.2,18.872232841016363 26 | 18.70425482397,19.038015164310572 27 | 17.9369635569582,18.295067992611187 28 | 16.8002785954493,17.564390298781408 29 | 16.0518241270717,16.573062622556982 30 | 15.6,15.781704763414194 31 | 15.8138566276697,15.480197481405373 32 | 15.6,15.7503120796962 33 | 15.5,15.636753921657327 34 | 16.9,15.663816113527362 35 | 16.9,17.088286566137484 36 | 16.5848711705622,17.048000985081455 37 | 16.5,16.6467199748151 38 | 16.6090626267992,16.468269041016402 39 | 16.9,16.41669718506015 40 | 16.3571232368073,16.671661868422134 41 | 15.8086066033803,16.130441885217525 42 | 15.3658176103493,15.633557127364343 43 | 14.6,15.192328128892955 44 | 15.1,14.594245221920854 45 | 15.8,15.194896743569341 46 | 16.8,15.966002689632791 47 | 16.6,17.00496022714329 48 | 16.7192483526601,16.744246579848742 49 | 17.2,16.806569123664733 50 | 17.1,17.142401409298195 51 | 16.8850347176091,16.965757434807646 52 | 16.5714711219633,16.651327367560803 53 | 16.0281222467056,16.24384477465082 54 | 15.6059489945569,15.778531428750904 55 | 15.0,15.461185684900913 56 | 15.8806300893339,14.964050565192757 57 | 16.2,15.952822795263426 58 | 16.6977119639493,16.409817934621746 59 | 17.3,16.904783694933087 60 | 17.3289357147014,17.482066607389164 61 | 17.7,17.4038461015356 62 | 17.578123001106,17.64600023195881 63 | 17.7,17.42619746103294 64 | 17.5275087352717,17.43686800707223 65 | 17.3452459710929,17.265700348721165 66 | 17.1739152242681,16.974029097018867 67 | 16.8,16.972867934023135 68 | 17.34778518988,16.764263481500645 69 | 17.45,17.376588688502764 70 | 18.0,17.559387074788923 71 | 16.9,18.155270837921677 72 | 18.7,17.08446097804233 73 | 18.1784009501267,18.68878296770303 74 | 18.5,18.113245544310676 75 | 18.2671751442735,18.293824079311115 76 | 18.1218438708835,17.89046946219338 77 | 17.9291754321507,17.87114982671677 78 | 17.8149956585753,17.66201674867633 79 | 18.2,17.64225416302829 80 | 17.6,18.107832265707373 81 | 16.5,17.597939175595904 82 | 17.1567953937229,16.647184234387943 83 | 16.9,17.31778670460955 84 | 17.6200491062091,17.087343346307428 85 | 18.2,17.726298135500816 86 | 18.4,18.140317266373028 87 | 18.1044639434209,18.22764185628568 88 | 17.9366033484543,17.739139175215627 89 | 17.667241628211,17.574409864968967 90 | 17.474775663671,17.476252838178624 91 | 18.3,17.329503449484505 92 | 16.4609846937963,18.213199770698573 93 | 15.0,16.523596460050317 94 | 16.3,15.183033380276928 95 | 15.4,16.49942862996417 96 | 15.7,15.614697901948663 97 | 16.1,15.805829796216315 98 | 15.8168477039739,16.06073799339549 99 | 15.6423294752424,15.629960659356904 100 | 15.3808070710038,15.44931528066115 101 | 15.2066309911184,15.194485900255561 102 | 14.6,15.060811060595583 103 | 15.5,14.507419430915434 104 | 16.0,15.473026761784016 105 | 15.6,16.068352283262687 106 | 15.5,15.728014709740116 107 | 15.8937543424538,15.645204552310643 108 | 16.35,16.036533011743067 109 | 16.2996618190614,16.449979973074836 110 | 16.7,16.28363628149073 111 | 16.2726142498836,16.51299186736312 112 | 15.894984389485,15.99218072347101 113 | 15.5360771799074,15.599082063902188 114 | 15.5,15.262332591408416 115 | 15.1510880927314,15.351498612107516 116 | 14.7,15.088585271053352 117 | 15.0,14.752941152238531 118 | 15.1,15.103960870968445 119 | 15.4,15.272650005979134 120 | 15.7,15.579268479286274 121 | 15.4,15.74131475580461 122 | 15.3248675513222,15.36488924564232 123 | 15.104028867257,15.142787883235329 124 | 14.7660248453752,14.915139851317937 125 | 14.5360627527944,14.517266923861207 126 | 14.2,14.296174362508243 127 | 14.5,14.095014257003339 128 | 14.8,14.461928764087764 129 | 14.9,14.897651378183554 130 | 14.921529369299,15.074725664037606 131 | 15.2,15.124481078931229 132 | 15.0492410961549,15.41702954487815 133 | 15.0075105436755,15.185635980708083 134 | 14.8798279675214,14.992737706743995 135 | 14.643167248577,14.74828892909457 136 | 14.4443489136043,14.453469153374456 137 | 14.3613876763705,14.233297748216721 138 | 14.3370582607855,14.203275898990713 139 | 14.5,14.20219041109823 140 | 14.1,14.40776166755982 141 | 13.7,14.099664038083507 142 | 14.5,13.843879078698254 143 | 15.1,14.725275807750455 144 | 15.3,15.343235552561104 145 | 14.9939275902318,15.400309384998096 146 | 14.990000692646,14.948389930729968 147 | 14.9746918327189,14.83271785153776 148 | 14.9192185430464,14.748117293012191 149 | 14.7615745551544,14.643896217370335 150 | 14.5100780711315,14.550006221687068 151 | 14.3314750855839,14.31274788861397 152 | 14.2641977998039,14.272063428516415 153 | 14.2451942335961,14.287591204084881 154 | 14.2402879565296,14.41246851490844 155 | 14.2,14.457053201591815 156 | 14.4,14.427300618032849 157 | 14.1,14.475052398128419 158 | 14.1412141093347,14.08338881731055 159 | 14.0106461728765,13.947595353853616 160 | 13.7750752910018,13.785270291013072 161 | 13.5833681335174,13.551257692230285 162 | 13.6,13.407990344695293 163 | 13.3054613919808,13.48055667000894 164 | 13.0924305720408,13.228954521433113 165 | 12.8,13.147607170870325 166 | 12.9,13.026899322459007 167 | 13.2,13.178928660278629 168 | 13.7,13.44000720377935 169 | 13.3587380221692,13.848650658889074 170 | 13.2966325749938,13.367877263398716 171 | 13.2,13.176185961731695 172 | 13.2350548811667,13.060103945345892 173 | 13.2274195567798,13.070151281520467 174 | 13.2145753796556,13.112281044564984 175 | 13.204919324076,13.075672279988328 176 | 13.2,13.194270893445163 177 | 13.2003242581381,13.306448406136685 178 | 13.197013358222,13.338643241514305 179 | 13.1841263046751,13.486517607677659 180 | 13.1377045735151,13.451216372621866 181 | 13.0079974484711,13.354635362525995 182 | 12.8076214439349,13.024298514686262 183 | 12.6700023275237,12.74913573696779 184 | 12.6191319871583,12.559469806423548 185 | 12.604861057744,12.503896981438526 186 | 12.601181683043,12.539381176535333 187 | 12.6002547842224,12.557411691578613 188 | 12.4,12.60628780057744 189 | 12.4,12.519776055091624 190 | 13.1,12.622008920232735 191 | 13.6,13.39787349046967 192 | 13.1,13.851065248972159 193 | 13.2,13.207274588318754 194 | 13.1528392877472,13.21951084560337 195 | 13.0796842305178,13.116476908160164 196 | 12.8911606979675,12.968535800164734 197 | 12.6388634226875,12.76376489601359 198 | 12.4880545265582,12.574602632613837 199 | 12.2,12.481182933091315 200 | 12.7829781166698,12.252108706203833 201 | 13.1819172763924,12.911040879023615 202 | 13.2,13.352990092061887 203 | 13.9,13.503287728723818 204 | 13.7861950356139,14.195457158884537 205 | 14.1,13.939714893999213 206 | 13.8544162632731,14.096790778403415 207 | 13.6419857762598,13.679130624642305 208 | 13.3277380753334,13.427783969789225 209 | 13.1211198655446,13.16706266842884 210 | 13.0,12.946712754961903 211 | 13.0539359994559,12.904059141270263 212 | 12.8,13.062041658036355 213 | 13.1,12.906172633369469 214 | 13.6288169434596,13.291712991697077 215 | 14.0,13.928052623058928 216 | 14.7,14.271749684920794 217 | 14.3486549990859,14.818009383614037 218 | 14.5,14.33533121124099 219 | 14.3020102554118,14.419438826225015 220 | 14.1313515598198,14.099402453449319 221 | 13.9909028554985,13.940426128821674 222 | 14.0,13.83187378098988 223 | 13.4,13.804856335338577 224 | 13.7,13.370629997760785 225 | 14.5,13.730528356459628 226 | 15.1,14.711442310970588 227 | 15.7,15.38789373415555 228 | 15.346162854953,15.906584210846828 229 | 15.7,15.451949867284565 230 | 15.5179720440997,15.611005954384616 231 | 15.5009124654594,15.376610909472387 232 | 15.4477785671348,15.3092422785335 233 | 15.3454290413963,15.231375207166954 234 | 15.2553100888423,15.11312720659603 235 | 15.216521385185,15.12127546985668 236 | 15.205010419036,15.19449654152189 237 | 14.8,15.250878760484277 238 | 15.2,14.92740052521638 239 | 16.3,15.465297822899865 240 | 16.1184951339248,16.531469612678883 241 | 16.6,16.20659414337462 242 | 16.4,16.619254191772587 243 | 16.278073427268,16.200161235126732 244 | 16.117665350169,16.080668479230134 245 | 15.9433578974052,15.863607305165122 246 | 15.6,15.72695264533875 247 | 16.0498735628995,15.491664567532524 248 | 16.0,16.038865610991433 249 | 16.3,16.059588834652207 250 | 16.6,16.40879386884056 251 | 17.0,16.79598735651754 252 | 17.2,17.144833594120378 253 | 16.3,17.299833821884835 254 | 16.3,16.294453968984513 255 | 16.3453511924736,16.17708998159287 256 | 16.1432884858134,16.157327700272404 257 | 15.9022898672602,15.886865177462532 258 | 16.4,15.740735815824618 259 | 15.0691594571634,16.268850252022418 260 | 14.2,15.043225802373117 261 | 13.8,14.235505888471785 262 | 15.0,13.980802251793927 263 | 15.7,15.24634038761647 264 | 14.5543386023874,15.90420858608373 265 | 13.8,14.719325763251597 266 | 13.7,13.809452605119168 267 | 13.8896483510843,13.621139813776102 268 | 13.7890229499165,13.705881690958556 269 | 13.5839308505074,13.678403218379799 270 | 13.3899341417418,13.418600787317033 271 | 13.3022285052679,13.272232714989126 272 | 12.7,13.332098864775016 273 | 13.45,12.83899349631584 274 | 14.2,13.688321710088461 275 | 14.8,14.529236980733044 276 | 14.4826255918855,15.023745344540533 277 | 14.8,14.546392712530398 278 | 14.6023302743422,14.756512886257383 279 | 14.5147689429952,14.484920765146775 280 | 14.3848450220177,14.347760307660016 281 | 14.2991584480612,14.168264660774536 282 | 14.5,14.073015697972057 283 | 14.0222573597417,14.419385988292664 284 | 13.4,13.971009484395857 285 | 13.8,13.522828242846929 286 | 14.4,13.986971456228012 287 | 14.9,14.68495913612609 288 | 15.0,15.14446072277402 289 | 13.45,15.134934963356633 290 | 13.2,13.473843922179636 291 | 13.3,13.123193170245703 292 | 13.2993145662978,13.136319617492452 293 | 13.1096064557998,13.125678541710752 294 | 13.2,12.959609203095585 295 | 12.5,13.11259149174817 296 | 12.6341514522027,12.51094552346546 297 | 12.1,12.717289374627628 298 | 12.5,12.341301074156336 299 | 13.1,12.838770142508663 300 | 13.7,13.369024089519181 301 | 14.0,13.812242778953454 302 | 13.4547551045143,13.987993818732651 303 | 13.5,13.361074558808848 304 | 12.9975930451471,13.308390845441723 305 | 12.6,12.854782443695983 306 | 12.7,12.511336969112309 307 | 12.2974977973568,12.63164672782481 308 | 11.5,12.301659711451974 309 | 12.2,11.71978858036329 310 | 12.2,12.424139733876938 311 | 12.6,12.503224636714325 312 | 12.8,12.870044222903246 313 | 12.7409305407432,12.976749919080207 314 | 13.0,12.697861033340013 315 | 12.9,12.953232077281978 316 | 12.6771478426761,12.742329451873195 317 | 12.2776428535496,12.550892910137215 318 | 11.7176409072298,12.18440105683723 319 | 11.3681372886751,11.751804894030558 320 | 11.1,11.51702686155697 321 | 10.9,11.330187880244637 322 | 11.4,11.295382811453301 323 | 11.6,11.844670079838975 324 | 12.7,12.043273358697142 325 | 13.0,12.879277140407575 326 | 12.5386383789887,13.061939763626572 327 | 12.5115212365414,12.479337272164765 328 | 12.4125619039931,12.404631366824797 329 | 12.1251338891721,12.306319628790805 330 | 11.6466453599453,12.051068647746776 331 | 11.291018573972,11.66316015936662 332 | 11.1534619878547,11.42051328149955 333 | 11.1142367864006,11.365064498173234 334 | 11.1,11.48119565718727 335 | 11.0,11.622058428490222 336 | 11.2239685280845,11.489702586935527 337 | 11.5,11.52984000257395 338 | 11.3894192408488,11.66222756264304 339 | 11.4328330983346,11.455951691679166 340 | 11.5522030226402,11.45877281354499 341 | 11.7311879742601,11.568280272755857 342 | 11.8504657654395,11.721519973254386 343 | 11.8938295427823,11.97146963973884 344 | 11.5,12.026130609766774 345 | 12.3966921063662,11.745943999176989 346 | 12.8,12.72486626693961 347 | 13.4,13.225774405170158 348 | 13.0626541241436,13.686159962172786 349 | 13.0486866035674,13.247438212878222 350 | 13.0089929317165,13.055284477691064 351 | 12.9456672770358,12.959886905553477 352 | 12.9006962928851,12.816901520962448 353 | 12.8837546582,12.764163949641834 354 | 13.1,12.767797945224917 355 | 12.6418183842658,13.001786277652531 356 | 12.5,12.61312645434954 357 | 11.8,12.625341767907528 358 | 12.9,12.095227327860538 359 | 12.7774251623597,13.216337300831999 360 | 13.2,13.095141658090384 361 | 12.9660122711608,13.4064075421377 362 | 12.8901205796801,12.959320910414533 363 | 12.7186028712442,12.804158906149212 364 | 12.5315930142552,12.587434958396829 365 | 12.4376173279073,12.423036529860758 366 | 12.6,12.331630385850591 367 | 12.1078662343822,12.53135360133632 368 | 11.6,12.164721035910526 369 | 11.8207053118801,11.784492688669864 370 | 11.8532338651321,12.086552865987855 371 | 11.9075269014991,12.21353504886543 372 | 11.9479950687359,12.24570743782241 373 | 12.0,12.174022529612637 374 | 11.8860752919821,12.120296805399892 375 | 11.6702018825185,11.866121099890758 376 | 11.3394272502943,11.660803771307688 377 | 11.1141235155597,11.364342520655637 378 | 11.0,11.180417002953341 379 | -------------------------------------------------------------------------------- /output/USA_GW/USA_GW_0svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW/USA_GW_0svm.png -------------------------------------------------------------------------------- /output/USA_GW/USA_GW_hyper_param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW/USA_GW_hyper_param.png -------------------------------------------------------------------------------- /output/USA_GW/USA_GW_ivs_forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW/USA_GW_ivs_forward.png -------------------------------------------------------------------------------- /output/USA_GW/USA_GW_scatter_svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW/USA_GW_scatter_svm.png -------------------------------------------------------------------------------- /output/USA_GW/results.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW/results.pkl -------------------------------------------------------------------------------- /output/USA_GW_deep/USA_GW_deep_0svm.csv: -------------------------------------------------------------------------------- 1 | Observed,Predicted 2 | 131.5,129.05093578964562 3 | 132.454285714286,131.66733663703383 4 | 131.333333333333,132.56996226207016 5 | 129.5,131.5106993066664 6 | 129.405714285714,129.7827907632395 7 | 128.166666666667,129.72331158772855 8 | 132.26,128.71439006437674 9 | 129.5,132.38537316919582 10 | 129.166666666667,129.97577750636893 11 | 129.981428571429,129.50500746158178 12 | 131.0,130.25223970892088 13 | 133.666666666667,131.1983725253568 14 | 135.022857142857,133.72957912512004 15 | 135.333333333333,135.04114591953217 16 | 135.0,135.3432119932214 17 | 133.217142857143,135.0189322009502 18 | 134.666666666667,133.29806518450587 19 | 135.666666666667,134.69537854886354 20 | 136.785,135.56623995230933 21 | 136.8,136.74132914414326 22 | 137.666666666667,136.82356134746587 23 | 137.928571428571,137.63358383249692 24 | 137.833333333333,137.88702801682265 25 | 138.166666666667,137.79322431591743 26 | 138.322857142857,138.12159264480294 27 | 138.5,138.27553129100647 28 | 138.833333333333,138.4501736836693 29 | 137.666666666667,138.77891924774056 30 | 137.0,137.46145491334303 31 | 135.4,136.78157286974343 32 | 135.111428571429,135.43274788390735 33 | 135.4,135.28695629489556 34 | 135.0,135.8570670909908 35 | 135.707142857143,134.97433257186873 36 | 136.0,135.70766606576856 37 | 137.333333333333,136.0098705528879 38 | 137.988571428571,137.30125234847165 39 | 139.0,137.94612629148745 40 | 139.166666666667,138.94335390234932 41 | 139.685714285714,139.10780261916685 42 | 139.666666666667,139.62001320268155 43 | 139.0,139.56096473676658 44 | 138.847142857143,138.92729564834877 45 | 138.4,138.79749517341205 46 | 138.0,138.35222864394888 47 | 139.28,137.9577306112476 48 | 139.9225,139.22003109288283 49 | 140.5,139.85368289906626 50 | 141.522857142857,140.42341233052707 51 | 142.0,141.43120996113123 52 | 142.5,141.90039718847487 53 | 141.835714285714,142.39118649771942 54 | 141.666666666667,141.7388239253448 55 | 142.833333333333,141.467206395663 56 | 143.42,142.56857162988325 57 | 143.429398831405,143.22421020129187 58 | 143.996630136353,143.2969210947566 59 | 143.9625,143.85681275211232 60 | 144.333333333333,143.81991025089968 61 | 145.0,144.18017746605398 62 | 146.292857142857,144.8253990288053 63 | 147.333333333333,146.06630546506852 64 | 148.166666666667,147.05335713816544 65 | 148.811428571429,147.83531405894578 66 | 147.5,148.4345563384542 67 | 146.5,147.19093696607217 68 | 146.0,146.25788920100717 69 | 146.5,145.60940101108474 70 | 148.25,146.26447373713182 71 | 149.333333333333,147.88241328670472 72 | 150.0,148.91566824761577 73 | 150.0,149.52478723559932 74 | 149.0,149.52478723559932 75 | 149.0,148.60880724525498 76 | 148.428571428571,148.60880724525498 77 | 148.0,148.0793592834352 78 | 147.166666666667,147.60565817338434 79 | 146.0,146.7749779033118 80 | 145.5,145.63620222249358 81 | 144.975714285714,145.2712419658849 82 | 144.0,144.79243233988575 83 | 143.833333333333,143.85965869409864 84 | 143.666666666667,143.69458999766823 85 | 143.0,143.53187011711057 86 | 143.0,142.88090164443983 87 | 143.443932733349,142.88090164443983 88 | 144.0,143.31465586301582 89 | 144.0,143.85638306943147 90 | 143.333333333333,143.85638306943147 91 | 142.833333333333,142.95020500384496 92 | 142.0,142.71065407499248 93 | 139.956,141.80121332557587 94 | 139.166666666667,139.88775723108603 95 | 139.0,139.1206953980793 96 | 138.864285714286,138.9433699105732 97 | 138.833333333333,138.80946245225948 98 | 138.833333333333,138.7789283351393 99 | 138.841428571429,138.7789283351393 100 | 138.0,138.78691408329178 101 | 138.0,137.95738405617416 102 | 137.52,137.9573751972842 103 | 137.0,137.4654595204571 104 | 136.857142857143,136.95281375076016 105 | 136.331428571429,136.83625470788172 106 | 136.0,136.2577405334368 107 | 136.072676528809,135.99393732467382 108 | 135.9275,136.08502309481256 109 | 135.5,135.9228814706824 110 | 136.0,135.50560823906568 111 | 135.833333333333,135.9937452168139 112 | 135.333333333333,135.83088130994602 113 | 135.0,135.3432119932214 114 | 135.021428571429,135.01890011973256 115 | 135.0,135.03494646842287 116 | 135.0,135.0018428175437 117 | 134.983333333333,135.01987138077237 118 | 134.833333333333,135.00657435009552 119 | 134.033333333333,134.85706542762594 120 | 134.021428571429,134.08283853361849 121 | 134.666666666667,134.07132290831163 122 | 135.0,134.69537854886354 123 | 134.677142857143,135.0189322009502 124 | 134.5,134.70553581844175 125 | 134.0,134.53388988202897 126 | 133.99,134.02477943063323 127 | 134.166666666667,134.0391962810005 128 | 134.0,134.16493611330998 129 | 133.771428571429,134.0510076740317 130 | 133.0,133.85516817489665 131 | 133.666666666667,133.0904125227202 132 | 134.077142857143,133.72957912512004 133 | 134.0,134.1250542588282 134 | 134.333333333333,134.05065198388107 135 | 134.291428571429,134.3726017249067 136 | 134.333333333333,134.33208155933508 137 | 133.833333333333,134.3538977180362 138 | 133.041428571429,133.88867394035944 139 | 133.666666666667,133.13476581211253 140 | 133.3,133.69315633617717 141 | 133.37,133.37747063036784 142 | 133.347354295919,133.45845908497205 143 | 133.257561929859,133.42282911038632 144 | 133.13,133.33679285203092 145 | 133.151665270389,133.21462832666933 146 | 133.115769177212,133.23535860882586 147 | 133.1,133.2009980137624 148 | 133.053592084113,133.18590706896123 149 | 133.004780738392,133.1415082778568 150 | 133.03,133.0904260385162 151 | 132.905177006145,133.07192285859023 152 | 133.6,132.9755421778928 153 | 132.0,133.68732669605117 154 | 132.0,132.14772173714442 155 | 132.0,132.13936613005407 156 | 132.0,132.13906150417014 157 | 132.130797464011,132.13909841183437 158 | 132.270029039153,132.2629199892674 159 | 132.476271336491,132.3949432136117 160 | 132.612058422829,132.59086827177254 161 | 133.0,132.71882348183544 162 | 132.666666666667,133.09024187588597 163 | 132.166666666667,132.7721278405782 164 | 131.785714285714,132.2913934715113 165 | 132.0,131.925044190613 166 | 132.0,132.13920880709398 167 | 132.0,132.1421939321113 168 | 132.0,132.13929914991067 169 | 132.666666666667,132.139076415566 170 | 133.5,132.77211254408397 171 | 134.333333333333,133.56938695891728 172 | 135.0,134.3726017249067 173 | 135.0,135.0189322009502 174 | 134.833333333333,135.0189322009502 175 | 135.0,134.8570614294984 176 | 134.3,134.99204384282154 177 | 134.75,134.33993665331576 178 | 134.166666666667,134.77869834695713 179 | 134.0,134.21160254597203 180 | 134.0,134.05157935638493 181 | 134.0,134.05065198388107 182 | 134.5,134.05065198388107 183 | 134.5,134.53388988202897 184 | 134.0,134.53388988202897 185 | 134.0,134.05065198388107 186 | 133.5,134.05065198388107 187 | 133.166666666667,133.5620521708522 188 | 133.166666666667,133.24592919994345 189 | 133.0,133.2403388671069 190 | 133.0,133.08778163042555 191 | 133.0,133.09291682342206 192 | 133.5,133.09093714079478 193 | 133.333333333333,133.56938695891728 194 | 134.0,133.4094324704893 195 | 134.0,134.05065198388107 196 | 133.333333333333,134.05065198388107 197 | 133.0,133.4094198881655 198 | 133.0,133.09026084272108 199 | 133.0,133.09026323931135 200 | 133.0,133.02560991425977 201 | 132.78,133.08842038869255 202 | 132.56,132.95006259591523 203 | 132.727646080918,132.6709151677825 204 | 132.708721750007,132.83023538595123 205 | 132.637199920295,132.8121899401521 206 | 132.404751990788,132.7440389918366 207 | 131.910800704458,132.52287848826896 208 | 131.416852520986,132.05472213187278 209 | 131.184410856916,131.58915373409602 210 | 131.112893230644,131.37096917649552 211 | 131.093971732198,131.2697832886392 212 | 130.8,131.28626914711845 213 | 131.41,131.0322477107649 214 | 131.015,131.58234838571326 215 | 131.125,131.21247434984622 216 | 131.375,131.31541826015345 217 | 132.54,131.5498294136832 218 | 134.315,132.65149446023614 219 | 133.095,134.3548725381554 220 | 132.73,133.1811225902337 221 | 132.255,132.83247975064023 222 | 132.135,132.38068273443247 223 | 131.89,132.26524425275937 224 | 131.74,132.03498801734185 225 | 131.465,131.89376968503515 226 | 131.366666666667,131.6344316054789 227 | 131.435,131.5420484662369 228 | 131.22,131.6062110774753 229 | 131.185,131.404355866829 230 | 131.325,131.37155607310908 231 | 131.7,131.5028756381319 232 | 131.445,131.85570806559946 233 | 131.255,131.61561194271889 234 | 131.24,131.43718791960083 235 | 131.05,131.41576333603044 236 | 130.98,131.245006004484 237 | 130.793333333333,131.1815214173634 238 | 130.83,131.00535669586782 239 | 131.115,131.03957864306116 240 | 132.33,131.30609445645086 241 | 131.48,132.45187013251834 242 | 132.6,131.64850311499873 243 | 133.62,132.70860989656862 244 | 134.21,133.68470171370663 245 | 133.505,134.25338126356093 246 | 133.52459564446,133.54736737356578 247 | 133.542469551127,133.59282912461222 248 | 133.600555977718,133.543906296231 249 | 133.723984646998,133.96127176837646 250 | 133.847408116098,133.85907333104058 251 | 133.905485784741,133.9156382523508 252 | 133.92335391787,133.95951948898352 253 | 133.47,133.9767299653971 254 | 134.735,133.54057743326035 255 | 134.31,134.76164539860005 256 | 134.17,134.35003774073775 257 | 133.615,134.2147399063915 258 | 133.23,133.60507301322986 259 | 133.16,133.3091180873808 260 | 133.115,133.25302969448038 261 | 132.935,133.2005109911271 262 | 132.815,133.10388163167602 263 | 132.74,132.9135590987955 264 | 132.86,132.84201492814955 265 | 132.895,132.95651114973856 266 | 133.195,132.98993145460804 267 | 133.355,133.27685511133402 268 | 133.255,133.43021289841963 269 | 132.965,133.33433769922257 270 | 132.875,133.05680883843596 271 | 132.685,132.96447856381096 272 | 132.81,132.7895828242123 273 | 132.705,132.91022309181923 274 | 132.605,132.80864599171326 275 | 132.15,132.7133673067118 276 | 132.385,132.2811165469821 277 | 132.495,132.5041102521656 278 | 132.45,132.60868102573028 279 | 132.445,132.56588755699755 280 | 132.48,132.56113396463488 281 | 132.21,132.5944143120002 282 | 132.298213574772,132.33799770453635 283 | 132.085,132.42167933487445 284 | 132.583333333333,132.21953713077167 285 | 132.37,132.65753971607054 286 | 132.315,132.48985508665402 287 | 132.045,132.45340117315882 288 | 132.05,132.1816648123329 289 | 132.34,132.18639712330878 290 | 132.1,132.461366076685 291 | 132.13,132.23374388604307 292 | 131.945,132.26216438486884 293 | 131.98,132.08705391731849 294 | 132.195,132.1201769665402 295 | 131.9,132.27640003806783 296 | 131.23,132.0445096031634 297 | 131.855,131.44983384684784 298 | 131.735,132.0019951481125 299 | 132.01,131.88872036237123 300 | 131.71,132.14853846519964 301 | 132.07,131.86513765648172 302 | 132.03,132.20533271057695 303 | 131.855,132.16746570484284 304 | 131.735,132.0019951481125 305 | 131.75,131.88871799633105 306 | 131.765,131.9028691679554 307 | 131.77,131.91702276861812 308 | 131.353333333333,131.9217459132834 309 | 131.555,131.5292209750698 310 | 132.678591545837,131.7190939981898 311 | 133.46,132.78352904963214 312 | 133.73,133.53097597548987 313 | 134.505,133.79051319558852 314 | 134.85,134.5387316560011 315 | 134.48,134.87324015611716 316 | 134.19,134.5145245924438 317 | 133.98,134.23405905751 318 | 134.47,134.00565332539347 319 | 134.03,134.5032289324458 320 | 134.25,134.08882585437811 321 | 133.063333333333,134.28315843960902 322 | 133.89,133.59752857750135 323 | 133.785,133.9421879457166 324 | 133.64854887949,133.8434564912285 325 | 133.347158322728,133.71306218061625 326 | 132.706717635951,133.42269149698492 327 | 132.066299961852,132.81028330901333 328 | 131.764937218721,132.20182927699372 329 | 131.62,131.9169635248185 330 | 131.695,131.76737519016234 331 | 131.59,131.84571220866331 332 | 131.525,131.75297391502735 333 | 131.42,131.69206153198152 334 | 131.59,131.58636592548785 335 | 132.385,131.75206587240828 336 | 132.755,132.50562163222497 337 | 133.465,132.8563161512986 338 | 133.355,133.53577659684245 339 | 133.075,133.43021289841963 340 | 132.925,133.1619869486892 341 | 132.685,133.0185865449587 342 | 132.855,132.78958329958107 343 | 133.045,132.95173775833163 344 | 132.4,133.13142096668003 345 | 133.695,132.51836281615564 346 | 133.44,133.75610178940428 347 | 133.27,133.5117756440234 348 | 134.555,133.3490365305558 349 | 134.445,134.58715928373948 350 | 133.905,134.48062103775106 351 | 134.225,133.959054714176 352 | 134.12,134.26787491642168 353 | 134.605,134.1664554645314 354 | 134.25,134.63560477082476 355 | 133.85,134.25384344756506 356 | 133.48,133.89791900449924 357 | 134.16,133.50389965166855 358 | 134.96,134.20494826289197 359 | 135.53,135.01144304412998 360 | 135.64,135.5352413266449 361 | 135.8,135.64214680917303 362 | 136.15,135.79832625209056 363 | 137.345,136.14044512499723 364 | 137.02,137.31272347796215 365 | 137.4,136.99334342886021 366 | 137.265,137.36680755006535 367 | 137.17,137.2340732627833 368 | 137.173333333333,137.14070402595235 369 | 138.09,137.0793417817626 370 | 137.96,138.04607291612032 371 | 138.695,137.9329406098182 372 | 139.205,138.6424763014606 373 | 139.81,139.14562838705172 374 | 139.94,139.7426671758365 375 | 140.095,139.87095139165453 376 | 140.8,140.02389273127835 377 | 140.065,140.71921376476553 378 | 139.615,139.99430815805576 379 | 139.77,139.454449430887 380 | 141.43,139.7007977158441 381 | 140.2,141.25212990990914 382 | 140.895,140.12795998386906 383 | 140.625,140.82230426724118 384 | 141.28,140.54666985207464 385 | 141.025,141.19212885493133 386 | 140.32,140.940960306785 387 | 140.11,140.24586992389226 388 | 140.345,140.03869261028268 389 | 140.005,140.27053092013637 390 | 139.84,139.93513073814609 391 | 140.695,139.7722785259895 392 | 140.83,140.57951004542292 393 | 140.77,140.74875245594944 394 | 140.755,140.69491900685236 395 | 141.39,140.67494479617608 396 | 141.54,141.3005301535245 397 | 141.915,141.4480645350202 398 | 142.245,141.8168523813218 399 | 142.285,142.14100670032846 400 | 143.53,142.180268083946 401 | 143.96,143.3986244570989 402 | 142.8,143.67805922838807 403 | 142.365,142.64601985184547 404 | 141.6225,142.26254138066716 405 | 141.85,141.48664063573773 406 | 141.46,141.752983662878 407 | 141.77,141.37520569702568 408 | 142.575,141.67432364772873 409 | 143.29,142.4646948095531 410 | 143.28,143.16437256390014 411 | 143.02,143.15460512418304 412 | 143.06,142.90046533312454 413 | 142.38,142.93958663495155 414 | 142.54,142.2696887854402 415 | 143.45,142.4239584625153 416 | 142.223333333333,142.79331947940267 417 | 142.23,142.04643453291035 418 | 141.85,142.04999977684395 419 | 141.76,141.75969421688157 420 | 141.865,141.66492606066132 421 | 141.79,141.76771321148854 422 | 141.56,141.69398925556777 423 | 141.725,141.46775900399706 424 | 142.65,141.63007132419736 425 | 142.325,142.53822211764412 426 | 142.32,141.7771992578291 427 | 142.01,142.05482063768935 428 | -------------------------------------------------------------------------------- /output/USA_GW_deep/USA_GW_deep_0svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_deep/USA_GW_deep_0svm.png -------------------------------------------------------------------------------- /output/USA_GW_deep/USA_GW_deep_hyper_param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_deep/USA_GW_deep_hyper_param.png -------------------------------------------------------------------------------- /output/USA_GW_deep/USA_GW_deep_ivs_forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_deep/USA_GW_deep_ivs_forward.png -------------------------------------------------------------------------------- /output/USA_GW_deep/USA_GW_deep_scatter_svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_deep/USA_GW_deep_scatter_svm.png -------------------------------------------------------------------------------- /output/USA_GW_deep/results.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_deep/results.pkl -------------------------------------------------------------------------------- /output/USA_GW_north/USA_GW_north_0svm.csv: -------------------------------------------------------------------------------- 1 | Observed,Predicted 2 | 166.1714286,162.13486421163827 3 | 167.9683333,165.72110919531798 4 | 159.4566667,160.63288037212538 5 | 157.2214286,155.5004429983719 6 | 156.1966667,154.58830409266088 7 | 155.5771429,156.02544455299793 8 | 155.045,156.109620608251 9 | 154.57,154.21279444214076 10 | 154.03,153.64516857221264 11 | 155.0614286,152.85465232800314 12 | 154.1657143,154.62903666954722 13 | 158.5457143,157.61186583504318 14 | 165.3016667,161.63560016110844 15 | 158.7214286,161.7223908151665 16 | 157.0371429,155.1153691136339 17 | 156.1871429,154.04445822435483 18 | 155.3942857,154.58703425526568 19 | 155.0642857,154.50687900396585 20 | 154.7183333,153.33364542297898 21 | 154.52,153.4395689675612 22 | 153.9742857,152.8792156741905 23 | 153.7571429,151.64959595452584 24 | 153.3557143,152.77768806565948 25 | 153.2971429,151.62652605244975 26 | 154.6485714,155.45854864641242 27 | 158.9783333,160.23109048019668 28 | 154.9114286,154.70189568724027 29 | 153.4828571,151.61631385847144 30 | 153.0114286,151.6349540743479 31 | 152.6428571,152.0386263913273 32 | 152.44,152.85776556353787 33 | 152.0585714,152.48910768136352 34 | 151.7071429,151.04122324862894 35 | 151.4742857,150.09697087039774 36 | 151.3833333,150.90616056778848 37 | 151.2328571,152.46885436150237 38 | 155.9742857,154.85651983748048 39 | 161.9328571,162.00624849752583 40 | 157.7114286,157.85175203501464 41 | 154.2742857,153.22807365386205 42 | 153.0985714,152.02090742575996 43 | 152.71,151.2437150390492 44 | 152.15,153.41309671557983 45 | 151.63,153.1558472760875 46 | 151.2671429,151.28002547497056 47 | 150.7785714,149.4915540290352 48 | 150.2214286,149.01655972971338 49 | 150.2514286,150.5866669889677 50 | 158.6542857,153.49248967580397 51 | 166.3828571,164.17303839597832 52 | 158.1975,159.8197567320778 53 | 154.4057143,153.24919284924786 54 | 153.1528571,151.51534061404962 55 | 152.4385714,152.8899503686328 56 | 151.6557143,153.63029587210517 57 | 151.0957143,152.32313452574738 58 | 150.6485714,150.5589507572335 59 | 150.2042857,149.7742295790542 60 | 149.9142857,148.77978952721432 61 | 150.6342857,151.44340325712503 62 | 163.172,155.93540102110285 63 | 157.35,161.48707387767695 64 | 154.3816667,153.16059431318698 65 | 152.2514286,150.62859930425404 66 | 151.3428571,150.80700857534111 67 | 150.7333333,152.54296714936496 68 | 150.2228571,151.9352155358319 69 | 149.8016667,150.82286624944425 70 | 149.4842857,149.36556392920514 71 | 149.2085714,148.45835131485816 72 | 149.0285714,149.32292576997918 73 | 148.8216667,151.64565351711863 74 | 153.1714286,153.15666984700783 75 | 149.86,155.92587104676363 76 | 149.0433333,149.08145116588818 77 | 148.7128571,147.90867425745523 78 | 148.3,148.74697943767333 79 | 148.005,150.15719444686744 80 | 147.66,149.281937822338 81 | 147.5257143,149.19668035385024 82 | 147.29,147.5063567667812 83 | 147.1485714,145.4316329786371 84 | 146.9216667,148.37422419842136 85 | 149.6016667,150.4654767012071 86 | 161.752,156.3290408204005 87 | 158.4975,160.46897059578643 88 | 151.68,152.8621141021638 89 | 150.3157143,150.19284918751353 90 | 149.58,150.6235744527823 91 | 148.8914286,151.87198645782826 92 | 148.4728571,150.71651695839864 93 | 148.1542857,149.6701464938844 94 | 147.7114286,148.90187938190917 95 | 147.4571429,148.46344971345556 96 | 147.3928571,149.64267906858018 97 | 152.1714286,153.51437315800877 98 | 164.632,158.72078705967726 99 | 160.6257143,165.3669732362066 100 | 154.3133333,156.21377163390463 101 | 152.0414286,151.06231426375092 102 | 150.75,152.01505071777422 103 | 150.2928571,153.07233420110893 104 | 149.6257143,151.9738689239407 105 | 149.3514286,150.04258119592856 106 | 148.8571429,149.529377124365 107 | 148.7483333,150.5221597657249 108 | 151.5328571,152.0251009243745 109 | 152.6428571,155.26729372064253 110 | 166.025,159.65456171782307 111 | 170.0066667,166.99779505383526 112 | 156.52,159.40440683742187 113 | 155.2066667,153.06748292338034 114 | 153.8257143,153.6596957161925 115 | 152.8414286,154.37283246141345 116 | 152.08,155.1244761816551 117 | 151.825,152.31506518836778 118 | 151.235,151.18748593416194 119 | 151.18,150.8528097874259 120 | 150.8916667,151.69073987915968 121 | 150.3533333,152.25684240831413 122 | 160.98,155.70177642414066 123 | 162.1985714,162.6521716333971 124 | 160.77,159.07592550161348 125 | 154.9942857,155.39465653008952 126 | 153.545,152.76951431782146 127 | 152.9285714,153.21834763098553 128 | 152.3457143,153.3413363303299 129 | 151.7342857,153.360359266537 130 | 151.25,150.92711891275837 131 | 151.1133333,150.44073598648782 132 | 150.835,151.02305984307287 133 | 152.7142857,153.88118329374527 134 | 170.6428571,160.09189434355255 135 | 172.1628571,169.96418742323348 136 | 163.61,163.0245517601311 137 | 158.5471429,156.84696564209182 138 | 156.8914286,154.10519454000035 139 | 155.94,156.9879510181769 140 | 155.0028571,156.68679751707262 141 | 154.3528571,154.69649659085982 142 | 153.9114286,152.47469310667825 143 | 153.3228571,153.35327303461634 144 | 156.5766667,154.36909106471077 145 | 153.2366667,155.70259381313303 146 | 153.3114286,155.25190826429352 147 | 152.8,155.00332476397404 148 | 152.1914286,150.71337627013415 149 | 151.88,151.5554964941397 150 | 151.6085714,151.07042986007392 151 | 151.3514286,151.24806394818125 152 | 150.92,151.0118797067293 153 | 150.64,150.70267208988702 154 | 148.46,150.52993482514987 155 | 148.0,148.23990871236512 156 | 147.8,148.82804719999385 157 | 147.55,149.81806015705615 158 | 147.18,151.0769030362826 159 | 152.29,154.87653860505134 160 | 148.57,149.5359725043013 161 | 147.52,147.48265123815597 162 | 147.01,148.03641014095143 163 | 146.72,148.15062934637368 164 | 148.0,148.57584447846034 165 | 148.01,148.77126641845146 166 | 147.15,147.9935130569502 167 | 143.15,147.18089579491976 168 | 143.27,146.28090580914363 169 | 148.78,148.42095962602173 170 | -------------------------------------------------------------------------------- /output/USA_GW_north/USA_GW_north_0svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_north/USA_GW_north_0svm.png -------------------------------------------------------------------------------- /output/USA_GW_north/USA_GW_north_hyper_param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_north/USA_GW_north_hyper_param.png -------------------------------------------------------------------------------- /output/USA_GW_north/USA_GW_north_ivs_forward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_north/USA_GW_north_ivs_forward.png -------------------------------------------------------------------------------- /output/USA_GW_north/USA_GW_north_scatter_svm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_north/USA_GW_north_scatter_svm.png -------------------------------------------------------------------------------- /output/USA_GW_north/results.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/output/USA_GW_north/results.pkl -------------------------------------------------------------------------------- /output/readme.txt: -------------------------------------------------------------------------------- 1 | In the output folder (currently empty) will be saved all the modeling output produced by B-AMA -------------------------------------------------------------------------------- /protocol/LSTM_module_config.txt: -------------------------------------------------------------------------------- 1 | [dimensionality] 2 | n_param = 2 3 | 4 | [param_values] 5 | p0 = 2, 6, 8, 10 6 | p1 = 50, 100, 200 7 | 8 | 9 | -------------------------------------------------------------------------------- /protocol/__pycache__/data_division.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/data_division.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/data_transformation.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/data_transformation.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/ivs.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/ivs.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/model_testing.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/model_testing.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/model_training.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/model_training.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/postprocess.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/postprocess.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/__pycache__/utils.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alessandroamaranto/B-AMA/3c8c7504ef54dff417cc3de32bc8a34b90a379be/protocol/__pycache__/utils.cpython-38.pyc -------------------------------------------------------------------------------- /protocol/advanced_configurations.txt: -------------------------------------------------------------------------------- 1 | [Data_Division] 2 | split_custom = 0 3 | split_period: 0 4 | 5 | [Data_Transformation] 6 | mode = min_max 7 | ma_f = 10 8 | 9 | [IVS] 10 | ivs_method = forward_selection 11 | max_predictor = 4 12 | max_tol = 0.001 13 | n_split = 4 14 | 15 | [Model_Training] 16 | n_C : 0.8, 0.9, 1, 2, 2.5, 2.75 17 | n_eps : 0.001, 0.005, 0.01, 0.025, 0.05, 0.1 18 | n_tol : 0.005, 0.01, 0.02, 0.04 19 | krn : linear, rbf, sigmoid 20 | activation: logistic, tanh, relu 21 | neurons: 3, 5, 7, 9, 11 22 | iter: 500, 1000, 1500, 2000, 4000 23 | alfa: 0.0001, 0.001, 0.01, 0.1 24 | learning: constant, invscaling, adaptive 25 | 26 | -------------------------------------------------------------------------------- /protocol/data_division.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 14:41:27 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import os 10 | import configparser 11 | 12 | import protocol.utils as ut 13 | 14 | class data_division(): 15 | 16 | """ 17 | data_division performs the split between training and test set. 18 | At the current state, two methods are employed: 19 | - Optimal split (default): as described by Amaranto et al; (2020) 20 | - User defined split 21 | To change the default configurations, switch the split_custom parameter in the 22 | split_config.txt configuration file and specify the index of the first period in 23 | the test set. 24 | """ 25 | 26 | def __init__(self): 27 | 28 | Config = configparser.ConfigParser() 29 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 30 | 31 | # Split between calibration and validation parameters 32 | self.custom_split = int(Config.get('Data_Division','split_custom')) # Default: 0, optimal split is performed 33 | 34 | # If custom split == 1, then specify the firts period to be included in the validation set 35 | self.split_index = np.array([e.strip() for e in Config.get('Data_Division', 36 | 'split_period').split(',')]).astype( 37 | int) 38 | def compute_split_stats(self, yc, yv): 39 | 40 | """ 41 | compute_split_stats(self, yc, yv) 42 | 43 | Compute the objective function value for the t-th split. 44 | The objective function is defined as s_mean + s_sd, where: 45 | - s_mean is the squared difference between the training and the test 46 | set mean 47 | - s_sd is the quared difference between the training and the test 48 | set standard deviation. 49 | 50 | Input: 51 | - yc: training set - output 52 | - yv: test set - output 53 | Returns: 54 | - d: objective function value (to be minimized) 55 | """ 56 | 57 | s_mean = np.power(np.mean(yc) - np.mean(yv), 2) 58 | s_sd = np.power(np.power(np.var(yc) - np.var(yv), 2), 0.5) 59 | 60 | d = s_mean + s_sd 61 | 62 | return(d) 63 | 64 | def split_data(self, x, nY, n_v, period): 65 | 66 | """ 67 | split_data(self, x, nY, n_v, period) 68 | 69 | Iteratively split the dataset into training and validation, and extracts 70 | the split minimizing the objective function d, defined as the sum between the 71 | normalized average difference and the normalized standard deviation difference 72 | computed between the training and test set. 73 | For further information, consult Amaranto et al., 2020 (https://doi.org/10.1016/j.jhydrol.2020.124957) 74 | 75 | Input: 76 | - x: the whole dataset 77 | - nY: number of periods 78 | - n_v: number of periods in the test set 79 | - period: time series periodicity 80 | Returns: 81 | - yS: optimal starting period in the test set 82 | """ 83 | 84 | # Extract dependent variable and normalize 85 | xi = x[:, x.shape[1]-1] 86 | y = ut.normalize_vector(xi) 87 | 88 | # Initialize the objective function 89 | dummy_d = np.Inf 90 | 91 | # Iterate across all possible splits 92 | for t in range(0, nY - n_v + 1): 93 | 94 | # Define iteration-specific training and test set 95 | cut = np.arange(t*period, (t + n_v)*period) 96 | yv = y[cut] 97 | yc = np.delete(y, cut) 98 | 99 | # Compute objective function 100 | d = self.compute_split_stats(yc, yv) 101 | 102 | # Extract optimal split 103 | if d < dummy_d: 104 | 105 | dummy_d = d 106 | yS = t 107 | 108 | return(yS) 109 | 110 | def optimal_split(self, x, period, i): 111 | 112 | """ 113 | optimal_split(self, x, period, i) 114 | 115 | split data between training and test set according to the user's choice. 116 | 117 | Input: 118 | - x = the whole dataset 119 | - period = periodicity of the time series. 120 | - self.split_index = if custom split == 1, the first period in the 121 | test set 122 | - i = time series index (only if multiple files are in the 'input/ 123 | case-study folder, otherwise always 0') 124 | Returns: 125 | - oS = index of the firt period to be included in the test set 126 | - n_v = test set length 127 | """ 128 | 129 | nD = x.shape[0] # number of time steps 130 | nY = int(nD/period) # number of periods 131 | n_v = round((x.shape[0]/period)*0.3, 0) # test set length 132 | n_v = int(n_v) # convert from float to int 133 | 134 | # Compute optimal or custom-defined split 135 | if self.custom_split: 136 | oS = self.split_index[i] 137 | return(oS, n_v) 138 | else: 139 | oS = self.split_data(x, nY, int(n_v), period) 140 | 141 | return(oS, n_v) 142 | 143 | def split(self, xi, si, period, n_v): 144 | 145 | 146 | """ 147 | split(self, xi, si, period, n_v) 148 | 149 | split data between training and test set. 150 | Input: 151 | - xi = the whole dataset 152 | - si = index of the first period to be included in the test set 153 | - period = periodicity of the time series. 154 | - n_v = number of periods to be included in the test set 155 | Returns: 156 | - c = training set 157 | - v = test set 158 | """ 159 | 160 | # Create index of test set data 161 | cut = np.arange(si*period, (si + n_v)*period) 162 | 163 | # Extract index-corresponding data for test, delete for training 164 | v = xi[cut.astype(int), :] 165 | c = np.delete(xi, cut.astype(int), axis = 0) 166 | 167 | return(c, v) 168 | 169 | -------------------------------------------------------------------------------- /protocol/data_transformation.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 15:39:01 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import os 10 | import math 11 | import configparser 12 | import protocol.utils as ut 13 | 14 | class data_transformation(): 15 | 16 | """ 17 | data_tranformation prepares the data for the following step by either removing 18 | the seasonal component or normalizing them in the range 0-1. 19 | User-defined options, to be specified in the norm_config.txt: 20 | - min_max to normalize in the range 0-1 (default) 21 | - seasonal to remove the seasonal component 22 | """ 23 | 24 | def __init__(self): 25 | 26 | Config = configparser.ConfigParser() 27 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 28 | 29 | self.mode = str(Config.get('Data_Transformation','mode')) # user-defined transformation 30 | self.ma_f = int(Config.get('Data_Transformation','ma_f')) # user-defined transformation 31 | 32 | def normalize(self, c, v): 33 | 34 | """ 35 | normalize(self, c, v) 36 | 37 | Normalize the data in the range 0.1-0.9. 38 | 39 | Input: 40 | - c = training set 41 | - v = test set 42 | Returns: 43 | - cn = normalized training set 44 | - vn = normalized test set 45 | - mn = minimum (for time series reconstruction) 46 | - mX = maximum (for time series reconstruction) 47 | 48 | """ 49 | 50 | # compute minimum and maximum of each column 51 | mn = np.min(c, axis = 0) 52 | mX = np.max(c, axis = 0) 53 | 54 | # allocate memory for output data-sets 55 | cn = np.empty(c.shape) 56 | vn = np.empty(v.shape) 57 | 58 | # Normalize 59 | for i in range(0, cn.shape[1]): 60 | 61 | cn[:, i] = ((0.8*(c[:, i]-mn[i]))/(mX[i]-mn[i]))+0.1 62 | 63 | try: 64 | vn[:, i] = ((0.8*(v[:, i]-mn[i]))/(mX[i]-mn[i]))+0.1 65 | except: 66 | print('no' + str(i) + 'th column, check if this operation is carried in forecast mode') 67 | 68 | return(cn, vn, mn , mX) 69 | 70 | 71 | def moving_average(self, ci, nY, f, period): 72 | 73 | """ 74 | moving_average(self, ci, nY, f, period) 75 | 76 | Compute the moving average of a time series to remove noise. 77 | 78 | Input: 79 | - ci = ith column of the training set 80 | - nY = number of periods 81 | - f = moving window length 82 | - period = time-series periodicity 83 | Returns: 84 | - mi = moving average (size = len(period)) 85 | - m = moving average (size = len(ci)) 86 | """ 87 | 88 | # reshape the columns for it to be period, nY 89 | ci = np.reshape(ci, newshape = (period, nY), order = 'F') 90 | shp = ci.shape 91 | 92 | # Build the matrix to compute the moving average 93 | Y_up = np.c_[ 94 | ci[shp[0]-f:shp[0], shp[1]-1], 95 | ci[shp[0]-f:shp[0], :shp[1]-1] 96 | ] 97 | 98 | Y_d = np.c_[ 99 | ci[:f, 1:shp[1]], 100 | ci[:f, 0] 101 | ] 102 | 103 | Y = np.concatenate( 104 | (Y_up, ci, Y_d) 105 | ) 106 | 107 | # allocate memory for moving average 108 | mi = np.empty(shape = period) 109 | 110 | # compute moving average 111 | for k in range(0, period): 112 | 113 | mi[k] = np.mean(Y[k:k+2*f, :]) 114 | 115 | # repeat data for the same length of the training set. 116 | mm = np.tile(mi, nY ) 117 | 118 | return(mi, mm) 119 | 120 | 121 | 122 | def remove_season(self, c, v, period): 123 | 124 | """ 125 | remove_season(self, c, v, period) 126 | 127 | Remove seasonality from data, such that z = (x - ut)/sigmat, 128 | where ut is the ciclostationary average with moving windowm and 129 | sigmat is the ciclostationary variance. 130 | 131 | Input: 132 | - c = training set 133 | - v = test set 134 | - period = time-series periodicity 135 | Returns: 136 | - c_s = de-seasonalized training set 137 | - v_s = de-seasonalized test set 138 | - uc = ciclostationary mean 139 | - var_c = ciclostationary variance 140 | """ 141 | 142 | 143 | # Get the integers of the number of years in the training set 144 | nY = math.floor(c.shape[0]/period) 145 | ex = c.shape[0] - nY*period 146 | 147 | # Truncate training set and allocate memory for de-sesonalized training 148 | c_trunc = c[:c.shape[0]-ex,:] 149 | c_s = np.ndarray(shape = c.shape) 150 | 151 | # Get the integers of the number of years in the training set 152 | nY_v = math.floor(v.shape[0]/period) 153 | ex_v = v.shape[0] - nY_v*period 154 | 155 | # Allocate memory for de-seasonalized test set 156 | v_s = np.ndarray(shape = v.shape) 157 | 158 | # De-seasonalize each column 159 | for i in range(0, c_trunc.shape[1]): 160 | 161 | ci = c_trunc[:, i] 162 | 163 | # Compute ciclostazionary moving average and variance 164 | uc, uci = self.moving_average(ci, nY, self.ma_f, period) 165 | var_c, var_ci = self.moving_average( np.power(ci-uci, 2), nY, self.ma_f, period) 166 | 167 | var_c = np.sqrt(var_c) 168 | 169 | # Concatenate for the extra days 170 | uc_i = ut.concat_ciclo(uc, nY, ex) 171 | var_ci = ut.concat_ciclo(var_c, nY, ex) 172 | 173 | # Normalize data 174 | np.seterr(invalid='ignore') 175 | c_s[:, i] = (c[:, i] - uc_i)/var_ci 176 | c_s[:, i][np.isnan(c_s[:, i])] = 0 177 | 178 | # Concatenate for the extra days in the test set (using cyclo mean 179 | # and variance computed in the training set) 180 | uv = ut.concat_ciclo(uc, nY_v, ex_v) 181 | var_v = ut.concat_ciclo(var_c, nY_v, ex_v) 182 | 183 | # Transform test set 184 | v_s[:, i] = (v[:, i] - uv)/var_v 185 | v_s[:, i][np.isnan(v_s[:, i])] = 0 186 | 187 | return(c_s, v_s, uc, var_c) 188 | 189 | 190 | 191 | def transform_data(self, c, v, period): 192 | 193 | """ 194 | transform_data(self, c, v, period) 195 | 196 | Tranform the data accordig to the user's choice 197 | 198 | Input: 199 | - c = training set 200 | - v = test set 201 | - period = time-series periodicity 202 | Returns: 203 | - cn = normalized training set 204 | - vn = normalized test set 205 | - mn = minimum (for time series reconstruction) 206 | - mX = maximum (for time series reconstruction) 207 | """ 208 | 209 | # Switch and normalize 210 | if self.mode == 'seasonal': 211 | cn, vn, mn, mX = self.remove_season(c, v, period) 212 | elif self.mode == 'min_max': 213 | cn, vn, mn, mX = self.normalize(c, v) 214 | else: 215 | raise ValueError("normalization method should be either min_max or seasonal") 216 | 217 | return(cn, vn, mn, mX) 218 | 219 | 220 | -------------------------------------------------------------------------------- /protocol/ivs.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 12:32:18 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import os 10 | import math 11 | import numpy.matlib 12 | import configparser 13 | import matplotlib.pyplot as plt 14 | from sklearn.metrics import mean_squared_error 15 | from hydroeval import evaluator, nse 16 | from sklearn.svm import SVR 17 | 18 | import protocol.utils as ut 19 | 20 | class input_variable_selection(): 21 | 22 | """ 23 | input_variable_selection select the predictors for the data-driven models 24 | Three IVS methods are currently employed (below the method and the keyword 25 | to be specified in ivs_config.txt): 26 | - forward selection (forward_selection) - default; 27 | - model-based correlation (correlation); 28 | - exhaustive search (exhaustive) 29 | """ 30 | 31 | 32 | def __init__(self): 33 | 34 | Config = configparser.ConfigParser() 35 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 36 | 37 | # Ivs param 38 | self.ivs_mode = str(Config.get('IVS','ivs_method')) # IVS method 39 | self.np_max = int(Config.get('IVS','max_predictor')) # Maximum number of predictors 40 | self.tol_max = float(Config.get('IVS','max_tol')) # Tolerance stopping criteria 41 | self.n_folds = int(Config.get('IVS','n_split')) # Number of folds in cross-validation 42 | 43 | 44 | def run_model_fold(self, xc, yc): 45 | 46 | """ 47 | run_model_fold(xc, yc) 48 | 49 | Trains k (where k = number of folds) SVM models, and returns the average 50 | RMSE and NSE across the folds. 51 | 52 | Input: 53 | - xc = input subset 54 | - yc = output 55 | - n.folds = number of folds in the cross-validation 56 | 57 | Returns: 58 | - err = average RMSE across the folds 59 | - err_NS = average NSE across the folds 60 | """ 61 | 62 | err = 0 # RMSE 63 | err_NS = 0 # NSE 64 | l_fold = int(xc.shape[0]/self.n_folds) # Length of each fold 65 | 66 | # Iterate across folds 67 | for f in range(0, self.n_folds): 68 | 69 | # Extract input and output training data for the k-th fold 70 | cut = np.arange(f*l_fold, (f+1)*l_fold) 71 | 72 | x_tr = np.delete(xc, cut.astype(int), axis = 0) 73 | y_tr = np.delete(yc, cut.astype(int), axis = 0) 74 | 75 | # Extract input and output cv data for the k-th fold 76 | if xc.shape[1] > 1: 77 | x_cv = xc[cut.astype(int), :] 78 | else: 79 | x_cv = xc[cut.astype(int)] 80 | 81 | y_cv = yc[cut.astype(int)] 82 | 83 | # Train the SVM model 84 | ddm = SVR(kernel='rbf', 85 | gamma='auto', 86 | coef0=0.0, 87 | tol=0.01, 88 | C=3, 89 | shrinking=True, 90 | cache_size=200, 91 | verbose=False, 92 | max_iter=-1) 93 | 94 | mod = ddm.fit(x_tr, y_tr) 95 | y_theta = mod.predict(x_cv) 96 | 97 | # Compute the error metrics 98 | err = err + math.sqrt(mean_squared_error(y_theta, y_cv)) 99 | err_NS = err_NS + evaluator(nse, y_theta, y_cv)[0] 100 | 101 | # Extract the average 102 | err = err/self.n_folds 103 | err_NS = err_NS/self.n_folds 104 | 105 | return(err, err_NS) 106 | 107 | def ivs_correlation(self, cn, case_study): 108 | 109 | """ 110 | ivs_coorelation(self, cn) 111 | 112 | Iterative procedure, model-based. Input candidates are ranked based on the 113 | cross-correlation with the output. Each predictor is iteratively added 114 | to the input subset according with the ranking. 115 | The procedure stops when either no further improvements are achieved or 116 | the maximum number of predictors is reached. 117 | 118 | Input: 119 | - cn = training set 120 | - tol_max = objective function improvement for stopping criteria 121 | - np_max = maximum number of predictors 122 | 123 | Returns: 124 | - column index(es) of the selected predictor(s) 125 | """ 126 | 127 | performance = [] 128 | 129 | # Compute cross-correlation and sort input candidates 130 | c_mat = np.corrcoef(cn, rowvar = False)[cn.shape[1]-1, :cn.shape[1]-1] 131 | prev = c_mat.argsort()[::-1] 132 | 133 | n_pr = 0 134 | 135 | # Iterate through predictors 136 | while n_pr < self.np_max: 137 | 138 | yc = cn[:, cn.shape[1]-1] 139 | xc = cn[:, prev[0:n_pr+1]] 140 | 141 | performance.append(self.run_model_fold(xc, yc)[1]) 142 | 143 | # Check for stopping criteria 144 | if n_pr > 0: 145 | tol = performance[n_pr] - performance[n_pr-1] 146 | 147 | if tol < self.tol_max: 148 | 149 | n_pr -= 1 150 | break 151 | 152 | n_pr += 1 153 | 154 | ut.plot_improvement(prev[0:n_pr+1], np.array(performance[0:n_pr+1]), case_study) 155 | return(prev[0:n_pr+1]) 156 | 157 | def ivs_forward_selection(self, cn, case_study): 158 | 159 | """ 160 | ivs_forward_selection(self, cn) 161 | 162 | Iterative procedure. Trains n = cn.shape[1] -1 SISO models, and extracts 163 | the predictor maximising some performance criteria. It then trains n-1 model, 164 | combining the selected predictor with each of the remaining candidates. 165 | The procedure stops when no further improvements are achieved. 166 | 167 | Input: 168 | - cn = training set 169 | - tol_max = objective function improvement for stopping criteria 170 | 171 | Returns: 172 | - column index(es) of the selected predictor(s) 173 | """ 174 | 175 | # forward_selection parameters 176 | k = 0 # counter 177 | n_max = cn.shape[1] -1 # maximum number of predictor 178 | i_s = [] # selected columns 179 | p_box = [] # improvement vector 180 | 181 | # Iterate across all predictors 182 | while k < cn.shape[1] -1: 183 | 184 | # Train n SISO models, and select the best predictor 185 | if k == 0: 186 | 187 | performance = [] 188 | 189 | for i in range(0, n_max): 190 | 191 | yc = cn[:, cn.shape[1]-1] 192 | xc = cn[:, [i]] 193 | 194 | performance.append(self.run_model_fold(xc, yc)[1]) 195 | 196 | # Add to the selected input set the best predictor, save the performance 197 | i_s.append(np.argmax(performance)) 198 | p_max = np.max(performance) 199 | p_box.append(p_max) 200 | p_graph = np.array(performance) 201 | k = k + 1 202 | 203 | else: 204 | 205 | 206 | # Train n-len(i_s) MISO models ultis stopping criteria is met 207 | performance = [] 208 | 209 | var_range = np.arange(n_max) 210 | var_range = np.delete(var_range, i_s) 211 | 212 | # Iterate along the remaining candidates 213 | for i in var_range: 214 | 215 | yc = cn[:, cn.shape[1]-1] # Output 216 | e_c = np.hstack([i, i_s]) # Candidate input index 217 | xc = cn[:, e_c] # Candidate input set 218 | 219 | # Train the model and save input candidate performance 220 | performance.append(self.run_model_fold(xc, yc)[1]) 221 | 222 | k = k + 1 223 | 224 | # Extract the MISO best performance and performance index 225 | m_i = np.argmax(performance) 226 | p_max_new = np.max(performance) 227 | 228 | # Add new line to performance matrix 229 | pn = np.empty(n_max) 230 | pn[i_s] = float('nan') 231 | 232 | m = 0 233 | for j in range(0, n_max): 234 | 235 | if j not in i_s: 236 | pn[j] = performance[m] 237 | m = m + 1 238 | 239 | p_graph = np.vstack([p_graph, pn]) 240 | 241 | # Check if stopping criteia is met 242 | if p_max_new - p_max > self.tol_max: 243 | 244 | 245 | p_max = p_max_new 246 | p_box.append(p_max) 247 | i_s.append(var_range[m_i]) 248 | 249 | else: 250 | break 251 | 252 | # Plot IVS results 253 | ut.plot_ivs_forward(p_graph, i_s, p_box, case_study) 254 | 255 | return(i_s) 256 | 257 | 258 | def ivs_exhaustive_search(self, cn, case_study): 259 | 260 | """ 261 | ivs_exhaustive_search(self, cn) 262 | 263 | Tries all the possible input combination, extracts the one minimizing the 264 | RMSE in the cross-validation set. 265 | 266 | Input: 267 | - cn = training set 268 | 269 | Returns: 270 | - column index(es) of the selected predictor(s) 271 | """ 272 | 273 | # Generate set of all possible input combination 274 | i_sp = ut.exhaustive_set(cn) 275 | performance = np.empty(len(i_sp)) 276 | 277 | # Iterate through combinations 278 | for j in range(0, len(i_sp)): 279 | 280 | i_ss = i_sp[j] 281 | 282 | xc = cn[:, i_ss] 283 | yc = cn[:, cn.shape[1]-1] 284 | 285 | # Test the candidate input and return the RMSE 286 | performance[j] = self.run_model_fold(xc, yc)[0] 287 | 288 | # Extract the best performance 289 | idx = np.argmin(performance).astype(int) 290 | ut.plot_box(performance, case_study) 291 | 292 | return(i_sp[idx]) 293 | 294 | 295 | def select_input(self, cn, case_study): 296 | 297 | """ 298 | 299 | select_input(self, cn) 300 | 301 | Selects the predictors for the data-driven models 302 | 303 | Input: 304 | - self.ivs_mode = input variable selection method 305 | - cn = training set 306 | 307 | Returns: 308 | - i_s = column index(es) of the selected predictor(s) 309 | """ 310 | 311 | # Inspect the method and run the ivs procedure 312 | if self.ivs_mode == 'correlation': 313 | i_s = self.ivs_correlation(cn, case_study) 314 | 315 | elif self.ivs_mode == 'forward_selection': 316 | i_s = self.ivs_forward_selection(cn, case_study) 317 | 318 | elif self.ivs_mode == 'exhaustive': 319 | i_s = self.ivs_exhaustive_search(cn, case_study) 320 | 321 | else: 322 | i_s = np.arange(cn.shape[1]-1) 323 | 324 | return(i_s) 325 | 326 | 327 | -------------------------------------------------------------------------------- /protocol/model_testing.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 16:58:05 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import numpy.matlib 10 | import configparser 11 | import os 12 | import math 13 | from hydroeval import evaluator, nse 14 | import protocol.utils as ut 15 | 16 | class model_testing(): 17 | 18 | """ 19 | test the optimal models (defined by model_training) and evaluate their 20 | performance in the test set 21 | """ 22 | 23 | def __init__(self): 24 | 25 | Config = configparser.ConfigParser() 26 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 27 | 28 | 29 | self.mode = str(Config.get('Data_Transformation','mode')) 30 | 31 | def de_normalize(self, xn, a, b, mn, mX): 32 | 33 | """ 34 | de_normalize(self, xn, a, b, mn, mX) 35 | 36 | invert normalization 37 | 38 | Input: 39 | - xn = normalized data 40 | - a = upper bound 41 | - b = lower bound 42 | - mn = minimum 43 | - mX = maximum 44 | 45 | Returns: 46 | - x = de-normalized data 47 | """ 48 | 49 | x = mn + (((xn-b)/(a-b))*(mX-mn)) 50 | 51 | return(x) 52 | 53 | def reconstruct_season(self, y, mn, mX, period): 54 | 55 | """ 56 | reconstruct_season(self, y, mn, mX, period) 57 | 58 | add seasonality 59 | 60 | Input: 61 | - y = normalized data 62 | - mn = ciclostationary average 63 | - mX = ciclostationary variance 64 | - period = periodicity of the time series 65 | 66 | Returns: 67 | - yr = reconstrunced data 68 | """ 69 | 70 | # Season reconstruction parameters 71 | ln = y.shape[0] 72 | nY = math.floor(ln/period) 73 | ex = ln - nY*period 74 | 75 | # Ciclostationary mean and variance 76 | uv = ut.concat_ciclo(mn, nY, ex) 77 | var_v = ut.concat_ciclo(mX, nY, ex) 78 | 79 | # Reconstruct data 80 | yr = y*var_v + uv 81 | 82 | return(yr) 83 | 84 | 85 | def reconstruct_pattern(self, y_ext_c, y_ext_v, mn, mX, period): 86 | 87 | """ 88 | reconstruct_pattern(self, y_ext_c, y_ext_v, mn, mX, period) 89 | 90 | Reconstruct the original time-series to re-convert from data-transformation 91 | 92 | Input: 93 | - y_ext_c = predicted output in the training set 94 | - y_ext_v = predicted output in the test set 95 | - mn, mX = reconstruction parameters 96 | - period = seasonality of the time-series 97 | 98 | Output: 99 | - yc_theta = reconstructed training set 100 | - yv_theta = reconstructed test set 101 | """ 102 | 103 | if self.mode == 'seasonal': 104 | yc_theta = self.reconstruct_season(y_ext_c, mn, mX, period) 105 | yv_theta = self.reconstruct_season(y_ext_v, mn, mX, period) 106 | elif self.mode == 'min_max': 107 | yc_theta = self.de_normalize(y_ext_c, 0.9, 0.1, mn[len(mn)-1], mX[len(mX)-1]) 108 | yv_theta = self.de_normalize(y_ext_v, 0.9, 0.1, mn[len(mn)-1], mX[len(mX)-1]) 109 | else: 110 | raise ValueError("normalization method should be either min_max or seasonal") 111 | 112 | 113 | return(yc_theta, yv_theta) 114 | 115 | 116 | def test_model(self, ms, cn, vn, column_index, mn, mX, model, period): 117 | 118 | """ 119 | test_model(self, ms, cn, vn, column_index, mn, mX, period) 120 | 121 | Evaluate modelling performance in the test set, and returns the 122 | reconstructed predicted time-series 123 | 124 | Input: 125 | - ms = the modelling ensamble selected through model_training 126 | - cn = the normalized calibration set 127 | - vn = the normalized validation set 128 | - column_index = the selected input via IVS 129 | - mn, mX = time series reconstruction paramaeters 130 | - period = periodicity or seasonality of the time-series 131 | 132 | Returns: 133 | - yc_rec = reconstructed training set 134 | - yv_rec = reconstructed test set 135 | - eps_c = NSE in the training set 136 | - eps_v = NSE in the test set 137 | - res = residuals 138 | """ 139 | 140 | xc = cn[:, column_index] # Training input 141 | yc = cn[:, cn.shape[1]-1] # Training output 142 | 143 | xv = vn[:, column_index] # Test input 144 | yv = vn[:, vn.shape[1]-1] # Test output 145 | 146 | # Allocate memory for training and test output 147 | y_theta_v = np.ndarray(shape = (yv.shape[0], len(ms))) 148 | y_theta_c = np.ndarray(shape = (yc.shape[0], len(ms))) 149 | 150 | # Perform ensamble prediction 151 | for j in range(0, len(ms)): 152 | 153 | if model == 'ann' or model == 'svm': 154 | 155 | y_theta_v[:, j] = ms[j].predict(xv) 156 | y_theta_c[:, j] = ms[j].predict(xc) 157 | 158 | else: 159 | try: 160 | modulename = model + '_module' 161 | new_module = __import__(modulename) 162 | 163 | y_theta_v[:, j] = new_module.test_module(ms[j], xv) 164 | y_theta_c[:, j] = new_module.test_module(ms[j], xc) 165 | 166 | except: 167 | 168 | raise ValueError('No module for model' + model + 'specified') 169 | 170 | 171 | # Aggregate the forecast 172 | y_ext_v = np.mean(y_theta_v, axis = 1) 173 | y_ext_c = np.mean(y_theta_c, axis = 1) 174 | 175 | # Conpute the residuals 176 | res = yc - y_ext_c 177 | 178 | # Reconstruct time-series 179 | yc_rec, yv_rec = self.reconstruct_pattern(y_ext_c, y_ext_v, mn, mX, period) 180 | yc_o_rec, yv_o_rec = self.reconstruct_pattern(yc, yv, mn, mX, period) 181 | 182 | # Compute the error statistics 183 | eps_v = evaluator(nse, yv_rec, yv_o_rec)[0] 184 | eps_c = evaluator(nse, yc_rec, yc_o_rec)[0] 185 | 186 | return(yc_rec, yv_rec, eps_c, eps_v, res) -------------------------------------------------------------------------------- /protocol/model_training.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 16:09:53 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import math 10 | import numpy.matlib 11 | import configparser 12 | import os 13 | from sklearn.metrics import mean_squared_error 14 | from sklearn.svm import SVR 15 | from sklearn.neural_network import MLPRegressor 16 | import itertools 17 | 18 | import protocol.utils as ut 19 | 20 | class model_training(): 21 | 22 | """ 23 | model_training select the optimal model architecture and the optimal hyper- 24 | parameters value based on a k-fold cross-validation: 25 | """ 26 | 27 | def __init__(self): 28 | 29 | Config = configparser.ConfigParser() 30 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 31 | 32 | self.n_folds = int(Config.get('IVS','n_split')) 33 | 34 | 35 | def gen_hyperparam_space(self, model): 36 | 37 | """ 38 | gen_hyperparam_space(self, model) 39 | 40 | Generate the hyper-parameter space for the selected model. 41 | 42 | Input: 43 | model = the selected model 44 | 45 | Returns: 46 | hyper param = the models candidate hyper parameters: 47 | ANN {activation function, number of neurons, number of iterations, 48 | alfa and learning rate} 49 | SVM {kernel type, eps, C and tolerance} 50 | """ 51 | Config = configparser.ConfigParser() 52 | Config.read(os.path.join('protocol', 'advanced_configurations.txt')) 53 | 54 | # Read the hyper parameters set from the configuration settings file 55 | if model == 'svm': 56 | krn = np.array([e.strip() for e in Config.get('Model_Training', 'krn').split(',')]).tolist() 57 | eps = np.array([e.strip() for e in Config.get('Model_Training', 'n_eps').split(',')]).tolist() 58 | nodes = np.array([e.strip() for e in Config.get('Model_Training', 'n_C').split(',')]).tolist() 59 | tol = np.array([e.strip() for e in Config.get('Model_Training', 'n_tol').split(',')]).tolist() 60 | 61 | # Generate hyperparameters cartesian product 62 | ls = [krn, eps, nodes, tol] 63 | hyper_param = [] 64 | 65 | for element in itertools.product(*ls): 66 | hyper_param.append(element) 67 | 68 | 69 | elif model == 'ann': 70 | af = np.array([e.strip() for e in Config.get('Model_Training', 'activation').split(',')]).tolist() 71 | n_neu = np.array([e.strip() for e in Config.get('Model_Training', 'neurons').split(',')]).tolist() 72 | n_it = np.array([e.strip() for e in Config.get('Model_Training', 'iter').split(',')]).tolist() 73 | alpha = np.array([e.strip() for e in Config.get('Model_Training', 'alfa').split(',')]).tolist() 74 | learn = np.array([e.strip() for e in Config.get('Model_Training', 'learning').split(',')]).tolist() 75 | 76 | # Generate hyperparameters cartesian product 77 | ls = [af, n_neu, n_it, alpha, learn] 78 | hyper_param = [] 79 | 80 | for element in itertools.product(*ls): 81 | hyper_param.append(element) 82 | 83 | else: 84 | 85 | try: 86 | Config.read(os.path.join('protocol', model + '_module_config.txt')) 87 | n_param = int(Config.get('dimensionality', 'n_param')) 88 | 89 | ls = [] 90 | for c in range(n_param): 91 | p_name = 'p' + str(c) 92 | ls.append(np.array([e.strip() for e in Config.get('param_values', p_name).split(',')]).tolist()) 93 | 94 | hyper_param = [] 95 | 96 | for element in itertools.product(*ls): 97 | hyper_param.append(element) 98 | 99 | 100 | 101 | except: 102 | print('warning, no model-specific configuration settings file specified') 103 | 104 | return(hyper_param) 105 | 106 | 107 | def train_model_node(self, xc, yc, hp, model): 108 | 109 | """ 110 | train_model_node(self, model) 111 | 112 | Given a hyper-parameter configuration, trains k-models (one per each fold) 113 | 114 | Input: 115 | - xc = input subset 116 | - yc = output 117 | - hp = the hyper-parameters combination 118 | - model = the selected model 119 | 120 | Returns: 121 | - models_node = the trained model 122 | - error_node = the error computed in each fold 123 | """ 124 | 125 | # Allocate memory for the outputs 126 | models_node = np.ndarray(shape = (self.n_folds, ), dtype = 'object') # Trained models 127 | errors_node = np.ndarray(shape = (self.n_folds, )) # Corresponding error 128 | 129 | # Compute the length of each fold 130 | l_fold = int(xc.shape[0]/self.n_folds) 131 | 132 | 133 | for f in range(0, self.n_folds): 134 | 135 | cut = np.arange(f*l_fold, (f+1)*l_fold) 136 | 137 | # Prepare training and cv data 138 | x_tr = np.delete(xc, cut.astype(int), axis = 0) 139 | y_tr = np.delete(yc, cut.astype(int), axis = 0) 140 | 141 | if xc.shape[1] > 1: 142 | x_cv = xc[cut.astype(int), :] 143 | else: 144 | x_cv = xc[cut.astype(int)] 145 | 146 | y_cv = yc[cut.astype(int)] 147 | 148 | # Train the models 149 | if model == 'svm': 150 | 151 | ddm = SVR(kernel=hp[0], 152 | gamma='auto', 153 | coef0=0.0, 154 | tol=float(hp[3]), 155 | C=float(hp[2]), 156 | epsilon = float(hp[1]), 157 | shrinking=True, 158 | cache_size=200, 159 | verbose=False, 160 | max_iter=-1) 161 | models_node[f] = ddm.fit(x_tr, y_tr) 162 | y_theta = models_node[f].predict(x_cv) 163 | 164 | elif model == 'ann': 165 | 166 | ddm = MLPRegressor(hidden_layer_sizes = (int(hp[1])), 167 | activation = hp[0], 168 | learning_rate = hp[4], 169 | max_iter = int(hp[2]), 170 | alpha = float(hp[3]) 171 | ) 172 | models_node[f] = ddm.fit(x_tr, y_tr) 173 | y_theta = models_node[f].predict(x_cv) 174 | else: 175 | try: 176 | modulename = model + '_module' 177 | new_module = __import__(modulename) 178 | 179 | models_node[f], y_theta = new_module.train_module(x_tr, x_cv, y_tr, hp) 180 | 181 | except: 182 | 183 | raise ValueError('No module for model' + model + 'specified') 184 | 185 | # Compute the error in each fold 186 | 187 | errors_node[f] = math.sqrt(mean_squared_error(y_theta, y_cv)) 188 | 189 | 190 | return(models_node, errors_node) 191 | 192 | 193 | def train_model(self, cn, column_index, model, case_study): 194 | 195 | """ 196 | train_model(self, cn, column_index, model) 197 | 198 | Iterate across the hyper parameters space and models architectures, to select 199 | the optimal one. 200 | 201 | Input: 202 | - cn = calibration set 203 | - column_index = selected input from IVS 204 | - model = the selected modelling techinique 205 | 206 | Returns: 207 | - ms = the selected optimal modelling ensamble 208 | """ 209 | 210 | # Generate hyper parameters space 211 | hyper_param = self.gen_hyperparam_space(model) 212 | 213 | # Allocate memory for models and errors 214 | models = np.ndarray(shape = (len(hyper_param), self.n_folds), dtype = 'object') 215 | error = np.ndarray(shape = (len(hyper_param), self.n_folds)) 216 | 217 | # Divide input and output 218 | xc = cn[:, column_index] 219 | yc = cn[:, cn.shape[1]-1] 220 | 221 | # Iterate across the hyper parameter space 222 | for i in range(0, len(hyper_param)): 223 | 224 | models[i, :], error[i, :] = self.train_model_node(xc, yc, hyper_param[i], model) 225 | 226 | # Get the index of the minimum error for each fold 227 | idx = np.argmin(error, axis = 0) 228 | 229 | # Extract the optima model and corresponding error 230 | ms = np.ndarray(shape = (len(idx) ,), dtype = 'object' ) 231 | eps = np.ndarray(shape = (len(idx) ,)) 232 | 233 | ut.hyper_param_plot(hyper_param, error, case_study) 234 | for i in range(0, models.shape[1]): 235 | 236 | ms[i] = models[idx[i], i] 237 | eps[i] = error[idx[i], i] 238 | 239 | 240 | return(ms) -------------------------------------------------------------------------------- /protocol/postprocess.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 17:30:04 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import numpy.matlib 10 | import matplotlib.pyplot as plt 11 | import configparser 12 | import os 13 | import pandas as pd 14 | from hydroeval import evaluator, nse 15 | 16 | class postprocess(): 17 | 18 | """ 19 | save and plot the results 20 | """ 21 | 22 | def __init__(self): 23 | 24 | Config = configparser.ConfigParser() 25 | Config.read('configuration_settings.txt') 26 | 27 | self.output_name = str(Config.get('Case_study','name')) # Case study name 28 | 29 | def save_forecasts(self, yc_rec, yv_rec, c, v, si, i, n_v, period, model, case_study): 30 | 31 | """ 32 | save_forecasts(self, yc_rec, yv_rec, c, v, si, i, n_v, period, model, case_study) 33 | 34 | Save the modelling results. 35 | 36 | Input: 37 | - yc_rec = reconstructed training set 38 | - yv_rec = reconstructed test set 39 | - c = training set 40 | - v = test set 41 | - si = split index 42 | - i = file index (only if multiple files are in a single folder) 43 | - n_v = length of test set 44 | - period = time-series periodicity 45 | - model = selected modelling techniquqe 46 | - case_study = case study name 47 | 48 | Returns: 49 | - yr = predicted y 50 | - yo = observed y 51 | - cs, vs = training and test set index 52 | """ 53 | 54 | # Allocate memory for observed and predicted TS 55 | yr = np.ndarray(shape = (yc_rec.shape[0] + yv_rec.shape[0], )) 56 | yo = np.ndarray(shape = (yc_rec.shape[0] + yv_rec.shape[0], )) 57 | 58 | # Index training and test set 59 | vs = np.arange(si*period, (si+n_v)*period) 60 | cs = np.delete(np.arange(0, yr.shape[0]), vs.astype(int)) 61 | 62 | # Insert predicted data according to the index 63 | yr[vs.astype(int)] = yv_rec 64 | yr[cs.astype(int)] = yc_rec 65 | 66 | yo[vs.astype(int)] = v[:, v.shape[1]-1] 67 | yo[cs.astype(int)] = c[:, c.shape[1]-1] 68 | 69 | 70 | d = {'Observed': yo, 'Predicted' : yr} 71 | df = pd.DataFrame(data = d) 72 | 73 | # Save output 74 | fn = self.output_name + '_' + str(i) + model + '.csv' 75 | fs = os.path.join('output', case_study, fn) 76 | 77 | df.to_csv(fs, index = False) 78 | 79 | 80 | return(yr, yo, vs, cs) 81 | 82 | 83 | def plot_forecasts(self, yr, yo, vs, cs, i, period, model, case_study, start, stop): 84 | 85 | """ 86 | plot_forecasts(self, yr, yo, vs, cs, i, period, model, case_study, start, stop): 87 | 88 | Plot modelling results. 89 | 90 | Input: 91 | - yr = forecasts 92 | - yo = observation 93 | - cs = training set index 94 | - vs = test set index 95 | - i = file index (only if multiple files are in a single folder) 96 | - period = time-series periodicity 97 | - model = selected modelling techniquqe 98 | - case_study = case study name 99 | - start = initial year 100 | - stop = final year 101 | """ 102 | 103 | self.plot_ts(yr, yo, vs, cs, i, period, model, case_study, start, stop) 104 | self.plot_scatter(yr, yo, vs, cs, model, case_study) 105 | 106 | 107 | def plot_ts(self, yr, yo, vs, cs, i, period, model, case_study, start, stop): 108 | 109 | # Define time step 110 | if period == 12: 111 | step = 'months' 112 | elif period == 365: 113 | step = 'days' 114 | elif period == 8760: 115 | step = 'hours' 116 | else: 117 | raise ValueError("system periodiciti not included") 118 | 119 | # Initialize the plot 120 | fig, ax = plt.subplots(constrained_layout=True, figsize=(16, 6)) 121 | 122 | xlb = 'Time (' + step + ')' 123 | 124 | x = np.arange(0, len(yr)) 125 | 126 | # Line plot 127 | ax.plot(x, yo, label = 'Observed', color = '#3E065F') 128 | ax.plot(x, yr, label = 'Predicted', color = '#FF0075') 129 | 130 | # Scatter 131 | ax.scatter(x[cs.astype(int)], yr[cs.astype(int)], label = 'Training', color = '#69DADB') 132 | ax.scatter(x[vs.astype(int)], yr[vs.astype(int)], label = 'Test', color = '#FEE440') 133 | 134 | # Legend 135 | ax.legend(bbox_to_anchor=(0., 1.02, 1., .102), loc='lower left', ncol=4, mode="expand", borderaxespad=0.) 136 | 137 | # Error stats 138 | tx = 'NSE = ' + str(np.round(evaluator(nse, yr[vs.astype(int)], yo[vs.astype(int)])[0], 2)) 139 | tx_p = max(np.max(yo), np.max(yr)) 140 | ax.text(0.1, tx_p, tx, fontsize = 16, weight = 'bold') 141 | 142 | # Adjust graphics 143 | ax.set_xlabel(xlb, fontsize = 20) 144 | ax.set_ylabel('Discharge', fontsize = 20) 145 | 146 | ax.autoscale(enable=True, axis='x', tight=True) 147 | 148 | tck = np.arange(min(x), max(x), period) 149 | tck_l = np.arange(start, stop + 1) 150 | 151 | if len(tck) < 10: 152 | 153 | ax.set_xticks(tck) 154 | ax.set_xticklabels(tck_l) 155 | else: 156 | 157 | pl_stop = np.linspace(0, len(tck), num = 10).astype(int) 158 | ax.set_xticks(tck[pl_stop[:len(pl_stop)-1]]) 159 | ax.set_xticklabels(tck_l[pl_stop[:len(pl_stop)-1]]) 160 | 161 | # Save 162 | fn = self.output_name + '_' + str(i) + model + '.png' 163 | fs = os.path.join('output', case_study, fn) 164 | 165 | fig.savefig(fs) 166 | 167 | def plot_scatter(self, yr, yo, vs, cs, model, case_study): 168 | 169 | fig, ax = plt.subplots(1, 2, constrained_layout=True, figsize=(16,6)) 170 | 171 | X = np.vstack([yr, yo]) 172 | lb = np.min(X) 173 | ub = np.max(X) 174 | 175 | ax[0].scatter(yo[cs.astype(int)], yr[cs.astype(int)], color = '#69DADB', alpha = 0.8) 176 | ax[0].plot(np.arange(lb, ub + 1), np.arange(lb, ub + 1)) 177 | # Adjust graphics 178 | ax[0].set_xlabel('Observed', fontsize = 18) 179 | ax[0].set_ylabel('Predicted', fontsize = 18) 180 | ax[0].set_title('Training set', fontsize = 20) 181 | ax[0].set_xlim(lb, ub) 182 | ax[0].set_ylim(lb, ub) 183 | tx = 'NSE = ' + str(np.round(evaluator(nse, yr[cs.astype(int)], yo[cs.astype(int)])[0], 1)) 184 | ax[0].text(lb, ub-0.045*ub, tx, fontsize = 16, weight = 'bold') 185 | 186 | ax[1].scatter(yo[vs.astype(int)], yr[vs.astype(int)], color = '#FEE440', alpha = 0.8) 187 | ax[1].plot(np.arange(lb, ub + 1), np.arange(lb, ub + 1)) 188 | ax[1].set_xlabel('Observed', fontsize = 18) 189 | ax[1].set_ylabel('Predicted', fontsize = 18) 190 | ax[1].set_title('Test set', fontsize = 20) 191 | ax[1].set_xlim(lb, ub) 192 | ax[1].set_ylim(lb, ub) 193 | 194 | tx = 'NSE = ' + str(np.round(evaluator(nse, yr[vs.astype(int)], yo[vs.astype(int)])[0], 1)) 195 | ax[1].text(lb, ub-0.045*ub, tx, fontsize = 16, weight = 'bold') 196 | 197 | 198 | fn = self.output_name + '_scatter_' + model + '.png' 199 | fs = os.path.join('output', case_study, fn) 200 | 201 | fig.savefig(fs) 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | -------------------------------------------------------------------------------- /protocol/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Mon May 30 17:14:52 2022 4 | 5 | @author: AMARANTO 6 | """ 7 | 8 | import numpy as np 9 | import os 10 | import glob 11 | from scipy.stats import norm 12 | import matplotlib.pyplot as plt 13 | import configparser 14 | import pandas as pd 15 | import seaborn as sns 16 | 17 | def read_model_data(case_study, period): 18 | 19 | ext = '*.csv' 20 | fpth = os.path.join('input', case_study, ext) 21 | 22 | fls = glob.glob(fpth) 23 | 24 | X = np.ndarray(shape = (len(fls), 1), dtype = 'object') 25 | 26 | 27 | 28 | for i in range(0, len(fls)): 29 | 30 | X[i, 0] = np.genfromtxt(fls[i], delimiter = ',', skip_header = 1) 31 | 32 | 33 | return(X) 34 | 35 | def powerset(s): 36 | 37 | x = len(s) 38 | masks = [1 << i for i in range(x)] 39 | for i in range(1 << x): 40 | yield [ss for mask, ss in zip(masks, s) if i & mask] 41 | 42 | def exhaustive_set(cn): 43 | 44 | nCol = np.arange(cn.shape[1]-1) 45 | r = [x for x in powerset(nCol)] 46 | r.sort() 47 | 48 | return(r[1:]) 49 | 50 | def concat_ciclo(u, nY, ex): 51 | 52 | u_ex = u[:ex] 53 | ui = np.tile(u, nY) 54 | 55 | ui = np.hstack([ui, u_ex]) 56 | 57 | return(ui) 58 | 59 | def normalize_vector(x): 60 | 61 | xm = min(x) 62 | xM = max(x) 63 | 64 | y = (x - xm)/(xM - xm) 65 | 66 | return(y) 67 | 68 | 69 | def residuals_stats(r): 70 | 71 | 72 | mu, std = norm.fit(r) 73 | 74 | return(mu, std) 75 | 76 | def plot_ivs_forward(pg, i_s, perf, case_study): 77 | 78 | Config = configparser.ConfigParser() 79 | Config.read('configuration_settings.txt') 80 | 81 | # Output parameters 82 | output_location = 'output' # Output folder 83 | output_name = str(Config.get('Case_study','name')) # Case study name 84 | 85 | pg[pg<0] = 0 86 | fig, ax = plt.subplots(1, 2, figsize = (18,7), gridspec_kw={'width_ratios': [2.5, 1]}) 87 | 88 | sc = ax[0].imshow(pg, cmap='viridis', interpolation='nearest') 89 | #cax = fig.add_axes([ax[0].get_position().x1+0.01,ax[0].get_position().y0,0.02,ax[0].get_position().height]) 90 | cb = fig.colorbar(sc, ax = ax[0], location = 'bottom', shrink = 0.6) 91 | ax[0].set_xlabel('Input column', fontsize = 18) 92 | ax[0].set_ylabel('Iteration', fontsize = 18) 93 | ax[0].set_yticks(np.arange(pg.shape[0]), np.arange(pg.shape[0]) ) 94 | 95 | cb.set_label('NSE [-]') 96 | ax[0].set_title('Forward input selection results', fontsize = 20) 97 | 98 | plot_improvement_bar(i_s, perf, ax[1]) 99 | 100 | # Save 101 | fn = output_name + '_ivs_forward' + '.png' 102 | fs = os.path.join(output_location, case_study, fn) 103 | 104 | plt.subplots_adjust(wspace = 0.15) 105 | 106 | 107 | 108 | 109 | fig.savefig(fs) 110 | 111 | def hyper_param_plot(hyper_param, error, case_study): 112 | 113 | Config = configparser.ConfigParser() 114 | Config.read('configuration_settings.txt') 115 | 116 | # Output parameters 117 | output_location = 'output' # Output folder 118 | output_name = str(Config.get('Case_study','name')) # Case study name 119 | 120 | hp = np.asarray(hyper_param) 121 | e1 = np.mean(error, axis = 1) 122 | 123 | fig, ax = plt.subplots(1, hp.shape[1], constrained_layout=True, figsize = (18, 5)) 124 | 125 | for i in range(0, hp.shape[1]): 126 | 127 | hp_plot = np.transpose(np.vstack([hp[:, i], e1.astype(float)])) 128 | 129 | cnam = 'p_' + str(i) 130 | hp_plot = pd.DataFrame(hp_plot, columns = [cnam, 'RMSE_p']) 131 | hp_plot.iloc[:, 1] = pd.to_numeric(hp_plot["RMSE_p"]) 132 | sss = sns.boxplot(ax = ax[i], x=cnam, y='RMSE_p', data=hp_plot) 133 | sss.set_xlabel(cnam, fontsize = 18) 134 | sss.set_ylabel('RMSE_p', fontsize = 18) 135 | 136 | # Save 137 | fn = output_name + '_hyper_param' + '.png' 138 | fs = os.path.join(output_location, case_study, fn) 139 | 140 | fig.savefig(fs) 141 | 142 | def plot_improvement(i_s, perf, case_study): 143 | 144 | Config = configparser.ConfigParser() 145 | Config.read('configuration_settings.txt') 146 | 147 | # Output parameters 148 | output_location = 'output' # Output folder 149 | output_name = str(Config.get('Case_study','name')) # Case study name 150 | 151 | fig, ax = plt.subplots() 152 | 153 | plot_improvement_bar(i_s, perf, ax) 154 | 155 | fn = output_name + '_ivs_improvements' + '.png' 156 | fs = os.path.join(output_location, case_study, fn) 157 | 158 | fig.savefig(fs) 159 | 160 | 161 | def plot_improvement_bar(i_s, perf, ax): 162 | 163 | is_l = i_s 164 | i_s = ['{:.2f}'.format(x) for x in i_s] 165 | 166 | ax.bar(i_s, perf, width = 0.4) 167 | ax.plot(i_s, perf, color = 'black', linewidth=2) 168 | 169 | ax.set_xlabel('Input column', fontsize = 18) 170 | ax.set_xticks(i_s, np.array(is_l).astype(int).astype(str)) 171 | ax.set_ylabel('NSE [-]', fontsize = 18) 172 | #ax.set_title('Cross-validation performance improvements ', fontsize = 18) 173 | 174 | 175 | def plot_box(performance, case_study): 176 | 177 | Config = configparser.ConfigParser() 178 | Config.read('configuration_settings.txt') 179 | 180 | # Output parameters 181 | output_location = 'output' # Output folder 182 | output_name = str(Config.get('Case_study','name')) # Case study name 183 | 184 | fig, ax = plt.subplots() 185 | ax.boxplot(performance) 186 | ax.set_xlabel('Input combinations', fontsize = 18) 187 | ax.set_ylabel('RMSEp', fontsize = 18) 188 | ax.set_xticks([]) 189 | ax.set_title('Performance variability across input space', fontsize = 20) 190 | 191 | fn = output_name + '_ivs_exaustive' + '.png' 192 | fs = os.path.join(output_location, case_study, fn) 193 | 194 | 195 | 196 | 197 | fig.savefig(fs) 198 | 199 | 200 | 201 | 202 | 203 | 204 | --------------------------------------------------------------------------------