├── .gitignore ├── .project ├── .pydevproject ├── .travis.yml ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── abcpmc ├── __init__.py ├── mpi_util.py ├── sampler.py └── threshold.py ├── docs ├── Makefile ├── abcpmc.png ├── abcpmc.rst ├── authors.rst ├── check_sphinx.py ├── conf.py ├── contributing.rst ├── history.rst ├── index.rst ├── installation.rst ├── modules.rst └── usage.rst ├── notebooks ├── 2d_gauss.ipynb ├── README.rst ├── dual_abc_pmc.ipynb └── toy_model.ipynb ├── requirements.readthedocs.txt ├── requirements.txt ├── setup.py ├── test ├── __init__.py ├── test_MpiUtil.py ├── test_sampler.py └── test_threshold.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | 3 | # C extensions 4 | *.so 5 | 6 | # Packages 7 | *.egg 8 | *.egg-info 9 | dist 10 | build 11 | eggs 12 | parts 13 | bin 14 | var 15 | sdist 16 | develop-eggs 17 | .installed.cfg 18 | lib 19 | lib64 20 | 21 | # Installer logs 22 | pip-log.txt 23 | 24 | # Unit test / coverage reports 25 | .coverage 26 | .tox 27 | nosetests.xml 28 | coverage.xml 29 | junit*.xml 30 | 31 | # Translations 32 | *.mo 33 | 34 | # Mr Developer 35 | .mr.developer.cfg 36 | .project 37 | .pydevproject 38 | 39 | # Sphinx 40 | docs/_build 41 | 42 | # Coverage 43 | htmlcov/ 44 | 45 | # TextMate 46 | .tm_properties 47 | 48 | # Local env 49 | .envs 50 | .hope 51 | -------------------------------------------------------------------------------- /.project: -------------------------------------------------------------------------------- 1 | 2 | 3 | abcpmc 4 | 5 | 6 | 7 | 8 | 9 | org.python.pydev.PyDevBuilder 10 | 11 | 12 | 13 | 14 | 15 | org.python.pydev.pythonNature 16 | 17 | 18 | -------------------------------------------------------------------------------- /.pydevproject: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | /${PROJECT_DIR_NAME} 5 | 6 | python 2.7 7 | Default 8 | 9 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | - "3.3" 5 | 6 | before_install: 7 | - sudo apt-get update 8 | # We do this conditionally because it saves us some downloading if the 9 | # version is the same. 10 | - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then 11 | wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh; 12 | else 13 | wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh; 14 | fi 15 | - bash miniconda.sh -b -p $HOME/miniconda 16 | - export PATH="$HOME/miniconda/bin:$PATH" 17 | - conda update --yes conda 18 | - travis_retry conda install --yes python=$TRAVIS_PYTHON_VERSION pip numpy scipy 19 | 20 | install: 21 | - travis_retry pip install coveralls 22 | - travis_retry pip install -r requirements.txt 23 | 24 | # command to run tests, e.g. python setup.py test 25 | script: coverage run --source abcpmc setup.py test 26 | after_success: 27 | coveralls 28 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Credits 3 | ======= 4 | 5 | Development Lead 6 | ---------------- 7 | 8 | * Joel Akeret 9 | 10 | Contributors 11 | ------------ 12 | 13 | None yet. Why not be the first? 14 | 15 | Citations 16 | --------- 17 | 18 | As you use **abcpmc** for your exciting discoveries, please cite the paper that describes the package: 19 | 20 | `Akeret, J., Refregier, A., Amara, A, Seehars, S., and Hasner, C., Journal of Cosmology and Astroparticle Physics (2015) `_ 21 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Contributing 3 | ============ 4 | 5 | Contributions are welcome, and they are greatly appreciated! Every 6 | little bit helps, and credit will always be given. 7 | 8 | You can contribute in many ways: 9 | 10 | Types of Contributions 11 | ---------------------- 12 | 13 | Report Bugs 14 | ~~~~~~~~~~~ 15 | 16 | If you are reporting a bug, please include: 17 | 18 | * Your operating system name and version. 19 | * Any details about your local setup that might be helpful in troubleshooting. 20 | * Detailed steps to reproduce the bug. 21 | 22 | Fix Bugs 23 | ~~~~~~~~ 24 | 25 | Implement Features 26 | ~~~~~~~~~~~~~~~~~~ 27 | 28 | Write Documentation 29 | ~~~~~~~~~~~~~~~~~~~ 30 | 31 | abcpmc could always use more documentation, whether as part of the 32 | official abcpmc docs, in docstrings, or even on the web in blog posts, 33 | articles, and such. 34 | 35 | Submit Feedback 36 | ~~~~~~~~~~~~~~~ 37 | 38 | If you are proposing a feature: 39 | 40 | * Explain in detail how it would work. 41 | * Keep the scope as narrow as possible, to make it easier to implement. 42 | * Remember that this is a volunteer-driven project, and that contributions 43 | are welcome :) 44 | 45 | Pull Request Guidelines 46 | ----------------------- 47 | 48 | Before you submit a pull request, check that it meets these guidelines: 49 | 50 | 1. The pull request should include tests. 51 | 2. If the pull request adds functionality, the docs should be updated. Put 52 | your new functionality into a function with a docstring, and add the 53 | feature to the list in README.rst. 54 | 3. The pull request should work for Python 2.6, 2.7, and 3.3, and for PyPy. 55 | make sure that the tests pass for all supported Python versions. 56 | 57 | 58 | Tips 59 | ---- 60 | 61 | To run a subset of tests:: 62 | 63 | $ py.test test/test_abcpmc.py -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | .. :changelog: 2 | 3 | History 4 | ------- 5 | 0.x.x (2016-0x-xx) 6 | ++++++++++++++++++ 7 | 8 | * Fixed map issue under Py3 9 | * Replaced numpy.random with a RandomState 10 | * Restart sampling feature 11 | 12 | 0.1.2 (2016-01-27) 13 | ++++++++++++++++++ 14 | 15 | * Added support for sampling with multiple distance simultaneously 16 | * Clean setup.py 17 | * Simplifying the code 18 | * Improved documentation 19 | 20 | 21 | 0.1.1 (2015-05-03) 22 | ++++++++++++++++++ 23 | 24 | * Python 3 support 25 | * Minor fixes 26 | * Improved documentation 27 | 28 | 0.1.0 (2015-04-28) 29 | ++++++++++++++++++ 30 | 31 | * First release -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include AUTHORS.rst 2 | include CONTRIBUTING.rst 3 | include HISTORY.rst 4 | include LICENSE 5 | include README.rst 6 | include requirements.txt 7 | include Makefile 8 | include docs/* 9 | include test/* 10 | include notebooks/* 11 | include example/* -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: help clean clean-pyc clean-build list test test-all coverage docs release sdist 2 | 3 | help: 4 | @echo "clean-build - remove build artifacts" 5 | @echo "clean-pyc - remove Python file artifacts" 6 | @echo "lint - check style with flake8" 7 | @echo "test - run tests quickly with the default Python" 8 | @echo "test-all - run tests on every Python version with tox" 9 | @echo "coverage - check code coverage quickly with the default Python" 10 | @echo "docs - generate Sphinx HTML documentation, including API docs" 11 | @echo "sdist - package" 12 | 13 | clean: clean-build clean-pyc 14 | 15 | clean-build: 16 | rm -fr build/ 17 | rm -fr dist/ 18 | rm -fr *.egg-info 19 | 20 | clean-pyc: 21 | find . -name '*.pyc' -exec rm -f {} + 22 | find . -name '*.pyo' -exec rm -f {} + 23 | find . -name '*~' -exec rm -f {} + 24 | find . -name '__pycache__' -exec rm -rf {} + 25 | 26 | lint: 27 | flake8 abcpmc test 28 | 29 | test: 30 | py.test 31 | 32 | test-all: 33 | tox 34 | 35 | coverage: 36 | coverage run --source abcpmc setup.py test 37 | coverage report -m 38 | coverage html 39 | open htmlcov/index.html 40 | 41 | docs: 42 | #rm -f docs/abcpmc.rst 43 | #rm -f docs/modules.rst 44 | sphinx-apidoc -o docs/ abcpmc 45 | $(MAKE) -C docs clean 46 | $(MAKE) -C docs html 47 | open docs/_build/html/index.html 48 | 49 | sdist: clean 50 | #pip freeze > requirements.rst 51 | python setup.py sdist 52 | ls -l dist -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ============================= 2 | abcpmc 3 | ============================= 4 | 5 | .. image:: https://badge.fury.io/py/abcpmc.svg 6 | :target: http://badge.fury.io/py/abcpmc 7 | 8 | .. image:: https://travis-ci.org/jakeret/abcpmc.svg?branch=master 9 | :target: https://travis-ci.org/jakeret/abcpmc 10 | 11 | .. image:: https://coveralls.io/repos/jakeret/abcpmc/badge.svg?branch=master 12 | :target: https://coveralls.io/r/jakeret/abcpmc?branch=master 13 | 14 | .. image:: https://img.shields.io/badge/docs-latest-blue.svg?style=flat 15 | :target: http://abcpmc.readthedocs.org/en/latest 16 | 17 | .. image:: http://img.shields.io/badge/arXiv-1504.07245-orange.svg?style=flat 18 | :target: http://arxiv.org/abs/1504.07245 19 | 20 | 21 | 22 | A Python Approximate Bayesian Computing (ABC) Population Monte Carlo (PMC) implementation based on Sequential Monte Carlo (SMC) with Particle Filtering techniques. 23 | 24 | .. image:: https://raw.githubusercontent.com/jakeret/abcpmc/master/docs/abcpmc.png 25 | :alt: approximated 2d posterior (created with triangle.py). 26 | :align: center 27 | 28 | The **abcpmc** package has been developed at ETH Zurich in the `Software Lab of the Cosmology Research Group `_ of the `ETH Institute of Astronomy `_. 29 | 30 | The development is coordinated on `GitHub `_ and contributions are welcome. The documentation of **abcpmc** is available at `readthedocs.org `_ and the package is distributed over `PyPI `_. 31 | 32 | Features 33 | -------- 34 | 35 | * Entirely implemented in Python and easy to extend 36 | 37 | * Follows Beaumont et al. 2009 PMC algorithm 38 | 39 | * Parallelized with muliprocessing or message passing interface (MPI) 40 | 41 | * Extendable with k-nearest neighbour (KNN) or optimal local covariance matrix (OLCM) pertubation kernels (Fillipi et al. 2012) 42 | 43 | * Detailed examples in IPython notebooks 44 | 45 | * A `2D gauss `_ case study 46 | 47 | * A `Multi distance `_ case study 48 | 49 | * A `toy model `_ including a comparison to theoretical predictions 50 | 51 | 52 | -------------------------------------------------------------------------------- /abcpmc/__init__.py: -------------------------------------------------------------------------------- 1 | # abcpmc is free software: you can redistribute it and/or modify 2 | # it under the terms of the GNU General Public License as published by 3 | # the Free Software Foundation, either version 3 of the License, or 4 | # (at your option) any later version. 5 | # 6 | # abcpmc is distributed in the hope that it will be useful, 7 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 8 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 9 | # GNU General Public License for more details. 10 | # 11 | # You should have received a copy of the GNU General Public License 12 | # along with abcpmc. If not, see . 13 | 14 | __author__ = 'Joel Akeret' 15 | __email__ = 'jakeret@phys.ethz.ch' 16 | __version__ = '0.1.2' 17 | __credits__ = 'ETH Zurich, Institute for Astronomy' 18 | 19 | from abcpmc.sampler import * 20 | from abcpmc.threshold import * -------------------------------------------------------------------------------- /abcpmc/mpi_util.py: -------------------------------------------------------------------------------- 1 | # abcpmc is free software: you can redistribute it and/or modify 2 | # it under the terms of the GNU General Public License as published by 3 | # the Free Software Foundation, either version 3 of the License, or 4 | # (at your option) any later version. 5 | # 6 | # abcpmc is distributed in the hope that it will be useful, 7 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 8 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 9 | # GNU General Public License for more details. 10 | # 11 | # You should have received a copy of the GNU General Public License 12 | # along with abcpmc. If not, see . 13 | 14 | import itertools 15 | 16 | try: 17 | from mpi4py import MPI 18 | MPI = MPI 19 | except ImportError: 20 | MPI = None 21 | 22 | __all__ = ["MpiPool", "mpiBCast"] 23 | 24 | class MpiPool(object): 25 | def __init__(self, mapFunction=map): 26 | self.rank = MPI.COMM_WORLD.Get_rank() 27 | self.size = MPI.COMM_WORLD.Get_size() 28 | self.mapFunction = mapFunction 29 | 30 | def map(self, function, sequence): 31 | (rank,size) = (MPI.COMM_WORLD.Get_rank(),MPI.COMM_WORLD.Get_size()) 32 | sequence = mpiBCast(sequence) 33 | mergedList = _mergeList(MPI.COMM_WORLD.allgather( 34 | self.mapFunction(function, _splitList(sequence,size)[rank]))) 35 | return mergedList 36 | 37 | def isMaster(self): 38 | """ 39 | Returns true if the rank is 0 40 | """ 41 | return (self.rank==0) 42 | 43 | def mpiBCast(value): 44 | """ 45 | Mpi bcasts the value and returns the value from the master (rank = 0). 46 | """ 47 | return MPI.COMM_WORLD.bcast(value) 48 | 49 | def _splitList(list, n): 50 | blockLen = len(list) / float(n) 51 | return [list[int(round(blockLen * i)): int(round(blockLen * (i + 1)))] for i in range(n)] 52 | 53 | def _mergeList(lists): 54 | return list(itertools.chain(*lists)) 55 | 56 | 57 | 58 | -------------------------------------------------------------------------------- /abcpmc/sampler.py: -------------------------------------------------------------------------------- 1 | # abcpmc is free software: you can redistribute it and/or modify 2 | # it under the terms of the GNU General Public License as published by 3 | # the Free Software Foundation, either version 3 of the License, or 4 | # (at your option) any later version. 5 | # 6 | # abcpmc is distributed in the hope that it will be useful, 7 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 8 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 9 | # GNU General Public License for more details. 10 | # 11 | # You should have received a copy of the GNU General Public License 12 | # along with abcpmc. If not, see . 13 | 14 | 15 | ''' 16 | Created on Oct 9, 2014 17 | 18 | author: jakeret 19 | ''' 20 | from __future__ import print_function, division, absolute_import, unicode_literals 21 | 22 | from multiprocessing.pool import Pool 23 | from collections import namedtuple 24 | 25 | import numpy as np 26 | from scipy import stats 27 | from scipy import spatial 28 | 29 | __all__ = ["GaussianPrior", 30 | "TophatPrior", 31 | "ParticleProposal", 32 | "KNNParticleProposal", 33 | "OLCMParticleProposal", 34 | "Sampler", 35 | "PoolSpec", 36 | "weighted_cov", 37 | "weighted_avg_and_std" 38 | ] 39 | 40 | class GaussianPrior(object): 41 | """ 42 | Normal gaussian prior 43 | 44 | :param mu: scalar or vector of means 45 | :param sigma: scalar variance or covariance matrix 46 | """ 47 | 48 | def __init__(self, mu, sigma): 49 | self.mu = mu 50 | self.sigma = sigma 51 | self._random = np.random.mtrand.RandomState() 52 | 53 | def __call__(self, theta=None): 54 | if theta is None: 55 | return self._random.multivariate_normal(self.mu, self.sigma) 56 | else: 57 | return stats.multivariate_normal.pdf(theta, self.mu, self.sigma) 58 | 59 | 60 | class TophatPrior(object): 61 | """ 62 | Tophat prior 63 | 64 | :param min: scalar or array of min values 65 | :param max: scalar or array of max values 66 | """ 67 | 68 | def __init__(self, min, max): 69 | self.min = np.atleast_1d(min) 70 | self.max = np.atleast_1d(max) 71 | self._random = np.random.mtrand.RandomState() 72 | assert self.min.shape == self.max.shape 73 | assert np.all(self.min < self.max) 74 | 75 | def __call__(self, theta=None): 76 | if theta is None: 77 | return np.array([self._random.uniform(mi, ma) for (mi, ma) in zip(self.min, self.max)]) 78 | else: 79 | return 1 if np.all(theta < self.max) and np.all(theta >= self.min) else 0 80 | 81 | class ParticleProposal(object): 82 | """ 83 | Creates new particles using twice the weighted covariance matrix (Beaumont et al. 2009) 84 | """ 85 | def __init__(self, sampler, eps, pool, kwargs): 86 | self.postfn = sampler.postfn 87 | self.distfn = sampler.dist 88 | self._random = sampler._random 89 | self.Y = sampler.Y 90 | self.N = sampler.N 91 | self.eps = np.asanyarray(eps) 92 | self.pool = pool 93 | self.kwargs = kwargs 94 | 95 | self.sigma = 2 * weighted_cov(pool.thetas, pool.ws) 96 | 97 | def __call__(self, i): 98 | # setting seed to prevent problem with multiprocessing 99 | self._random.seed(i) 100 | cnt = 1 101 | while True: 102 | idx = self._random.choice(range(self.N), 1, p= self.pool.ws/np.sum(self.pool.ws))[0] 103 | theta = self.pool.thetas[idx] 104 | sigma = self._get_sigma(theta, **self.kwargs) 105 | sigma = np.atleast_2d(sigma) 106 | thetap = self._random.multivariate_normal(theta, sigma) 107 | X = self.postfn(thetap) 108 | p = np.asarray(self.distfn(X, self.Y)) 109 | 110 | if np.all(p <= self.eps): 111 | break 112 | cnt+=1 113 | return thetap, p, cnt 114 | 115 | def _get_sigma(self, theta): 116 | return self.sigma 117 | 118 | class KNNParticleProposal(ParticleProposal): 119 | """ 120 | Creates new particles using a covariance matrix from the K-nearest neighbours (Fillipi et al. 2012) 121 | Set `k` as key-word arguement in `abcpmc.Sampler.particle_proposal_kwargs` 122 | """ 123 | 124 | def _get_sigma(self, theta, k): 125 | tree = spatial.cKDTree(self.pool.thetas) 126 | _, idxs = tree.query(theta, k, p=2) 127 | sigma = np.cov(self.pool.thetas[idxs].T) 128 | return sigma 129 | 130 | class OLCMParticleProposal(ParticleProposal): 131 | """ 132 | Creates new particles using an optimal loacl covariance matrix (Fillipi et al. 2012) 133 | """ 134 | 135 | def _get_sigma(self, theta): 136 | if len(self.eps.shape) == 0: 137 | idx = self.pool.dists < self.eps 138 | else: 139 | idx = np.all(self.pool.dists < self.eps, axis=1) 140 | thetas = self.pool.thetas[idx] 141 | weights = self.pool.ws[idx] 142 | weights = weights/np.sum(weights) 143 | 144 | m = np.sum((weights * thetas.T).T, axis=0) 145 | n = thetas.shape[1] 146 | 147 | sigma = np.empty((n, n)) 148 | for i in range(n): 149 | for j in range(n): 150 | sigma[i, j] = np.sum(weights * (thetas[:, i] - m[i]) * (thetas[:, j] - m[j]).T) + (m[i] - theta[i]) * (m[j] - theta[j]) 151 | return sigma 152 | 153 | 154 | """Namedtuple representing a pool of one sampling iteration""" 155 | PoolSpec = namedtuple("PoolSpec", ["t", "eps", "ratio", "thetas", "dists", "ws"]) 156 | 157 | class Sampler(object): 158 | """ 159 | ABC population monte carlo sampler 160 | 161 | :param N: number of particles 162 | :param Y: observed data set 163 | :param postfn: model function (a callable), which creates a new dataset x for a given theta 164 | :param dist: distance function rho(X, Y) (a callable) 165 | :param threads: (optional) number of threads. If >1 and no pool is given multiprocesses will be started 166 | :param pool: (optional) a pool instance which has a function 167 | """ 168 | 169 | particle_proposal_cls = ParticleProposal 170 | particle_proposal_kwargs = {} 171 | 172 | def __init__(self, N, Y, postfn, dist, threads=1, pool=None): 173 | self.N = N 174 | self.Y = Y 175 | self.postfn = postfn 176 | self.dist = dist 177 | self._random = np.random.mtrand.RandomState() 178 | 179 | if pool is not None: 180 | self.pool = pool 181 | self.mapFunc = self.pool.map 182 | 183 | elif threads == 1: 184 | self.mapFunc = map 185 | else: 186 | self.pool = Pool(threads) 187 | self.mapFunc = self.pool.map 188 | 189 | 190 | def sample(self, prior, eps_proposal, pool=None): 191 | """ 192 | Launches the sampling process. Yields the intermediate results per iteration. 193 | 194 | :param prior: instance of a prior definition (or an other callable) see :py:class:`sampler.GaussianPrior` 195 | :param eps_proposal: an instance of a threshold proposal (or an other callable) see :py:class:`sampler.ConstEps` 196 | :param pool: (optional) a PoolSpec instance,if not None the initial rejection sampling 197 | will be skipped and the pool is used for the further sampling 198 | 199 | :yields pool: yields a namedtuple representing the values of one iteration 200 | """ 201 | if pool is None: 202 | eps = eps_proposal.next() 203 | wrapper = _RejectionSamplingWrapper(self, eps, prior) 204 | 205 | res = list(self.mapFunc(wrapper, self._random.randint(0, np.iinfo(np.uint32).max, self.N))) 206 | thetas = np.array([theta for (theta, _, _) in res]) 207 | dists = np.array([dist for (_, dist, _) in res]) 208 | cnts = np.sum([cnt for (_, _, cnt) in res]) 209 | ws = np.ones(self.N) / self.N 210 | 211 | pool = PoolSpec(0, eps, self.N/cnts, thetas, dists, ws) 212 | yield pool 213 | 214 | for t, eps in enumerate(eps_proposal, pool.t + 1): 215 | particleProposal = self.particle_proposal_cls(self, eps, pool, self.particle_proposal_kwargs) 216 | 217 | res = list(self.mapFunc(particleProposal, self._random.randint(0, np.iinfo(np.uint32).max, self.N))) 218 | thetas = np.array([theta for (theta, _, _) in res]) 219 | dists = np.array([dist for (_, dist, _) in res]) 220 | cnts = np.sum([cnt for (_, _, cnt) in res]) 221 | 222 | sigma = 2 * weighted_cov(pool.thetas, pool.ws) 223 | wrapper = _WeightWrapper(prior, sigma, pool.ws, pool.thetas) 224 | 225 | wt = np.array(list(self.mapFunc(wrapper, thetas))) 226 | ws = wt/np.sum(wt) 227 | 228 | pool = PoolSpec(t, eps, self.N/cnts, thetas, dists, ws) 229 | yield pool 230 | 231 | 232 | def close(self): 233 | """ 234 | Tries to close the pool (avoid hanging threads) 235 | """ 236 | if hasattr(self, "pool") and self.pool is not None: 237 | try: 238 | self.pool.close() 239 | except: pass 240 | 241 | 242 | class _WeightWrapper(object): # @DontTrace 243 | """ 244 | Wraps the computation of new particle weights. 245 | Allows for pickling the functionality. 246 | """ 247 | 248 | def __init__(self, prior, sigma, ws, thetas): 249 | self.prior = prior 250 | self.sigma = sigma 251 | self.ws = ws 252 | self.thetas = thetas 253 | 254 | def __call__(self, theta): 255 | kernel = stats.multivariate_normal(theta, self.sigma).pdf 256 | w = self.prior(theta) / np.sum(self.ws * kernel(self.thetas)) 257 | return w 258 | 259 | class _RejectionSamplingWrapper(object): # @DontTrace 260 | """ 261 | Wraps the computation of new particles in the first iteration (simple rejection sampling). 262 | Allows for pickling the functionality. 263 | """ 264 | 265 | def __init__(self, sampler, eps, prior): 266 | self.postfn = sampler.postfn 267 | self.distfn = sampler.dist 268 | self._random = sampler._random 269 | self.Y = sampler.Y 270 | self.eps = np.asarray(eps) 271 | self.prior = prior 272 | 273 | def __call__(self, i): 274 | # setting seed to prevent problem with multiprocessing 275 | self._random.seed(i) 276 | try: 277 | self.prior._random = self._random 278 | except: pass 279 | 280 | cnt = 1 281 | while True: 282 | thetai = self.prior() 283 | X = self.postfn(thetai) 284 | p = np.asarray(self.distfn(X, self.Y)) 285 | if np.all(p <= self.eps): 286 | break 287 | cnt+=1 288 | return thetai, p, cnt 289 | 290 | def weighted_cov(values, weights): 291 | """ 292 | Computes a weighted covariance matrix 293 | 294 | :param values: the array of values 295 | :param weights: array of weights for each entry of the values 296 | 297 | :returns sigma: the weighted covariance matrix 298 | """ 299 | 300 | n = values.shape[1] 301 | sigma = np.empty((n, n)) 302 | w = weights.sum() / (weights.sum()**2 - (weights**2).sum()) 303 | average = np.average(values, axis=0, weights=weights) 304 | for j in range(n): 305 | for k in range(n): 306 | sigma[j, k] = w * np.sum(weights * ((values[:, j] - average[j]) * (values[:, k] - average[k]))) 307 | return sigma 308 | 309 | 310 | def weighted_avg_and_std(values, weights, axis=None): 311 | """ 312 | Return the weighted avg and standard deviation. 313 | 314 | :param values: Array with the values 315 | :param weights: Array with the same shape as values containing the weights 316 | :param axis: (optional) the axis to be used for the computation 317 | 318 | :returns avg, sigma: weighted average and standard deviation 319 | """ 320 | #http://stackoverflow.com/a/2415343/4067032 321 | avg = np.average(values, weights=weights, axis=axis) 322 | # Fast and numerically precise 323 | variance = np.average((values-avg)**2, weights=weights, axis=axis) 324 | return (avg, np.sqrt(variance)) 325 | -------------------------------------------------------------------------------- /abcpmc/threshold.py: -------------------------------------------------------------------------------- 1 | # abcpmc is free software: you can redistribute it and/or modify 2 | # it under the terms of the GNU General Public License as published by 3 | # the Free Software Foundation, either version 3 of the License, or 4 | # (at your option) any later version. 5 | # 6 | # abcpmc is distributed in the hope that it will be useful, 7 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 8 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 9 | # GNU General Public License for more details. 10 | # 11 | # You should have received a copy of the GNU General Public License 12 | # along with abcpmc. If not, see . 13 | ''' 14 | Created on Jan 19, 2015 15 | 16 | author: jakeret 17 | ''' 18 | from __future__ import print_function, division, absolute_import, unicode_literals 19 | 20 | import numpy as np 21 | 22 | class EpsProposal(object): 23 | 24 | def __init__(self, T): 25 | self.T = T 26 | self.reset() 27 | 28 | def __iter__(self): 29 | return self 30 | 31 | def __next__(self): 32 | return self.next() 33 | 34 | def next(self): 35 | if(self.t>=self.T): 36 | raise StopIteration() 37 | 38 | eps_val = self(self.t) 39 | self.t += 1 40 | return eps_val 41 | 42 | def reset(self): 43 | self.t = 0 44 | 45 | class ListEps(EpsProposal): 46 | 47 | def __init__(self, T, eps_vals): 48 | super(ListEps, self).__init__(T) 49 | self.eps_vals = eps_vals 50 | 51 | def __call__(self, t): 52 | return self.eps_vals[t] 53 | 54 | class ConstEps(EpsProposal): 55 | """ 56 | Constant threshold. Can be used to apply alpha-percentile threshold decrease 57 | :param eps: epsilon value 58 | """ 59 | 60 | def __init__(self, T, eps): 61 | super(ConstEps, self).__init__(T) 62 | self.eps = eps 63 | 64 | def __call__(self, t): 65 | return self.eps 66 | 67 | class LinearEps(EpsProposal): 68 | """ 69 | Linearly decreasing threshold 70 | 71 | :param max: epsilon at t=0 72 | :param min: epsilon at t=T 73 | :param T: number of iterations 74 | """ 75 | 76 | def __init__(self, T, max, min): 77 | super(LinearEps, self).__init__(T) 78 | self.eps_vals = np.linspace(max, min, T) 79 | 80 | def __call__(self, t): 81 | return self.eps_vals[t] 82 | 83 | class LinearConstEps(EpsProposal): 84 | """ 85 | Linearly decreasing threshold until T1, then constant until T2 86 | 87 | :param max: epsilon at t=0 88 | :param min: epsilon at t=T 89 | :param T1: number of iterations for decrease 90 | :param T2: number of iterations for constant behavior 91 | """ 92 | 93 | def __init__(self, max, min, T1, T2): 94 | super(LinearConstEps, self).__init__(T1+T2) 95 | self.eps_vals = np.r_[np.linspace(max, min, T1), [min]*T2] 96 | 97 | def __call__(self, t): 98 | return self.eps_vals[t] 99 | 100 | class ExponentialEps(EpsProposal): 101 | """ 102 | Exponentially decreasing threshold 103 | 104 | :param max: epsilon at t=0 105 | :param min: epsilon at t=T 106 | :param T: number of iterations 107 | """ 108 | 109 | def __init__(self, T, max, min): 110 | super(ExponentialEps, self).__init__(T) 111 | self.eps_vals = np.logspace(np.log10(max), np.log10(min), T) 112 | 113 | def __call__(self, t): 114 | return self.eps_vals[t] 115 | 116 | class ExponentialConstEps(EpsProposal): 117 | def __init__(self, max, min, T1, T2): 118 | super(ExponentialConstEps, self).__init__(T1+T2) 119 | self.eps_vals = np.r_[np.logspace(np.log10(max), np.log10(min), T1), [min]*T2] 120 | 121 | def __call__(self, t): 122 | return self.eps_vals[t] 123 | 124 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/complexity.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/complexity.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/complexity" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/complexity" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." -------------------------------------------------------------------------------- /docs/abcpmc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jakeret/abcpmc/bd4e22f5e84ed0941e6cfccf589390986700e0e4/docs/abcpmc.png -------------------------------------------------------------------------------- /docs/abcpmc.rst: -------------------------------------------------------------------------------- 1 | abcpmc Package 2 | ============== 3 | 4 | :mod:`sampler` Module 5 | --------------------- 6 | 7 | .. autoclass:: abcpmc.sampler.Sampler 8 | :members: 9 | 10 | .. autoclass:: abcpmc.sampler.GaussianPrior 11 | :members: 12 | 13 | .. autoclass:: abcpmc.sampler.TophatPrior 14 | :members: 15 | 16 | .. autoclass:: abcpmc.sampler.ParticleProposal 17 | :members: 18 | 19 | .. autoclass:: abcpmc.sampler.OLCMParticleProposal 20 | :members: 21 | :show-inheritance: 22 | 23 | .. autoclass:: abcpmc.sampler.KNNParticleProposal 24 | :members: 25 | :show-inheritance: 26 | 27 | .. autofunction:: abcpmc.sampler.weighted_cov 28 | 29 | .. autofunction:: abcpmc.sampler.weighted_avg_and_std 30 | 31 | :mod:`threshold` Module 32 | ----------------------- 33 | 34 | Various different threshold implementations 35 | 36 | .. automodule:: abcpmc.threshold 37 | :members: 38 | :undoc-members: 39 | :show-inheritance: 40 | 41 | 42 | :mod:`mpi_util` Module 43 | ---------------------- 44 | 45 | A helper util for message passing interface (MPI) handlings 46 | 47 | .. automodule:: abcpmc.mpi_util 48 | :members: 49 | :undoc-members: 50 | :show-inheritance: 51 | 52 | -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.rst -------------------------------------------------------------------------------- /docs/check_sphinx.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Created on Dec 2, 2013 3 | 4 | @author: jakeret 5 | ''' 6 | import py 7 | import subprocess 8 | def test_linkcheck(tmpdir): 9 | doctrees = tmpdir.join("doctrees") 10 | htmldir = tmpdir.join("html") 11 | subprocess.check_call( 12 | ["sphinx-build", "-blinkcheck", 13 | "-d", str(doctrees), ".", str(htmldir)]) 14 | 15 | def test_build_docs(tmpdir): 16 | doctrees = tmpdir.join("doctrees") 17 | htmldir = tmpdir.join("html") 18 | subprocess.check_call([ 19 | "sphinx-build", "-bhtml", 20 | "-d", str(doctrees), ".", str(htmldir)]) -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # complexity documentation build configuration file, created by 4 | # sphinx-quickstart on Tue Jul 9 22:26:36 2013. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | #sys.path.insert(0, os.path.abspath('.')) 20 | 21 | cwd = os.getcwd() 22 | parent = os.path.dirname(cwd) 23 | sys.path.insert(0, parent) 24 | 25 | import mock 26 | 27 | MOCK_MODULES = ['numpy', 'scipy', 'matplotlib', 'scipy.stats'] 28 | for mod_name in MOCK_MODULES: 29 | sys.modules[mod_name] = mock.Mock() 30 | 31 | import abcpmc 32 | 33 | # -- General configuration ----------------------------------------------------- 34 | 35 | # If your documentation needs a minimal Sphinx version, state it here. 36 | #needs_sphinx = '1.0' 37 | 38 | # Add any Sphinx extension module names here, as strings. They can be extensions 39 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 40 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.viewcode'] 41 | 42 | # Add any paths that contain templates here, relative to this directory. 43 | templates_path = ['_templates'] 44 | 45 | # The suffix of source filenames. 46 | source_suffix = '.rst' 47 | 48 | # The encoding of source files. 49 | #source_encoding = 'utf-8-sig' 50 | 51 | # The master toctree document. 52 | master_doc = 'index' 53 | 54 | # General information about the project. 55 | project = u'abcpmc' 56 | copyright = u'2014, ETH Zurich, Institute for Astronomy' 57 | 58 | # The version info for the project you're documenting, acts as replacement for 59 | # |version| and |release|, also used in various other places throughout the 60 | # built documents. 61 | # 62 | # The short X.Y version. 63 | version = abcpmc.__version__ 64 | # The full version, including alpha/beta/rc tags. 65 | release = abcpmc.__version__ 66 | 67 | # The language for content autogenerated by Sphinx. Refer to documentation 68 | # for a list of supported languages. 69 | #language = None 70 | 71 | # There are two options for replacing |today|: either, you set today to some 72 | # non-false value, then it is used: 73 | #today = '' 74 | # Else, today_fmt is used as the format for a strftime call. 75 | #today_fmt = '%B %d, %Y' 76 | 77 | # List of patterns, relative to source directory, that match files and 78 | # directories to ignore when looking for source files. 79 | exclude_patterns = ['_build'] 80 | 81 | # The reST default role (used for this markup: `text`) to use for all documents. 82 | #default_role = None 83 | 84 | # If true, '()' will be appended to :func: etc. cross-reference text. 85 | #add_function_parentheses = True 86 | 87 | # If true, the current module name will be prepended to all description 88 | # unit titles (such as .. function::). 89 | #add_module_names = True 90 | 91 | # If true, sectionauthor and moduleauthor directives will be shown in the 92 | # output. They are ignored by default. 93 | #show_authors = False 94 | 95 | # The name of the Pygments (syntax highlighting) style to use. 96 | pygments_style = 'sphinx' 97 | 98 | # A list of ignored prefixes for module index sorting. 99 | #modindex_common_prefix = [] 100 | 101 | # If true, keep warnings as "system message" paragraphs in the built documents. 102 | #keep_warnings = False 103 | 104 | 105 | # -- Options for HTML output --------------------------------------------------- 106 | 107 | # The theme to use for HTML and HTML Help pages. See the documentation for 108 | # a list of builtin themes. 109 | # html_theme = 'default' 110 | 111 | # Theme options are theme-specific and customize the look and feel of a theme 112 | # further. For a list of options available for each theme, see the 113 | # documentation. 114 | #html_theme_options = {} 115 | 116 | # Add any paths that contain custom themes here, relative to this directory. 117 | #html_theme_path = [] 118 | 119 | # The name for this set of Sphinx documents. If None, it defaults to 120 | # " v documentation". 121 | #html_title = None 122 | 123 | # A shorter title for the navigation bar. Default is the same as html_title. 124 | #html_short_title = None 125 | 126 | # The name of an image file (relative to this directory) to place at the top 127 | # of the sidebar. 128 | #html_logo = None 129 | 130 | # The name of an image file (within the static path) to use as favicon of the 131 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 132 | # pixels large. 133 | #html_favicon = None 134 | 135 | # Add any paths that contain custom static files (such as style sheets) here, 136 | # relative to this directory. They are copied after the builtin static files, 137 | # so a file named "default.css" will overwrite the builtin "default.css". 138 | html_static_path = ['_static'] 139 | 140 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 141 | # using the given strftime format. 142 | #html_last_updated_fmt = '%b %d, %Y' 143 | 144 | # If true, SmartyPants will be used to convert quotes and dashes to 145 | # typographically correct entities. 146 | #html_use_smartypants = True 147 | 148 | # Custom sidebar templates, maps document names to template names. 149 | #html_sidebars = {} 150 | 151 | # Additional templates that should be rendered to pages, maps page names to 152 | # template names. 153 | #html_additional_pages = {} 154 | 155 | # If false, no module index is generated. 156 | #html_domain_indices = True 157 | 158 | # If false, no index is generated. 159 | #html_use_index = True 160 | 161 | # If true, the index is split into individual pages for each letter. 162 | #html_split_index = False 163 | 164 | # If true, links to the reST sources are added to the pages. 165 | #html_show_sourcelink = True 166 | 167 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 168 | #html_show_sphinx = True 169 | 170 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 171 | #html_show_copyright = True 172 | 173 | # If true, an OpenSearch description file will be output, and all pages will 174 | # contain a tag referring to it. The value of this option must be the 175 | # base URL from which the finished HTML is served. 176 | #html_use_opensearch = '' 177 | 178 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 179 | #html_file_suffix = None 180 | 181 | # Output file base name for HTML help builder. 182 | htmlhelp_basename = 'abcpmcdoc' 183 | 184 | 185 | # -- Options for LaTeX output -------------------------------------------------- 186 | 187 | latex_elements = { 188 | # The paper size ('letterpaper' or 'a4paper'). 189 | #'papersize': 'letterpaper', 190 | 191 | # The font size ('10pt', '11pt' or '12pt'). 192 | #'pointsize': '10pt', 193 | 194 | # Additional stuff for the LaTeX preamble. 195 | #'preamble': '', 196 | } 197 | 198 | # Grouping the document tree into LaTeX files. List of tuples 199 | # (source start file, target name, title, author, documentclass [howto/manual]). 200 | latex_documents = [ 201 | ('index', 'abcpmc.tex', u'abcpmc Documentation', 202 | u'Joel Akeret', 'manual'), 203 | ] 204 | 205 | # The name of an image file (relative to this directory) to place at the top of 206 | # the title page. 207 | #latex_logo = None 208 | 209 | # For "manual" documents, if this is true, then toplevel headings are parts, 210 | # not chapters. 211 | #latex_use_parts = False 212 | 213 | # If true, show page references after internal links. 214 | #latex_show_pagerefs = False 215 | 216 | # If true, show URL addresses after external links. 217 | #latex_show_urls = False 218 | 219 | # Documents to append as an appendix to all manuals. 220 | #latex_appendices = [] 221 | 222 | # If false, no module index is generated. 223 | #latex_domain_indices = True 224 | 225 | 226 | # -- Options for manual page output -------------------------------------------- 227 | 228 | # One entry per manual page. List of tuples 229 | # (source start file, name, description, authors, manual section). 230 | man_pages = [ 231 | ('index', 'abcpmc', u'abcpmc Documentation', 232 | [u'Joel Akeret'], 1) 233 | ] 234 | 235 | # If true, show URL addresses after external links. 236 | #man_show_urls = False 237 | 238 | 239 | # -- Options for Texinfo output ------------------------------------------------ 240 | 241 | # Grouping the document tree into Texinfo files. List of tuples 242 | # (source start file, target name, title, author, 243 | # dir menu entry, description, category) 244 | texinfo_documents = [ 245 | ('index', 'abcpmc', u'abcpmc Documentation', 246 | u'Joel Akeret', 'abcpmc', 'One line description of project.', 247 | 'Miscellaneous'), 248 | ] 249 | 250 | # Documents to append as an appendix to all manuals. 251 | #texinfo_appendices = [] 252 | 253 | # If false, no module index is generated. 254 | #texinfo_domain_indices = True 255 | 256 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 257 | #texinfo_show_urls = 'footnote' 258 | 259 | # If true, do not generate a @detailmenu in the "Top" node's menu. 260 | #texinfo_no_detailmenu = False 261 | 262 | try: 263 | import sphinx_eth_theme 264 | html_theme = "sphinx_eth_theme" 265 | html_theme_path = [sphinx_eth_theme.get_html_theme_path()] 266 | except ImportError: 267 | html_theme = 'default' -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst -------------------------------------------------------------------------------- /docs/history.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../HISTORY.rst -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. complexity documentation master file, created by 2 | sphinx-quickstart on Tue Jul 9 22:26:36 2013. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | .. include:: ../README.rst 7 | 8 | Contents: 9 | ========= 10 | 11 | .. toctree:: 12 | :maxdepth: 2 13 | 14 | installation 15 | usage 16 | abcpmc 17 | contributing 18 | authors 19 | history 20 | 21 | Feedback 22 | ======== 23 | 24 | If you have any suggestions or questions about **abcpmc** feel free to email me 25 | at jakeret@phys.ethz.ch. 26 | 27 | If you encounter any errors or problems with **abcpmc**, please let me know! -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Installation 3 | ============ 4 | 5 | At the command line via pip:: 6 | 7 | $ pip install abcpmc 8 | 9 | This will install the package and all of the required dependencies. 10 | 11 | .. note:: If you wish to use `abcpmc` on a cluster with MPI you need to manually install `mpi4py `_. 12 | 13 | -------------------------------------------------------------------------------- /docs/modules.rst: -------------------------------------------------------------------------------- 1 | abcpmc 2 | ====== 3 | 4 | .. toctree:: 5 | :maxdepth: 4 6 | 7 | abcpmc 8 | -------------------------------------------------------------------------------- /docs/usage.rst: -------------------------------------------------------------------------------- 1 | ======== 2 | Usage 3 | ======== 4 | 5 | IPython notebook examples 6 | -------------------------- 7 | 8 | Detailed examples are available in the project and online: A `2D gauss `_ example and a `toy model `_ including a comparison to theoretical predictions. 9 | 10 | General usage of abcpmc 11 | ------------------------ 12 | 13 | In generall the use of abcpmc in simple:: 14 | 15 | import abcpmc 16 | import numpy as np 17 | 18 | #create "observed" data set 19 | size = 5000 20 | sigma = np.eye(4) * 0.25 21 | means = np.array([1.1, 1.5, 1.1, 1.5]) 22 | data = np.random.multivariate_normal(means, sigma, size) 23 | #------- 24 | 25 | #distance function: sum of abs mean differences 26 | def dist(x, y): 27 | return np.sum(np.abs(np.mean(x, axis=0) - np.mean(y, axis=0))) 28 | 29 | #our "model", a gaussian with varying means 30 | def postfn(theta): 31 | return np.random.multivariate_normal(theta, sigma, size) 32 | 33 | eps = abcpmc.LinearEps(20, 5, 0.075) 34 | prior = abcpmc.GaussianPrior(means*1.1, sigma*2) #our best guess 35 | 36 | sampler = abcpmc.Sampler(N=10, Y=data, postfn=postfn, dist=dist) 37 | 38 | for pool in sampler.sample(prior, eps): 39 | print("T: {0}, eps: {1:>.4f}, ratio: {2:>.4f}".format(pool.t, pool.eps, pool.ratio)) 40 | for i, (mean, std) in enumerate(zip(np.mean(pool.thetas, axis=0), np.std(pool.thetas, axis=0))): 41 | print(u" theta[{0}]: {1:>.4f} \u00B1 {2:>.4f}".format(i, mean,std)) 42 | 43 | 44 | the resulting output would look something like this:: 45 | 46 | T: 0, eps: 2.0000, ratio: 0.2439 47 | theta[0]: 0.9903 ± 0.5435 48 | theta[1]: 1.6050 ± 0.4912 49 | theta[2]: 1.0567 ± 0.4548 50 | theta[3]: 1.2859 ± 0.5213 51 | T: 1, eps: 1.8987, ratio: 0.3226 52 | theta[0]: 1.1666 ± 0.4129 53 | theta[1]: 1.6597 ± 0.5227 54 | theta[2]: 1.1263 ± 0.3366 55 | theta[3]: 1.4711 ± 0.2150 56 | T: 2, eps: 1.7974, ratio: 0.3030 57 | theta[0]: 1.1263 ± 0.2505 58 | theta[1]: 1.4832 ± 0.5057 59 | theta[2]: 1.0585 ± 0.3387 60 | theta[3]: 1.4782 ± 0.2808 61 | T: 3, eps: 1.6961, ratio: 0.4167 62 | theta[0]: 1.1265 ± 0.1845 63 | theta[1]: 1.2032 ± 0.4470 64 | theta[2]: 1.0248 ± 0.2074 65 | theta[3]: 1.4689 ± 0.4250 66 | 67 | ... 68 | 69 | T: 19, eps: 0.0750, ratio: 0.0441 70 | theta[0]: 1.1108 ± 0.0172 71 | theta[1]: 1.4832 ± 0.0166 72 | theta[2]: 1.0895 ± 0.0202 73 | theta[3]: 1.5016 ± 0.0097 74 | 75 | 76 | 77 | Parallelisation on cluster with MPI 78 | ------------------------------------ 79 | 80 | `abcpmc` has an built-in support for massively parallelized sampling on a cluster using MPI. 81 | 82 | To make use of this parallelization the abcpmc Sampler need to be initialized with an instance of the `MpiPool`: 83 | 84 | .. code-block:: python 85 | 86 | import abcpmc 87 | from abcpmc import mpi_util 88 | 89 | ... 90 | 91 | mpi_pool = mpi_util.MpiPool() 92 | sampler = abcpmc.Sampler(N, Y, postfn, dist, pool=mpi_pool) #pass the mpi_pool 93 | 94 | if mpi_pool.isMaster(): print("Start sampling") 95 | 96 | for pool in sampler.sample(prior, eps): 97 | 98 | ... 99 | 100 | If the threshold is dynamically adapted the user has to make sure that the state is synchonized among all MPI task with a broadcast: 101 | 102 | .. code-block:: python 103 | 104 | eps.eps = mpi_util.mpiBCast(new_threshold) 105 | 106 | 107 | Finally, the job has to be launched as follows to run on `N` tasks in parallel (might depend on your system):: 108 | 109 | $ mpirun -np N python 110 | 111 | -------------------------------------------------------------------------------- /notebooks/README.rst: -------------------------------------------------------------------------------- 1 | ========== 2 | NOTEBOOKS 3 | ========== 4 | 5 | * A `2D gauss `_ case study 6 | 7 | * A `toy model `_ including a comparison to theoretical predictions 8 | 9 | * A `multi distance sampling `_ example 10 | 11 | 12 | -------------------------------------------------------------------------------- /requirements.readthedocs.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jakeret/abcpmc/bd4e22f5e84ed0941e6cfccf589390986700e0e4/requirements.readthedocs.txt -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | scipy>=0.15.0 3 | Sphinx>=1.2.3 4 | mock>=1.0.1 5 | pytest>=2.6.3 6 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | import sys 5 | from setuptools.command.test import test as TestCommand 6 | from setuptools import find_packages 7 | 8 | try: 9 | from setuptools import setup 10 | except ImportError: 11 | from distutils.core import setup 12 | 13 | 14 | class PyTest(TestCommand): 15 | def finalize_options(self): 16 | TestCommand.finalize_options(self) 17 | # self.test_args 18 | # self.test_suite = True 19 | 20 | def run_tests(self): 21 | import pytest 22 | errno = pytest.main(self.test_args) 23 | sys.exit(errno) 24 | 25 | 26 | readme = open('README.rst').read() 27 | 28 | history = open('HISTORY.rst').read().replace('.. :changelog:', '') 29 | 30 | #during runtime 31 | requires = ["numpy", 32 | "scipy>=0.15"] 33 | 34 | #for testing 35 | tests_require=['pytest>=2.3', 36 | 'mock'] 37 | 38 | PACKAGE_PATH = os.path.abspath(os.path.join(__file__, os.pardir)) 39 | 40 | setup( 41 | name='abcpmc', 42 | version='0.1.2', 43 | description='approximate bayesian computing with population monte carlo', 44 | long_description=readme + '\n\n' + history, 45 | author='Joel Akeret', 46 | author_email='jakeret@phys.ethz.ch', 47 | url='http://www.cosmology.ethz.ch/research/software-lab/abcpmc.html', 48 | packages=find_packages(PACKAGE_PATH, "test"), 49 | package_dir={'abcpmc': 'abcpmc'}, 50 | include_package_data=True, 51 | install_requires=requires, 52 | license="GPLv3", 53 | zip_safe=False, 54 | keywords=["abcpmc", 55 | "approximate bayesian computing ", 56 | "population monte carlo"], 57 | classifiers=[ 58 | "Development Status :: 4 - Beta", 59 | "Intended Audience :: Science/Research", 60 | 'Intended Audience :: Developers', 61 | "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)", 62 | "Natural Language :: English", 63 | "Operating System :: MacOS", 64 | "Operating System :: POSIX", 65 | "Topic :: Scientific/Engineering", 66 | "Topic :: Scientific/Engineering :: Mathematics", 67 | "Topic :: Scientific/Engineering :: Physics", 68 | "Topic :: Scientific/Engineering :: Astronomy", 69 | 'Programming Language :: Python :: 2.6', 70 | 'Programming Language :: Python :: 2.7', 71 | 'Programming Language :: Python :: 3', 72 | 'Programming Language :: Python :: 3.3', 73 | ], 74 | tests_require=tests_require, 75 | cmdclass = {'test': PyTest}, 76 | ) -------------------------------------------------------------------------------- /test/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jakeret/abcpmc/bd4e22f5e84ed0941e6cfccf589390986700e0e4/test/__init__.py -------------------------------------------------------------------------------- /test/test_MpiUtil.py: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2014 ETH Zurich, Institute for Astronomy 2 | 3 | ''' 4 | Created on Jul 29, 2014 5 | 6 | author: jakeret 7 | ''' 8 | from __future__ import print_function, division, absolute_import, unicode_literals 9 | 10 | import numpy as np 11 | from mock import patch 12 | 13 | from abcpmc import mpi_util 14 | 15 | class TestMpiUtil(object): 16 | 17 | @patch("abcpmc.mpi_util.MPI") 18 | def test_splitlist_1(self, mpi_mock): 19 | sequence = self._get_sequence(7) 20 | n = 1 21 | sList = mpi_util._splitList(sequence, n) 22 | assert len(sList) == n 23 | for i in range(len(sequence)): 24 | assert np.all(sList[0][i] == sequence[i]) 25 | 26 | @patch("abcpmc.mpi_util.MPI") 27 | def test_splitlist_2(self, mpi_mock): 28 | sequence = self._get_sequence(7) 29 | n = 2 30 | sList = mpi_util._splitList(sequence, n) 31 | assert len(sList) == n 32 | 33 | assert len(sList[0]) == 4 34 | for i in range(4): 35 | assert np.all(sList[0][i] == sequence[0+i]) 36 | 37 | assert len(sList[1]) == 3 38 | for i in range(3): 39 | assert np.all(sList[1][i] == sequence[4+i]) 40 | 41 | @patch("abcpmc.mpi_util.MPI") 42 | def test_splitlist_3(self, mpi_mock): 43 | sequence = self._get_sequence(7) 44 | n = 3 45 | sList = mpi_util._splitList(sequence, n) 46 | assert len(sList) == n 47 | 48 | l0 = 2 49 | assert len(sList[0]) == l0 50 | for i in range(l0): 51 | assert np.all(sList[0][i] == sequence[0+i]) 52 | 53 | l1 = 3 54 | assert len(sList[1]) == l1 55 | for i in range(l1): 56 | assert np.all(sList[1][i] == sequence[2+i]) 57 | 58 | l2 = 2 59 | assert len(sList[2]) == l2 60 | for i in range(l2): 61 | assert np.all(sList[2][i] == sequence[5+i]) 62 | 63 | @patch("abcpmc.mpi_util.MPI") 64 | def test_splitlist_equal(self, mpi_mock): 65 | sequence = self._get_sequence(10) 66 | 67 | n = 10 68 | sList = mpi_util._splitList(sequence, n) 69 | assert len(sList) == n 70 | 71 | l0 = 1 72 | for k in range(n): 73 | assert len(sList[k]) == l0 74 | for i in range(l0): 75 | assert np.all(sList[k][i] == sequence[(k*l0)+i]) 76 | 77 | @patch("abcpmc.mpi_util.MPI") 78 | def test_splitlist_80(self, mpi_mock): 79 | sequence = self._get_sequence(160) 80 | 81 | n = 80 82 | sList = mpi_util._splitList(sequence, n) 83 | assert len(sList) == n 84 | 85 | l0 = 2 86 | for k in range(n): 87 | assert len(sList[k]) == l0 88 | for i in range(l0): 89 | assert np.all(sList[k][i] == sequence[(k*l0)+i]) 90 | 91 | 92 | def _get_sequence(self, lenght): 93 | sequence = (np.ones((lenght,4)).T * np.arange(lenght)).T 94 | sequence = [sequence[i] for i in range(len(sequence))] 95 | return sequence -------------------------------------------------------------------------------- /test/test_sampler.py: -------------------------------------------------------------------------------- 1 | 2 | # Copyright (C) 2014 ETH Zurich, Institute for Astronomy 3 | 4 | """ 5 | Tests for `abcpmc` module. 6 | """ 7 | from __future__ import print_function, division, absolute_import, unicode_literals 8 | 9 | import abcpmc 10 | import pytest 11 | import numpy as np 12 | from scipy import stats 13 | from mock.mock import Mock 14 | 15 | 16 | class TestTophatPrior(object): 17 | 18 | def test_tophat_univariate(self): 19 | min=1 20 | max=5 21 | with pytest.raises(AssertionError): 22 | abcpmc.TophatPrior(max, min) 23 | 24 | prior = abcpmc.TophatPrior(min, max) 25 | vals = prior() 26 | assert vals>min 27 | assert valsmin) 44 | assert np.all(vals<=max) 45 | 46 | assert prior(theta=[0, 0]) == 0 47 | assert prior(theta=[1, 2]) == 1 48 | assert prior(theta=[2, 3]) == 1 49 | assert prior(theta=[5, 5]) == 0 50 | assert prior(theta=[6, 6]) == 0 51 | 52 | class TestGaussianPrior(object): 53 | 54 | def test_normal(self): 55 | prior = abcpmc.GaussianPrior([0,0], [[1,0],[0,1]]) 56 | 57 | rngs = np.array([prior() for _ in range(10000)]) 58 | assert len(rngs.shape) == 2 59 | D, p = stats.kstest(rngs[:,0], "norm") 60 | 61 | assert D < 0.015 62 | 63 | def test_pdf(self): 64 | try: 65 | from scipy.stats import multivariate_normal 66 | except ImportError: 67 | pytest.skip("Scipy.stats.multivariate_normal is not available") 68 | 69 | prior = abcpmc.GaussianPrior([0,0], [[1,0],[0,1]]) 70 | 71 | theta = prior([0,0]) 72 | assert np.allclose(theta, 0.15915, 1e-4) 73 | 74 | class TestRejectionSamplingWrapperWrapper(object): 75 | 76 | def test_new_particle(self): 77 | eps = 1 78 | prior = lambda : 1 79 | thetai = 1 80 | postfn = lambda theta: thetai 81 | p = 0.5 82 | dist = lambda x,y: p 83 | Y = None 84 | 85 | sampler = abcpmc.Sampler(2, Y, postfn, dist) 86 | wrapper = abcpmc.sampler._RejectionSamplingWrapper(sampler, eps, prior) 87 | rthetai, rp, cnt = wrapper(0) 88 | assert thetai == rthetai 89 | assert p == rp 90 | assert cnt == 1 91 | 92 | # swrapper = pickle.dumps(wrapper) 93 | # uppwrapper = pickle.loads(swrapper) 94 | 95 | def test_new_particle_multidist(self): 96 | eps = 1. 97 | threshold = [eps, eps] 98 | prior = lambda : 1 99 | thetai = 1 100 | postfn = lambda theta: thetai 101 | dist = Mock() 102 | 103 | distances = [[eps*2, eps*2], 104 | [eps/2, eps*2], 105 | [eps*2, eps/2], 106 | [eps, eps]] 107 | 108 | dist.side_effect = distances 109 | Y = None 110 | sampler = abcpmc.Sampler(2, Y, postfn, dist) 111 | wrapper = abcpmc.sampler._RejectionSamplingWrapper(sampler, threshold, prior) 112 | _, _, cnt = wrapper(0) 113 | assert cnt == len(distances) 114 | 115 | 116 | class TestWeightWrapper(object): 117 | 118 | def test_compute_weights(self): 119 | try: 120 | from scipy.stats import multivariate_normal 121 | except ImportError: 122 | pytest.skip("Scipy.stats.multivariate_normal is not available") 123 | 124 | 125 | prior = lambda theta: 1 126 | samples = [1] * 10 127 | weights = [1/len(samples)] * len(samples) 128 | sigma = 1 129 | 130 | wrapper = abcpmc.sampler._WeightWrapper(prior, sigma, weights, samples) 131 | rweight = wrapper(theta=0) 132 | 133 | assert rweight is not None 134 | assert rweight > 0.0 135 | 136 | def test_weighted_cov(): 137 | N = 4 138 | values = np.eye(N) 139 | weights = np.ones(N) 140 | wcov = abcpmc.weighted_cov(values, weights) 141 | 142 | assert np.all(wcov == np.cov(values)) 143 | 144 | class TestParticleProposal(object): 145 | 146 | def test_propose(self): 147 | eps = 1 148 | thetai = 1 149 | postfn = lambda theta: thetai 150 | p = 0.5 151 | dist = lambda x,y: p 152 | Y = None 153 | sampler = abcpmc.Sampler(2, Y, postfn, dist) 154 | 155 | thetas = np.array([[0.5], [1]]) 156 | weights = np.array([0.75, 0.25]) 157 | pool = abcpmc.sampler.PoolSpec(1, eps, 1, thetas, None, weights) 158 | 159 | wrapper = abcpmc.sampler.ParticleProposal(sampler, eps, pool, {}) 160 | 161 | sigma = 0.25 162 | assert wrapper._get_sigma(None) == sigma 163 | 164 | rthetai, rp, cnt = wrapper(0) 165 | assert rthetai > (thetai - 5*sigma) and rthetai < (thetai + 5*sigma) 166 | assert p == rp 167 | assert cnt == 1 168 | 169 | def test_propose_multidist(self): 170 | eps = 1. 171 | threshold = [eps, eps] 172 | thetai = 1 173 | postfn = lambda theta: thetai 174 | dist = Mock() 175 | 176 | distances = [[eps*2, eps*2], 177 | [eps/2, eps*2], 178 | [eps*2, eps/2], 179 | [eps, eps]] 180 | 181 | dist.side_effect = distances 182 | Y = None 183 | sampler = abcpmc.Sampler(2, Y, postfn, dist) 184 | 185 | thetas = np.array([[0.5], [1]]) 186 | weights = np.array([0.75, 0.25]) 187 | pool = abcpmc.sampler.PoolSpec(1, threshold, 1, thetas, None, weights) 188 | 189 | wrapper = abcpmc.sampler.ParticleProposal(sampler, threshold, pool, {}) 190 | 191 | _, _, cnt = wrapper(0) 192 | assert cnt == len(distances) 193 | 194 | class TestOLCMParticleProposal(object): 195 | 196 | def test_get_sigma(self): 197 | eps = 1.1 198 | thetai = 1 199 | postfn = lambda theta: thetai 200 | p = 0.5 201 | dist = lambda x,y: p 202 | Y = None 203 | sampler = abcpmc.Sampler(1, Y, postfn, dist) 204 | 205 | 206 | thetas = np.array([[1], [1], [2]]) 207 | dists = np.array([1, 1, 2]) 208 | ws = np.array([1, 1, 1]) 209 | pool = abcpmc.sampler.PoolSpec(1, eps, 1, thetas, dists, ws) 210 | 211 | wrapper = abcpmc.sampler.OLCMParticleProposal(sampler, eps, pool, {}) 212 | 213 | sigma = np.var(thetas[:1]) 214 | assert wrapper._get_sigma(thetas[0]) == sigma 215 | 216 | def test_get_sigma_multidist(self): 217 | eps = 1. 218 | threshold = [eps, eps] 219 | 220 | thetai = 1 221 | postfn = lambda theta: thetai 222 | p = 0.5 223 | dist = lambda x,y: p 224 | Y = None 225 | sampler = abcpmc.Sampler(1, Y, postfn, dist) 226 | 227 | 228 | dists = np.array([[eps/2, eps/2], 229 | [eps/2, eps/2], 230 | [eps*2, eps*2], 231 | [eps/2, eps*2], 232 | [eps*2, eps/2] ]) 233 | 234 | thetas = np.array([[1], [1], [2], [2], [2]]) 235 | ws = np.array([1]*len(dists)) 236 | pool = abcpmc.sampler.PoolSpec(1, threshold, 1, thetas, dists, ws) 237 | 238 | wrapper = abcpmc.sampler.OLCMParticleProposal(sampler, threshold, pool, {}) 239 | 240 | sigma = np.var(thetas[:1]) 241 | assert wrapper._get_sigma(thetas[0]) == sigma 242 | 243 | class TestKNNParticleProposal(object): 244 | 245 | def test_propose(self): 246 | eps = 1 247 | thetai = 1 248 | postfn = lambda theta: thetai 249 | p = 0.5 250 | dist = lambda x,y: p 251 | Y = None 252 | sampler = abcpmc.Sampler(1, Y, postfn, dist) 253 | sigma = 0 254 | 255 | thetas = np.array([[1], [1], [2]]) 256 | dists = np.array([1, 1, 2]) 257 | ws = np.array([1, 1, 1]) 258 | pool = abcpmc.sampler.PoolSpec(1, eps, 1, thetas, dists, ws) 259 | 260 | wrapper = abcpmc.sampler.KNNParticleProposal(sampler, eps, pool, {}) 261 | 262 | assert wrapper._get_sigma(thetas[0], 2) == sigma 263 | 264 | class TestSampler(object): 265 | 266 | def test_sample(self): 267 | N = 10 268 | T = 2 269 | postfn = lambda theta: None 270 | 271 | dist = 1.0 272 | distfn = lambda X, Y: dist 273 | prior = abcpmc.TophatPrior([0], [100]) 274 | sampler = abcpmc.Sampler(N, 0, postfn, distfn) 275 | 276 | eps = 10 277 | eps_proposal = abcpmc.ConstEps(T, eps) 278 | for i, pool in enumerate(sampler.sample(prior, eps_proposal)): 279 | assert pool is not None 280 | assert pool.t == i 281 | assert pool.ratio == 1.0 282 | assert pool.eps == eps 283 | assert len(pool.thetas) == N 284 | assert np.all(pool.thetas != 0.0) 285 | assert len(pool.dists) == N 286 | assert np.all(pool.dists == dist) 287 | assert len(pool.ws) == N 288 | assert np.allclose(np.sum(pool.ws), 1.0) 289 | 290 | 291 | assert i+1 == T 292 | 293 | def test_weighted_avg_and_std(): 294 | values = np.random.normal(size=1000) 295 | weights = np.ones((1000)) 296 | 297 | avg, std = abcpmc.weighted_avg_and_std(values, weights) 298 | 299 | assert np.allclose(avg, np.average(values)) 300 | assert np.allclose(std, np.std(values)) 301 | 302 | -------------------------------------------------------------------------------- /test/test_threshold.py: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2014 ETH Zurich, Institute for Astronomy 2 | 3 | ''' 4 | Created on Jan 19, 2015 5 | 6 | author: jakeret 7 | ''' 8 | from __future__ import print_function, division, absolute_import, unicode_literals 9 | import abcpmc 10 | import numpy as np 11 | 12 | def test_ConstEps(): 13 | eps_val = 0.5 14 | T = 5 15 | eps = abcpmc.ConstEps(T, eps_val) 16 | 17 | for i, e in enumerate(eps): 18 | assert e == eps_val 19 | 20 | assert i+1 == T 21 | 22 | def test_ListEps(): 23 | eps_vals = np.arange(0, 5, 1) 24 | T = 5 25 | eps = abcpmc.ListEps(T, eps_vals) 26 | 27 | for e1,e2 in zip(eps, eps_vals): 28 | assert e1 == e2 29 | 30 | assert e1 == eps_vals[-1] 31 | 32 | def test_LinearEps(): 33 | eps_vals = np.arange(0, 5, 1) 34 | T = 5 35 | eps = abcpmc.LinearEps(T, eps_vals[0], eps_vals[-1]) 36 | 37 | for e1,e2 in zip(eps, eps_vals): 38 | assert e1 == e2 39 | 40 | assert e1 == eps_vals[-1] 41 | 42 | def test_LinearConstEps(): 43 | T1 = 6 44 | T2 = 2 45 | eps_vals = np.r_[np.linspace(10, 5, T1), [5]*T2] 46 | eps = abcpmc.LinearConstEps(eps_vals[0], eps_vals[-1], T1, T2) 47 | 48 | for e1,e2 in zip(eps, eps_vals): 49 | assert e1 == e2 50 | 51 | assert e1 == eps_vals[-1] 52 | 53 | def test_ExponentialEps(): 54 | T = 5 55 | eps_vals = np.logspace(np.log10(10), np.log10(5), T) 56 | eps = abcpmc.ExponentialEps(T, eps_vals[0], eps_vals[-1]) 57 | 58 | for e1,e2 in zip(eps, eps_vals): 59 | assert e1 == e2 60 | 61 | assert e1 == eps_vals[-1] 62 | 63 | def test_ExponentialConstEps(): 64 | T1 = 6 65 | T2 = 2 66 | min = 5 67 | max = 10 68 | eps_vals = np.r_[np.logspace(np.log10(max), np.log10(min), T1), [min]*T2] 69 | eps = abcpmc.ExponentialConstEps(eps_vals[0], eps_vals[-1], T1, T2) 70 | 71 | for e1,e2 in zip(eps, eps_vals): 72 | assert e1 == e2 -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py26, py27, py33, py34, docs-ci 3 | 4 | [testenv] 5 | setenv = 6 | PYTHONPATH = {toxinidir}:{toxinidir}/abcpmc 7 | deps = 8 | -r{toxinidir}/requirements.txt 9 | pytest-cov 10 | commands = 11 | py.test --basetemp={envtmpdir} --junitxml=junit-{envname}.xml --cov-report xml --cov abcpmc 12 | 13 | [testenv:style] 14 | deps = 15 | -r{toxinidir}/requirements.txt 16 | flake8 17 | commands = 18 | python setup.py flake8 19 | 20 | [testenv:docs] 21 | changedir=docs/ 22 | deps = 23 | -r{toxinidir}/requirements.txt 24 | sphinx 25 | commands = 26 | sphinx-build -b linkcheck ./ _build/ 27 | sphinx-build -b html ./ _build/ 28 | 29 | [testenv:docs-ci] 30 | whitelist_externals = mv 31 | changedir=docs/ 32 | deps = 33 | -r{toxinidir}/requirements.txt 34 | pytest-cov 35 | commands= 36 | py.test --tb=line -v --junitxml=junit-{envname}.xml check_sphinx.py --cov-report xml --cov abcpmc 37 | mv coverage.xml ../. --------------------------------------------------------------------------------