├── .gitignore ├── CONTRIBUTING.rst ├── LICENSE.md ├── README.md ├── contributors.md ├── examples ├── constrained │ ├── branin_con.py │ └── config.json ├── distributed │ ├── branin.py │ └── config.json ├── noisy │ ├── branin_noisy.py │ └── config.json └── simple │ ├── branin.py │ ├── config.json │ └── make_plots.py ├── setup.py └── spearmint ├── __init__.py ├── choosers ├── __init__.py ├── acquisition_functions.py └── default_chooser.py ├── cleanup.sh ├── grids ├── __init__.py ├── sobol.py └── sobol_grid.py ├── kernels ├── __init__.py ├── abstract_kernel.py ├── kernel_utils.py ├── matern.py ├── noise.py ├── product.py ├── product_kernel.py ├── scale.py ├── sum_kernel.py └── transform_kernel.py ├── launcher.py ├── main.py ├── models ├── __init__.py ├── abstract_model.py ├── gp.py └── gp_classifier.py ├── resources ├── __init__.py └── resource.py ├── sampling ├── __init__.py ├── abstract_sampler.py ├── elliptical_slice_sampler.py ├── mcmc.py ├── slice_sampler.py └── whitened_prior_slice_sampler.py ├── schedulers ├── PBS.py ├── SGE.py ├── SLURM.py ├── __init__.py ├── abstract_scheduler.py ├── cluster_scheduler.py └── local.py ├── tasks ├── __init__.py ├── base_task.py ├── task.py └── task_group.py ├── tests ├── __init__.py ├── grids │ ├── __init__.py │ └── test_sobol_grid.py ├── kernels │ ├── test_matern.py │ ├── test_product_kernel.py │ ├── test_scale.py │ ├── test_subset.py │ ├── test_sum_kernel.py │ └── test_transform_kernel.py ├── models │ ├── config.json │ ├── in_progress │ │ └── gp.py │ ├── test_gp.py │ └── test_gp_classifier.py ├── tasks │ ├── __init__.py │ └── test_task.py └── transformations │ ├── test_beta_warp.py │ ├── test_kumar_warp.py │ ├── test_linear.py │ ├── test_norm_lin.py │ ├── test_normalization.py │ └── test_transformer.py ├── transformations ├── __init__.py ├── abstract_transformation.py ├── beta_warp.py ├── ignore_dims.py ├── kumar_warp.py ├── linear.py ├── norm_lin.py ├── normalization.py └── transformer.py └── utils ├── __init__.py ├── cleanup.py ├── compression.py ├── database ├── __init__.py ├── abstractdb.py └── mongodb.py ├── grad_check.py ├── linalg.py ├── locker.py ├── param.py ├── parsing.py └── priors.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.log 3 | *.out 4 | *.blg 5 | *.aux 6 | *.bib 7 | #*# 8 | *.DS_Store 9 | *.pdf 10 | *.py~ 11 | 12 | 13 | 14 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | Spearmint 2 | ========= 3 | 4 | Academic and Non-Commercial Research Use Software License and Terms of Use 5 | -------------------------------------------------------------------------- 6 | 7 | Spearmint is a software package to perform Bayesian optimization according to specific algorithms (the “Software”). The Software is designed to automatically run experiments (thus the code name *spearmint*) in a manner that iteratively adjusts a number of parameters so as to minimize some objective in as few runs as possible. 8 | 9 | The Software was developed by Ryan P. Adams, Michael Gelbart, and Jasper Snoek and at Harvard University, Kevin Swersky at the University of Toronto (“Toronto”), and Hugo Larochelle at the Université de Sherbrooke (“Sherbrooke”), which assigned its rights in the Software to Socpra Sciences et Génie S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement between the parties, it is distributed for free academic and non-commercial research use by the President and Fellows of Harvard College (“Harvard”). 10 | 11 | Using the Software indicates your agreement to be bound by the terms of this Software Use Agreement (“Agreement”). Absent your agreement to the terms below, you (the “End User”) have no rights to hold or use the Software whatsoever. 12 | Harvard agrees to grant hereunder the limited non-exclusive license to End User for the use of the Software in the performance of End User’s internal, non-commercial research and academic use at End User’s academic or not-for-profit research institution (“Institution”) on the following terms and conditions: 13 | 14 | **1. NO REDISTRIBUTION.** The Software remains the property of Harvard, Toronto and Socpra, and except as set forth in Section 4, End User shall not publish, distribute, or otherwise transfer or make available the Software to any other party. 15 | 16 | **2. NO COMMERCIAL USE.** End User shall not use the Software for commercial purposes and any such use of the Software is expressly prohibited. This includes, but is not limited to, use of the Software in fee-for-service arrangements, core facilities or laboratories or to provide research services to (or in collaboration with) third parties for a fee, and in industry-sponsored collaborative research projects where any commercial rights are granted to the sponsor. If End User wishes to use the Software for commercial purposes or for any other restricted purpose, End User must execute a separate license agreement with Harvard. 17 | 18 | Requests for use of the Software for commercial purposes, please contact: 19 | 20 | :: 21 | 22 | Office of Technology Development 23 | Harvard University 24 | Smith Campus Center, Suite 727E 25 | 1350 Massachusetts Avenue 26 | Cambridge, MA 02138 USA 27 | Telephone: (617) 495-3067 28 | Facsimile: (617) 495-9568 29 | E-mail: otd@harvard.edu 30 | 31 | **3. OWNERSHIP AND COPYRIGHT NOTICE.** Harvard, Toronto and Socpra own all intellectual property in the Software. End User shall gain no ownership to the Software. End User shall not remove or delete and shall retain in the Software, in any modifications to Software and in any Derivative Works, the copyright, trademark, or other notices pertaining to Software as provided with the Software. 32 | 33 | **4. DERIVATIVE WORKS.** End User may create and use Derivative Works, as such term is defined under U.S. copyright laws, provided that any such Derivative Works shall be restricted to non-commercial, internal research and academic use at End User’s Institution. End User may distribute Derivative Works to other Institutions solely for the performance of non-commercial, internal research and academic use on terms substantially similar to this License and Terms of Use. 34 | 35 | **5. FEEDBACK.** In order to improve the Software, comments from End Users may be useful. End User agrees to provide Harvard with feedback on the End User’s use of the Software (e.g., any bugs in the Software, the user experience, etc.). Harvard is permitted to use such information provided by End User in making changes and improvements to the Software without compensation or an accounting to End User. 36 | 37 | **6. NON ASSERT.** End User acknowledges that Harvard, Toronto and/or Sherbrooke or Socpra may develop modifications to the Software that may be based on the feedback provided by End User under Section 5 above. Harvard, Toronto and Sherbrooke/Socpra shall not be restricted in any way by End User regarding their use of such information. End User acknowledges the right of Harvard, Toronto and Sherbrooke/Socpra to prepare, publish, display, reproduce, transmit and or use modifications to the Software that may be substantially similar or functionally equivalent to End User’s modifications and/or improvements if any. In the event that End User obtains patent protection for any modification or improvement to Software, End User agrees not to allege or enjoin infringement of End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, or any of the researchers, medical or research staff, officers, directors and employees of those institutions. 38 | 39 | **7. PUBLICATION & ATTRIBUTION.** End User has the right to publish, present, or share results from the use of the Software.  In accordance with customary academic practice, End User will acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers of the Software and may cite the relevant reference(s) from the following list of publications: 40 | 41 | :: 42 | 43 | Practical Bayesian Optimization of Machine Learning Algorithms 44 | Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 45 | Advances in Neural Information Processing Systems, 2012 46 | 47 | Multi-Task Bayesian Optimization 48 | Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 49 | Advances in Neural Information Processing Systems, 2013 50 | 51 | Input Warping for Bayesian Optimization of Non-stationary Functions 52 | Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 53 | International Conference on Machine Learning, 2014 54 | 55 | Bayesian Optimization and Semiparametric Models with Applications to Assistive Technology 56 | Jasper Snoek, PhD Thesis, University of Toronto, 2013 57 | 58 | Bayesian Optimization with Unknown Constraints 59 | Michael Gelbart, Jasper Snoek and Ryan Prescott Adams 60 | Uncertainty in Artificial Intelligence, 2014 61 | 62 | **8. NO WARRANTIES.** THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE and Socpra HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND SHERBROOKE and Socpra MAKE NO WARRANTY ABOUT THE ACCURACY, RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE SOFTWARE. HARVARD, TORONTO AND SHERBROOKE and Socpra DO NOT WARRANT THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 63 | 64 | **9. Limitations of Liability and Remedies.** USE OF THE SOFTWARE IS AT END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL HARVARD, TORONTO OR SHERBROOKE OR Socpra BE LIABLE TO END USER OR ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR Socpra IS NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, TORONTO OR SHERBROOKE OR Socpra IS ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 65 | 66 | **10. INDEMNIFICATION.** To the extent permitted by law, End User shall indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke and Socpra, their corporate affiliates, current or future directors, trustees, officers, faculty, medical and professional staff, employees, students and agents and their respective successors, heirs and assigns (the "Indemnitees"), against any liability, damage, loss or expense (including reasonable attorney's fees and expenses of litigation) incurred by or imposed upon the Indemnitees or any one of them in connection with any claims, suits, actions, demands or judgments arising from End User’s breach of this Agreement or its Institution’s use of the Software except to the extent caused by the gross negligence or willful misconduct of Harvard, Toronto or Sherbrooke or Socpra. This indemnification provision shall survive expiration or termination of this Agreement. 67 | 68 | **11. GOVERNING LAW.** This Agreement shall be construed and governed by the laws of the Commonwealth of Massachusetts regardless of otherwise applicable choice of law standards. 69 | 70 | **12. NON-USE OF NAME.** Nothing in this License and Terms of Use shall be construed as granting End Users or their Institutions any rights or licenses to use any trademarks, service marks or logos associated with the Software. You may not use the terms “Harvard” or “University of Toronto” or “Université de Sherbrooke” or “Socpra Sciences et Génie S.E.C.” (or a substantially similar term) in any way that is inconsistent with the permitted uses described herein. You agree not to use any name or emblem of Harvard, Toronto or Sherbrooke, or any of their subdivisions for any purpose, or to falsely suggest any relationship between End User (or its Institution) and Harvard, Toronto and/or Sherbrooke, or in any manner that would infringe or violate any of their rights. 71 | 72 | **13.** End User represents and warrants that it has the legal authority to enter into this License and Terms of Use on behalf of itself and its Institution. 73 | 74 | *** 75 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # Spearmint 2 | 3 | ###Academic and Non-Commercial Research Use Software License and Terms of Use 4 | 5 | Spearmint is a software package to perform Bayesian optimization 6 | according to specific algorithms (the “Software”). The Software is 7 | designed to automatically run experiments (thus the code name 8 | *spearmint*) in a manner that iteratively adjusts a number of 9 | parameters so as to minimize some objective in as few runs as 10 | possible. 11 | 12 | The Software was developed by Ryan P. Adams, Michael Gelbart, and 13 | Jasper Snoek at Harvard University, Kevin Swersky at the 14 | University of Toronto (“Toronto”), and Hugo Larochelle at the 15 | Université de Sherbrooke (“Sherbrooke”), which assigned its rights 16 | in the Software to Socpra Sciences et Génie 17 | S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 18 | between the parties, it is distributed for free academic and 19 | non-commercial research use by the President and Fellows of Harvard 20 | College (“Harvard”). 21 | 22 | Using the Software indicates your agreement to be bound by the terms 23 | of this Software Use Agreement (“Agreement”). Absent your agreement 24 | to the terms below, you (the “End User”) have no rights to hold or 25 | use the Software whatsoever. 26 | 27 | Harvard agrees to grant hereunder the limited non-exclusive license 28 | to End User for the use of the Software in the performance of End 29 | User’s internal, non-commercial research and academic use at End 30 | User’s academic or not-for-profit research institution 31 | (“Institution”) on the following terms and conditions: 32 | 33 | **1. NO REDISTRIBUTION.** The Software remains the property Harvard, 34 | Toronto and Socpra, and except as set forth in Section 4, End User 35 | shall not publish, distribute, or otherwise transfer or make 36 | available the Software to any other party. 37 | 38 | **2. NO COMMERCIAL USE.** End User shall not use the Software for 39 | commercial purposes and any such use of the Software is expressly 40 | prohibited. This includes, but is not limited to, use of the 41 | Software in fee-for-service arrangements, core facilities or 42 | laboratories or to provide research services to (or in collaboration 43 | with) third parties for a fee, and in industry-sponsored 44 | collaborative research projects where any commercial rights are 45 | granted to the sponsor. If End User wishes to use the Software for 46 | commercial purposes or for any other restricted purpose, End User 47 | must execute a separate license agreement with Harvard. 48 | 49 | Requests for use of the Software for commercial purposes, please 50 | contact: 51 | 52 | Office of Technology Development 53 | Harvard University 54 | Smith Campus Center, Suite 727E 55 | 1350 Massachusetts Avenue 56 | Cambridge, MA 02138 USA 57 | Telephone: (617) 495-3067 58 | Facsimile: (617) 495-9568 59 | E-mail: otd@harvard.edu 60 | 61 | **3. OWNERSHIP AND COPYRIGHT NOTICE.** Harvard, Toronto and Socpra own 62 | all intellectual property in the Software. End User shall gain no 63 | ownership to the Software. End User shall not remove or delete and 64 | shall retain in the Software, in any modifications to Software and 65 | in any Derivative Works, the copyright, trademark, or other notices 66 | pertaining to Software as provided with the Software. 67 | 68 | **4. DERIVATIVE WORKS.** End User may create and use Derivative Works, 69 | as such term is defined under U.S. copyright laws, provided that any 70 | such Derivative Works shall be restricted to non-commercial, 71 | internal research and academic use at End User’s Institution. End 72 | User may distribute Derivative Works to other Institutions solely 73 | for the performance of non-commercial, internal research and 74 | academic use on terms substantially similar to this License and 75 | Terms of Use. 76 | 77 | **5. FEEDBACK.** In order to improve the Software, comments from End 78 | Users may be useful. End User agrees to provide Harvard with 79 | feedback on the End User’s use of the Software (e.g., any bugs in 80 | the Software, the user experience, etc.). Harvard is permitted to 81 | use such information provided by End User in making changes and 82 | improvements to the Software without compensation or an accounting 83 | to End User. 84 | 85 | **6. NON ASSERT.** End User acknowledges that Harvard, Toronto and/or 86 | Sherbrooke or Socpra may develop modifications to the Software that 87 | may be based on the feedback provided by End User under Section 5 88 | above. Harvard, Toronto and Sherbrooke/Socpra shall not be 89 | restricted in any way by End User regarding their use of such 90 | information. End User acknowledges the right of Harvard, Toronto 91 | and Sherbrooke/Socpra to prepare, publish, display, reproduce, 92 | transmit and or use modifications to the Software that may be 93 | substantially similar or functionally equivalent to End User’s 94 | modifications and/or improvements if any. In the event that End 95 | User obtains patent protection for any modification or improvement 96 | to Software, End User agrees not to allege or enjoin infringement of 97 | End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 98 | or any of the researchers, medical or research staff, officers, 99 | directors and employees of those institutions. 100 | 101 | **7. PUBLICATION & ATTRIBUTION.** End User has the right to publish, 102 | present, or share results from the use of the Software. In 103 | accordance with customary academic practice, End User will 104 | acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 105 | of the Software and may cite the relevant reference(s) from the 106 | following list of publications: 107 | 108 | Practical Bayesian Optimization of Machine Learning Algorithms 109 | Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 110 | *Advances in Neural Information Processing Systems*, 2012 111 | 112 | Multi-Task Bayesian Optimization 113 | Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 114 | *Advances in Neural Information Processing Systems*, 2013 115 | 116 | Input Warping for Bayesian Optimization of Non-stationary Functions 117 | Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 118 | *International Conference on Machine Learning*, 2013 119 | 120 | Bayesian Optimization and Semiparametric Models with Applications to Assistive Technology 121 | Jasper Snoek, PhD Thesis, University of Toronto, 2013 122 | 123 | **8. NO WARRANTIES.** THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 124 | EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 125 | HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 126 | OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 127 | IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 128 | PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 129 | SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 130 | RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 131 | SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 132 | THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 133 | 134 | **9. LIMITATIONS OF LIABILITY AND REMEDIES.** USE OF THE SOFTWARE IS AT 135 | END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 136 | ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 137 | HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 138 | ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 139 | INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 140 | DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 141 | THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 142 | NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 143 | TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 144 | SUCH DAMAGES. 145 | 146 | **10. INDEMNIFICATION.** To the extent permitted by law, End User shall 147 | indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 148 | and Socpra, their corporate affiliates, current or future directors, 149 | trustees, officers, faculty, medical and professional staff, 150 | employees, students and agents and their respective successors, 151 | heirs and assigns (the "Indemnitees"), against any liability, 152 | damage, loss or expense (including reasonable attorney's fees and 153 | expenses of litigation) incurred by or imposed upon the Indemnitees 154 | or any one of them in connection with any claims, suits, actions, 155 | demands or judgments arising from End User’s breach of this 156 | Agreement or its Institution’s use of the Software except to the 157 | extent caused by the gross negligence or willful misconduct of 158 | Harvard, Toronto or Sherbrooke or Socpra. This indemnification 159 | provision shall survive expiration or termination of this Agreement. 160 | 161 | **11. GOVERNING LAW.** This Agreement shall be construed and governed by 162 | the laws of the Commonwealth of Massachusetts regardless of 163 | otherwise applicable choice of law standards. 164 | 165 | **12. NON-USE OF NAME.** Nothing in this License and Terms of Use shall 166 | be construed as granting End Users or their Institutions any rights 167 | or licenses to use any trademarks, service marks or logos associated 168 | with the Software. You may not use the terms “Harvard” or 169 | “University of Toronto” or “Université de Sherbrooke” or “Socpra 170 | Sciences et Génie S.E.C.” (or a substantially similar term) in any 171 | way that is inconsistent with the permitted uses described 172 | herein. You agree not to use any name or emblem of Harvard, Toronto 173 | or Sherbrooke, or any of their subdivisions for any purpose, or to 174 | falsely suggest any relationship between End User (or its 175 | Institution) and Harvard, Toronto and/or Sherbrooke, or in any 176 | manner that would infringe or violate any of their rights. 177 | 178 | **13.** End User represents and warrants that it has the legal authority 179 | to enter into this License and Terms of Use on behalf of itself and 180 | its Institution. 181 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Spearmint 2 | ========================================= 3 | 4 | Spearmint is a software package to perform Bayesian optimization. The Software is designed to automatically run experiments (thus the code name spearmint) in a manner that iteratively adjusts a number of parameters so as to minimize some objective in as few runs as possible. 5 | 6 | **IMPORTANT:** Spearmint is under an **Academic and Non-Commercial Research Use License**. Before using spearmint please be aware of the [license](LICENSE.md). If you do not qualify to use spearmint you can ask to obtain a license as detailed in the [license](LICENSE.md) or you can use the older open source code version (which is somewhat outdated) at https://github.com/JasperSnoek/spearmint. 7 | 8 | ### Relevant Publications 9 | 10 | Spearmint implements a combination of the algorithms detailed in the following publications: 11 | 12 | Practical Bayesian Optimization of Machine Learning Algorithms 13 | Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 14 | Advances in Neural Information Processing Systems, 2012 15 | 16 | Multi-Task Bayesian Optimization 17 | Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 18 | Advances in Neural Information Processing Systems, 2013 19 | 20 | Input Warping for Bayesian Optimization of Non-stationary Functions 21 | Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 22 | International Conference on Machine Learning, 2014 23 | 24 | Bayesian Optimization and Semiparametric Models with Applications to Assistive Technology 25 | Jasper Snoek, PhD Thesis, University of Toronto, 2013 26 | 27 | Bayesian Optimization with Unknown Constraints 28 | Michael Gelbart, Jasper Snoek and Ryan Prescott Adams 29 | Uncertainty in Artificial Intelligence, 2014 30 | 31 | ### Setting up Spearmint 32 | 33 | **STEP 1: Installation** 34 | 35 | 1. Install [python](https://www.python.org/), [numpy](http://www.numpy.org/), [scipy](http://www.numpy.org/), [pymongo](https://api.mongodb.org/python/current/). For academic users, the [anaconda](http://continuum.io/downloads) distribution is great. Use numpy 1.8 or higher. We use python 2.7. 36 | 2. Download/clone the spearmint code 37 | 3. Install the spearmint package using pip: `pip install -e \` (the -e means changes will be reflected automatically) 38 | 4. Download and install MongoDB: https://www.mongodb.org/ 39 | 5. Install the pymongo package using e.g., pip `pip install pymongo` or anaconda `conda install pymongo` 40 | 41 | **STEP 2: Setting up your experiment** 42 | 1. Create a callable objective function. See `./examples/simple/branin.py` as an example 43 | 2. Create a config file. There are 3 example config files in the ../examples directory. Note 1: There are more parameters that can be set in the config files than what is shown in the examples, but these parameters all have default values. Note 2: By default Spearmint assumes your function is noisy (non-deterministic). If it is noise-free, you should set this explicitly as in the ../examples/simple/config.json file. 44 | 45 | **STEP 3: Running spearmint** 46 | 1. Start up a MongoDB daemon instance: 47 | `mongod --fork --logpath --dbpath ` 48 | 2. Run spearmint: `python main.py \` 49 | 50 | **STEP 4: Looking at your results** 51 | Spearmint will output results to standard out / standard err. You can also load the results from the database and manipulate them directly. 52 | -------------------------------------------------------------------------------- /contributors.md: -------------------------------------------------------------------------------- 1 | Contributors 2 | ------------ 3 | We have verified that the following individuals have have submitted 4 | Contributor License Agreements. 5 | 6 | [ivoanastacio](https://github.com/HIPS/Spearmint/pull/4) 7 | [kyunghyuncho](https://github.com/HIPS/Spearmint/pull/5) 8 | Peter Sadowski (psadowsk@uci.edu) 9 | Jacob Stevenson [js850](https://github.com/HIPS/Spearmint/pull/7) 10 | [lzamparo](https://github.com/HIPS/Spearmint/pull/9) 11 | [liori](https://github.com/HIPS/Spearmint/pull/13) 12 | [JesseLivezey](https://github.com/HIPS/Spearmint/pull/16) 13 | [ajschumacher](https://github.com/HIPS/Spearmint/pull/40) 14 | [danielhernandezlobato](https://github.com/HIPS/Spearmint/issues/53) 15 | -------------------------------------------------------------------------------- /examples/constrained/branin_con.py: -------------------------------------------------------------------------------- 1 | import math 2 | import numpy as np 3 | 4 | def evaluate(job_id, params): 5 | 6 | x = params['X'] 7 | y = params['Y'] 8 | 9 | print 'Evaluating at (%f, %f)' % (x, y) 10 | 11 | if x < 0 or x > 5.0 or y > 5.0: 12 | return np.nan 13 | # Feasible region: x in [0,5] and y in [0,5] 14 | 15 | obj = float(np.square(y - (5.1/(4*np.square(math.pi)))*np.square(x) + (5/math.pi)*x- 6) + 10*(1-(1./(8*math.pi)))*np.cos(x) + 10) 16 | 17 | con1 = float(y-x) # y >= x 18 | 19 | con2 = float(10.0-y) # y <= 10 20 | 21 | return { 22 | "branin" : obj, 23 | "y_at_least_x" : con1, 24 | "y_at_most_10" : con2 25 | } 26 | 27 | # True minimum is at 2.945, 2.945, with a value of 0.8447 28 | 29 | def main(job_id, params): 30 | try: 31 | return evaluate(job_id, params) 32 | except Exception as ex: 33 | print ex 34 | print 'An error occurred in branin_con.py' 35 | return np.nan 36 | -------------------------------------------------------------------------------- /examples/constrained/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "language" : "PYTHON", 3 | "main-file" : "branin_con", 4 | "experiment-name" : "branin-constrained-example", 5 | "variables": { 6 | "X": { 7 | "type": "FLOAT", 8 | "size": 1, 9 | "min": -5, 10 | "max": 10 11 | }, 12 | "Y": { 13 | "type": "FLOAT", 14 | "size": 1, 15 | "min": 0, 16 | "max": 15 17 | } 18 | }, 19 | "tasks": { 20 | "branin" : { 21 | "type" : "OBJECTIVE", 22 | "likelihood" : "NOISELESS" 23 | }, 24 | "y_at_least_x" : { 25 | "type" : "CONSTRAINT", 26 | "likelihood" : "NOISELESS" 27 | }, 28 | "y_at_most_10" : { 29 | "type" : "CONSTRAINT", 30 | "likelihood" : "NOISELESS" 31 | } 32 | }, 33 | "polling-time" : 1 34 | } 35 | 36 | -------------------------------------------------------------------------------- /examples/distributed/branin.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import sys 3 | import math 4 | import time 5 | 6 | def branin(x, y): 7 | 8 | result = np.square(y - (5.1/(4*np.square(math.pi)))*np.square(x) + 9 | (5/math.pi)*x - 6) + 10*(1-(1./(8*math.pi)))*np.cos(x) + 10 10 | 11 | result = float(result) 12 | 13 | #if np.random.rand > 0.75: 14 | # raise Exception('Blah!') 15 | 16 | print 'Result = %f' % result 17 | time.sleep(np.random.randint(30)) 18 | return {'branin' : result} 19 | 20 | # Write a function like this called 'main' 21 | def main(job_id, params): 22 | print 'Anything printed here will end up in the output directory for job #%d' % job_id 23 | print params 24 | return branin(params['x'], params['y']) 25 | -------------------------------------------------------------------------------- /examples/distributed/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "language" : "PYTHON", 3 | "experiment-name" : "branin-distributed-example", 4 | "polling-time" : 1, 5 | "resources" : { 6 | "my-machine" : { 7 | "scheduler" : "local", 8 | "max-concurrent" : 1, 9 | "max-finished-jobs" : 100 10 | }, 11 | "cluster" : { 12 | "scheduler" : "SLURM", 13 | "max-concurrent" : 3, 14 | "max-finished-jobs" : 1000 15 | } 16 | }, 17 | "tasks": { 18 | "branin" : { 19 | "type" : "OBJECTIVE", 20 | "likelihood" : "NOISELESS", 21 | "main-file" : "branin", 22 | "resources" : ["my-machine", "cluster"] 23 | } 24 | }, 25 | "variables": { 26 | "x" : { 27 | "type" : "FLOAT", 28 | "size" : 1, 29 | "min" : -5, 30 | "max" : 10 31 | }, 32 | "y" : { 33 | "type" : "FLOAT", 34 | "size" : 1, 35 | "min" : 0, 36 | "max" : 15 37 | } 38 | } 39 | } 40 | 41 | -------------------------------------------------------------------------------- /examples/noisy/branin_noisy.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import math 3 | 4 | def branin(x, y): 5 | 6 | result = np.square(y - (5.1/(4*np.square(math.pi)))*np.square(x) + 7 | (5/math.pi)*x - 6) + 10*(1-(1./(8*math.pi)))*np.cos(x) + 10 8 | 9 | result = float(result) 10 | noise = np.random.normal() * 50. 11 | 12 | print 'Result = %f, noise %f, total %f' % (result, noise, result+noise) 13 | #time.sleep(np.random.randint(60)) 14 | return result + noise 15 | 16 | # Write a function like this called 'main' 17 | def main(job_id, params): 18 | print 'Anything printed here will end up in the output directory for job #%d' % job_id 19 | print params 20 | return branin(params['x'], params['y']) 21 | -------------------------------------------------------------------------------- /examples/noisy/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "language" : "PYTHON", 3 | "main-file" : "branin_noisy.py", 4 | "experiment-name" : "braninhoo_noisy", 5 | "likelihood" : "GAUSSIAN", 6 | "variables" : { 7 | "x" : { 8 | "type" : "FLOAT", 9 | "size" : 1, 10 | "min" : -5, 11 | "max" : 10 12 | }, 13 | "y" : { 14 | "type" : "FLOAT", 15 | "size" : 1, 16 | "min" : 0, 17 | "max" : 15 18 | } 19 | } 20 | } 21 | 22 | -------------------------------------------------------------------------------- /examples/simple/branin.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import math 3 | 4 | def branin(x, y): 5 | 6 | result = np.square(y - (5.1/(4*np.square(math.pi)))*np.square(x) + 7 | (5/math.pi)*x - 6) + 10*(1-(1./(8*math.pi)))*np.cos(x) + 10 8 | 9 | result = float(result) 10 | 11 | print 'Result = %f' % result 12 | #time.sleep(np.random.randint(60)) 13 | return result 14 | 15 | # Write a function like this called 'main' 16 | def main(job_id, params): 17 | print 'Anything printed here will end up in the output directory for job #%d' % job_id 18 | print params 19 | return branin(params['x'], params['y']) 20 | -------------------------------------------------------------------------------- /examples/simple/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "language" : "PYTHON", 3 | "main-file" : "branin.py", 4 | "experiment-name" : "simple-braninhoo-example", 5 | "likelihood" : "NOISELESS", 6 | "variables" : { 7 | "x" : { 8 | "type" : "FLOAT", 9 | "size" : 1, 10 | "min" : -5, 11 | "max" : 10 12 | }, 13 | "y" : { 14 | "type" : "FLOAT", 15 | "size" : 1, 16 | "min" : 0, 17 | "max" : 15 18 | } 19 | } 20 | } 21 | 22 | -------------------------------------------------------------------------------- /examples/simple/make_plots.py: -------------------------------------------------------------------------------- 1 | import importlib 2 | import sys 3 | from itertools import izip 4 | 5 | import numpy as np 6 | import matplotlib.pyplot as plt 7 | import mpl_toolkits.mplot3d.axes3d as axes3d 8 | 9 | 10 | from spearmint.utils.database.mongodb import MongoDB 11 | 12 | from spearmint.main import get_options, parse_resources_from_config, load_jobs, remove_broken_jobs, \ 13 | load_task_group, load_hypers 14 | 15 | def print_dict(d, level=1): 16 | if isinstance(d, dict): 17 | if level > 1: print "" 18 | for k, v in d.iteritems(): 19 | print " " * level, k, 20 | print_dict(v, level=level+1) 21 | else: 22 | print d 23 | 24 | def main(): 25 | """ 26 | Usage: python make_plots.py PATH_TO_DIRECTORY 27 | 28 | TODO: Some aspects of this function are specific to the simple branin example 29 | We should clean this up so that interpretation of plots are more clear and 30 | so that it works in more general cases 31 | (e.g. if objective likelihood is binomial then values should not be 32 | unstandardized) 33 | """ 34 | options, expt_dir = get_options() 35 | print "options:" 36 | print_dict(options) 37 | 38 | # reduce the grid size 39 | options["grid_size"] = 400 40 | 41 | resources = parse_resources_from_config(options) 42 | 43 | # Load up the chooser. 44 | chooser_module = importlib.import_module('spearmint.choosers.' + options['chooser']) 45 | chooser = chooser_module.init(options) 46 | print "chooser", chooser 47 | experiment_name = options.get("experiment-name", 'unnamed-experiment') 48 | 49 | # Connect to the database 50 | db_address = options['database']['address'] 51 | sys.stderr.write('Using database at %s.\n' % db_address) 52 | db = MongoDB(database_address=db_address) 53 | 54 | # testing below here 55 | jobs = load_jobs(db, experiment_name) 56 | remove_broken_jobs(db, jobs, experiment_name, resources) 57 | 58 | print "resources:", resources 59 | print_dict(resources) 60 | resource = resources.itervalues().next() 61 | 62 | task_options = { task: options["tasks"][task] for task in resource.tasks } 63 | print "task_options:" 64 | print_dict(task_options) # {'main': {'likelihood': u'NOISELESS', 'type': 'OBJECTIVE'}} 65 | 66 | task_group = load_task_group(db, options, resource.tasks) 67 | print "task_group", task_group # TaskGroup 68 | print "tasks:" 69 | print_dict(task_group.tasks) # {'main': } 70 | 71 | 72 | hypers = load_hypers(db, experiment_name) 73 | print "loaded hypers", hypers # from GP.to_dict() 74 | 75 | hypers = chooser.fit(task_group, hypers, task_options) 76 | print "\nfitted hypers:" 77 | print_dict(hypers) 78 | 79 | lp, x = chooser.best() 80 | x = x.flatten() 81 | print "best", lp, x 82 | bestp = task_group.paramify(task_group.from_unit(x)) 83 | print "expected best position", bestp 84 | 85 | # get the grid of points 86 | grid = chooser.grid 87 | # print "chooser objectives:", 88 | # print_dict(chooser.objective) 89 | print "chooser models:", chooser.models 90 | print_dict(chooser.models) 91 | obj_model = chooser.models[chooser.objective['name']] 92 | obj_mean, obj_var = obj_model.function_over_hypers(obj_model.predict, grid) 93 | 94 | # un-normalize the function values and variances 95 | obj_task = task_group.tasks['main'] 96 | obj_mean = [obj_task.unstandardize_mean(obj_task.unstandardize_variance(v)) for v in obj_mean] 97 | obj_std = [obj_task.unstandardize_variance(np.sqrt(v)) for v in obj_var] 98 | 99 | 100 | # for xy, m, v in izip(grid, obj_mean, obj_var): 101 | # print xy, m, v 102 | 103 | grid = map(task_group.from_unit, grid) 104 | # return 105 | 106 | xymv = [(xy[0], xy[1], m, v) for xy, m, v in izip(grid, obj_mean, obj_std)]# if .2 < xy[0] < .25] 107 | 108 | x = map(lambda x:x[0], xymv) 109 | y = map(lambda x:x[1], xymv) 110 | m = map(lambda x:x[2], xymv) 111 | sig = map(lambda x:x[3], xymv) 112 | # print y 113 | 114 | fig = plt.figure(dpi=100) 115 | ax = fig.add_subplot(111, projection='3d') 116 | 117 | ax.plot(x, y, m, marker='.', linestyle="None") 118 | 119 | # plot errorbars 120 | for i in np.arange(0, len(x)): 121 | ax.plot([x[i], x[i]], [y[i], y[i]], [m[i]+sig[i], m[i]-sig[i]], marker="_", color='k') 122 | 123 | # get the observed points 124 | task = task_group.tasks['main'] 125 | idata = task.valid_normalized_data_dict 126 | xy = idata["inputs"] 127 | xy = map(task_group.from_unit, xy) 128 | xy = np.array(xy) 129 | vals = idata["values"] 130 | vals = [obj_task.unstandardize_mean(obj_task.unstandardize_variance(v)) for v in vals] 131 | 132 | ax.plot(xy[:,0], xy[:,1], vals, marker='o', color="r", linestyle="None") 133 | 134 | plt.show() 135 | 136 | 137 | 138 | if __name__ == "__main__": 139 | main() 140 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from setuptools import setup 3 | 4 | # Utility function to read the README file. 5 | # Used for the long_description. It's nice, because now 1) we have a top level 6 | # README file and 2) it's easier to type in the README file than to put a raw 7 | # string in below ... 8 | def read(fname): 9 | return open(os.path.join(os.path.dirname(__file__), fname)).read() 10 | 11 | setup( 12 | name = "spearmint", 13 | version = "0.1", 14 | author = "Jasper Snoek, Ryan Adams, Kevin Swersky, Michael Gelbart, and Hugo Larochelle", 15 | author_email = "rpa@seas.harvard.edu, jsnoek@seas.harvard.edu, kswersky@cs.toronto.edu, mgelbart@seas.harvard.edu, hugo.larochelle@usherbrooke.ca", 16 | description = ("A package for Bayesian optimization."), 17 | keywords = "Bayesian Optimization, Magic, Minty Freshness", 18 | packages=['spearmint', 19 | 'spearmint.choosers', 20 | 'spearmint.grids', 21 | 'spearmint.kernels', 22 | 'spearmint.models', 23 | 'spearmint.resources', 24 | 'spearmint.sampling', 25 | 'spearmint.schedulers', 26 | 'spearmint.tasks', 27 | 'spearmint.transformations', 28 | 'spearmint.utils', 29 | 'spearmint.utils.database',], 30 | long_description=read('README.md'), 31 | ) 32 | -------------------------------------------------------------------------------- /spearmint/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/__init__.py -------------------------------------------------------------------------------- /spearmint/choosers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/choosers/__init__.py -------------------------------------------------------------------------------- /spearmint/cleanup.sh: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | #! /bin/bash 186 | # This is a simple script to cleanup the intermediate files in 187 | # spearmint experiment directories 188 | [[ -n "$1" ]] || { echo "Usage: ./cleanup.sh "; exit 0 ; } 189 | if [ -d $1 ] 190 | then 191 | python -c "import spearmint.utils.cleanup as cleanup; import sys; cleanup.cleanup(sys.argv[1])" $1 192 | cd $1 193 | rm output/* 194 | else 195 | echo "$1 is not a valid directory" 196 | fi 197 | -------------------------------------------------------------------------------- /spearmint/grids/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/grids/__init__.py -------------------------------------------------------------------------------- /spearmint/grids/sobol_grid.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | from .sobol import sobol 187 | 188 | def generate(num_dims, grid_size=20000, grid_seed=0): 189 | return sobol(grid_size+grid_seed, num_dims)[grid_seed:] 190 | 191 | -------------------------------------------------------------------------------- /spearmint/kernels/__init__.py: -------------------------------------------------------------------------------- 1 | from matern import Matern52 2 | from sum_kernel import SumKernel 3 | from product_kernel import ProductKernel 4 | from noise import Noise 5 | from scale import Scale 6 | from transform_kernel import TransformKernel 7 | 8 | __all__ = ["Matern52", "SumKernel", "ProductKernel", "Noise", "Scale", "TransformKernel"] 9 | -------------------------------------------------------------------------------- /spearmint/kernels/abstract_kernel.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import numpy as np 186 | 187 | from abc import ABCMeta, abstractmethod 188 | 189 | class AbstractKernel(object): 190 | __metaclass__ = ABCMeta 191 | 192 | @property 193 | def hypers(self): 194 | return None 195 | 196 | @abstractmethod 197 | def cov(self, inputs): 198 | pass 199 | 200 | @abstractmethod 201 | def diag_cov(self, inputs): 202 | pass 203 | 204 | @abstractmethod 205 | def cross_cov(self, inputs_1, inputs_2): 206 | pass 207 | 208 | @abstractmethod 209 | def cross_cov_grad_data(self, inputs_1, inputs_2): 210 | pass 211 | 212 | 213 | 214 | -------------------------------------------------------------------------------- /spearmint/kernels/sum_kernel.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | from .abstract_kernel import AbstractKernel 187 | 188 | 189 | class SumKernel(AbstractKernel): 190 | def __init__(self, *kernels): 191 | self.kernels = kernels 192 | 193 | def cov(self, inputs): 194 | return reduce(lambda K1, K2: K1+K2, [kernel.cov(inputs) for kernel in self.kernels]) 195 | 196 | def diag_cov(self, inputs): 197 | return reduce(lambda K1, K2: K1+K2, [kernel.diag_cov(inputs) for kernel in self.kernels]) 198 | 199 | def cross_cov(self, inputs_1, inputs_2): 200 | return reduce(lambda K1, K2: K1+K2, [kernel.cross_cov(inputs_1,inputs_2) for kernel in self.kernels]) 201 | 202 | # This is the gradient wrt **inputs_2** 203 | def cross_cov_grad_data(self, inputs_1, inputs_2): 204 | return reduce(lambda dK1, dK2: dK1+dK2, [kernel.cross_cov_grad_data(inputs_1,inputs_2) for kernel in self.kernels]) 205 | 206 | -------------------------------------------------------------------------------- /spearmint/models/__init__.py: -------------------------------------------------------------------------------- 1 | from gp import GP 2 | from gp_classifier import GPClassifier 3 | 4 | __all__ = ["GP", "GPClassifier"] 5 | -------------------------------------------------------------------------------- /spearmint/resources/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/resources/__init__.py -------------------------------------------------------------------------------- /spearmint/sampling/__init__.py: -------------------------------------------------------------------------------- 1 | from abstract_sampler import AbstractSampler 2 | from slice_sampler import SliceSampler 3 | from whitened_prior_slice_sampler import WhitenedPriorSliceSampler 4 | from elliptical_slice_sampler import EllipticalSliceSampler 5 | 6 | __all__ = ["AbstractSampler", "SliceSampler", "WhitenedPriorSliceSampler", "EllipticalSliceSampler"] -------------------------------------------------------------------------------- /spearmint/schedulers/SGE.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | from cluster_scheduler import AbstractClusterScheduler 186 | 187 | def init(*args, **kwargs): 188 | return SGEScheduler(*args, **kwargs) 189 | 190 | class SGEScheduler(AbstractClusterScheduler): 191 | def submit_command(self, output_file, job_name): 192 | return 'qsub -S /bin/bash -e %s -o %s -j -N %s' % (output_file, output_file, job_name) 193 | def output_regexp(self): 194 | return r'Your job (\d+)' 195 | -------------------------------------------------------------------------------- /spearmint/schedulers/SLURM.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import spearmint 186 | from cluster_scheduler import AbstractClusterScheduler 187 | 188 | def init(*args, **kwargs): 189 | return SLURMScheduler(*args, **kwargs) 190 | 191 | class SLURMScheduler(AbstractClusterScheduler): 192 | 193 | def submit_command(self, output_file, job_name): 194 | return 'sbatch -e %s -o %s -J %s' % (output_file, output_file, job_name) 195 | 196 | def output_regexp(self): 197 | return r'Submitted batch job (\d+)' 198 | -------------------------------------------------------------------------------- /spearmint/schedulers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/schedulers/__init__.py -------------------------------------------------------------------------------- /spearmint/schedulers/abstract_scheduler.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | from abc import ABCMeta, abstractmethod 187 | 188 | class AbstractScheduler(object): 189 | __metaclass__ = ABCMeta 190 | 191 | def __init__(self, options): 192 | self.options = options 193 | 194 | @abstractmethod 195 | def submit(self, job_id, experiment_name, experiment_dir, database_address): 196 | pass 197 | 198 | @abstractmethod 199 | def alive(self): 200 | pass 201 | 202 | 203 | 204 | 205 | -------------------------------------------------------------------------------- /spearmint/tasks/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/tasks/__init__.py -------------------------------------------------------------------------------- /spearmint/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/tests/__init__.py -------------------------------------------------------------------------------- /spearmint/tests/grids/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/tests/grids/__init__.py -------------------------------------------------------------------------------- /spearmint/tests/grids/test_sobol_grid.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | import numpy as np 187 | 188 | from collections import OrderedDict 189 | from spearmint.grids import sobol_grid 190 | 191 | def test_generate(): 192 | grid = sobol_grid.generate(10, grid_size=100, grid_seed=1) 193 | 194 | assert grid.shape == (100,10) 195 | assert np.all(grid[0] == 0.5) 196 | 197 | -------------------------------------------------------------------------------- /spearmint/tests/kernels/test_matern.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import numpy as np 186 | import numpy.random as npr 187 | 188 | from spearmint.kernels import Matern52 189 | 190 | def test_matern_grad(): 191 | npr.seed(1) 192 | 193 | eps = 1e-5 194 | N = 10 195 | M = 5 196 | D = 3 197 | 198 | kernel = Matern52(D) 199 | 200 | data1 = npr.randn(N,D) 201 | data2 = npr.randn(M,D) 202 | 203 | loss = np.sum(kernel.cross_cov(data1, data2)) 204 | dloss = kernel.cross_cov_grad_data(data1, data2).sum(0) 205 | 206 | dloss_est = np.zeros(dloss.shape) 207 | for i in xrange(M): 208 | for j in xrange(D): 209 | data2[i,j] += eps 210 | loss_1 = np.sum(kernel.cross_cov(data1, data2)) 211 | data2[i,j] -= 2*eps 212 | loss_2 = np.sum(kernel.cross_cov(data1, data2)) 213 | data2[i,j] += eps 214 | dloss_est[i,j] = ((loss_1 - loss_2) / (2*eps)) 215 | 216 | assert np.linalg.norm(dloss - dloss_est) < 1e-6 217 | 218 | 219 | 220 | -------------------------------------------------------------------------------- /spearmint/tests/kernels/test_scale.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import numpy as np 186 | import numpy.random as npr 187 | 188 | from spearmint.kernels import Matern52, Scale 189 | 190 | def test_grad(): 191 | npr.seed(1) 192 | 193 | eps = 1e-5 194 | N = 10 195 | M = 5 196 | D = 3 197 | 198 | kernel = Scale(Matern52(D)) 199 | kernel.amp2.value = 5.75 200 | 201 | data1 = npr.randn(N,D) 202 | data2 = npr.randn(M,D) 203 | 204 | loss = np.sum(kernel.cross_cov(data1, data2)) 205 | dloss = kernel.cross_cov_grad_data(data1, data2).sum(0) 206 | 207 | dloss_est = np.zeros(dloss.shape) 208 | for i in xrange(M): 209 | for j in xrange(D): 210 | data2[i,j] += eps 211 | loss_1 = np.sum(kernel.cross_cov(data1, data2)) 212 | data2[i,j] -= 2*eps 213 | loss_2 = np.sum(kernel.cross_cov(data1, data2)) 214 | data2[i,j] += eps 215 | dloss_est[i,j] = ((loss_1 - loss_2) / (2*eps)) 216 | 217 | assert np.linalg.norm(dloss - dloss_est) < 1e-6 218 | 219 | 220 | 221 | -------------------------------------------------------------------------------- /spearmint/tests/models/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "language" : "PYTHON", 3 | "function-name" : "branin", 4 | "experiment-name" : "braninhoo-example", 5 | "variables": { 6 | "x" : { 7 | "type" : "FLOAT", 8 | "size" : 1, 9 | "min" : -5, 10 | "max" : 10 11 | }, 12 | "y" : { 13 | "type" : "FLOAT", 14 | "size" : 1, 15 | "min" : 0, 16 | "max" : 15 17 | } 18 | } 19 | } 20 | 21 | -------------------------------------------------------------------------------- /spearmint/tests/tasks/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/tests/tasks/__init__.py -------------------------------------------------------------------------------- /spearmint/tests/transformations/test_linear.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import warnings 186 | 187 | import numpy as np 188 | import numpy.random as npr 189 | 190 | from spearmint.transformations import Linear 191 | from spearmint.utils import priors 192 | from spearmint.utils.param import Param as Hyperparameter 193 | 194 | def test_backward_pass(): 195 | npr.seed(1) 196 | 197 | eps = 1e-5 198 | N = 10 199 | D = 5 200 | 201 | lin = Linear(D) 202 | 203 | data = 0.5*npr.rand(N,D) 204 | new_data = lin.forward_pass(data) 205 | loss = np.sum(new_data**2) 206 | V = 2*new_data 207 | 208 | dloss = lin.backward_pass(V) 209 | 210 | dloss_est = np.zeros(dloss.shape) 211 | for i in xrange(N): 212 | for j in xrange(D): 213 | data[i,j] += eps 214 | loss_1 = np.sum(lin.forward_pass(data)**2) 215 | data[i,j] -= 2*eps 216 | loss_2 = np.sum(lin.forward_pass(data)**2) 217 | data[i,j] += eps 218 | dloss_est[i,j] = ((loss_1 - loss_2) / (2*eps)) 219 | 220 | assert np.linalg.norm(dloss - dloss_est) < 1e-6 221 | 222 | 223 | 224 | -------------------------------------------------------------------------------- /spearmint/tests/transformations/test_norm_lin.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import warnings 186 | 187 | import numpy as np 188 | import numpy.random as npr 189 | 190 | from spearmint.transformations import NormLin 191 | from spearmint.utils import priors 192 | from spearmint.utils.param import Param as Hyperparameter 193 | 194 | def test_backward_pass(): 195 | npr.seed(1) 196 | 197 | eps = 1e-5 198 | N = 10 199 | D = 5 200 | 201 | nl = NormLin(D) 202 | 203 | data = 0.5*npr.rand(N,D) 204 | new_data = nl.forward_pass(data) 205 | loss = np.sum(new_data**2) 206 | V = 2*new_data 207 | 208 | dloss = nl.backward_pass(V) 209 | 210 | dloss_est = np.zeros(dloss.shape) 211 | for i in xrange(N): 212 | for j in xrange(D): 213 | data[i,j] += eps 214 | loss_1 = np.sum(nl.forward_pass(data)**2) 215 | data[i,j] -= 2*eps 216 | loss_2 = np.sum(nl.forward_pass(data)**2) 217 | data[i,j] += eps 218 | dloss_est[i,j] = ((loss_1 - loss_2) / (2*eps)) 219 | 220 | assert np.linalg.norm(dloss - dloss_est) < 1e-6 221 | 222 | 223 | 224 | -------------------------------------------------------------------------------- /spearmint/tests/transformations/test_normalization.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | import warnings 186 | 187 | import numpy as np 188 | import numpy.random as npr 189 | 190 | from spearmint.transformations import Normalization 191 | from spearmint.utils import priors 192 | from spearmint.utils.param import Param as Hyperparameter 193 | 194 | def test_backward_pass(): 195 | npr.seed(1) 196 | 197 | eps = 1e-5 198 | N = 10 199 | D = 5 200 | 201 | n = Normalization(D) 202 | 203 | data = 0.5*npr.rand(N,D) 204 | new_data = n.forward_pass(data) 205 | loss = np.sum(new_data**2) 206 | V = 2*new_data 207 | 208 | dloss = n.backward_pass(V) 209 | 210 | dloss_est = np.zeros(dloss.shape) 211 | for i in xrange(N): 212 | for j in xrange(D): 213 | data[i,j] += eps 214 | loss_1 = np.sum(n.forward_pass(data)**2) 215 | data[i,j] -= 2*eps 216 | loss_2 = np.sum(n.forward_pass(data)**2) 217 | data[i,j] += eps 218 | dloss_est[i,j] = ((loss_1 - loss_2) / (2*eps)) 219 | 220 | assert np.linalg.norm(dloss - dloss_est) < 1e-5 221 | 222 | 223 | 224 | -------------------------------------------------------------------------------- /spearmint/transformations/__init__.py: -------------------------------------------------------------------------------- 1 | from beta_warp import BetaWarp 2 | from ignore_dims import IgnoreDims 3 | from kumar_warp import KumarWarp 4 | from normalization import Normalization 5 | from linear import Linear 6 | from transformer import Transformer 7 | from norm_lin import NormLin 8 | 9 | __all__ = ["BetaWarp", "IgnoreDims", "KumarWarp", "Normalization", "Linear", "Transformer", "NormLin"] 10 | -------------------------------------------------------------------------------- /spearmint/transformations/abstract_transformation.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | from abc import ABCMeta, abstractmethod 187 | 188 | 189 | class AbstractTransformation(object): 190 | __metaclass__ = ABCMeta 191 | 192 | @property 193 | def hypers(self): 194 | return None 195 | 196 | @abstractmethod 197 | def forward_pass(self, inputs): 198 | pass 199 | 200 | @abstractmethod 201 | def backward_pass(self, V): 202 | pass 203 | 204 | def output_num_dims(self): 205 | return self.num_dims 206 | 207 | -------------------------------------------------------------------------------- /spearmint/transformations/ignore_dims.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | import numpy as np 187 | 188 | from .abstract_transformation import AbstractTransformation 189 | 190 | 191 | class IgnoreDims(AbstractTransformation): 192 | def __init__(self, num_dims, to_ignore, name="IgnoreDims"): 193 | self.name = name 194 | self.num_dims = num_dims 195 | self.ign = np.array(to_ignore) 196 | 197 | self.validate_ign() 198 | 199 | def validate_ign(self): 200 | assert len(set(self.ign)) == len(self.ign), 'Each index may only be used once.' 201 | assert max(self.ign) < self.num_dims, 'Maximum index exceeds number of dimensions.' 202 | 203 | def forward_pass(self, inputs): 204 | new_inputs = inputs.copy() 205 | 206 | new_inputs[:,self.ign] = 0.0 207 | 208 | return new_inputs 209 | 210 | def backward_pass(self, V): 211 | JV = V.copy() 212 | JV[:,self.ign] = 0.0 213 | 214 | return JV 215 | 216 | 217 | 218 | -------------------------------------------------------------------------------- /spearmint/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/utils/__init__.py -------------------------------------------------------------------------------- /spearmint/utils/cleanup.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | import os 187 | import sys 188 | import pymongo 189 | import json 190 | from parsing import parse_db_address 191 | 192 | 193 | def cleanup(path): 194 | 195 | if not os.path.isdir(path): 196 | raise Exception("%s is not a valid directory" % path) 197 | 198 | with open(os.path.join(path, 'config.json'), 'r') as f: 199 | cfg = json.load(f) 200 | 201 | db_address = parse_db_address(cfg) 202 | print 'Cleaning up experiment %s in database at %s' % (cfg["experiment-name"], db_address) 203 | 204 | client = pymongo.MongoClient(db_address) 205 | 206 | db = client.spearmint 207 | db[cfg["experiment-name"]]['jobs'].drop() 208 | db[cfg["experiment-name"]]['hypers'].drop() 209 | 210 | if __name__ == '__main__': 211 | cleanup(sys.argv[1]) -------------------------------------------------------------------------------- /spearmint/utils/database/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HIPS/Spearmint/990d27d4477bbc9b0d5cfb2c950fc387decf6ea2/spearmint/utils/database/__init__.py -------------------------------------------------------------------------------- /spearmint/utils/database/abstractdb.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Spearmint 3 | # 4 | # Academic and Non-Commercial Research Use Software License and Terms 5 | # of Use 6 | # 7 | # Spearmint is a software package to perform Bayesian optimization 8 | # according to specific algorithms (the “Software”). The Software is 9 | # designed to automatically run experiments (thus the code name 10 | # 'spearmint') in a manner that iteratively adjusts a number of 11 | # parameters so as to minimize some objective in as few runs as 12 | # possible. 13 | # 14 | # The Software was developed by Ryan P. Adams, Michael Gelbart, and 15 | # Jasper Snoek at Harvard University, Kevin Swersky at the 16 | # University of Toronto (“Toronto”), and Hugo Larochelle at the 17 | # Université de Sherbrooke (“Sherbrooke”), which assigned its rights 18 | # in the Software to Socpra Sciences et Génie 19 | # S.E.C. (“Socpra”). Pursuant to an inter-institutional agreement 20 | # between the parties, it is distributed for free academic and 21 | # non-commercial research use by the President and Fellows of Harvard 22 | # College (“Harvard”). 23 | # 24 | # Using the Software indicates your agreement to be bound by the terms 25 | # of this Software Use Agreement (“Agreement”). Absent your agreement 26 | # to the terms below, you (the “End User”) have no rights to hold or 27 | # use the Software whatsoever. 28 | # 29 | # Harvard agrees to grant hereunder the limited non-exclusive license 30 | # to End User for the use of the Software in the performance of End 31 | # User’s internal, non-commercial research and academic use at End 32 | # User’s academic or not-for-profit research institution 33 | # (“Institution”) on the following terms and conditions: 34 | # 35 | # 1. NO REDISTRIBUTION. The Software remains the property Harvard, 36 | # Toronto and Socpra, and except as set forth in Section 4, End User 37 | # shall not publish, distribute, or otherwise transfer or make 38 | # available the Software to any other party. 39 | # 40 | # 2. NO COMMERCIAL USE. End User shall not use the Software for 41 | # commercial purposes and any such use of the Software is expressly 42 | # prohibited. This includes, but is not limited to, use of the 43 | # Software in fee-for-service arrangements, core facilities or 44 | # laboratories or to provide research services to (or in collaboration 45 | # with) third parties for a fee, and in industry-sponsored 46 | # collaborative research projects where any commercial rights are 47 | # granted to the sponsor. If End User wishes to use the Software for 48 | # commercial purposes or for any other restricted purpose, End User 49 | # must execute a separate license agreement with Harvard. 50 | # 51 | # Requests for use of the Software for commercial purposes, please 52 | # contact: 53 | # 54 | # Office of Technology Development 55 | # Harvard University 56 | # Smith Campus Center, Suite 727E 57 | # 1350 Massachusetts Avenue 58 | # Cambridge, MA 02138 USA 59 | # Telephone: (617) 495-3067 60 | # Facsimile: (617) 495-9568 61 | # E-mail: otd@harvard.edu 62 | # 63 | # 3. OWNERSHIP AND COPYRIGHT NOTICE. Harvard, Toronto and Socpra own 64 | # all intellectual property in the Software. End User shall gain no 65 | # ownership to the Software. End User shall not remove or delete and 66 | # shall retain in the Software, in any modifications to Software and 67 | # in any Derivative Works, the copyright, trademark, or other notices 68 | # pertaining to Software as provided with the Software. 69 | # 70 | # 4. DERIVATIVE WORKS. End User may create and use Derivative Works, 71 | # as such term is defined under U.S. copyright laws, provided that any 72 | # such Derivative Works shall be restricted to non-commercial, 73 | # internal research and academic use at End User’s Institution. End 74 | # User may distribute Derivative Works to other Institutions solely 75 | # for the performance of non-commercial, internal research and 76 | # academic use on terms substantially similar to this License and 77 | # Terms of Use. 78 | # 79 | # 5. FEEDBACK. In order to improve the Software, comments from End 80 | # Users may be useful. End User agrees to provide Harvard with 81 | # feedback on the End User’s use of the Software (e.g., any bugs in 82 | # the Software, the user experience, etc.). Harvard is permitted to 83 | # use such information provided by End User in making changes and 84 | # improvements to the Software without compensation or an accounting 85 | # to End User. 86 | # 87 | # 6. NON ASSERT. End User acknowledges that Harvard, Toronto and/or 88 | # Sherbrooke or Socpra may develop modifications to the Software that 89 | # may be based on the feedback provided by End User under Section 5 90 | # above. Harvard, Toronto and Sherbrooke/Socpra shall not be 91 | # restricted in any way by End User regarding their use of such 92 | # information. End User acknowledges the right of Harvard, Toronto 93 | # and Sherbrooke/Socpra to prepare, publish, display, reproduce, 94 | # transmit and or use modifications to the Software that may be 95 | # substantially similar or functionally equivalent to End User’s 96 | # modifications and/or improvements if any. In the event that End 97 | # User obtains patent protection for any modification or improvement 98 | # to Software, End User agrees not to allege or enjoin infringement of 99 | # End User’s patent against Harvard, Toronto or Sherbrooke or Socpra, 100 | # or any of the researchers, medical or research staff, officers, 101 | # directors and employees of those institutions. 102 | # 103 | # 7. PUBLICATION & ATTRIBUTION. End User has the right to publish, 104 | # present, or share results from the use of the Software. In 105 | # accordance with customary academic practice, End User will 106 | # acknowledge Harvard, Toronto and Sherbrooke/Socpra as the providers 107 | # of the Software and may cite the relevant reference(s) from the 108 | # following list of publications: 109 | # 110 | # Practical Bayesian Optimization of Machine Learning Algorithms 111 | # Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams 112 | # Neural Information Processing Systems, 2012 113 | # 114 | # Multi-Task Bayesian Optimization 115 | # Kevin Swersky, Jasper Snoek and Ryan Prescott Adams 116 | # Advances in Neural Information Processing Systems, 2013 117 | # 118 | # Input Warping for Bayesian Optimization of Non-stationary Functions 119 | # Jasper Snoek, Kevin Swersky, Richard Zemel and Ryan Prescott Adams 120 | # Preprint, arXiv:1402.0929, http://arxiv.org/abs/1402.0929, 2013 121 | # 122 | # Bayesian Optimization and Semiparametric Models with Applications to 123 | # Assistive Technology Jasper Snoek, PhD Thesis, University of 124 | # Toronto, 2013 125 | # 126 | # 8. NO WARRANTIES. THE SOFTWARE IS PROVIDED "AS IS." TO THE FULLEST 127 | # EXTENT PERMITTED BY LAW, HARVARD, TORONTO AND SHERBROOKE AND SOCPRA 128 | # HEREBY DISCLAIM ALL WARRANTIES OF ANY KIND (EXPRESS, IMPLIED OR 129 | # OTHERWISE) REGARDING THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY 130 | # IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 131 | # PURPOSE, OWNERSHIP, AND NON-INFRINGEMENT. HARVARD, TORONTO AND 132 | # SHERBROOKE AND SOCPRA MAKE NO WARRANTY ABOUT THE ACCURACY, 133 | # RELIABILITY, COMPLETENESS, TIMELINESS, SUFFICIENCY OR QUALITY OF THE 134 | # SOFTWARE. HARVARD, TORONTO AND SHERBROOKE AND SOCPRA DO NOT WARRANT 135 | # THAT THE SOFTWARE WILL OPERATE WITHOUT ERROR OR INTERRUPTION. 136 | # 137 | # 9. LIMITATIONS OF LIABILITY AND REMEDIES. USE OF THE SOFTWARE IS AT 138 | # END USER’S OWN RISK. IF END USER IS DISSATISFIED WITH THE SOFTWARE, 139 | # ITS EXCLUSIVE REMEDY IS TO STOP USING IT. IN NO EVENT SHALL 140 | # HARVARD, TORONTO OR SHERBROOKE OR SOCPRA BE LIABLE TO END USER OR 141 | # ITS INSTITUTION, IN CONTRACT, TORT OR OTHERWISE, FOR ANY DIRECT, 142 | # INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR OTHER 143 | # DAMAGES OF ANY KIND WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH 144 | # THE SOFTWARE, EVEN IF HARVARD, TORONTO OR SHERBROOKE OR SOCPRA IS 145 | # NEGLIGENT OR OTHERWISE AT FAULT, AND REGARDLESS OF WHETHER HARVARD, 146 | # TORONTO OR SHERBROOKE OR SOCPRA IS ADVISED OF THE POSSIBILITY OF 147 | # SUCH DAMAGES. 148 | # 149 | # 10. INDEMNIFICATION. To the extent permitted by law, End User shall 150 | # indemnify, defend and hold harmless Harvard, Toronto and Sherbrooke 151 | # and Socpra, their corporate affiliates, current or future directors, 152 | # trustees, officers, faculty, medical and professional staff, 153 | # employees, students and agents and their respective successors, 154 | # heirs and assigns (the "Indemnitees"), against any liability, 155 | # damage, loss or expense (including reasonable attorney's fees and 156 | # expenses of litigation) incurred by or imposed upon the Indemnitees 157 | # or any one of them in connection with any claims, suits, actions, 158 | # demands or judgments arising from End User’s breach of this 159 | # Agreement or its Institution’s use of the Software except to the 160 | # extent caused by the gross negligence or willful misconduct of 161 | # Harvard, Toronto or Sherbrooke or Socpra. This indemnification 162 | # provision shall survive expiration or termination of this Agreement. 163 | # 164 | # 11. GOVERNING LAW. This Agreement shall be construed and governed by 165 | # the laws of the Commonwealth of Massachusetts regardless of 166 | # otherwise applicable choice of law standards. 167 | # 168 | # 12. NON-USE OF NAME. Nothing in this License and Terms of Use shall 169 | # be construed as granting End Users or their Institutions any rights 170 | # or licenses to use any trademarks, service marks or logos associated 171 | # with the Software. You may not use the terms “Harvard” or 172 | # “University of Toronto” or “Université de Sherbrooke” or “Socpra 173 | # Sciences et Génie S.E.C.” (or a substantially similar term) in any 174 | # way that is inconsistent with the permitted uses described 175 | # herein. You agree not to use any name or emblem of Harvard, Toronto 176 | # or Sherbrooke, or any of their subdivisions for any purpose, or to 177 | # falsely suggest any relationship between End User (or its 178 | # Institution) and Harvard, Toronto and/or Sherbrooke, or in any 179 | # manner that would infringe or violate any of their rights. 180 | # 181 | # 13. End User represents and warrants that it has the legal authority 182 | # to enter into this License and Terms of Use on behalf of itself and 183 | # its Institution. 184 | 185 | 186 | from abc import ABCMeta, abstractmethod 187 | 188 | class AbstractDB(object): 189 | __metaclass__ = ABCMeta 190 | 191 | @abstractmethod 192 | def save(self, collection_name): 193 | pass 194 | 195 | @abstractmethod 196 | def load(self, collection_name, expt_id): 197 | pass --------------------------------------------------------------------------------