├── .circleci └── config.yml ├── .gitattributes ├── .gitignore ├── CHANGELOG.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── LICENSES ├── GLMNET_LICENSE └── SCIKIT_LEARN_LICENSE ├── MANIFEST.in ├── README.rst ├── dev-requirements.txt ├── glmnet ├── __init__.py ├── _glmnet.pyf ├── doc.py ├── errors.py ├── linear.py ├── logistic.py ├── scorer.py ├── src │ └── glmnet │ │ ├── DESCRIPTION │ │ ├── NOTICE │ │ └── glmnet5.f90 ├── tests │ ├── __init__.py │ ├── test_errors.py │ ├── test_linear.py │ ├── test_logistic.py │ ├── test_pandas.py │ ├── test_util.py │ └── util.py └── util.py └── setup.py /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | 3 | workflows: 4 | version: 2 5 | test: 6 | jobs: 7 | - test-3.6 8 | - test-3.7 9 | - test-3.8 10 | 11 | jobs: 12 | test-3.8: &unit-tests 13 | working_directory: ~/python-glmnet 14 | docker: 15 | - image: circleci/python:3.8 16 | steps: 17 | - checkout 18 | - restore_cache: 19 | key: deps1-{{ .Environment.CIRCLE_JOB }}-{{ checksum "dev-requirements.txt" }} 20 | - run: 21 | name: Install environment 22 | command: | 23 | python3 -m venv venv 24 | . venv/bin/activate 25 | pip install -r dev-requirements.txt 26 | - save_cache: 27 | key: deps1-{{ .Environment.CIRCLE_JOB }}-{{ checksum "dev-requirements.txt" }} 28 | paths: 29 | - "venv" 30 | - run: 31 | name: Install Fortran compiler 32 | command: sudo apt-get install -y gfortran 33 | - run: 34 | name: Build package under test and check for PyPI compliance via twine 35 | command: | 36 | . venv/bin/activate 37 | pip install twine 38 | python setup.py sdist 39 | twine check dist/`ls dist/ | grep .tar.gz` 40 | - run: 41 | name: Install package under test 42 | command: | 43 | . venv/bin/activate 44 | pip install . 45 | - run: 46 | name: Run test suite 47 | command: | 48 | . venv/bin/activate 49 | cd ~ 50 | pytest -v python-glmnet/glmnet 51 | test-3.6: 52 | <<: *unit-tests 53 | docker: 54 | - image: circleci/python:3.6 55 | test-3.7: 56 | <<: *unit-tests 57 | docker: 58 | - image: circleci/python:3.7 59 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | glmnet/_version.py export-subst 2 | glmnet/src/glmnet/* linguist-vendored 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | build/ 2 | *.pyc 3 | *.so 4 | *.egg-info 5 | *.so.dSYM 6 | _glmnetmodule.c 7 | dist 8 | MANIFEST 9 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Change Log 2 | All notable changes to this project will be documented in this file. 3 | This project adheres to [Semantic Versioning](http://semver.org/). 4 | 5 | ## Unreleased 6 | 7 | ## 2.2.1 - 2020-06-30 8 | ### Fix 9 | * [#65](https://github.com/civisanalytics/python-glmnet/pull/65) 10 | Remove `six` dependency entirely. 11 | 12 | ## 2.2.0 - 2020-06-29 13 | ### Changed 14 | * [#57](https://github.com/civisanalytics/python-glmnet/pull/57) 15 | Mark the Fortran code as linguist-vendored so that GitHub classifies 16 | this project as Python (#37). 17 | * [#62](https://github.com/civisanalytics/python-glmnet/pull/62) 18 | Update the cross-validation for users to be able to define groups of 19 | observations, which is equivalent with *foldid* of *cvglmnet* in *R*. 20 | * [#64](https://github.com/civisanalytics/python-glmnet/pull/64) 21 | - Python version support: Add v3.8, and drop v3.4 + v3.5. 22 | - Maintenance: Drop versioneer; update and pin dependencies for development. 23 | 24 | ## 2.1.1 - 2019-03-11 25 | ### Fixed 26 | * [#55](https://github.com/civisanalytics/python-glmnet/pull/55) 27 | Include all Fortran source code in source tarball; exclude autogenerated C. 28 | 29 | ## 2.1.0 - 2019-03-11 30 | 31 | ### Added 32 | * [#29](https://github.com/civisanalytics/python-glmnet/pull/29) 33 | Provide understandable error messages for more glmnet solver errors. 34 | * [#31](https://github.com/civisanalytics/python-glmnet/pull/31) 35 | Expose `max_features` parameter in `ElasticNet` and `LogitNet`. 36 | * [#34](https://github.com/civisanalytics/python-glmnet/pull/34) 37 | Use sample weights in `LogitNet`. 38 | * [#41](https://github.com/civisanalytics/python-glmnet/pull/41) 39 | Add `lower_limits` and `upper_limits` parameters to `ElasticNet` 40 | and `LogitNet`, allowing users to restrict the range of fitted coefficients. 41 | 42 | ### Changed 43 | * [#44](https://github.com/civisanalytics/python-glmnet/pull/44) 44 | Change CircleCI configuration file from v1 to v2, switch to pytest, 45 | and test in Python versions 3.4 - 3.7. 46 | * [#36](https://github.com/civisanalytics/python-glmnet/pull/36) 47 | Convert README to .rst format for better display on PyPI (#35). 48 | * [#54](https://github.com/civisanalytics/python-glmnet/pull/54) 49 | Use `setuptools` in `setup.py` and update author in metadata. 50 | 51 | ### Fixed 52 | * [#24](https://github.com/civisanalytics/python-glmnet/pull/24) 53 | Use shuffled splits (controlled by input seed) for cross validation (#23). 54 | * [#47](https://github.com/civisanalytics/python-glmnet/pull/47) 55 | Remove inappropriate `__init__.py` from the root path (#46). 56 | * [#51](https://github.com/civisanalytics/python-glmnet/pull/51) 57 | Satisfy scikit-learn estimator checks. Includes: 58 | Allow one-sample predictions; allow list inputs for sample weights; 59 | Ensure scikit-learn Estimator compatibility. 60 | * [#53](https://github.com/civisanalytics/python-glmnet/pull/53) 61 | Return correct dimensions for 1-row predictions, with or without lambda 62 | path, in both `LogitNet` and `ElasticNet` (#52, #30, #25). 63 | 64 | ## 2.0.0 - 2017-03-01 65 | 66 | ### API Changes 67 | * [#10](https://github.com/civisanalytics/python-glmnet/pull/10) the parameter `n_folds` in the constructors of `LogitNet` and `ElasticNet` has been changed to `n_splits` for consistency with Scikit-Learn. 68 | 69 | ### Added 70 | * [#6](https://github.com/civisanalytics/python-glmnet/pull/6) expose relative penalty 71 | 72 | ### Changed 73 | * [#10](https://github.com/civisanalytics/python-glmnet/pull/10) update Scikit-Learn to 0.18 74 | 75 | ### Fixed 76 | * [#3](https://github.com/civisanalytics/python-glmnet/pull/3) ensure license and readme are included in sdist 77 | * [#8](https://github.com/civisanalytics/python-glmnet/pull/8) fix readme encoding 78 | * [#14](https://github.com/civisanalytics/python-glmnet/pull/14) fix reference to `lambda_best_` in docs 79 | * [#16](https://github.com/civisanalytics/python-glmnet/pull/16) fix import path for UndefinedMetricWarning 80 | 81 | ## 1.0.0 - 2016-06-03 82 | ### Added 83 | - Initial release 84 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Code of Conduct 2 | 3 | As contributors and maintainers of this project, and in the interest of 4 | fostering an open and welcoming community, we pledge to respect all people who 5 | contribute through reporting issues, posting feature requests, updating 6 | documentation, submitting pull requests or patches, and other activities. 7 | 8 | We are committed to making participation in this project a harassment-free 9 | experience for everyone, regardless of level of experience, gender, gender 10 | identity and expression, sexual orientation, disability, personal appearance, 11 | body size, race, ethnicity, age, religion, or nationality. 12 | 13 | Examples of unacceptable behavior by participants include: 14 | 15 | * The use of sexualized language or imagery 16 | * Personal attacks 17 | * Trolling or insulting/derogatory comments 18 | * Public or private harassment 19 | * Publishing other's private information, such as physical or electronic 20 | addresses, without explicit permission 21 | * Other unethical or unprofessional conduct 22 | 23 | Project maintainers have the right and responsibility to remove, edit, or 24 | reject comments, commits, code, wiki edits, issues, and other contributions 25 | that are not aligned to this Code of Conduct, or to ban temporarily or 26 | permanently any contributor for other behaviors that they deem inappropriate, 27 | threatening, offensive, or harmful. 28 | 29 | By adopting this Code of Conduct, project maintainers commit themselves to 30 | fairly and consistently applying these principles to every aspect of managing 31 | this project. Project maintainers who do not follow or enforce the Code of 32 | Conduct may be permanently removed from the project team. 33 | 34 | This Code of Conduct applies both within project spaces and in public spaces 35 | when an individual is representing the project or its community. 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 38 | reported by contacting a project maintainer at opensource@civisanalytics.com. 39 | All complaints will be reviewed and investigated and will result in a response 40 | that is deemed necessary and appropriate to the circumstances. Maintainers are 41 | obligated to maintain confidentiality with regard to the reporter of an 42 | incident. 43 | 44 | 45 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 46 | version 1.3.0, available at 47 | [http://contributor-covenant.org/version/1/3/0/][version] 48 | 49 | [homepage]: http://contributor-covenant.org 50 | [version]: http://contributor-covenant.org/version/1/3/0/ -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to python-glmnet 2 | 3 | We welcome bug reports and pull requests from everyone! 4 | This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct. 5 | 6 | 7 | ## Getting Started 8 | 9 | 1. Fork it ( https://github.com/civisanalytics/python-glmnet/fork ) 10 | 2. Install the dependencies (`pip install -r requirements.txt`) 11 | 3. Make sure you are able to run the test suite locally (`nosetests -v`) 12 | 4. Create a feature branch (`git checkout -b my-new-feature`) 13 | 5. Make your change. Don't forget tests 14 | 6. Make sure the test suite, including your new tests, passes (`nosetests -v`) 15 | 7. Commit your changes (`git commit -am 'Add some feature'`) 16 | 8. Push to the branch (`git push origin my-new-feature`) 17 | 9. Create a new pull request 18 | 10. If the CircleCI build fails, address any issues 19 | 20 | ## Tips 21 | 22 | - All pull requests must include test coverage. If you’re not sure how to test 23 | your changes, feel free to ask for help. 24 | - Contributions must conform to PEP 8 and the NumPy/SciPy Documentation standards: 25 | - [PEP 8 -- Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/) 26 | - [A Guide to NumPy/SciPy Documentation](https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt) 27 | - Don’t forget to add your change to the [CHANGELOG](CHANGELOG.md). See 28 | [Keep a CHANGELOG](http://keepachangelog.com/) for guidelines. 29 | 30 | Thank you for taking the time to contribute to python-glmnet! 31 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | You may use, distribute, and copy python-glmnet under the terms of GNU General 2 | Public License Version 2 (GPLv2) which is coplied below. Parts of Scikit-Learn 3 | and the R glmnet package are included. Their licenses may be found in the 4 | LICENSES folder. 5 | 6 | ------------------------------------------------------------------------------- 7 | GNU GENERAL PUBLIC LICENSE 8 | Version 2, June 1991 9 | 10 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 11 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 12 | Everyone is permitted to copy and distribute verbatim copies 13 | of this license document, but changing it is not allowed. 14 | 15 | Preamble 16 | 17 | The licenses for most software are designed to take away your 18 | freedom to share and change it. By contrast, the GNU General Public 19 | License is intended to guarantee your freedom to share and change free 20 | software--to make sure the software is free for all its users. This 21 | General Public License applies to most of the Free Software 22 | Foundation's software and to any other program whose authors commit to 23 | using it. (Some other Free Software Foundation software is covered by 24 | the GNU Lesser General Public License instead.) You can apply it to 25 | your programs, too. 26 | 27 | When we speak of free software, we are referring to freedom, not 28 | price. Our General Public Licenses are designed to make sure that you 29 | have the freedom to distribute copies of free software (and charge for 30 | this service if you wish), that you receive source code or can get it 31 | if you want it, that you can change the software or use pieces of it 32 | in new free programs; and that you know you can do these things. 33 | 34 | To protect your rights, we need to make restrictions that forbid 35 | anyone to deny you these rights or to ask you to surrender the rights. 36 | These restrictions translate to certain responsibilities for you if you 37 | distribute copies of the software, or if you modify it. 38 | 39 | For example, if you distribute copies of such a program, whether 40 | gratis or for a fee, you must give the recipients all the rights that 41 | you have. You must make sure that they, too, receive or can get the 42 | source code. And you must show them these terms so they know their 43 | rights. 44 | 45 | We protect your rights with two steps: (1) copyright the software, and 46 | (2) offer you this license which gives you legal permission to copy, 47 | distribute and/or modify the software. 48 | 49 | Also, for each author's protection and ours, we want to make certain 50 | that everyone understands that there is no warranty for this free 51 | software. If the software is modified by someone else and passed on, we 52 | want its recipients to know that what they have is not the original, so 53 | that any problems introduced by others will not reflect on the original 54 | authors' reputations. 55 | 56 | Finally, any free program is threatened constantly by software 57 | patents. We wish to avoid the danger that redistributors of a free 58 | program will individually obtain patent licenses, in effect making the 59 | program proprietary. To prevent this, we have made it clear that any 60 | patent must be licensed for everyone's free use or not licensed at all. 61 | 62 | The precise terms and conditions for copying, distribution and 63 | modification follow. 64 | 65 | GNU GENERAL PUBLIC LICENSE 66 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 67 | 68 | 0. This License applies to any program or other work which contains 69 | a notice placed by the copyright holder saying it may be distributed 70 | under the terms of this General Public License. The "Program", below, 71 | refers to any such program or work, and a "work based on the Program" 72 | means either the Program or any derivative work under copyright law: 73 | that is to say, a work containing the Program or a portion of it, 74 | either verbatim or with modifications and/or translated into another 75 | language. (Hereinafter, translation is included without limitation in 76 | the term "modification".) Each licensee is addressed as "you". 77 | 78 | Activities other than copying, distribution and modification are not 79 | covered by this License; they are outside its scope. The act of 80 | running the Program is not restricted, and the output from the Program 81 | is covered only if its contents constitute a work based on the 82 | Program (independent of having been made by running the Program). 83 | Whether that is true depends on what the Program does. 84 | 85 | 1. You may copy and distribute verbatim copies of the Program's 86 | source code as you receive it, in any medium, provided that you 87 | conspicuously and appropriately publish on each copy an appropriate 88 | copyright notice and disclaimer of warranty; keep intact all the 89 | notices that refer to this License and to the absence of any warranty; 90 | and give any other recipients of the Program a copy of this License 91 | along with the Program. 92 | 93 | You may charge a fee for the physical act of transferring a copy, and 94 | you may at your option offer warranty protection in exchange for a fee. 95 | 96 | 2. You may modify your copy or copies of the Program or any portion 97 | of it, thus forming a work based on the Program, and copy and 98 | distribute such modifications or work under the terms of Section 1 99 | above, provided that you also meet all of these conditions: 100 | 101 | a) You must cause the modified files to carry prominent notices 102 | stating that you changed the files and the date of any change. 103 | 104 | b) You must cause any work that you distribute or publish, that in 105 | whole or in part contains or is derived from the Program or any 106 | part thereof, to be licensed as a whole at no charge to all third 107 | parties under the terms of this License. 108 | 109 | c) If the modified program normally reads commands interactively 110 | when run, you must cause it, when started running for such 111 | interactive use in the most ordinary way, to print or display an 112 | announcement including an appropriate copyright notice and a 113 | notice that there is no warranty (or else, saying that you provide 114 | a warranty) and that users may redistribute the program under 115 | these conditions, and telling the user how to view a copy of this 116 | License. (Exception: if the Program itself is interactive but 117 | does not normally print such an announcement, your work based on 118 | the Program is not required to print an announcement.) 119 | 120 | These requirements apply to the modified work as a whole. If 121 | identifiable sections of that work are not derived from the Program, 122 | and can be reasonably considered independent and separate works in 123 | themselves, then this License, and its terms, do not apply to those 124 | sections when you distribute them as separate works. But when you 125 | distribute the same sections as part of a whole which is a work based 126 | on the Program, the distribution of the whole must be on the terms of 127 | this License, whose permissions for other licensees extend to the 128 | entire whole, and thus to each and every part regardless of who wrote it. 129 | 130 | Thus, it is not the intent of this section to claim rights or contest 131 | your rights to work written entirely by you; rather, the intent is to 132 | exercise the right to control the distribution of derivative or 133 | collective works based on the Program. 134 | 135 | In addition, mere aggregation of another work not based on the Program 136 | with the Program (or with a work based on the Program) on a volume of 137 | a storage or distribution medium does not bring the other work under 138 | the scope of this License. 139 | 140 | 3. You may copy and distribute the Program (or a work based on it, 141 | under Section 2) in object code or executable form under the terms of 142 | Sections 1 and 2 above provided that you also do one of the following: 143 | 144 | a) Accompany it with the complete corresponding machine-readable 145 | source code, which must be distributed under the terms of Sections 146 | 1 and 2 above on a medium customarily used for software interchange; or, 147 | 148 | b) Accompany it with a written offer, valid for at least three 149 | years, to give any third party, for a charge no more than your 150 | cost of physically performing source distribution, a complete 151 | machine-readable copy of the corresponding source code, to be 152 | distributed under the terms of Sections 1 and 2 above on a medium 153 | customarily used for software interchange; or, 154 | 155 | c) Accompany it with the information you received as to the offer 156 | to distribute corresponding source code. (This alternative is 157 | allowed only for noncommercial distribution and only if you 158 | received the program in object code or executable form with such 159 | an offer, in accord with Subsection b above.) 160 | 161 | The source code for a work means the preferred form of the work for 162 | making modifications to it. For an executable work, complete source 163 | code means all the source code for all modules it contains, plus any 164 | associated interface definition files, plus the scripts used to 165 | control compilation and installation of the executable. However, as a 166 | special exception, the source code distributed need not include 167 | anything that is normally distributed (in either source or binary 168 | form) with the major components (compiler, kernel, and so on) of the 169 | operating system on which the executable runs, unless that component 170 | itself accompanies the executable. 171 | 172 | If distribution of executable or object code is made by offering 173 | access to copy from a designated place, then offering equivalent 174 | access to copy the source code from the same place counts as 175 | distribution of the source code, even though third parties are not 176 | compelled to copy the source along with the object code. 177 | 178 | 4. You may not copy, modify, sublicense, or distribute the Program 179 | except as expressly provided under this License. Any attempt 180 | otherwise to copy, modify, sublicense or distribute the Program is 181 | void, and will automatically terminate your rights under this License. 182 | However, parties who have received copies, or rights, from you under 183 | this License will not have their licenses terminated so long as such 184 | parties remain in full compliance. 185 | 186 | 5. You are not required to accept this License, since you have not 187 | signed it. However, nothing else grants you permission to modify or 188 | distribute the Program or its derivative works. These actions are 189 | prohibited by law if you do not accept this License. Therefore, by 190 | modifying or distributing the Program (or any work based on the 191 | Program), you indicate your acceptance of this License to do so, and 192 | all its terms and conditions for copying, distributing or modifying 193 | the Program or works based on it. 194 | 195 | 6. Each time you redistribute the Program (or any work based on the 196 | Program), the recipient automatically receives a license from the 197 | original licensor to copy, distribute or modify the Program subject to 198 | these terms and conditions. You may not impose any further 199 | restrictions on the recipients' exercise of the rights granted herein. 200 | You are not responsible for enforcing compliance by third parties to 201 | this License. 202 | 203 | 7. If, as a consequence of a court judgment or allegation of patent 204 | infringement or for any other reason (not limited to patent issues), 205 | conditions are imposed on you (whether by court order, agreement or 206 | otherwise) that contradict the conditions of this License, they do not 207 | excuse you from the conditions of this License. If you cannot 208 | distribute so as to satisfy simultaneously your obligations under this 209 | License and any other pertinent obligations, then as a consequence you 210 | may not distribute the Program at all. For example, if a patent 211 | license would not permit royalty-free redistribution of the Program by 212 | all those who receive copies directly or indirectly through you, then 213 | the only way you could satisfy both it and this License would be to 214 | refrain entirely from distribution of the Program. 215 | 216 | If any portion of this section is held invalid or unenforceable under 217 | any particular circumstance, the balance of the section is intended to 218 | apply and the section as a whole is intended to apply in other 219 | circumstances. 220 | 221 | It is not the purpose of this section to induce you to infringe any 222 | patents or other property right claims or to contest validity of any 223 | such claims; this section has the sole purpose of protecting the 224 | integrity of the free software distribution system, which is 225 | implemented by public license practices. Many people have made 226 | generous contributions to the wide range of software distributed 227 | through that system in reliance on consistent application of that 228 | system; it is up to the author/donor to decide if he or she is willing 229 | to distribute software through any other system and a licensee cannot 230 | impose that choice. 231 | 232 | This section is intended to make thoroughly clear what is believed to 233 | be a consequence of the rest of this License. 234 | 235 | 8. If the distribution and/or use of the Program is restricted in 236 | certain countries either by patents or by copyrighted interfaces, the 237 | original copyright holder who places the Program under this License 238 | may add an explicit geographical distribution limitation excluding 239 | those countries, so that distribution is permitted only in or among 240 | countries not thus excluded. In such case, this License incorporates 241 | the limitation as if written in the body of this License. 242 | 243 | 9. The Free Software Foundation may publish revised and/or new versions 244 | of the General Public License from time to time. Such new versions will 245 | be similar in spirit to the present version, but may differ in detail to 246 | address new problems or concerns. 247 | 248 | Each version is given a distinguishing version number. If the Program 249 | specifies a version number of this License which applies to it and "any 250 | later version", you have the option of following the terms and conditions 251 | either of that version or of any later version published by the Free 252 | Software Foundation. If the Program does not specify a version number of 253 | this License, you may choose any version ever published by the Free Software 254 | Foundation. 255 | 256 | 10. If you wish to incorporate parts of the Program into other free 257 | programs whose distribution conditions are different, write to the author 258 | to ask for permission. For software which is copyrighted by the Free 259 | Software Foundation, write to the Free Software Foundation; we sometimes 260 | make exceptions for this. Our decision will be guided by the two goals 261 | of preserving the free status of all derivatives of our free software and 262 | of promoting the sharing and reuse of software generally. 263 | 264 | NO WARRANTY 265 | 266 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 267 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 268 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 269 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 270 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 271 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 272 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 273 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 274 | REPAIR OR CORRECTION. 275 | 276 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 277 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 278 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 279 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 280 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 281 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 282 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 283 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 284 | POSSIBILITY OF SUCH DAMAGES. 285 | 286 | END OF TERMS AND CONDITIONS 287 | 288 | How to Apply These Terms to Your New Programs 289 | 290 | If you develop a new program, and you want it to be of the greatest 291 | possible use to the public, the best way to achieve this is to make it 292 | free software which everyone can redistribute and change under these terms. 293 | 294 | To do so, attach the following notices to the program. It is safest 295 | to attach them to the start of each source file to most effectively 296 | convey the exclusion of warranty; and each file should have at least 297 | the "copyright" line and a pointer to where the full notice is found. 298 | 299 | {description} 300 | Copyright (C) {year} {fullname} 301 | 302 | This program is free software; you can redistribute it and/or modify 303 | it under the terms of the GNU General Public License as published by 304 | the Free Software Foundation; either version 2 of the License, or 305 | (at your option) any later version. 306 | 307 | This program is distributed in the hope that it will be useful, 308 | but WITHOUT ANY WARRANTY; without even the implied warranty of 309 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 310 | GNU General Public License for more details. 311 | 312 | You should have received a copy of the GNU General Public License along 313 | with this program; if not, write to the Free Software Foundation, Inc., 314 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 315 | 316 | Also add information on how to contact you by electronic and paper mail. 317 | 318 | If the program is interactive, make it output a short notice like this 319 | when it starts in an interactive mode: 320 | 321 | Gnomovision version 69, Copyright (C) year name of author 322 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 323 | This is free software, and you are welcome to redistribute it 324 | under certain conditions; type `show c' for details. 325 | 326 | The hypothetical commands `show w' and `show c' should show the appropriate 327 | parts of the General Public License. Of course, the commands you use may 328 | be called something other than `show w' and `show c'; they could even be 329 | mouse-clicks or menu items--whatever suits your program. 330 | 331 | You should also get your employer (if you work as a programmer) or your 332 | school, if any, to sign a "copyright disclaimer" for the program, if 333 | necessary. Here is a sample; alter the names: 334 | 335 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 336 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 337 | 338 | {signature of Ty Coon}, 1 April 1989 339 | Ty Coon, President of Vice 340 | 341 | This General Public License does not permit incorporating your program into 342 | proprietary programs. If your program is a subroutine library, you may 343 | consider it more useful to permit linking proprietary applications with the 344 | library. If this is what you want to do, use the GNU Lesser General 345 | Public License instead of this License. 346 | -------------------------------------------------------------------------------- /LICENSES/GLMNET_LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | {description} 294 | Copyright (C) {year} {fullname} 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | {signature of Ty Coon}, 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /LICENSES/SCIKIT_LEARN_LICENSE: -------------------------------------------------------------------------------- 1 | New BSD License 2 | 3 | Copyright (c) 2007–2016 The scikit-learn developers. 4 | All rights reserved. 5 | 6 | 7 | Redistribution and use in source and binary forms, with or without 8 | modification, are permitted provided that the following conditions are met: 9 | 10 | a. Redistributions of source code must retain the above copyright notice, 11 | this list of conditions and the following disclaimer. 12 | b. Redistributions in binary form must reproduce the above copyright 13 | notice, this list of conditions and the following disclaimer in the 14 | documentation and/or other materials provided with the distribution. 15 | c. Neither the name of the Scikit-learn Developers nor the names of 16 | its contributors may be used to endorse or promote products 17 | derived from this software without specific prior written 18 | permission. 19 | 20 | 21 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 22 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 23 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 24 | ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR 25 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 26 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 27 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 28 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 29 | LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 30 | OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH 31 | DAMAGE. 32 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.rst 2 | include LICENSE 3 | recursive-include LICENSES * 4 | include glmnet/*.pyf 5 | exclude glmnet/*.c 6 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Python GLMNET 2 | ============= 3 | 4 | Maintenance Warning 5 | ------------------- 6 | 7 | This package is not longer maintained. 8 | 9 | Description 10 | ----------- 11 | 12 | This is a Python wrapper for the fortran library used in the R package 13 | `glmnet `__. 14 | While the library includes linear, logistic, Cox, Poisson, and 15 | multiple-response Gaussian, only linear and logistic are implemented in 16 | this package. 17 | 18 | The API follows the conventions of 19 | `Scikit-Learn `__, so it is expected to 20 | work with tools from that ecosystem. 21 | 22 | Installation 23 | ------------ 24 | 25 | requirements 26 | ~~~~~~~~~~~~ 27 | 28 | ``python-glmnet`` requires Python version >= 3.6, ``scikit-learn``, ``numpy``, 29 | and ``scipy``. Installation from source or via ``pip`` requires a Fortran compiler. 30 | 31 | conda 32 | ~~~~~ 33 | 34 | .. code:: bash 35 | 36 | conda install -c conda-forge glmnet 37 | 38 | 39 | pip 40 | ~~~ 41 | 42 | .. code:: bash 43 | 44 | pip install glmnet 45 | 46 | 47 | source 48 | ~~~~~~ 49 | 50 | ``glmnet`` depends on numpy, scikit-learn and scipy. 51 | A working Fortran compiler is also required to build the package. 52 | For Mac users, ``brew install gcc`` will take care of this requirement. 53 | 54 | .. code:: bash 55 | 56 | git clone git@github.com:civisanalytics/python-glmnet.git 57 | cd python-glmnet 58 | python setup.py install 59 | 60 | Usage 61 | ----- 62 | 63 | General 64 | ~~~~~~~ 65 | 66 | By default, ``LogitNet`` and ``ElasticNet`` fit a series of models using 67 | the lasso penalty (α = 1) and up to 100 values for λ (determined by the 68 | algorithm). In addition, after computing the path of λ values, 69 | performance metrics for each value of λ are computed using 3-fold cross 70 | validation. The value of λ corresponding to the best performing model is 71 | saved as the ``lambda_max_`` attribute and the largest value of λ such 72 | that the model performance is within ``cut_point * standard_error`` of 73 | the best scoring model is saved as the ``lambda_best_`` attribute. 74 | 75 | The ``predict`` and ``predict_proba`` methods accept an optional 76 | parameter ``lamb`` which is used to select which model(s) will be used 77 | to make predictions. If ``lamb`` is omitted, ``lambda_best_`` is used. 78 | 79 | Both models will accept dense or sparse arrays. 80 | 81 | Regularized Logistic Regression 82 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 83 | 84 | .. code:: python 85 | 86 | from glmnet import LogitNet 87 | 88 | m = LogitNet() 89 | m = m.fit(x, y) 90 | 91 | Prediction is similar to Scikit-Learn: 92 | 93 | .. code:: python 94 | 95 | # predict labels 96 | p = m.predict(x) 97 | # or probability estimates 98 | p = m.predict_proba(x) 99 | 100 | Regularized Linear Regression 101 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 102 | 103 | .. code:: python 104 | 105 | from glmnet import ElasticNet 106 | 107 | m = ElasticNet() 108 | m = m.fit(x, y) 109 | 110 | Predict: 111 | 112 | .. code:: python 113 | 114 | p = m.predict(x) 115 | -------------------------------------------------------------------------------- /dev-requirements.txt: -------------------------------------------------------------------------------- 1 | # Pin exact dependency version numbers for local dev and CircleCI. 2 | numpy==1.19.0 3 | pytest==5.4.3 4 | scikit-learn==0.23.1 5 | scipy==1.5.0 6 | -------------------------------------------------------------------------------- /glmnet/__init__.py: -------------------------------------------------------------------------------- 1 | import pkg_resources 2 | 3 | from .logistic import LogitNet 4 | from .linear import ElasticNet 5 | 6 | __all__ = ['LogitNet', 'ElasticNet'] 7 | 8 | __version__ = pkg_resources.get_distribution("glmnet").version 9 | -------------------------------------------------------------------------------- /glmnet/_glmnet.pyf: -------------------------------------------------------------------------------- 1 | ! -*- f90 -*- 2 | ! Note: the context of this file is case sensitive. 3 | ! 4 | ! see http://docs.scipy.org/doc/numpy-dev/f2py/signature-file.html for what all 5 | ! the stuff below means. 6 | ! 7 | ! A note on precision: 8 | ! ------------------- 9 | ! glmnet is coded such that the size of `real` is an option for the compiler 10 | ! to save space, we are using 32-bit floats (real*8), to change to 64-bit, 11 | ! replace `real*4` with `real*8` below and add the flag `-fdefault-real-8` 12 | ! to the compiler args section in setup.py. In the 32-bit case, all arrays 13 | ! passed to glmnet must have a dtype of np.float32, otherwise numpy will assume 14 | ! a default np.float64 (this causes poor model performance). 15 | ! 16 | ! A note on concurrency: 17 | ! ---------------------- 18 | ! The glmnet functions have been marked as threadsafe in the signatures below, 19 | ! meaning Python will release the GIL while they are running. For performing 20 | ! cross validation, the models can be fit concurrently using either the threading 21 | ! or multiprocessing libraries. `ThreadPoolExecutor` in the concurrent.futures 22 | ! library appears to work well. 23 | ! 24 | python module _glmnet ! in 25 | interface ! in :_glmnet 26 | subroutine lognet(parm,no,ni,nc,x,y,g,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in _glmnet.f 27 | real*8 :: parm 28 | integer intent(hide), check(shape(x,0)==no), depend(x) :: no=shape(x,0) 29 | integer intent(hide), check(shape(x,1)==ni), depend(x) :: ni=shape(x,1) 30 | integer check(shape(y,1)==max(2,nc)), depend(y) :: nc 31 | real*8 dimension(no,ni) :: x 32 | real*8 dimension(no,max(2,nc)), depend(no) :: y 33 | real*8 dimension(no,shape(y,1)), depend(no) :: g 34 | integer dimension(*) :: jd 35 | real*8 dimension(ni), depend(ni) :: vp 36 | real*8 dimension(2,ni), depend(ni) :: cl 37 | integer optional, depend(x) :: ne=min(shape(x,1), nx) 38 | integer :: nx 39 | integer optional, check((flmin < 1.0 || len(ulam)==nlam)), depend(flmin,ulam) :: nlam=len(ulam) 40 | real*8 :: flmin 41 | real*8 dimension(nlam) :: ulam 42 | real*8 :: thr 43 | integer optional :: isd=1 44 | integer optional :: intr=1 45 | integer optional :: maxit=100000 46 | integer optional :: kopt=0 47 | 48 | ! output 49 | integer intent(out) :: lmu 50 | real*8 intent(out), dimension(nc,nlam), depend(nc,nlam) :: a0 51 | real*8 intent(out), dimension(nx,nc,nlam), depend(nx,nc,nlam) :: ca 52 | integer intent(out), dimension(nx), depend(nx) :: ia 53 | integer intent(out), dimension(nlam), depend(nlam) :: nin 54 | real*8 intent(out), dimension(nlam), depend(nlam) :: dev0 55 | real*8 intent(out), dimension(nlam), depend(nlam) :: dev 56 | real*8 intent(out), dimension(nlam), depend(nlam) :: alm 57 | integer intent(out) :: nlp 58 | integer intent(out) :: jerr 59 | 60 | threadsafe 61 | end subroutine lognet 62 | 63 | subroutine splognet(parm,no,ni,nc,x,ix,jx,y,g,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in _glmnet.f 64 | real*8 :: parm 65 | integer :: no 66 | integer check(len(vp)>=ni), depend(vp) :: ni 67 | integer check(shape(y,1)==max(2, nc)), depend(y) :: nc 68 | real*8 dimension(*) :: x 69 | integer dimension(*) :: ix 70 | integer dimension(*) :: jx 71 | real*8 dimension(no,max(2,nc)), depend(no) :: y 72 | real*8 dimension(no,shape(y,1)),depend(no) :: g 73 | integer dimension(*) :: jd 74 | real*8 dimension(ni) :: vp 75 | real*8 dimension(2,ni),depend(ni) :: cl 76 | integer :: ne 77 | integer :: nx 78 | integer optional, check((flmin < 1.0 || len(ulam)==nlam)), depend(flmin, ulam) :: nlam=len(ulam) 79 | real*8 :: flmin 80 | real*8 dimension(nlam) :: ulam 81 | real*8 :: thr 82 | integer optional :: isd=1 83 | integer optional :: intr=1 84 | integer optional :: maxit=100000 85 | integer optional :: kopt=0 86 | 87 | ! output 88 | integer intent(out) :: lmu 89 | real*8 intent(out), dimension(nc,nlam), depend(nc,nlam) :: a0 90 | real*8 intent(out), dimension(nx,nc,nlam), depend(nx,nc,nlam) :: ca 91 | integer intent(out), dimension(nx), depend(nx) :: ia 92 | integer intent(out), dimension(nlam), depend(nlam) :: nin 93 | real*8 intent(out), dimension(nlam), depend(nlam) :: dev0 94 | real*8 intent(out), dimension(nlam), depend(nlam) :: dev 95 | real*8 intent(out), dimension(nlam), depend(nlam) :: alm 96 | integer intent(out) :: nlp 97 | integer intent(out) :: jerr 98 | 99 | threadsafe 100 | end subroutine splognet 101 | 102 | subroutine lsolns(ni,nx,nc,lmu,ca,ia,nin,b) ! in _glmnet.f 103 | integer :: ni 104 | integer intent(hide), depend(ca) :: nx=shape(ca,0) 105 | integer intent(hide), depend(ca) :: nc=shape(ca,1) 106 | integer intent(hide), depend(ca) :: lmu=shape(ca,2) 107 | real*8 dimension(nx,nc,lmu) :: ca 108 | integer dimension(nx), depend(nx) :: ia 109 | integer dimension(lmu), depend(lmu) :: nin 110 | real*8 intent(out), dimension(ni,nc,lmu), depend(nc,lmu) :: b 111 | 112 | threadsafe 113 | end subroutine lsolns 114 | 115 | subroutine elnet(ka,parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in _glmnet.f 116 | integer :: ka 117 | real*8 :: parm 118 | integer intent(hide), check(shape(x,0)==no), depend(x) :: no=shape(x,0) 119 | integer intent(hide), check(shape(x,1)==ni), depend(x) :: ni=shape(x,1) 120 | real*8 dimension(no,ni) :: x 121 | real*8 dimension(no), depend(no) :: y 122 | real*8 dimension(no), depend(no) :: w 123 | integer dimension(*) :: jd 124 | real*8 dimension(ni), depend(ni) :: vp 125 | real*8 dimension(2,ni), depend(ni) :: cl 126 | integer optional, depend(x) :: ne=min(shape(x, 1), nx) 127 | integer :: nx 128 | integer, optional, check((flmin < 1.0 || len(ulam)==nlam)), depend(flmin,ulam) :: nlam=len(ulam) 129 | real*8 :: flmin 130 | real*8 dimension(nlam) :: ulam 131 | real*8 :: thr 132 | integer optional :: isd=1 133 | integer optional :: intr=1 134 | integer optional :: maxit=100000 135 | 136 | ! output 137 | integer intent(out) :: lmu 138 | real*8 intent(out), dimension(nlam), depend(nlam) :: a0 139 | real*8 intent(out), dimension(nx,nlam), depend(nlam) :: ca 140 | integer intent(out), dimension(nx), depend(nx) :: ia 141 | integer intent(out), dimension(nlam), depend(nlam) :: nin 142 | real*8 intent(out), dimension(nlam), depend(nlam) :: rsq 143 | real*8 intent(out), dimension(nlam), depend(nlam) :: alm 144 | integer intent(out) :: nlp 145 | integer intent(out) :: jerr 146 | 147 | threadsafe 148 | end subroutine elnet 149 | 150 | 151 | subroutine spelnet(ka,parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :_glmnet:_glmnet.f 152 | integer :: ka 153 | real*8 :: parm 154 | integer check(len(y)>=no), depend(y) :: no 155 | integer check(len(vp)>=ni), depend(vp) :: ni 156 | real*8 dimension(*) :: x 157 | integer dimension(*) :: ix 158 | integer dimension(*) :: jx 159 | real*8 dimension(no) :: y 160 | real*8 dimension(no),depend(no) :: w 161 | integer dimension(*) :: jd 162 | real*8 dimension(ni) :: vp 163 | real*8 dimension(2,ni),depend(ni) :: cl 164 | integer :: ne 165 | integer :: nx 166 | integer optional, check((flmin < 1.0 || len(ulam)==nlam)),depend(flmin,ulam) :: nlam=len(ulam) 167 | real*8 :: flmin 168 | real*8 dimension(nlam) :: ulam 169 | real*8 :: thr 170 | integer optional :: isd=1 171 | integer optional :: intr=1 172 | integer optional :: maxit=100000 173 | 174 | ! output 175 | integer intent(out) :: lmu 176 | real*8 intent(out), dimension(nlam),depend(nlam) :: a0 177 | real*8 intent(out), dimension(nx,nlam),depend(nlam) :: ca 178 | integer intent(out), dimension(nx),depend(nx) :: ia 179 | integer intent(out), dimension(nlam),depend(nlam) :: nin 180 | real*8 intent(out), dimension(nlam),depend(nlam) :: rsq 181 | real*8 intent(out), dimension(nlam),depend(nlam) :: alm 182 | integer intent(out) :: nlp 183 | integer intent(out) :: jerr 184 | 185 | threadsafe 186 | end subroutine spelnet 187 | 188 | subroutine solns(ni,nx,lmu,ca,ia,nin,b) ! in _glmnet.f 189 | integer :: ni 190 | integer intent(hide), depend(ca) :: nx=shape(ca,0) 191 | integer intent(hide), depend(ca) :: lmu=shape(ca,1) 192 | real*8 dimension(nx,lmu) :: ca 193 | integer dimension(nx), depend(nx) :: ia 194 | integer dimension(lmu), depend(lmu) :: nin 195 | real*8 intent(out), dimension(ni,lmu), depend(lmu) :: b 196 | 197 | threadsafe 198 | end subroutine solns 199 | 200 | end interface 201 | end python module _glmnet 202 | 203 | -------------------------------------------------------------------------------- /glmnet/doc.py: -------------------------------------------------------------------------------- 1 | """For convenience, documentation for the wrapped functions from glmnet: 2 | 3 | Notes: 4 | In general, numpy will handle sending the correct array representation to 5 | the wrapped functions. If the array is not already 'F Contiguous', a copy 6 | will be made and passed to the glmnet. The glmnet documentation suggests that 7 | x and y may be overwritten during the fitting process, so these arrays should 8 | be copied anyway (with order='F'). 9 | 10 | -------------------------------------------------------------------------------- 11 | lognet: binomial/multinomial logistic elastic net 12 | 13 | call: 14 | lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr = lognet(parm,nc,x,y,g,jd,vp,cl,nx,flmin,ulam,thr,[ne,nlam,isd,intr,maxit,kopt]) 15 | 16 | Parameters 17 | ---------- 18 | parm : float 19 | The elasticnet mixing parameter, 0 <= param <= 1. The penalty is defined 20 | as (1 - param) / 2 ||B|| + param |B|. param=1 is the lasso penalty and 21 | param=0 is the ridge penalty. 22 | 23 | nc : int 24 | The number of distinct classes in y. Note, for the binomial case this 25 | value should be 1. 26 | 27 | x : rank-2 array('float') with bounds (no,ni) 28 | Input matrix with shape [n_samples, n_features] 29 | 30 | y : rank-2 array('float') with bounds (no,max(2,nc)) 31 | Response variable with shape [n_samples, n_classes]. Note for the binomial 32 | case, this must be [n_samples, 2]. 33 | 34 | g : rank-2 array('float') with bounds (no,shape(y,1)) 35 | Offsets of shape [n_samples, n_classes]. Unless you know what why just 36 | pass np.zeros((n_samples, n_classes)). 37 | 38 | jd : rank-1 array('int') with bounds (*) 39 | Feature deletion flag, equivalent to applying an infinite penalty. To 40 | include all features in the model, jd=0. To exclude the ith and jth feature: 41 | jd=[1, i, j] 42 | Note fortran uses 1-based indexing so the 0th feature is 1. If you are 43 | excluding features, the first element of jd must be a 1 to signal glmnet. 44 | 45 | vp : rank-1 array('float') with bounds (ni) 46 | Relative penalty to apply to each feature, use np.ones(n_features) to 47 | uniformily apply the elasticnet penalty. 48 | 49 | cl : rank-2 array('float') with bounds (2,ni) 50 | Interval constraints for the model coefficients. vp[0, :] are lower bounds and 51 | vp[1, :] are the upper bounds. 52 | 53 | nx : int 54 | The maximum number of variables allowed to enter all models along the path 55 | of param. If ne (see below) is also supplied, nx > ne. This should typically 56 | be set to n_features. 57 | 58 | flmin : float 59 | Smallest value for lambda as a fraction of lambda_max (the lambda for which 60 | all coefficients are zero). If n_samples > n_features, this value should be 61 | 1e-4, for n_features > n_samples use 1e-2. Note, if the lambda path is explicitly 62 | provided (see ulam below), flmin will be ignored, but it must be > 1. 63 | 64 | ulam : rank-1 array('float') with bounds (nlam) 65 | User supplied lambda sequence. Note glmnet typically computes its own 66 | sequence of lambda values (when ulam = None). If a specific sequence of 67 | lambdas is desired, they should be passed in decreasing order. 68 | 69 | thr : float 70 | Convergence threshold for coordinate descent, a good value is 1e-7. 71 | 72 | ne : int, Default: min(shape(x, 1), nx) 73 | The maximum number of variables allowed to enter the largest model (stopping 74 | criterion), if provided, nx > ne. 75 | 76 | nlam : int, Default: len(ulam) 77 | Maximum number of lambda values. If ulam is not supplied, nlam must be 78 | provided, 100 is a good value. 79 | 80 | isd : int, Default: 1/True 81 | Standardize predictor variables prior to fitting model. Note, output coefficients 82 | will always reference the original variable locations and scales. 83 | 84 | intr : int, Default: 1/True 85 | Include an intercept term in the model. 86 | 87 | maxit : int, Default: 100000 88 | Maximum number of passes ofer the data for all lambda values. 89 | 90 | kopt : int, Default: 0 (Newton-Raphson) 91 | Optimization algorithm: 92 | 0: Newton-Raphson (exact hessian) 93 | 1: Modified Newton-Raphson (upper-bounded hessian, sometimes faster) 94 | 2: Non-zero coefficients same for each class (exact behavior is not documented) 95 | 96 | Returns 97 | ------- 98 | lmu : int 99 | Actual number of lambda values used, may not be equal to nlam. 100 | 101 | a0 : rank-2 array('float') with bounds (nc,nlam) 102 | Intercept values for each class at each value of lambda. 103 | 104 | ca : rank-3 array('float') with bounds (nx,nc,nlam) 105 | Compressed coefficients for each class at each value of lambda. Suggest 106 | using lsolns to convert to a usable layout [n_features, n_classes, n_lambda]. 107 | Note in the binomial case, there will be one set of coefficients instead 108 | of two. 109 | 110 | ia : rank-1 array('int') with bounds (nx) 111 | Pointers to compressed coefficients, used by lsolns to decompress the ca 112 | array. 113 | 114 | nin : rank-1 array('int') with bounds (nlam) 115 | Number of compressed coefficients for each value of lambda, used by lsolns 116 | to decompress the ca array. 117 | 118 | dev0 : rank-1 array('float') with bounds (nlam) 119 | Null deviance (intercept only model) for each value of lambda. 120 | 121 | dev : rank-1 array('float') with bounds (nlam) 122 | Fraction of deviance explained for each value of lambda. 123 | 124 | alm : rank-1 array('float') with bounds (nlam) 125 | Actual lambda values corresponding to each solution. 126 | 127 | nlp : int 128 | Number of passes over the data for all lambda values. 129 | 130 | jerr : int 131 | Error flag: 132 | = 0: no error 133 | > 0: fatal error - no output returned 134 | < 7777: memory allocation error 135 | = 7777: all used predictors have zero variance 136 | = 8000 + k: null probability < 1e-5 for class k 137 | = 9000 + k: null probability for class k > 1 - 1e-5 138 | = 10000: maxval(vp) <= 0 139 | = 90000: bounds adjustment non convergence 140 | < 0: non fatal error - partial output 141 | 142 | -------------------------------------------------------------------------------- 143 | lsolns: uncompress coefficient vectors for all solutions 144 | 145 | call: 146 | b = lsolns(ni, ca, ia, nin) 147 | 148 | Parameters 149 | ---------- 150 | ni : int 151 | Number of input features. 152 | 153 | ca : rank-3 array('float') with bounds (nx,nc,lmu) 154 | Compressed coefficient array, as returned by lognet. 155 | 156 | ia : rank-1 array('int') with bounds (nx) 157 | Pointers to compressed coefficients, as returned by lognet. 158 | 159 | nin : rank-1 array('int') with bounds (lmu) 160 | Number of compressed coefficients for each solution, as returned by 161 | lognet. 162 | 163 | Returns 164 | ------- 165 | b : rank-3 array('float') with bounds (ni,nc,lmu) 166 | Dense coefficient array referencing original variables. 167 | 168 | -------------------------------------------------------------------------------- 169 | elnet: regression with elastic net penalty and squared-error loss 170 | 171 | call: 172 | lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr = elnet(ka,parm,x,y,w,jd,vp,cl,nx,flmin,ulam,thr,[ne,nlam,isd,intr,maxit]) 173 | 174 | Parameters 175 | ---------- 176 | ka : int 177 | Algorithm flag, 1 for covariance updating algorithm, 2 for naive algorithm. 178 | When n_features (ni) < 500, the covariance updating algorithm should be used 179 | as it saves all inner-products ever computed, making it much faster than the 180 | naive algorithm which needs to loop through each sample every time an inner- 181 | product is computed. In the case of n_features >> n_samples, the reverse 182 | is true w.r.t. efficiency. 183 | 184 | parm : float 185 | The elasticnet mixing parameter, 0 <= param <= 1. The penalty is defined 186 | as (1 - param) / 2 ||B|| + param |B|. param=1 is the lasso penalty and 187 | param=0 is the ridge penalty. 188 | 189 | x : rank-2 array('float') with bounds (no,ni) 190 | Input matrix with shape [n_samples, n_features] 191 | 192 | y : rank-1 array('f') with bounds (no) 193 | Response variable 194 | 195 | w : rank-1 array('f') with bounds (no) 196 | Observation weights 197 | 198 | jd : rank-1 array('int') with bounds (*) 199 | Feature deletion flag, equivalent to applying an infinite penalty. To 200 | include all features in the model, jd=0. To exclude the ith and jth feature: 201 | jd=[1, i, j] 202 | Note fortran uses 1-based indexing so the 0th feature is 1. If you are 203 | excluding features, the first element of jd must be a 1 to signal glmnet. 204 | 205 | vp : rank-1 array('float') with bounds (ni) 206 | Relative penalty to apply to each feature, use np.ones(n_features) to 207 | uniformily apply the elasticnet penalty. 208 | 209 | cl : rank-2 array('float') with bounds (2,ni) 210 | Interval constraints for the model coefficients. vp[0, :] are lower bounds and 211 | vp[1, :] are the upper bounds. 212 | 213 | nx : int 214 | The maximum number of variables allowed to enter all models along the path 215 | of param. If ne (see below) is also supplied, nx > ne. This should typically 216 | be set to n_features. 217 | 218 | flmin : float 219 | Smallest value for lambda as a fraction of lambda_max (the lambda for which 220 | all coefficients are zero). If n_samples > n_features, this value should be 221 | 1e-4, for n_features > n_samples use 1e-2. Note, if the lambda path is explicitly 222 | provided (see ulam below), flmin will be ignored, but it must be > 1. 223 | 224 | ulam : rank-1 array('float') with bounds (nlam) 225 | User supplied lambda sequence. Note glmnet typically computes its own 226 | sequence of lambda values (when ulam = None). If a specific sequence of 227 | lambdas is desired, they should be passed in decreasing order. 228 | 229 | thr : float 230 | Convergence threshold for coordinate descent, a good value is 1e-7. 231 | 232 | ne : int, Default: min(shape(x, 1), nx) 233 | The maximum number of variables allowed to enter the largest model (stopping 234 | criterion), if provided, nx > ne. 235 | 236 | nlam : int, Default: len(ulam) 237 | Maximum number of lambda values. If ulam is not supplied, nlam must be 238 | provided, 100 is a good value. 239 | 240 | isd : int, Default: 1/True 241 | Standardize predictor variables prior to fitting model. Note, output coefficients 242 | will always reference the original variable locations and scales. 243 | 244 | intr : int, Default: 1/True 245 | Include an intercept term in the model. 246 | 247 | maxit : int, Default: 100000 248 | Maximum number of passes ofer the data for all lambda values. 249 | 250 | Returns 251 | ------- 252 | lmu : int 253 | Actual number of lambda values used, may not be equal to nlam. 254 | 255 | a0 : rank-2 array('float') with bounds (nc,nlam) 256 | Intercept values for each class at each value of lambda. 257 | 258 | ca : rank-2 array('float') with bounds (nx,nlam) 259 | Compressed coefficients at each value of lambda. Suggest using solns to 260 | convert to a usable layout. 261 | 262 | ia : rank-1 array('int') with bounds (nx) 263 | Pointers to compressed coefficients, used by solns to decompress the ca 264 | array. 265 | 266 | nin : rank-1 array('int') with bounds (nlam) 267 | Number of compressed coefficients for each value of lambda, used by solns 268 | to decompress the ca array. 269 | 270 | rsq : rank-1 array('float') with bounds (nlam) 271 | R^2 values for each lambda. 272 | 273 | alm : rank-1 array('float') with bounds (nlam) 274 | Actual lambda values corresponding to each solution. 275 | 276 | nlp : int 277 | Number of passes over the data for all lambda values. 278 | 279 | jerr : int 280 | Error flag: 281 | = 0: no error 282 | > 0: fatal error - no output returned 283 | < 0: non fatal error - partial output 284 | 285 | -------------------------------------------------------------------------------- 286 | solns: uncompress coefficient vectors for all solutions 287 | 288 | call: 289 | b = solns(ni,ca,ia,nin) 290 | 291 | Parameters 292 | ---------- 293 | ni : int 294 | Number of input features. 295 | 296 | ca : rank-2 array('float') with bounds (nx,lmu) 297 | Compressed coefficient array, as returned by elnet. 298 | 299 | ia : input rank-1 array('int') with bounds (nx) 300 | Pointers to compressed coefficients, as returned by elnet. 301 | 302 | nin : input rank-1 array('int') with bounds (lmu) 303 | Number of compressed coefficients for each solution, as returned by elnet. 304 | 305 | Returns 306 | ------- 307 | b : rank-2 array('float') with bounds (ni,lmu) 308 | Dense coefficient array referencing original variables. 309 | """ 310 | -------------------------------------------------------------------------------- /glmnet/errors.py: -------------------------------------------------------------------------------- 1 | import warnings 2 | 3 | 4 | def _check_error_flag(jerr): 5 | """Check the glmnet solver error flag and issue warnings or raise 6 | exceptions as appropriate. 7 | 8 | The codes break down roughly as follows: 9 | 10 | jerr == 0: everything is fine 11 | jerr > 0: fatal errors such as memory allocation problems 12 | jerr < 0: non-fatal errors such as convergence warnings 13 | """ 14 | if jerr == 0: 15 | return 16 | 17 | if jerr > 0: 18 | _fatal_errors(jerr) 19 | 20 | if jerr < 0: 21 | _convergence_errors(jerr) 22 | 23 | 24 | def _fatal_errors(jerr): 25 | if jerr == 7777: 26 | raise ValueError("All predictors have zero variance " 27 | "(glmnet error no. 7777).") 28 | 29 | if jerr == 10000: 30 | raise ValueError("At least one value of relative_penalties must be " 31 | "positive (glmnet error no. 10000).") 32 | 33 | if jerr == 90000: 34 | raise ValueError("Solver did not converge (glmnet error no. 90000).") 35 | 36 | if jerr < 7777: 37 | raise RuntimeError("Memory allocation error (glmnet error no. {})." 38 | .format(jerr)) 39 | 40 | if jerr > 8000 and jerr < 9000: 41 | k = jerr - 8000 42 | raise ValueError("Probability for class {} close to 0.".format(k)) 43 | 44 | if jerr > 9000: 45 | k = jerr - 9000 46 | raise ValueError("Probability for class {} close to 1.".format(k)) 47 | 48 | else: 49 | raise RuntimeError("Fatal glmnet error no. {}.".format(jerr)) 50 | 51 | 52 | def _convergence_errors(jerr): 53 | if jerr < -20000: 54 | k = abs(20000 + jerr) 55 | warnings.warn("Predicted probability close to 0 or 1 for " 56 | "lambda no. {}.".format(k), RuntimeWarning) 57 | 58 | if jerr > -20000 and jerr < -10000: 59 | # This indicates the number of non-zero coefficients in a model 60 | # exceeded a user-specified bound. We don't expose this parameter to 61 | # the user, so there is not much sense in exposing the error either. 62 | warnings.warn("Non-fatal glmnet error no. {}.".format(jerr), 63 | RuntimeWarning) 64 | 65 | if jerr > -10000: 66 | warnings.warn("Model did not converge for smaller values of lambda, " 67 | "returning solution for the largest {} values." 68 | .format(-1 * (jerr - 1)), RuntimeWarning) 69 | -------------------------------------------------------------------------------- /glmnet/linear.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from scipy.sparse import issparse, csc_matrix 4 | from scipy import stats 5 | 6 | from sklearn.base import BaseEstimator 7 | from sklearn.metrics import r2_score 8 | from sklearn.model_selection import KFold, GroupKFold 9 | from sklearn.utils import check_array, check_X_y 10 | 11 | from .errors import _check_error_flag 12 | from _glmnet import elnet, spelnet, solns 13 | from glmnet.util import (_fix_lambda_path, 14 | _check_user_lambda, 15 | _interpolate_model, 16 | _score_lambda_path) 17 | 18 | 19 | class ElasticNet(BaseEstimator): 20 | """Elastic Net with squared error loss. 21 | 22 | This is a wrapper for the glmnet function elnet. 23 | 24 | Parameters 25 | ---------- 26 | alpha : float, default 1 27 | The alpha parameter, 0 <= alpha <= 1, 0 for ridge, 1 for lasso 28 | 29 | n_lambda : int, default 100 30 | Maximum number of lambda values to compute 31 | 32 | min_lambda_ratio : float, default 1e-4 33 | In combination with n_lambda, the ratio of the smallest and largest 34 | values of lambda computed. 35 | 36 | lambda_path : array, default None 37 | In place of supplying n_lambda, provide an array of specific values 38 | to compute. The specified values must be in decreasing order. When 39 | None, the path of lambda values will be determined automatically. A 40 | maximum of `n_lambda` values will be computed. 41 | 42 | standardize : bool, default True 43 | Standardize input features prior to fitting. The final coefficients 44 | will be on the scale of the original data regardless of the value 45 | of standardize. 46 | 47 | lower_limits : array, (shape n_features,) default -infinity 48 | Array of lower limits for each coefficient, must be non-positive. 49 | Can be a single value (which is then replicated), else an array 50 | corresponding to the number of features. 51 | 52 | upper_limits : array, (shape n_features,) default +infinity 53 | Array of upper limits for each coefficient, must be positive. 54 | See lower_limits. 55 | 56 | fit_intercept : bool, default True 57 | Include an intercept term in the model. 58 | 59 | cut_point : float, default 1 60 | The cut point to use for selecting lambda_best. 61 | arg_max lambda cv_score(lambda) >= cv_score(lambda_max) - cut_point * standard_error(lambda_max) 62 | 63 | n_splits : int, default 3 64 | Number of cross validation folds for computing performance metrics 65 | (including determination of `lambda_best_` and `lambda_max_`). If 66 | non-zero, must be at least 3. 67 | 68 | scoring : string, callable, or None, default None 69 | Scoring method for model selection during cross validation. When None, 70 | defaults to r^2 score. Valid options are `r2`, `mean_squared_error`, 71 | `mean_absolute_error`, `median_absolute_error`. Alternatively, supply 72 | a function or callable object with the following signature 73 | ``scorer(estimator, X, y)``. Note, the scoring function affects the 74 | selection of `lambda_best_` and `lambda_max_`, fitting the same data 75 | with different scoring methods will result in the selection of 76 | different models. 77 | 78 | n_jobs : int, default 1 79 | Maximum number of threads for computing cross validation metrics. 80 | 81 | tol : float, default 1e-7 82 | Convergence tolerance. 83 | 84 | max_iter : int, default 100000 85 | Maximum passes over the data for all values of lambda. 86 | 87 | random_state : number, default None 88 | Seed for the random number generator. The glmnet solver is not 89 | deterministic, this seed is used for determining the cv folds. 90 | 91 | max_features : int 92 | Optional maximum number of features with nonzero coefficients after 93 | regularization. If not set, defaults to X.shape[1] during fit 94 | Note, this will be ignored if the user specifies lambda_path. 95 | 96 | verbose : bool, default False 97 | When True some warnings and log messages are suppressed. 98 | 99 | Attributes 100 | ---------- 101 | n_lambda_ : int 102 | The number of lambda values found by glmnet. Note, this may be less 103 | than the number specified via n_lambda. 104 | 105 | lambda_path_ : array, shape (n_lambda_,) 106 | The values of lambda found by glmnet, in decreasing order. 107 | 108 | coef_path_ : array, shape (n_features, n_lambda_) 109 | The set of coefficients for each value of lambda in lambda_path_. 110 | 111 | coef_ : array, shape (n_features,) 112 | The coefficients corresponding to lambda_best_. 113 | 114 | intercept_ : float 115 | The intercept corresponding to lambda_best_. 116 | 117 | intercept_path_ : array, shape (n_lambda_,) 118 | The intercept for each value of lambda in lambda_path_. 119 | 120 | cv_mean_score_ : array, shape (n_lambda_,) 121 | The mean cv score for each value of lambda. This will be set by fit_cv. 122 | 123 | cv_standard_error_ : array, shape (n_lambda_,) 124 | The standard error of the mean cv score for each value of lambda, this 125 | will be set by fit_cv. 126 | 127 | lambda_max_ : float 128 | The value of lambda that gives the best performance in cross 129 | validation. 130 | 131 | lambda_best_ : float 132 | The largest value of lambda which is greater than lambda_max_ and 133 | performs within cut_point * standard error of lambda_max_. 134 | """ 135 | 136 | def __init__(self, alpha=1, n_lambda=100, min_lambda_ratio=1e-4, 137 | lambda_path=None, standardize=True, fit_intercept=True, 138 | lower_limits=-np.inf, upper_limits=np.inf, 139 | cut_point=1.0, n_splits=3, scoring=None, n_jobs=1, tol=1e-7, 140 | max_iter=100000, random_state=None, max_features=None, verbose=False): 141 | 142 | self.alpha = alpha 143 | self.n_lambda = n_lambda 144 | self.min_lambda_ratio = min_lambda_ratio 145 | self.lambda_path = lambda_path 146 | self.standardize = standardize 147 | self.lower_limits = lower_limits 148 | self.upper_limits = upper_limits 149 | self.fit_intercept = fit_intercept 150 | self.cut_point = cut_point 151 | self.n_splits = n_splits 152 | self.scoring = scoring 153 | self.n_jobs = n_jobs 154 | self.tol = tol 155 | self.max_iter = max_iter 156 | self.random_state = random_state 157 | self.max_features = max_features 158 | self.verbose = verbose 159 | 160 | def fit(self, X, y, sample_weight=None, relative_penalties=None, groups=None): 161 | """Fit the model to training data. If n_splits > 1 also run n-fold cross 162 | validation on all values in lambda_path. 163 | 164 | The model will be fit n+1 times. On the first pass, the lambda_path 165 | will be determined, on the remaining passes, the model performance for 166 | each value of lambda. After cross validation, the attribute 167 | `cv_mean_score_` will contain the mean score over all folds for each 168 | value of lambda, and `cv_standard_error_` will contain the standard 169 | error of `cv_mean_score_` for each value of lambda. The value of lambda 170 | which achieves the best performance in cross validation will be saved 171 | to `lambda_max_` additionally, the largest value of lambda s.t.: 172 | cv_score(l) >= cv_score(lambda_max_) -\ 173 | cut_point * standard_error(lambda_max_) 174 | will be saved to `lambda_best_`. 175 | 176 | Parameters 177 | ---------- 178 | X : array, shape (n_samples, n_features) 179 | Input features 180 | 181 | y : array, shape (n_samples,) 182 | Target values 183 | 184 | sample_weight : array, shape (n_samples,) 185 | Optional weight vector for observations 186 | 187 | relative_penalties: array, shape (n_features,) 188 | Optional relative weight vector for penalty. 189 | 0 entries remove penalty. 190 | 191 | groups: array, shape (n_samples,) 192 | Group labels for the samples used while splitting the dataset into train/test set. 193 | If the groups are specified, the groups will be passed to sklearn.model_selection.GroupKFold. 194 | If None, then data will be split randomly for K-fold cross-validation via sklearn.model_selection.KFold. 195 | 196 | Returns 197 | ------- 198 | self : object 199 | Returns self. 200 | """ 201 | 202 | X, y = check_X_y(X, y, accept_sparse='csr', ensure_min_samples=2) 203 | if sample_weight is None: 204 | sample_weight = np.ones(X.shape[0]) 205 | else: 206 | sample_weight = np.asarray(sample_weight) 207 | 208 | if not np.isscalar(self.lower_limits): 209 | self.lower_limits = np.asarray(self.lower_limits) 210 | if len(self.lower_limits) != X.shape[1]: 211 | raise ValueError("lower_limits must equal number of features") 212 | 213 | if not np.isscalar(self.upper_limits): 214 | self.upper_limits = np.asarray(self.upper_limits) 215 | if len(self.upper_limits) != X.shape[1]: 216 | raise ValueError("upper_limits must equal number of features") 217 | 218 | if any(self.lower_limits > 0) if isinstance(self.lower_limits, np.ndarray) else self.lower_limits > 0: 219 | raise ValueError("lower_limits must be non-positive") 220 | 221 | if any(self.upper_limits < 0) if isinstance(self.upper_limits, np.ndarray) else self.upper_limits < 0: 222 | raise ValueError("upper_limits must be positive") 223 | 224 | if self.alpha > 1 or self.alpha < 0: 225 | raise ValueError("alpha must be between 0 and 1") 226 | 227 | if self.n_splits > 0 and self.n_splits < 3: 228 | raise ValueError("n_splits must be at least 3") 229 | 230 | self._fit(X, y, sample_weight, relative_penalties) 231 | 232 | if self.n_splits >= 3: 233 | if groups is None: 234 | self._cv = KFold(n_splits=self.n_splits, shuffle=True, random_state=self.random_state) 235 | else: 236 | self._cv = GroupKFold(n_splits=self.n_splits) 237 | 238 | cv_scores = _score_lambda_path(self, X, y, groups, 239 | sample_weight, 240 | relative_penalties, 241 | self.scoring, 242 | n_jobs=self.n_jobs, 243 | verbose=self.verbose) 244 | 245 | self.cv_mean_score_ = np.atleast_1d(np.mean(cv_scores, axis=0)) 246 | self.cv_standard_error_ = np.atleast_1d(stats.sem(cv_scores)) 247 | 248 | self.lambda_max_inx_ = np.argmax(self.cv_mean_score_) 249 | self.lambda_max_ = self.lambda_path_[self.lambda_max_inx_] 250 | 251 | target_score = self.cv_mean_score_[self.lambda_max_inx_] -\ 252 | self.cut_point * self.cv_standard_error_[self.lambda_max_inx_] 253 | 254 | self.lambda_best_inx_ = np.argwhere(self.cv_mean_score_ >= target_score)[0] 255 | self.lambda_best_ = self.lambda_path_[self.lambda_best_inx_] 256 | 257 | self.coef_ = self.coef_path_[..., self.lambda_best_inx_] 258 | self.coef_ = self.coef_.squeeze(axis=self.coef_.ndim-1) 259 | self.intercept_ = self.intercept_path_[..., self.lambda_best_inx_].squeeze() 260 | if self.intercept_.shape == (): # convert 0d array to scalar 261 | self.intercept_ = float(self.intercept_) 262 | 263 | return self 264 | 265 | def _fit(self, X, y, sample_weight, relative_penalties): 266 | 267 | if self.lambda_path is not None: 268 | n_lambda = len(self.lambda_path) 269 | min_lambda_ratio = 1.0 270 | else: 271 | n_lambda = self.n_lambda 272 | min_lambda_ratio = self.min_lambda_ratio 273 | 274 | _y = y.astype(dtype=np.float64, order='F', copy=True) 275 | _sample_weight = sample_weight.astype(dtype=np.float64, order='F', 276 | copy=True) 277 | 278 | exclude_vars = 0 279 | 280 | if relative_penalties is None: 281 | relative_penalties = np.ones(X.shape[1], dtype=np.float64, 282 | order='F') 283 | 284 | coef_bounds = np.empty((2, X.shape[1]), dtype=np.float64, order='F') 285 | coef_bounds[0, :] = self.lower_limits 286 | coef_bounds[1, :] = self.upper_limits 287 | 288 | if X.shape[1] > X.shape[0]: 289 | # the glmnet docs suggest using a different algorithm for the case 290 | # of p >> n 291 | algo_flag = 2 292 | else: 293 | algo_flag = 1 294 | 295 | # This is a stopping criterion (nx) 296 | # R defaults to nx = num_features, and ne = num_features + 1 297 | if self.max_features is None: 298 | max_features = X.shape[1] 299 | else: 300 | max_features = self.max_features 301 | 302 | if issparse(X): 303 | _x = csc_matrix(X, dtype=np.float64, copy=True) 304 | 305 | (self.n_lambda_, 306 | self.intercept_path_, 307 | ca, 308 | ia, 309 | nin, 310 | _, # rsq 311 | self.lambda_path_, 312 | _, # nlp 313 | jerr) = spelnet(algo_flag, 314 | self.alpha, 315 | _x.shape[0], 316 | _x.shape[1], 317 | _x.data, 318 | _x.indptr + 1, # Fortran uses 1-based indexing 319 | _x.indices + 1, 320 | _y, 321 | _sample_weight, 322 | exclude_vars, 323 | relative_penalties, 324 | coef_bounds, 325 | max_features, 326 | X.shape[1] + 1, 327 | min_lambda_ratio, 328 | self.lambda_path, 329 | self.tol, 330 | n_lambda, 331 | self.standardize, 332 | self.fit_intercept, 333 | self.max_iter) 334 | else: 335 | _x = X.astype(dtype=np.float64, order='F', copy=True) 336 | 337 | (self.n_lambda_, 338 | self.intercept_path_, 339 | ca, 340 | ia, 341 | nin, 342 | _, # rsq 343 | self.lambda_path_, 344 | _, # nlp 345 | jerr) = elnet(algo_flag, 346 | self.alpha, 347 | _x, 348 | _y, 349 | _sample_weight, 350 | exclude_vars, 351 | relative_penalties, 352 | coef_bounds, 353 | X.shape[1] + 1, 354 | min_lambda_ratio, 355 | self.lambda_path, 356 | self.tol, 357 | max_features, 358 | n_lambda, 359 | self.standardize, 360 | self.fit_intercept, 361 | self.max_iter) 362 | 363 | # raises RuntimeError if self.jerr_ is nonzero 364 | self.jerr_ = jerr 365 | _check_error_flag(self.jerr_) 366 | 367 | self.lambda_path_ = self.lambda_path_[:self.n_lambda_] 368 | self.lambda_path_ = _fix_lambda_path(self.lambda_path_) 369 | # trim the pre-allocated arrays returned by glmnet to match the actual 370 | # number of values found for lambda 371 | self.intercept_path_ = self.intercept_path_[:self.n_lambda_] 372 | ca = ca[:, :self.n_lambda_] 373 | nin = nin[:self.n_lambda_] 374 | self.coef_path_ = solns(_x.shape[1], ca, ia, nin) 375 | 376 | return self 377 | 378 | def decision_function(self, X, lamb=None): 379 | lambda_best = None 380 | if hasattr(self, 'lambda_best_'): 381 | lambda_best = self.lambda_best_ 382 | 383 | lamb = _check_user_lambda(self.lambda_path_, lambda_best, lamb) 384 | coef, intercept = _interpolate_model(self.lambda_path_, 385 | self.coef_path_, 386 | self.intercept_path_, lamb) 387 | 388 | X = check_array(X, accept_sparse='csr') 389 | z = X.dot(coef) + intercept 390 | 391 | # drop last dimension (lambda path) when we are predicting for a 392 | # single value of lambda 393 | if lamb.shape[0] == 1: 394 | z = z.squeeze(axis=-1) 395 | return z 396 | 397 | def predict(self, X, lamb=None): 398 | """Predict the response Y for each sample in X 399 | 400 | Parameters 401 | ---------- 402 | X : array, shape (n_samples, n_features) 403 | 404 | lamb : array, shape (n_lambda,) 405 | Values of lambda from lambda_path_ from which to make predictions. 406 | If no values are provided, the returned predictions will be those 407 | corresponding to lambda_best_. The values of lamb must also be in 408 | the range of lambda_path_, values greater than max(lambda_path_) 409 | or less than min(lambda_path_) will be clipped. 410 | 411 | Returns 412 | ------- 413 | C : array, shape (n_samples,) or (n_samples, n_lambda) 414 | Predicted response value for each sample given each value of lambda 415 | """ 416 | return self.decision_function(X, lamb) 417 | 418 | def score(self, X, y, lamb=None): 419 | """Returns the coefficient of determination R^2 for each value of lambda. 420 | 421 | Parameters 422 | ---------- 423 | X : array, shape (n_samples, n_features) 424 | Test samples 425 | 426 | y : array, shape (n_samples,) 427 | True values for X 428 | 429 | lamb : array, shape (n_lambda,) 430 | Values from lambda_path_ for which to score predictions. 431 | 432 | Returns 433 | ------- 434 | scores : array, shape (n_lambda,) 435 | R^2 of predictions for each lambda. 436 | """ 437 | 438 | # pred will have shape (n_samples, n_lambda) 439 | pred = self.predict(X, lamb=lamb) 440 | 441 | # Reverse the args of the r2_score function from scikit-learn. The 442 | # function np.apply_along_axis passes an array as the first argument to 443 | # the provided function and any extra args as the subsequent arguments. 444 | def r2_reverse(y_pred, y_true): 445 | return r2_score(y_true, y_pred) 446 | 447 | # compute the score for each value of lambda 448 | return np.apply_along_axis(r2_reverse, 0, pred, y) 449 | -------------------------------------------------------------------------------- /glmnet/logistic.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | from scipy.special import expit 4 | from scipy.sparse import issparse, csc_matrix 5 | from scipy import stats 6 | 7 | from sklearn.base import BaseEstimator 8 | from sklearn.metrics import accuracy_score 9 | from sklearn.model_selection import StratifiedKFold, GroupKFold 10 | from sklearn.utils import check_array, check_X_y 11 | from sklearn.utils.multiclass import check_classification_targets 12 | 13 | from .errors import _check_error_flag 14 | from _glmnet import lognet, splognet, lsolns 15 | from glmnet.util import (_fix_lambda_path, 16 | _check_user_lambda, 17 | _interpolate_model, 18 | _score_lambda_path) 19 | 20 | 21 | class LogitNet(BaseEstimator): 22 | """Logistic Regression with elastic net penalty. 23 | 24 | This is a wrapper for the glmnet function lognet. 25 | 26 | Parameters 27 | ---------- 28 | alpha : float, default 1 29 | The alpha parameter, 0 <= alpha <= 1, 0 for ridge, 1 for lasso 30 | 31 | n_lambda : int, default 100 32 | Maximum number of lambda values to compute 33 | 34 | min_lambda_ratio : float, default 1e-4 35 | In combination with n_lambda, the ratio of the smallest and largest 36 | values of lambda computed. 37 | 38 | lambda_path : array, default None 39 | In place of supplying n_lambda, provide an array of specific values 40 | to compute. The specified values must be in decreasing order. When 41 | None, the path of lambda values will be determined automatically. A 42 | maximum of `n_lambda` values will be computed. 43 | 44 | standardize : bool, default True 45 | Standardize input features prior to fitting. The final coefficients 46 | will be on the scale of the original data regardless of the value 47 | of standardize. 48 | 49 | fit_intercept : bool, default True 50 | Include an intercept term in the model. 51 | 52 | lower_limits : array, (shape n_features,) default -infinity 53 | Array of lower limits for each coefficient, must be non-positive. 54 | Can be a single value (which is then replicated), else an array 55 | corresponding to the number of features. 56 | 57 | upper_limits : array, (shape n_features,) default +infinity 58 | Array of upper limits for each coefficient, must be positive. 59 | See lower_limits. 60 | 61 | cut_point : float, default 1 62 | The cut point to use for selecting lambda_best. 63 | arg_max lambda cv_score(lambda) >= cv_score(lambda_max) - cut_point * standard_error(lambda_max) 64 | 65 | n_splits : int, default 3 66 | Number of cross validation folds for computing performance metrics and 67 | determining `lambda_best_` and `lambda_max_`. If non-zero, must be 68 | at least 3. 69 | 70 | scoring : string, callable or None, default None 71 | Scoring method for model selection during cross validation. When None, 72 | defaults to classification score. Valid options are `accuracy`, 73 | `roc_auc`, `average_precision`, `precision`, `recall`. Alternatively, 74 | supply a function or callable object with the following signature 75 | ``scorer(estimator, X, y)``. Note, the scoring function affects the 76 | selection of `lambda_best_` and `lambda_max_`, fitting the same data 77 | with different scoring methods will result in the selection of 78 | different models. 79 | 80 | n_jobs : int, default 1 81 | Maximum number of threads for computing cross validation metrics. 82 | 83 | tol : float, default 1e-7 84 | Convergence tolerance. 85 | 86 | max_iter : int, default 100000 87 | Maximum passes over the data. 88 | 89 | random_state : number, default None 90 | Seed for the random number generator. The glmnet solver is not 91 | deterministic, this seed is used for determining the cv folds. 92 | 93 | max_features : int 94 | Optional maximum number of features with nonzero coefficients after 95 | regularization. If not set, defaults to X.shape[1] during fit 96 | Note, this will be ignored if the user specifies lambda_path. 97 | 98 | verbose : bool, default False 99 | When True some warnings and log messages are suppressed. 100 | 101 | Attributes 102 | ---------- 103 | classes_ : array, shape(n_classes,) 104 | The distinct classes/labels found in y. 105 | 106 | n_lambda_ : int 107 | The number of lambda values found by glmnet. Note, this may be less 108 | than the number specified via n_lambda. 109 | 110 | lambda_path_ : array, shape (n_lambda_,) 111 | The values of lambda found by glmnet, in decreasing order. 112 | 113 | coef_path_ : array, shape (n_classes, n_features, n_lambda_) 114 | The set of coefficients for each value of lambda in lambda_path_. 115 | 116 | coef_ : array, shape (n_clases, n_features) 117 | The coefficients corresponding to lambda_best_. 118 | 119 | intercept_ : array, shape (n_classes,) 120 | The intercept corresponding to lambda_best_. 121 | 122 | intercept_path_ : array, shape (n_classes, n_lambda_) 123 | The set of intercepts for each value of lambda in lambda_path_. 124 | 125 | cv_mean_score_ : array, shape (n_lambda_,) 126 | The mean cv score for each value of lambda. This will be set by fit_cv. 127 | 128 | cv_standard_error_ : array, shape (n_lambda_,) 129 | The standard error of the mean cv score for each value of lambda, this 130 | will be set by fit_cv. 131 | 132 | lambda_max_ : float 133 | The value of lambda that gives the best performance in cross 134 | validation. 135 | 136 | lambda_best_ : float 137 | The largest value of lambda which is greater than lambda_max_ and 138 | performs within cut_point * standard error of lambda_max_. 139 | """ 140 | 141 | def __init__(self, alpha=1, n_lambda=100, min_lambda_ratio=1e-4, 142 | lambda_path=None, standardize=True, fit_intercept=True, 143 | lower_limits=-np.inf, upper_limits=np.inf, 144 | cut_point=1.0, n_splits=3, scoring=None, n_jobs=1, tol=1e-7, 145 | max_iter=100000, random_state=None, max_features=None, verbose=False): 146 | 147 | self.alpha = alpha 148 | self.n_lambda = n_lambda 149 | self.min_lambda_ratio = min_lambda_ratio 150 | self.lambda_path = lambda_path 151 | self.standardize = standardize 152 | self.lower_limits = lower_limits 153 | self.upper_limits = upper_limits 154 | self.fit_intercept = fit_intercept 155 | self.cut_point = cut_point 156 | self.n_splits = n_splits 157 | self.scoring = scoring 158 | self.n_jobs = n_jobs 159 | self.tol = tol 160 | self.max_iter = max_iter 161 | self.random_state = random_state 162 | self.max_features = max_features 163 | self.verbose = verbose 164 | 165 | def fit(self, X, y, sample_weight=None, relative_penalties=None, groups=None): 166 | """Fit the model to training data. If n_splits > 1 also run n-fold cross 167 | validation on all values in lambda_path. 168 | 169 | The model will be fit n+1 times. On the first pass, the lambda_path 170 | will be determined, on the remaining passes, the model performance for 171 | each value of lambda. After cross validation, the attribute 172 | `cv_mean_score_` will contain the mean score over all folds for each 173 | value of lambda, and `cv_standard_error_` will contain the standard 174 | error of `cv_mean_score_` for each value of lambda. The value of lambda 175 | which achieves the best performance in cross validation will be saved 176 | to `lambda_max_` additionally, the largest value of lambda s.t.: 177 | cv_score(l) >= cv_score(lambda_max_) -\ 178 | cut_point * standard_error(lambda_max_) 179 | will be saved to `lambda_best_`. 180 | 181 | Parameters 182 | ---------- 183 | X : array, shape (n_samples, n_features) 184 | Input features 185 | 186 | y : array, shape (n_samples,) 187 | Target values 188 | 189 | sample_weight : array, shape (n_samples,) 190 | Optional weight vector for observations 191 | 192 | relative_penalties: array, shape (n_features,) 193 | Optional relative weight vector for penalty. 194 | 0 entries remove penalty. 195 | 196 | groups: array, shape (n_samples,) 197 | Group labels for the samples used while splitting the dataset into train/test set. 198 | If the groups are specified, the groups will be passed to sklearn.model_selection.GroupKFold. 199 | If None, then data will be split randomly for K-fold cross-validation via sklearn.model_selection.KFold. 200 | 201 | Returns 202 | ------- 203 | self : object 204 | Returns self. 205 | """ 206 | X, y = check_X_y(X, y, accept_sparse='csr', ensure_min_samples=2) 207 | if sample_weight is None: 208 | sample_weight = np.ones(X.shape[0]) 209 | else: 210 | sample_weight = np.asarray(sample_weight) 211 | 212 | if y.shape != sample_weight.shape: 213 | raise ValueError('the shape of weights is not the same with the shape of y') 214 | 215 | if not np.isscalar(self.lower_limits): 216 | self.lower_limits = np.asarray(self.lower_limits) 217 | if len(self.lower_limits) != X.shape[1]: 218 | raise ValueError("lower_limits must equal number of features") 219 | 220 | if not np.isscalar(self.upper_limits): 221 | self.upper_limits = np.asarray(self.upper_limits) 222 | if len(self.upper_limits) != X.shape[1]: 223 | raise ValueError("upper_limits must equal number of features") 224 | 225 | if any(self.lower_limits > 0) if isinstance(self.lower_limits, np.ndarray) else self.lower_limits > 0: 226 | raise ValueError("lower_limits must be non-positive") 227 | 228 | if any(self.upper_limits < 0) if isinstance(self.upper_limits, np.ndarray) else self.upper_limits < 0: 229 | raise ValueError("upper_limits must be positive") 230 | 231 | if self.alpha > 1 or self.alpha < 0: 232 | raise ValueError("alpha must be between 0 and 1") 233 | 234 | # fit the model 235 | self._fit(X, y, sample_weight, relative_penalties) 236 | 237 | # score each model on the path of lambda values found by glmnet and 238 | # select the best scoring 239 | if self.n_splits >= 3: 240 | if groups is None: 241 | self._cv = StratifiedKFold(n_splits=self.n_splits, shuffle=True, random_state=self.random_state) 242 | else: 243 | self._cv = GroupKFold(n_splits=self.n_splits) 244 | 245 | cv_scores = _score_lambda_path(self, X, y, groups, 246 | sample_weight, 247 | relative_penalties, 248 | self.scoring, 249 | n_jobs=self.n_jobs, 250 | verbose=self.verbose) 251 | 252 | self.cv_mean_score_ = np.atleast_1d(np.mean(cv_scores, axis=0)) 253 | self.cv_standard_error_ = np.atleast_1d(stats.sem(cv_scores)) 254 | 255 | self.lambda_max_inx_ = np.argmax(self.cv_mean_score_) 256 | self.lambda_max_ = self.lambda_path_[self.lambda_max_inx_] 257 | 258 | target_score = self.cv_mean_score_[self.lambda_max_inx_] -\ 259 | self.cut_point * self.cv_standard_error_[self.lambda_max_inx_] 260 | 261 | self.lambda_best_inx_ = np.argwhere(self.cv_mean_score_ >= target_score)[0] 262 | self.lambda_best_ = self.lambda_path_[self.lambda_best_inx_] 263 | 264 | self.coef_ = self.coef_path_[..., self.lambda_best_inx_] 265 | self.coef_ = self.coef_.squeeze(axis=self.coef_.ndim-1) 266 | self.intercept_ = self.intercept_path_[..., self.lambda_best_inx_].squeeze() 267 | if self.intercept_.shape == (): # convert 0d array to scalar 268 | self.intercept_ = float(self.intercept_) 269 | 270 | return self 271 | 272 | def _fit(self, X, y, sample_weight=None, relative_penalties=None): 273 | if self.lambda_path is not None: 274 | n_lambda = len(self.lambda_path) 275 | min_lambda_ratio = 1.0 276 | else: 277 | n_lambda = self.n_lambda 278 | min_lambda_ratio = self.min_lambda_ratio 279 | 280 | check_classification_targets(y) 281 | self.classes_ = np.unique(y) # the output of np.unique is sorted 282 | n_classes = len(self.classes_) 283 | if n_classes < 2: 284 | raise ValueError("Training data need to contain at least 2 " 285 | "classes.") 286 | 287 | # glmnet requires the labels a one-hot-encoded array of 288 | # (n_samples, n_classes) 289 | if n_classes == 2: 290 | # Normally we use 1/0 for the positive and negative classes. Since 291 | # np.unique sorts the output, the negative class will be in the 0th 292 | # column. We want a model predicting the positive class, not the 293 | # negative class, so we flip the columns here (the != condition). 294 | # 295 | # Broadcast comparison of self.classes_ to all rows of y. See the 296 | # numpy rules on broadcasting for more info, essentially this 297 | # "reshapes" y to (n_samples, n_classes) and self.classes_ to 298 | # (n_samples, n_classes) and performs an element-wise comparison 299 | # resulting in _y with shape (n_samples, n_classes). 300 | _y = (y[:, None] != self.classes_).astype(np.float64, order='F') 301 | else: 302 | # multinomial case, glmnet uses the entire array so we can 303 | # keep the original order. 304 | _y = (y[:, None] == self.classes_).astype(np.float64, order='F') 305 | 306 | # use sample weights, making sure all weights are positive 307 | # this is inspired by the R wrapper for glmnet, in lognet.R 308 | if sample_weight is not None: 309 | weight_gt_0 = sample_weight > 0 310 | sample_weight = sample_weight[weight_gt_0] 311 | _y = _y[weight_gt_0, :] 312 | X = X[weight_gt_0, :] 313 | _y = _y * np.expand_dims(sample_weight, 1) 314 | 315 | # we need some sort of "offset" array for glmnet 316 | # an array of shape (n_examples, n_classes) 317 | offset = np.zeros((X.shape[0], n_classes), dtype=np.float64, 318 | order='F') 319 | 320 | # You should have thought of that before you got here. 321 | exclude_vars = 0 322 | 323 | # how much each feature should be penalized relative to the others 324 | # this may be useful to expose to the caller if there are vars that 325 | # must be included in the final model or there is some prior knowledge 326 | # about how important some vars are relative to others, see the glmnet 327 | # vignette: 328 | # http://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html 329 | if relative_penalties is None: 330 | relative_penalties = np.ones(X.shape[1], dtype=np.float64, 331 | order='F') 332 | 333 | coef_bounds = np.empty((2, X.shape[1]), dtype=np.float64, order='F') 334 | coef_bounds[0, :] = self.lower_limits 335 | coef_bounds[1, :] = self.upper_limits 336 | 337 | if n_classes == 2: 338 | # binomial, tell glmnet there is only one class 339 | # otherwise we will get a coef matrix with two dimensions 340 | # where each pair are equal in magnitude and opposite in sign 341 | # also since the magnitudes are constrained to sum to one, the 342 | # returned coefficients would be one half of the proper values 343 | n_classes = 1 344 | 345 | 346 | # This is a stopping criterion (nx) 347 | # R defaults to nx = num_features, and ne = num_features + 1 348 | if self.max_features is None: 349 | max_features = X.shape[1] 350 | else: 351 | max_features = self.max_features 352 | 353 | # for documentation on the glmnet function lognet, see doc.py 354 | if issparse(X): 355 | _x = csc_matrix(X, dtype=np.float64, copy=True) 356 | 357 | (self.n_lambda_, 358 | self.intercept_path_, 359 | ca, 360 | ia, 361 | nin, 362 | _, # dev0 363 | _, # dev 364 | self.lambda_path_, 365 | _, # nlp 366 | jerr) = splognet(self.alpha, 367 | _x.shape[0], 368 | _x.shape[1], 369 | n_classes, 370 | _x.data, 371 | _x.indptr + 1, # Fortran uses 1-based indexing 372 | _x.indices + 1, 373 | _y, 374 | offset, 375 | exclude_vars, 376 | relative_penalties, 377 | coef_bounds, 378 | max_features, 379 | X.shape[1] + 1, 380 | min_lambda_ratio, 381 | self.lambda_path, 382 | self.tol, 383 | n_lambda, 384 | self.standardize, 385 | self.fit_intercept, 386 | self.max_iter, 387 | 0) 388 | else: # not sparse 389 | # some notes: glmnet requires both x and y to be float64, the two 390 | # arrays 391 | # may also be overwritten during the fitting process, so they need 392 | # to be copied prior to calling lognet. The fortran wrapper will 393 | # copy any arrays passed to a wrapped function if they are not in 394 | # the fortran layout, to avoid making extra copies, ensure x and y 395 | # are `F_CONTIGUOUS` prior to calling lognet. 396 | _x = X.astype(dtype=np.float64, order='F', copy=True) 397 | 398 | (self.n_lambda_, 399 | self.intercept_path_, 400 | ca, 401 | ia, 402 | nin, 403 | _, # dev0 404 | _, # dev 405 | self.lambda_path_, 406 | _, # nlp 407 | jerr) = lognet(self.alpha, 408 | n_classes, 409 | _x, 410 | _y, 411 | offset, 412 | exclude_vars, 413 | relative_penalties, 414 | coef_bounds, 415 | X.shape[1] + 1, 416 | min_lambda_ratio, 417 | self.lambda_path, 418 | self.tol, 419 | max_features, 420 | n_lambda, 421 | self.standardize, 422 | self.fit_intercept, 423 | self.max_iter, 424 | 0) 425 | 426 | # raises RuntimeError if self.jerr_ is nonzero 427 | self.jerr_ = jerr 428 | _check_error_flag(self.jerr_) 429 | 430 | # glmnet may not return the requested number of lambda values, so we 431 | # need to trim the trailing zeros from the returned path so 432 | # len(lambda_path_) is equal to n_lambda_ 433 | self.lambda_path_ = self.lambda_path_[:self.n_lambda_] 434 | # also fix the first value of lambda 435 | self.lambda_path_ = _fix_lambda_path(self.lambda_path_) 436 | self.intercept_path_ = self.intercept_path_[:, :self.n_lambda_] 437 | # also trim the compressed coefficient matrix 438 | ca = ca[:, :, :self.n_lambda_] 439 | # and trim the array of n_coef per lambda (may or may not be non-zero) 440 | nin = nin[:self.n_lambda_] 441 | # decompress the coefficients returned by glmnet, see doc.py 442 | self.coef_path_ = lsolns(X.shape[1], ca, ia, nin) 443 | # coef_path_ has shape (n_features, n_classes, n_lambda), we should 444 | # match shape for scikit-learn models: 445 | # (n_classes, n_features, n_lambda) 446 | self.coef_path_ = np.transpose(self.coef_path_, axes=(1, 0, 2)) 447 | 448 | return self 449 | 450 | def decision_function(self, X, lamb=None): 451 | lambda_best = None 452 | if hasattr(self, 'lambda_best_'): 453 | lambda_best = self.lambda_best_ 454 | 455 | lamb = _check_user_lambda(self.lambda_path_, lambda_best, lamb) 456 | coef, intercept = _interpolate_model(self.lambda_path_, 457 | self.coef_path_, 458 | self.intercept_path_, lamb) 459 | 460 | # coef must be (n_classes, n_features, n_lambda) 461 | if coef.ndim != 3: 462 | # we must be working with an intercept only model 463 | coef = coef[:, :, np.newaxis] 464 | # intercept must be (n_classes, n_lambda) 465 | if intercept.ndim != 2: 466 | intercept = intercept[:, np.newaxis] 467 | 468 | X = check_array(X, accept_sparse='csr') 469 | # return (n_samples, n_classes, n_lambda) 470 | z = np.empty((X.shape[0], coef.shape[0], coef.shape[-1])) 471 | # well... sometimes we just need a for loop 472 | for c in range(coef.shape[0]): # all classes 473 | for l in range(coef.shape[-1]): # all values of lambda 474 | z[:, c, l] = X.dot(coef[c, :, l]) 475 | z += intercept 476 | 477 | # drop the last dimension (lambda) when we are predicting for a single 478 | # value of lambda, and drop the middle dimension (class) when we are 479 | # predicting from a binomial model (for consistency with scikit-learn) 480 | return z.squeeze() 481 | 482 | def predict_proba(self, X, lamb=None): 483 | """Probability estimates for each class given X. 484 | 485 | The returned estimates are in the same order as the values in 486 | classes_. 487 | 488 | Parameters 489 | ---------- 490 | X : array, shape (n_samples, n_features) 491 | 492 | lamb : array, shape (n_lambda,) 493 | Values of lambda from lambda_path_ from which to make predictions. 494 | If no values are provided, the returned predictions will be those 495 | corresponding to lambda_best_. The values of lamb must also be in 496 | the range of lambda_path_, values greater than max(lambda_path_) 497 | or less than min(lambda_path_) will be clipped. 498 | 499 | Returns 500 | ------- 501 | T : array, shape (n_samples, n_classes) or (n_samples, n_classes, n_lambda) 502 | """ 503 | z = self.decision_function(X, lamb) 504 | expit(z, z) 505 | 506 | # reshape z to (n_samples, n_classes, n_lambda) 507 | n_lambda = len(np.atleast_1d(lamb)) 508 | z = z.reshape(X.shape[0], -1, n_lambda) 509 | 510 | if z.shape[1] == 1: 511 | # binomial, for consistency and to match scikit-learn, add the 512 | # complement so z has shape (n_samples, 2, n_lambda) 513 | z = np.concatenate((1-z, z), axis=1) 514 | else: 515 | # normalize for multinomial 516 | z /= np.expand_dims(z.sum(axis=1), axis=1) 517 | 518 | if n_lambda == 1: 519 | z = z.squeeze(axis=-1) 520 | return z 521 | 522 | def predict(self, X, lamb=None): 523 | """Predict class labels for samples in X. 524 | 525 | Parameters 526 | ---------- 527 | X : array, shape (n_samples, n_features) 528 | 529 | lamb : array, shape (n_lambda,) 530 | Values of lambda from lambda_path_ from which to make predictions. 531 | If no values are provided for lamb, the returned predictions will 532 | be those corresponding to lambda_best_. The values of lamb must 533 | also be in the range of lambda_path_, values greater than 534 | max(lambda_path_) or less than min(lambda_path_) will be clipped. 535 | 536 | Returns 537 | ------- 538 | T : array, shape (n_samples,) or (n_samples, n_lambda) 539 | Predicted class labels for each sample given each value of lambda 540 | """ 541 | 542 | scores = self.predict_proba(X, lamb) 543 | indices = scores.argmax(axis=1) 544 | 545 | return self.classes_[indices] 546 | 547 | def score(self, X, y, lamb=None): 548 | """Returns the mean accuracy on the given test data and labels. 549 | 550 | Parameters 551 | ---------- 552 | X : array, shape (n_samples, n_features) 553 | Test samples 554 | 555 | y : array, shape (n_samples,) 556 | True labels for X 557 | 558 | lamb : array, shape (n_lambda,) 559 | Values from lambda_path_ for which to score predictions. 560 | 561 | Returns 562 | ------- 563 | score : array, shape (n_lambda,) 564 | Mean accuracy for each value of lambda. 565 | """ 566 | pred = self.predict(X, lamb=lamb) 567 | return np.apply_along_axis(accuracy_score, 0, pred, y) 568 | -------------------------------------------------------------------------------- /glmnet/scorer.py: -------------------------------------------------------------------------------- 1 | """ 2 | The code below is a modified version of sklearn.metrics.scorer to allow for scoring 3 | the entire lambda path of a glmnet model. 4 | 5 | - lambda parameter added to the scorers 6 | - scorers return an array of scores, [n_lambda,] 7 | """ 8 | 9 | # Authors: Andreas Mueller 10 | # Lars Buitinck 11 | # Arnaud Joly 12 | # License: Simplified BSD 13 | 14 | from abc import ABCMeta, abstractmethod 15 | from functools import partial 16 | 17 | import numpy as np 18 | 19 | from sklearn.metrics import (r2_score, median_absolute_error, mean_absolute_error, 20 | mean_squared_error, accuracy_score, f1_score, 21 | roc_auc_score, average_precision_score, 22 | precision_score, recall_score, log_loss) 23 | from sklearn.utils.multiclass import type_of_target 24 | 25 | 26 | class _BaseScorer(metaclass=ABCMeta): 27 | def __init__(self, score_func, sign, kwargs): 28 | self._kwargs = kwargs 29 | self._score_func = score_func 30 | self._sign = sign 31 | 32 | @abstractmethod 33 | def __call__(self, estimator, X, y, sample_weight=None): 34 | pass 35 | 36 | def __repr__(self): 37 | kwargs_string = "".join([", %s=%s" % (str(k), str(v)) 38 | for k, v in self._kwargs.items()]) 39 | return ("make_scorer(%s%s%s%s)" 40 | % (self._score_func.__name__, 41 | "" if self._sign > 0 else ", greater_is_better=False", 42 | self._factory_args(), kwargs_string)) 43 | 44 | def _factory_args(self): 45 | """Return non-default make_scorer arguments for repr.""" 46 | return "" 47 | 48 | 49 | class _PredictScorer(_BaseScorer): 50 | def __call__(self, estimator, X, y_true, sample_weight=None, lamb=None): 51 | """Evaluate predicted target values for X relative to y_true and one or 52 | more values for lambda. 53 | 54 | Parameters 55 | ---------- 56 | estimator : object 57 | Trained estimator to use for scoring. Must have a predict_proba 58 | method; the output of that is used to compute the score. 59 | 60 | X : array-like or sparse matrix 61 | Test data that will be fed to estimator.predict. 62 | 63 | y_true : array-like 64 | Gold standard target values for X. 65 | 66 | sample_weight : array-like, optional (default=None) 67 | Sample weights. 68 | 69 | lamb : array, shape (n_lambda,) 70 | Values of lambda from lambda_path_ from which to score predictions. 71 | 72 | Returns 73 | ------- 74 | score : array, shape (n_lambda,) 75 | Score function applied to prediction of estimator on X. 76 | """ 77 | y_pred = estimator.predict(X, lamb=lamb) 78 | if sample_weight is not None: 79 | scores = np.apply_along_axis(lambda y_hat: self._score_func(y_true, y_hat, sample_weight=sample_weight, **self._kwargs), 0, y_pred) 80 | else: 81 | scores = np.apply_along_axis(lambda y_hat: self._score_func(y_true, y_hat, **self._kwargs), 0, y_pred) 82 | return self._sign * scores 83 | 84 | class _ProbaScorer(_BaseScorer): 85 | def __call__(self, clf, X, y_true, sample_weight=None, lamb=None): 86 | """Evaluate predicted probabilities for X relative to y_true. 87 | 88 | Parameters 89 | ---------- 90 | clf : object 91 | Trained classifier to use for scoring. Must have a predict_proba 92 | method; the output of that is used to compute the score. 93 | 94 | X : array-like or sparse matrix 95 | Test data that will be fed to clf.predict_proba. 96 | 97 | y_true : array-like 98 | Gold standard target values for X. These must be class labels, 99 | not probabilities. 100 | 101 | sample_weight : array-like, optional (default=None) 102 | Sample weights. 103 | 104 | lamb : array, shape (n_lambda,) 105 | Values of lambda from lambda_path_ from which to score predictions. 106 | 107 | Returns 108 | ------- 109 | score : array, shape (n_lambda,) 110 | Score function applied to prediction of estimator on X. 111 | """ 112 | y_pred = clf.predict_proba(X, lamb=lamb) # y_pred shape (n_samples, n_classes, n_lambda) 113 | 114 | if sample_weight is not None: 115 | score_func = lambda y_hat: self._score_func(y_true, y_hat, sample_weight=sample_weight, **self._kwargs) 116 | else: 117 | score_func = lambda y_hat: self._score_func(y_true, y_hat, **self._kwargs) 118 | 119 | scores = np.zeros(y_pred.shape[-1]) 120 | for i in range(len(scores)): 121 | scores[i] = score_func(y_pred[...,i]) 122 | 123 | return self._sign * scores 124 | 125 | def _factory_args(self): 126 | return ", needs_proba=True" 127 | 128 | 129 | class _ThresholdScorer(_BaseScorer): 130 | def __call__(self, clf, X, y_true, sample_weight=None, lamb=None): 131 | """Evaluate decision function output for X relative to y_true. 132 | 133 | Parameters 134 | ---------- 135 | clf : object 136 | Trained classifier to use for scoring. Must have either a 137 | decision_function method or a predict_proba method; the output of 138 | that is used to compute the score. 139 | 140 | X : array-like or sparse matrix 141 | Test data that will be fed to clf.decision_function or 142 | clf.predict_proba. 143 | 144 | y_true : array-like 145 | Gold standard target values for X. These must be class labels, 146 | not decision function values. 147 | 148 | sample_weight : array-like, optional (default=None) 149 | Sample weights. 150 | 151 | lamb : array, shape (n_lambda,) 152 | Values of lambda from lambda_path_ from which to score predictions. 153 | 154 | Returns 155 | ------- 156 | score : array, shape (n_lambda,) 157 | Score function applied to prediction of estimator on X. 158 | """ 159 | y_type = type_of_target(y_true) 160 | if y_type not in ("binary", "multilabel-indicator"): 161 | raise ValueError("{0} format is not supported".format(y_type)) 162 | 163 | y_pred = clf.decision_function(X, lamb=lamb) 164 | if sample_weight is not None: 165 | scores = np.apply_along_axis(lambda y_hat: self._score_func(y_true, y_hat, sample_weight=sample_weight, **self._kwargs), 0, y_pred) 166 | else: 167 | scores = np.apply_along_axis(lambda y_hat: self._score_func(y_true, y_hat, **self._kwargs), 0, y_pred) 168 | return self._sign * scores 169 | 170 | def _factory_args(self): 171 | return ", needs_threshold=True" 172 | 173 | 174 | def get_scorer(scoring): 175 | if isinstance(scoring, str): 176 | try: 177 | scorer = SCORERS[scoring] 178 | except KeyError: 179 | raise ValueError('%r is not a valid scoring value. ' 180 | 'Valid options are %s' 181 | % (scoring, sorted(SCORERS.keys()))) 182 | else: 183 | scorer = scoring 184 | return scorer 185 | 186 | 187 | def _passthrough_scorer(estimator, *args, **kwargs): 188 | """Function that wraps estimator.score""" 189 | return estimator.score(*args, **kwargs) 190 | 191 | 192 | def check_scoring(estimator, scoring=None, allow_none=False): 193 | """Determine scorer from user options. 194 | 195 | A TypeError will be thrown if the estimator cannot be scored. 196 | 197 | Parameters 198 | ---------- 199 | estimator : estimator object implementing 'fit' 200 | The object to use to fit the data. 201 | 202 | scoring : string, callable or None, optional, default: None 203 | A string (see model evaluation documentation) or 204 | a scorer callable object / function with signature 205 | ``scorer(estimator, X, y)``. 206 | 207 | allow_none : boolean, optional, default: False 208 | If no scoring is specified and the estimator has no score function, we 209 | can either return None or raise an exception. 210 | 211 | Returns 212 | ------- 213 | scoring : callable 214 | A scorer callable object / function with signature 215 | ``scorer(estimator, X, y)``. 216 | """ 217 | has_scoring = scoring is not None 218 | if not hasattr(estimator, 'fit'): 219 | raise TypeError("estimator should a be an estimator implementing " 220 | "'fit' method, %r was passed" % estimator) 221 | elif has_scoring: 222 | return get_scorer(scoring) 223 | elif hasattr(estimator, 'score'): 224 | return _passthrough_scorer 225 | elif allow_none: 226 | return None 227 | else: 228 | raise TypeError( 229 | "If no scoring is specified, the estimator passed should " 230 | "have a 'score' method. The estimator %r does not." % estimator) 231 | 232 | 233 | def make_scorer(score_func, greater_is_better=True, needs_proba=False, 234 | needs_threshold=False, **kwargs): 235 | """Make a scorer from a performance metric or loss function. 236 | 237 | This factory function wraps scoring functions for use in GridSearchCV 238 | and cross_val_score. It takes a score function, such as ``accuracy_score``, 239 | ``mean_squared_error``, ``adjusted_rand_index`` or ``average_precision`` 240 | and returns a callable that scores an estimator's output. 241 | 242 | Read more in the :ref:`User Guide `. 243 | 244 | Parameters 245 | ---------- 246 | score_func : callable, 247 | Score function (or loss function) with signature 248 | ``score_func(y, y_pred, **kwargs)``. 249 | 250 | greater_is_better : boolean, default=True 251 | Whether score_func is a score function (default), meaning high is good, 252 | or a loss function, meaning low is good. In the latter case, the 253 | scorer object will sign-flip the outcome of the score_func. 254 | 255 | needs_proba : boolean, default=False 256 | Whether score_func requires predict_proba to get probability estimates 257 | out of a classifier. 258 | 259 | needs_threshold : boolean, default=False 260 | Whether score_func takes a continuous decision certainty. 261 | This only works for binary classification using estimators that 262 | have either a decision_function or predict_proba method. 263 | 264 | For example ``average_precision`` or the area under the roc curve 265 | can not be computed using discrete predictions alone. 266 | 267 | **kwargs : additional arguments 268 | Additional parameters to be passed to score_func. 269 | 270 | Returns 271 | ------- 272 | scorer : callable 273 | Callable object that returns a scalar score; greater is better. 274 | 275 | Examples 276 | -------- 277 | >>> from sklearn.metrics import fbeta_score, make_scorer 278 | >>> ftwo_scorer = make_scorer(fbeta_score, beta=2) 279 | >>> ftwo_scorer 280 | make_scorer(fbeta_score, beta=2) 281 | >>> from sklearn.grid_search import GridSearchCV 282 | >>> from sklearn.svm import LinearSVC 283 | >>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]}, 284 | ... scoring=ftwo_scorer) 285 | """ 286 | sign = 1 if greater_is_better else -1 287 | if needs_proba and needs_threshold: 288 | raise ValueError("Set either needs_proba or needs_threshold to True," 289 | " but not both.") 290 | if needs_proba: 291 | cls = _ProbaScorer 292 | elif needs_threshold: 293 | cls = _ThresholdScorer 294 | else: 295 | cls = _PredictScorer 296 | return cls(score_func, sign, kwargs) 297 | 298 | 299 | # Standard regression scores 300 | r2_scorer = make_scorer(r2_score) 301 | mean_squared_error_scorer = make_scorer(mean_squared_error, 302 | greater_is_better=False) 303 | mean_absolute_error_scorer = make_scorer(mean_absolute_error, 304 | greater_is_better=False) 305 | median_absolute_error_scorer = make_scorer(median_absolute_error, 306 | greater_is_better=False) 307 | 308 | # Standard Classification Scores 309 | accuracy_scorer = make_scorer(accuracy_score) 310 | f1_scorer = make_scorer(f1_score) 311 | 312 | # Score functions that need decision values 313 | roc_auc_scorer = make_scorer(roc_auc_score, greater_is_better=True, 314 | needs_threshold=True) 315 | average_precision_scorer = make_scorer(average_precision_score, 316 | needs_threshold=True) 317 | precision_scorer = make_scorer(precision_score) 318 | recall_scorer = make_scorer(recall_score) 319 | 320 | # Score function for probabilistic classification 321 | log_loss_scorer = make_scorer(log_loss, greater_is_better=False, 322 | needs_proba=True) 323 | 324 | SCORERS = dict(r2=r2_scorer, 325 | median_absolute_error=median_absolute_error_scorer, 326 | mean_absolute_error=mean_absolute_error_scorer, 327 | mean_squared_error=mean_squared_error_scorer, 328 | accuracy=accuracy_scorer, roc_auc=roc_auc_scorer, 329 | average_precision=average_precision_scorer, 330 | log_loss=log_loss_scorer) 331 | 332 | for name, metric in [('precision', precision_score), 333 | ('recall', recall_score), ('f1', f1_score)]: 334 | SCORERS[name] = make_scorer(metric) 335 | for average in ['macro', 'micro', 'samples', 'weighted']: 336 | qualified_name = '{0}_{1}'.format(name, average) 337 | SCORERS[qualified_name] = make_scorer(partial(metric, pos_label=None, 338 | average=average)) 339 | -------------------------------------------------------------------------------- /glmnet/src/glmnet/DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: glmnet 2 | Type: Package 3 | Title: Lasso and Elastic-Net Regularized Generalized Linear Models 4 | Version: 2.0-5 5 | Date: 2016-3-15 6 | Author: Jerome Friedman, Trevor Hastie, Noah Simon, Rob Tibshirani 7 | Maintainer: Trevor Hastie 8 | Depends: Matrix (>= 1.0-6), utils, foreach 9 | Imports: methods 10 | Suggests: survival, knitr, lars 11 | Description: Extremely efficient procedures for fitting the entire lasso or elastic-net regularization path for linear regression, logistic and multinomial regression models, Poisson regression and the Cox model. Two recent additions are the multiple-response Gaussian, and the grouped multinomial. The algorithm uses cyclical coordinate descent in a path-wise fashion, as described in the paper linked to via the URL below. 12 | License: GPL-2 13 | VignetteBuilder: knitr 14 | URL: http://www.jstatsoft.org/v33/i01/. 15 | NeedsCompilation: yes 16 | Packaged: 2016-03-16 21:07:09 UTC; hastie 17 | Repository: CRAN 18 | Date/Publication: 2016-03-17 14:00:48 -------------------------------------------------------------------------------- /glmnet/src/glmnet/NOTICE: -------------------------------------------------------------------------------- 1 | The files glmnet5.f90 and DESCRIPTION were copied from the CRAN github mirror 2 | for the R package named glmnet (https://github.com/cran/glmnet) on June 1, 2016. 3 | See DESCRIPTION for license and attribution information. 4 | -------------------------------------------------------------------------------- /glmnet/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/civisanalytics/python-glmnet/b3f9ef220807828ce6ab9afd3e61a3e2f1675313/glmnet/tests/__init__.py -------------------------------------------------------------------------------- /glmnet/tests/test_errors.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | from glmnet.errors import _check_error_flag 4 | 5 | 6 | class TestErrors(unittest.TestCase): 7 | 8 | def test_zero_jerr(self): 9 | # This should not raise any warnings or exceptions. 10 | _check_error_flag(0) 11 | 12 | def test_convergence_err(self): 13 | msg = ("Model did not converge for smaller values of lambda, " 14 | "returning solution for the largest 75 values.") 15 | with self.assertWarns(RuntimeWarning, msg=msg): 16 | _check_error_flag(-76) 17 | 18 | def test_zero_var_err(self): 19 | msg = "All predictors have zero variance (glmnet error no. 7777)." 20 | with self.assertRaises(ValueError, msg=msg): 21 | _check_error_flag(7777) 22 | 23 | def test_all_negative_rel_penalty(self): 24 | msg = ("At least one value of relative_penalties must be positive, " 25 | "(glmnet error no. 10000).") 26 | with self.assertRaises(ValueError, msg=msg): 27 | _check_error_flag(10000) 28 | 29 | def test_memory_allocation_err(self): 30 | msg = "Memory allocation error (glmnet error no. 1234)." 31 | with self.assertRaises(RuntimeError, msg=msg): 32 | _check_error_flag(1234) 33 | 34 | def test_other_fatal_err(self): 35 | msg = "Fatal glmnet error no. 7778." 36 | with self.assertRaises(RuntimeError, msg=msg): 37 | _check_error_flag(7778) 38 | 39 | def test_class_prob_close_to_1(self): 40 | msg = "Probability for class 2 close to 0." 41 | with self.assertRaises(ValueError, msg=msg): 42 | _check_error_flag(8002) 43 | 44 | def test_class_prob_close_to_0(self): 45 | msg = "Probability for class 4 close to 0." 46 | with self.assertRaises(ValueError, msg=msg): 47 | _check_error_flag(8004) 48 | 49 | def test_predicted_class_close_to_0_or_1(self): 50 | msg = "Predicted probability close to 0 or 1 for lambda no. 7." 51 | with self.assertWarns(RuntimeWarning, msg=msg): 52 | _check_error_flag(-20007) 53 | 54 | def test_did_not_converge(self): 55 | msg = "Solver did not converge (glmnet error no. 90000)." 56 | with self.assertRaises(ValueError, msg=msg): 57 | _check_error_flag(90000) 58 | 59 | -------------------------------------------------------------------------------- /glmnet/tests/test_linear.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import numpy as np 4 | 5 | from scipy.sparse import csr_matrix 6 | 7 | from sklearn.datasets import make_regression 8 | from sklearn.metrics import r2_score 9 | from sklearn.utils import estimator_checks 10 | from sklearn.utils.testing import ignore_warnings 11 | 12 | from glmnet.tests.util import sanity_check_regression 13 | 14 | from glmnet import ElasticNet 15 | 16 | 17 | class TestElasticNet(unittest.TestCase): 18 | 19 | def setUp(self): 20 | np.random.seed(488881) 21 | x, y = make_regression(n_samples=1000, random_state=561) 22 | x_sparse = csr_matrix(x) 23 | 24 | x_wide, y_wide = make_regression(n_samples=100, n_features=150, 25 | random_state=1105) 26 | x_wide_sparse = csr_matrix(x_wide) 27 | 28 | self.inputs = [(x,y), (x_sparse, y), (x_wide, y_wide), 29 | (x_wide_sparse, y_wide)] 30 | self.alphas = [0., 0.25, 0.50, 0.75, 1.] 31 | self.n_splits = [-1, 0, 5] 32 | self.scoring = [ 33 | "r2", 34 | "mean_squared_error", 35 | "mean_absolute_error", 36 | "median_absolute_error", 37 | ] 38 | 39 | @ignore_warnings(category=RuntimeWarning) 40 | def test_estimator_interface(self): 41 | estimator_checks.check_estimator(ElasticNet) 42 | 43 | def test_with_defaults(self): 44 | m = ElasticNet(random_state=2821) 45 | for x, y in self.inputs: 46 | m = m.fit(x, y) 47 | sanity_check_regression(m, x) 48 | 49 | # check selection of lambda_best 50 | self.assertTrue(m.lambda_best_inx_ <= m.lambda_max_inx_) 51 | 52 | # check full path predict 53 | p = m.predict(x, lamb=m.lambda_path_) 54 | self.assertEqual(p.shape[-1], m.lambda_path_.size) 55 | 56 | def test_one_row_predict(self): 57 | # Verify that predicting on one row gives only one row of output 58 | m = ElasticNet(random_state=42) 59 | for X, y in self.inputs: 60 | m.fit(X, y) 61 | p = m.predict(X[0].reshape((1, -1))) 62 | assert p.shape == (1,) 63 | 64 | def test_one_row_predict_with_lambda(self): 65 | # One row to predict along with lambdas should give 2D output 66 | m = ElasticNet(random_state=42) 67 | for X, y in self.inputs: 68 | m.fit(X, y) 69 | p = m.predict(X[0].reshape((1, -1)), lamb=[20, 10]) 70 | assert p.shape == (1, 2) 71 | 72 | def test_with_single_var(self): 73 | x = np.random.rand(500,1) 74 | y = (1.3 * x).ravel() 75 | 76 | m = ElasticNet(random_state=449065) 77 | m = m.fit(x, y) 78 | self.check_r2_score(y, m.predict(x), 0.90) 79 | 80 | def test_with_no_predictor_variance(self): 81 | x = np.ones((500, 1)) 82 | y = np.random.rand(500) 83 | 84 | m = ElasticNet(random_state=561) 85 | msg = "All predictors have zero variance (glmnet error no. 7777)." 86 | with self.assertRaises(ValueError, msg=msg): 87 | m.fit(x, y) 88 | 89 | def test_relative_penalties(self): 90 | m1 = ElasticNet(random_state=4328) 91 | m2 = ElasticNet(random_state=4328) 92 | for x, y in self.inputs: 93 | p = x.shape[1] 94 | 95 | # m1 no relative penalties applied 96 | m1.fit(x, y) 97 | 98 | # find the nonzero indices from LASSO 99 | nonzero = np.nonzero(m1.coef_) 100 | 101 | # unpenalize those nonzero coefs 102 | penalty = np.repeat(1, p) 103 | penalty[nonzero] = 0 104 | 105 | # refit the model with the unpenalized coefs 106 | m2.fit(x, y, relative_penalties=penalty) 107 | 108 | # verify that the unpenalized coef ests exceed the penalized ones 109 | # in absolute value 110 | assert(np.all(np.abs(m1.coef_) <= np.abs(m2.coef_))) 111 | 112 | def test_alphas(self): 113 | x, y = self.inputs[0] 114 | for alpha in self.alphas: 115 | m = ElasticNet(alpha=alpha, random_state=2465) 116 | m = m.fit(x, y) 117 | self.check_r2_score(y, m.predict(x), 0.90, alpha=alpha) 118 | 119 | def test_coef_limits(self): 120 | x, y = self.inputs[0] 121 | lower_limits = np.repeat(-1, x.shape[1]) 122 | upper_limits = 0 123 | m = ElasticNet(lower_limits=lower_limits, upper_limits=upper_limits, random_state=5934, alpha=0) 124 | m = m.fit(x, y) 125 | assert(np.all(m.coef_ >= -1)) 126 | assert(np.all(m.coef_ <= 0)) 127 | 128 | def test_n_splits(self): 129 | x, y = self.inputs[0] 130 | for n in self.n_splits: 131 | m = ElasticNet(n_splits=n, random_state=6601) 132 | if n > 0 and n < 3: 133 | with self.assertRaisesRegexp(ValueError, 134 | "n_splits must be at least 3"): 135 | m = m.fit(x, y) 136 | else: 137 | m = m.fit(x, y) 138 | sanity_check_regression(m, x) 139 | 140 | def test_cv_scoring(self): 141 | x, y = self.inputs[0] 142 | for method in self.scoring: 143 | m = ElasticNet(scoring=method, random_state=1729) 144 | m = m.fit(x, y) 145 | self.check_r2_score(y, m.predict(x), 0.90, scoring=method) 146 | 147 | def test_predict_without_cv(self): 148 | x, y = self.inputs[0] 149 | m = ElasticNet(n_splits=0, random_state=340561) 150 | m = m.fit(x, y) 151 | 152 | # should not make prediction unless value is passed for lambda 153 | with self.assertRaises(ValueError): 154 | m.predict(x) 155 | 156 | def test_coef_interpolation(self): 157 | x, y = self.inputs[0] 158 | m = ElasticNet(n_splits=0, random_state=1729) 159 | m = m.fit(x, y) 160 | 161 | # predict for a value of lambda between two values on the computed path 162 | lamb_lo = m.lambda_path_[1] 163 | lamb_hi = m.lambda_path_[2] 164 | 165 | # a value not equal to one on the computed path 166 | lamb_mid = (lamb_lo + lamb_hi) / 2.0 167 | 168 | pred_lo = m.predict(x, lamb=lamb_lo) 169 | pred_hi = m.predict(x, lamb=lamb_hi) 170 | pred_mid = m.predict(x, lamb=lamb_mid) 171 | 172 | self.assertFalse(np.allclose(pred_lo, pred_mid)) 173 | self.assertFalse(np.allclose(pred_hi, pred_mid)) 174 | 175 | def test_lambda_clip_warning(self): 176 | x, y = self.inputs[0] 177 | m = ElasticNet(n_splits=0, random_state=1729) 178 | m = m.fit(x, y) 179 | 180 | # we should get a warning when we ask for predictions at values of 181 | # lambda outside the range of lambda_path_ 182 | with self.assertWarns(RuntimeWarning): 183 | # note, lambda_path_ is in decreasing order 184 | m.predict(x, lamb=m.lambda_path_[0] + 1) 185 | 186 | with self.assertWarns(RuntimeWarning): 187 | m.predict(x, lamb=m.lambda_path_[-1] - 1) 188 | 189 | def check_r2_score(self, y_true, y, at_least, **other_params): 190 | score = r2_score(y_true, y) 191 | msg = "expected r2 of {}, got: {}, with: {}".format(at_least, score, other_params) 192 | self.assertTrue(score > at_least, msg) 193 | 194 | def test_random_state_cv(self): 195 | random_state = 133 196 | m = ElasticNet(random_state=random_state) 197 | x, y = self.inputs[0] 198 | m.fit(x, y) 199 | print(dir(m._cv)) 200 | assert m._cv.random_state == random_state 201 | 202 | def test_max_features(self): 203 | x, y = self.inputs[3] 204 | max_features = 5 205 | m = ElasticNet(n_splits=3, random_state=42, max_features=max_features) 206 | m = m.fit(x, y) 207 | num_features = np.count_nonzero(m.coef_) 208 | self.assertTrue(num_features <= max_features) 209 | 210 | if __name__ == "__main__": 211 | unittest.main() 212 | -------------------------------------------------------------------------------- /glmnet/tests/test_logistic.py: -------------------------------------------------------------------------------- 1 | import itertools 2 | import unittest 3 | 4 | import numpy as np 5 | 6 | from scipy.sparse import csr_matrix 7 | 8 | from sklearn.datasets import make_classification 9 | from sklearn.metrics import accuracy_score, f1_score 10 | from sklearn.utils import estimator_checks, class_weight 11 | from sklearn.utils.testing import ignore_warnings 12 | 13 | from glmnet.tests.util import sanity_check_logistic 14 | 15 | from glmnet import LogitNet 16 | 17 | 18 | class TestLogitNet(unittest.TestCase): 19 | 20 | def setUp(self): 21 | np.random.seed(488881) 22 | # binomial 23 | x, y = make_classification(n_samples=300, random_state=6601) 24 | x_sparse = csr_matrix(x) 25 | 26 | x_wide, y_wide = make_classification(n_samples=100, n_features=150, 27 | random_state=8911) 28 | x_wide_sparse = csr_matrix(x_wide) 29 | self.binomial = [(x, y), (x_sparse, y), (x_wide, y_wide), 30 | (x_wide_sparse, y_wide)] 31 | 32 | # multinomial 33 | x, y = make_classification(n_samples=400, n_classes=3, n_informative=15, 34 | n_features=25, random_state=10585) 35 | x_sparse = csr_matrix(x) 36 | 37 | x_wide, y_wide = make_classification(n_samples=400, n_classes=3, 38 | n_informative=15, n_features=500, 39 | random_state=15841) 40 | x_wide_sparse = csr_matrix(x_wide) 41 | self.multinomial = [(x, y), (x_sparse, y), (x_wide, y_wide), 42 | (x_wide_sparse, y_wide)] 43 | 44 | self.alphas = [0., 0.25, 0.50, 0.75, 1.] 45 | self.n_splits = [-1, 0, 5] 46 | self.scoring = [ 47 | "accuracy", 48 | "roc_auc", 49 | "average_precision", 50 | "log_loss", 51 | "precision_macro", 52 | "precision_micro", 53 | "precision_weighted", 54 | "f1_micro", 55 | "f1_macro", 56 | "f1_weighted", 57 | ] 58 | self.multinomial_scoring = [ 59 | "accuracy", 60 | "log_loss", 61 | "precision_macro", 62 | "precision_micro", 63 | "precision_weighted", 64 | "f1_micro", 65 | "f1_macro", 66 | "f1_weighted" 67 | ] 68 | 69 | @ignore_warnings(category=RuntimeWarning) # convergence warnings from glmnet 70 | def test_estimator_interface(self): 71 | estimator_checks.check_estimator(LogitNet) 72 | 73 | def test_with_defaults(self): 74 | m = LogitNet(random_state=29341) 75 | for x, y in itertools.chain(self.binomial, self.multinomial): 76 | m = m.fit(x, y) 77 | sanity_check_logistic(m, x) 78 | 79 | # check selection of lambda_best 80 | assert m.lambda_best_inx_ <= m.lambda_max_inx_ 81 | 82 | # check full path predict 83 | p = m.predict(x, lamb=m.lambda_path_) 84 | assert p.shape[-1] == m.lambda_path_.size 85 | 86 | def test_one_row_predict(self): 87 | # Verify that predicting on one row gives only one row of output 88 | m = LogitNet(random_state=42) 89 | for X, y in itertools.chain(self.binomial, self.multinomial): 90 | m.fit(X, y) 91 | p = m.predict(X[0].reshape((1, -1))) 92 | assert p.shape == (1,) 93 | 94 | def test_one_row_predict_proba(self): 95 | # Verify that predict_proba on one row gives 2D output 96 | m = LogitNet(random_state=42) 97 | for X, y in itertools.chain(self.binomial, self.multinomial): 98 | m.fit(X, y) 99 | p = m.predict_proba(X[0].reshape((1, -1))) 100 | assert p.shape == (1, len(np.unique(y))) 101 | 102 | def test_one_row_predict_with_lambda(self): 103 | # One row to predict along with lambdas should give 2D output 104 | m = LogitNet(random_state=42) 105 | lamb = [0.01, 0.02, 0.04, 0.1] 106 | for X, y in itertools.chain(self.binomial, self.multinomial): 107 | m.fit(X, y) 108 | p = m.predict(X[0].reshape((1, -1)), lamb=lamb) 109 | assert p.shape == (1, len(lamb)) 110 | 111 | def test_one_row_predict_proba_with_lambda(self): 112 | # One row to predict_proba along with lambdas should give 3D output 113 | m = LogitNet(random_state=42) 114 | lamb = [0.01, 0.02, 0.04, 0.1] 115 | for X, y in itertools.chain(self.binomial, self.multinomial): 116 | m.fit(X, y) 117 | p = m.predict_proba(X[0].reshape((1, -1)), lamb=lamb) 118 | assert p.shape == (1, len(np.unique(y)), len(lamb)) 119 | 120 | def test_alphas(self): 121 | x, y = self.binomial[0] 122 | for alpha in self.alphas: 123 | m = LogitNet(alpha=alpha, random_state=41041) 124 | m = m.fit(x, y) 125 | check_accuracy(y, m.predict(x), 0.85, alpha=alpha) 126 | 127 | def test_coef_limits(self): 128 | x, y = self.binomial[0] 129 | lower_limits = np.repeat(-1, x.shape[1]) 130 | upper_limits = 0 131 | m = LogitNet(lower_limits=lower_limits, upper_limits=upper_limits, random_state=69265, alpha=0) 132 | m = m.fit(x, y) 133 | assert(np.all(m.coef_ >= -1)) 134 | assert(np.all(m.coef_ <= 0)) 135 | 136 | def test_relative_penalties(self): 137 | x, y = self.binomial[0] 138 | p = x.shape[1] 139 | 140 | # m1 no relative penalties applied 141 | m1 = LogitNet(alpha=1) 142 | m1.fit(x, y) 143 | 144 | # find the nonzero indices from LASSO 145 | nonzero = np.nonzero(m1.coef_[0]) 146 | 147 | # unpenalize those nonzero coefs 148 | penalty = np.repeat(1, p) 149 | penalty[nonzero] = 0 150 | 151 | # refit the model with the unpenalized coefs 152 | m2 = LogitNet(alpha=1) 153 | m2.fit(x, y, relative_penalties=penalty) 154 | 155 | # verify that the unpenalized coef ests exceed the penalized ones 156 | # in absolute value 157 | assert(np.all(np.abs(m1.coef_[0]) <= np.abs(m2.coef_[0]))) 158 | 159 | def test_n_splits(self): 160 | x, y = self.binomial[0] 161 | for n in self.n_splits: 162 | m = LogitNet(n_splits=n, random_state=46657) 163 | if n > 0 and n < 3: 164 | with self.assertRaisesRegexp(ValueError, 165 | "n_splits must be at least 3"): 166 | m = m.fit(x, y) 167 | else: 168 | m = m.fit(x, y) 169 | sanity_check_logistic(m, x) 170 | 171 | def test_cv_scoring(self): 172 | x, y = self.binomial[0] 173 | for method in self.scoring: 174 | m = LogitNet(scoring=method, random_state=52633) 175 | m = m.fit(x, y) 176 | check_accuracy(y, m.predict(x), 0.85, scoring=method) 177 | 178 | def test_cv_scoring_multinomial(self): 179 | x, y = self.multinomial[0] 180 | for method in self.scoring: 181 | m = LogitNet(scoring=method, random_state=488881) 182 | 183 | if method in self.multinomial_scoring: 184 | m = m.fit(x, y) 185 | check_accuracy(y, m.predict(x), 0.65, scoring=method) 186 | else: 187 | with self.assertRaises(ValueError): 188 | m.fit(x, y) 189 | 190 | def test_predict_without_cv(self): 191 | x, y = self.binomial[0] 192 | m = LogitNet(n_splits=0, random_state=399001) 193 | m = m.fit(x, y) 194 | 195 | # should not make prediction unless value is passed for lambda 196 | with self.assertRaises(ValueError): 197 | m.predict(x) 198 | 199 | def test_coef_interpolation(self): 200 | x, y = self.binomial[0] 201 | m = LogitNet(n_splits=0, random_state=561) 202 | m = m.fit(x, y) 203 | 204 | # predict for a value of lambda between two values on the computed path 205 | lamb_lo = m.lambda_path_[1] 206 | lamb_hi = m.lambda_path_[2] 207 | 208 | # a value not equal to one on the computed path 209 | lamb_mid = (lamb_lo + lamb_hi) / 2.0 210 | 211 | pred_lo = m.predict_proba(x, lamb=lamb_lo) 212 | pred_hi = m.predict_proba(x, lamb=lamb_hi) 213 | pred_mid = m.predict_proba(x, lamb=lamb_mid) 214 | 215 | self.assertFalse(np.allclose(pred_lo, pred_mid)) 216 | self.assertFalse(np.allclose(pred_hi, pred_mid)) 217 | 218 | def test_lambda_clip_warning(self): 219 | x, y = self.binomial[0] 220 | m = LogitNet(n_splits=0, random_state=1729) 221 | m = m.fit(x, y) 222 | 223 | with self.assertWarns(RuntimeWarning): 224 | m.predict(x, lamb=m.lambda_path_[0] + 1) 225 | 226 | with self.assertWarns(RuntimeWarning): 227 | m.predict(x, lamb=m.lambda_path_[-1] - 1) 228 | 229 | def test_single_class_exception(self): 230 | x, y = self.binomial[0] 231 | y = np.ones_like(y) 232 | m = LogitNet() 233 | 234 | with self.assertRaises(ValueError) as e: 235 | m.fit(x, y) 236 | 237 | self.assertEqual("Training data need to contain at least 2 classes.", 238 | str(e.exception)) 239 | 240 | def test_random_state_cv(self): 241 | random_state = 133 242 | m = LogitNet(random_state=random_state) 243 | x, y = self.binomial[0] 244 | m.fit(x, y) 245 | print(dir(m._cv)) 246 | assert m._cv.random_state == random_state 247 | 248 | def test_max_features(self): 249 | max_features = 5 250 | m = LogitNet(random_state=1, max_features=max_features) 251 | x, y = self.multinomial[3] 252 | m = m.fit(x, y) 253 | num_features = np.count_nonzero(m.coef_, axis=1) 254 | self.assertTrue(np.all(num_features <= max_features)) 255 | 256 | def test_use_sample_weights(self): 257 | x, y = self.multinomial[1] 258 | class_0_idx = np.where(y==0) 259 | to_drop = class_0_idx[0][:-3] 260 | to_keep = np.ones(len(y), dtype=bool) 261 | to_keep[to_drop] = False 262 | y = y[to_keep] 263 | x = x[to_keep, :] 264 | sample_weight = class_weight.compute_sample_weight('balanced', y) 265 | sample_weight[0] = 0. 266 | 267 | unweighted = LogitNet(random_state=2, scoring='f1_micro') 268 | unweighted = unweighted.fit(x, y) 269 | unweighted_acc = f1_score(y, unweighted.predict(x), sample_weight=sample_weight, 270 | average='micro') 271 | 272 | weighted = LogitNet(random_state=2, scoring='f1_micro') 273 | weighted = weighted.fit(x, y, sample_weight=sample_weight) 274 | weighted_acc = f1_score(y, weighted.predict(x), sample_weight=sample_weight, 275 | average='micro') 276 | 277 | self.assertTrue(weighted_acc >= unweighted_acc) 278 | 279 | 280 | def check_accuracy(y, y_hat, at_least, **other_params): 281 | score = accuracy_score(y, y_hat) 282 | msg = "expected accuracy of {}, got: {} with {}".format(at_least, score, other_params) 283 | assert score > at_least, msg 284 | 285 | 286 | if __name__ == "__main__": 287 | unittest.main() 288 | -------------------------------------------------------------------------------- /glmnet/tests/test_pandas.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | from sklearn.datasets import make_regression, make_classification 4 | from glmnet import LogitNet, ElasticNet 5 | 6 | from glmnet.tests.util import sanity_check_logistic, sanity_check_regression 7 | 8 | pd = None 9 | try: 10 | import pandas as pd 11 | except: 12 | pass 13 | 14 | 15 | class TestElasticNetPandas(unittest.TestCase): 16 | 17 | @unittest.skipUnless(pd, "pandas not available") 18 | def test_with_pandas_df(self): 19 | x, y = make_regression(random_state=561) 20 | df = pd.DataFrame(x) 21 | df['y'] = y 22 | 23 | m = ElasticNet(n_splits=3, random_state=123) 24 | m = m.fit(df.drop(['y'], axis=1), df.y) 25 | sanity_check_regression(m, x) 26 | 27 | 28 | class TestLogitNetPandas(unittest.TestCase): 29 | 30 | @unittest.skipUnless(pd, "pandas not available") 31 | def test_with_pandas_df(self): 32 | x, y = make_classification(random_state=1105) 33 | df = pd.DataFrame(x) 34 | df['y'] = y 35 | 36 | m = LogitNet(n_splits=3, random_state=123) 37 | m = m.fit(df.drop(['y'], axis=1), df.y) 38 | sanity_check_logistic(m, x) 39 | 40 | 41 | if __name__ == "__main__": 42 | unittest.main() 43 | -------------------------------------------------------------------------------- /glmnet/tests/test_util.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import numpy as np 4 | from numpy.testing import assert_warns 5 | 6 | from glmnet.util import _interpolate_model 7 | 8 | 9 | class TestUtils(unittest.TestCase): 10 | 11 | def test_interpolate_model_intercept_only(self): 12 | lambda_path = np.array((0.99,)) 13 | coef_path = np.random.random(size=(5, 1)) 14 | intercept_path = np.random.random(size=(1,)) 15 | 16 | # would be nice to use assertWarnsRegex to check the message, but this 17 | # fails due to http://bugs.python.org/issue20484 18 | assert_warns(RuntimeWarning, _interpolate_model, lambda_path, 19 | coef_path, intercept_path, 0.99) 20 | 21 | 22 | if __name__ == "__main__": 23 | unittest.main() 24 | -------------------------------------------------------------------------------- /glmnet/tests/util.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | 4 | def sanity_check_logistic(m, x): 5 | sanity_check_model_attributes(m) 6 | sanity_check_cv_attrs(m, is_clf=True) 7 | 8 | assert m.classes_ is not None 9 | assert m.coef_path_.ndim == 3, "wrong number of dimensions for coef_path_" 10 | 11 | n_classes = len(m.classes_) 12 | if len(m.classes_) == 2: # binomial is a special case 13 | n_classes = 1 14 | assert m.coef_path_.shape[0] == n_classes, "wrong size for coef_path_" 15 | 16 | assert m.intercept_path_.ndim == 2, "wrong number of dimensions for intercept_path_" 17 | 18 | # check preds at random value of lambda 19 | l = np.random.choice(m.lambda_path_) 20 | p = m.predict(x, lamb=l) 21 | check_logistic_predict(m, x, p) 22 | 23 | p = m.predict_proba(x, lamb=l) 24 | check_logistic_predict_proba(m, x, p) 25 | 26 | # if cv ran, check default behavior of predict and predict_proba 27 | if m.n_splits >= 3: 28 | p = m.predict(x) 29 | check_logistic_predict(m, x, p) 30 | 31 | p = m.predict_proba(x) 32 | check_logistic_predict_proba(m, x, p) 33 | 34 | 35 | def check_logistic_predict(m, x, p): 36 | assert p.shape[0] == x.shape[0], "%r != %r" % (p.shape[0], x.shape[0]) 37 | assert np.all(np.in1d(np.unique(p),m.classes_)) 38 | 39 | 40 | def check_logistic_predict_proba(m, x, p): 41 | assert p.shape[0] == x.shape[0] 42 | assert p.shape[1] == len(m.classes_) 43 | assert np.all(p >= 0) and np.all(p <= 1.), "predict_proba values outside [0,1]" 44 | 45 | 46 | def sanity_check_regression(m, x): 47 | sanity_check_model_attributes(m) 48 | sanity_check_cv_attrs(m) 49 | 50 | assert m.coef_path_.ndim == 2, "wrong number of dimensions for coef_path_" 51 | assert m.intercept_path_.ndim == 1, "wrong number of dimensions for intercept_path_" 52 | 53 | # check predict at random value of lambda 54 | l = np.random.choice(m.lambda_path_) 55 | p = m.predict(x, lamb=l) 56 | assert p.shape[0] == x.shape[0] 57 | 58 | # if cv ran, check default behavior of predict 59 | if m.n_splits >= 3: 60 | p = m.predict(x) 61 | assert p.shape[0] == x.shape[0] 62 | 63 | 64 | def sanity_check_model_attributes(m): 65 | assert m.n_lambda_ > 0, "n_lambda_ is not set" 66 | assert m.lambda_path_.size == m.n_lambda_, "lambda_path_ does not have length n_lambda_" 67 | assert m.coef_path_.shape[-1] == m.n_lambda_, "wrong size for coef_path_" 68 | assert m.intercept_path_.shape[-1] == m.n_lambda_, "wrong size for intercept_path_" 69 | assert m.jerr_ == 0, "jerr is non-zero" 70 | 71 | 72 | def sanity_check_cv_attrs(m, is_clf=False): 73 | if m.n_splits >= 3: 74 | if is_clf: 75 | assert m.coef_.shape[-1] == m.coef_path_.shape[1], "wrong size for coef_" 76 | else: 77 | assert m.coef_.size == m.coef_path_.shape[0], "wrong size for coef_" 78 | assert m.intercept_ is not None, "intercept_ is not set" 79 | assert m.cv_mean_score_.size == m.n_lambda_, "wrong size for cv_mean_score_" 80 | assert m.cv_standard_error_.size == m.n_lambda_, "wrong size for cv_standard_error_" 81 | assert m.lambda_max_ is not None, "lambda_max_ is not set" 82 | assert (m.lambda_max_inx_ >= 0 and m.lambda_max_inx_ < m.n_lambda_, 83 | "lambda_max_inx_ is outside bounds of lambda_path_") 84 | assert m.lambda_best_ is not None, "lambda_best_ is not set" 85 | assert (m.lambda_best_inx_ >= 0 and m.lambda_best_inx_ < m.n_lambda_, 86 | "lambda_best_inx_ is outside bounds of lambda_path_") 87 | -------------------------------------------------------------------------------- /glmnet/util.py: -------------------------------------------------------------------------------- 1 | import math 2 | import warnings 3 | 4 | import numpy as np 5 | 6 | from scipy.interpolate import interp1d 7 | 8 | from sklearn.base import clone 9 | from sklearn.exceptions import UndefinedMetricWarning 10 | from joblib import Parallel, delayed 11 | 12 | from glmnet.scorer import check_scoring 13 | 14 | 15 | def _score_lambda_path(est, X, y, groups, sample_weight, relative_penalties, 16 | scoring, n_jobs, verbose): 17 | """Score each model found by glmnet using cross validation. 18 | 19 | Parameters 20 | ---------- 21 | est : estimator 22 | The previously fitted estimator. 23 | 24 | X : array, shape (n_samples, n_features) 25 | Input features 26 | 27 | y : array, shape (n_samples,) 28 | Target values. 29 | 30 | groups: array, shape (n_samples,) 31 | Group labels for the samples used while splitting the dataset into train/test set. 32 | 33 | sample_weight : array, shape (n_samples,) 34 | Weight of each row in X. 35 | 36 | relative_penalties: array, shape (n_features,) 37 | Relative weight vector for penalty. 38 | 0 entries remove penalty. 39 | 40 | scoring : string, callable or None 41 | Scoring method to apply to each model. 42 | 43 | n_jobs: int 44 | Maximum number of threads to use for scoring models. 45 | 46 | verbose : bool 47 | Emit logging data and warnings when True. 48 | 49 | Returns 50 | ------- 51 | scores : array, shape (n_lambda,) 52 | Scores for each value of lambda over all cv folds. 53 | """ 54 | scorer = check_scoring(est, scoring) 55 | cv_split = est._cv.split(X, y, groups) 56 | 57 | # We score the model for every value of lambda, for classification 58 | # models, this will be an intercept-only model, meaning it predicts 59 | # the same class regardless of the input. Obviously, this makes some of 60 | # the scikit-learn metrics unhappy, so we are silencing these warnings. 61 | # Also note, catch_warnings is not thread safe. 62 | with warnings.catch_warnings(): 63 | action = 'always' if verbose else 'ignore' 64 | warnings.simplefilter(action, UndefinedMetricWarning) 65 | 66 | scores = Parallel(n_jobs=n_jobs, verbose=verbose, backend='threading')( 67 | delayed(_fit_and_score)(est, scorer, X, y, sample_weight, relative_penalties, 68 | est.lambda_path_, train_idx, test_idx) 69 | for (train_idx, test_idx) in cv_split) 70 | 71 | return scores 72 | 73 | 74 | def _fit_and_score(est, scorer, X, y, sample_weight, relative_penalties, 75 | score_lambda_path, train_inx, test_inx): 76 | """Fit and score a single model. 77 | 78 | Parameters 79 | ---------- 80 | est : estimator 81 | The previously fitted estimator. 82 | 83 | scorer : callable 84 | The scoring function to apply to each model. 85 | 86 | X : array, shape (n_samples, n_features) 87 | Input features 88 | 89 | y : array, shape (n_samples,) 90 | Target values. 91 | 92 | sample_weight : array, shape (n_samples,) 93 | Weight of each row in X. 94 | 95 | relative_penalties: array, shape (n_features,) 96 | Relative weight vector for penalty. 97 | 0 entries remove penalty. 98 | 99 | score_lambda_path : array, shape (n_lambda,) 100 | The lambda values to evaluate/score. 101 | 102 | train_inx : array, shape (n_train,) 103 | Array of integers indicating which rows from X, y are in the training 104 | set for this fold. 105 | 106 | test_inx : array, shape (n_test,) 107 | Array of integers indicating which rows from X, y are in the test 108 | set for this fold. 109 | 110 | Returns 111 | ------- 112 | scores : array, shape (n_lambda,) 113 | Scores for each value of lambda for a single cv fold. 114 | """ 115 | m = clone(est) 116 | m = m._fit(X[train_inx, :], y[train_inx], sample_weight[train_inx], relative_penalties) 117 | 118 | lamb = np.clip(score_lambda_path, m.lambda_path_[-1], m.lambda_path_[0]) 119 | return scorer(m, X[test_inx, :], y[test_inx], lamb=lamb) 120 | 121 | 122 | def _fix_lambda_path(lambda_path): 123 | """Replace the first value in lambda_path (+inf) with something more 124 | reasonable. The method below matches what is done in the R/glmnent wrapper.""" 125 | if lambda_path.shape[0] > 2: 126 | lambda_0 = math.exp(2 * math.log(lambda_path[1]) - math.log(lambda_path[2])) 127 | lambda_path[0] = lambda_0 128 | return lambda_path 129 | 130 | 131 | def _check_user_lambda(lambda_path, lambda_best=None, lamb=None): 132 | """Verify the user-provided value of lambda is acceptable and ensure this 133 | is a 1-d array. 134 | 135 | Parameters 136 | ---------- 137 | lambda_path : array, shape (n_lambda,) 138 | The path of lambda values as found by glmnet. This must be in 139 | decreasing order. 140 | 141 | lambda_best : float, optional 142 | The value of lambda producing the highest scoring model (found via 143 | cross validation). 144 | 145 | lamb : float, array, or None 146 | The value(s) of lambda for which predictions are desired. This must 147 | be provided if `lambda_best` is None, meaning the model was fit with 148 | cv_folds < 1. 149 | 150 | Returns 151 | ------- 152 | lamb : array, shape (n_lambda,) 153 | The value(s) of lambda, potentially clipped to the range of values in 154 | lambda_path. 155 | """ 156 | 157 | if lamb is None: 158 | if lambda_best is None: 159 | raise ValueError("You must specify a value for lambda or run " 160 | "with cv_folds > 1 to select a value " 161 | "automatically.") 162 | lamb = lambda_best 163 | 164 | # ensure numpy math works later 165 | lamb = np.array(lamb, ndmin=1) 166 | if np.any(lamb < lambda_path[-1]) or np.any(lamb > lambda_path[0]): 167 | warnings.warn("Some values of lamb are outside the range of " 168 | "lambda_path_ [{}, {}]".format(lambda_path[-1], 169 | lambda_path[0]), 170 | RuntimeWarning) 171 | np.clip(lamb, lambda_path[-1], lambda_path[0], lamb) 172 | 173 | return lamb 174 | 175 | 176 | def _interpolate_model(lambda_path, coef_path, intercept_path, lamb): 177 | """Interpolate coefficients and intercept between values of lambda. 178 | 179 | Parameters 180 | ---------- 181 | lambda_path : array, shape (n_lambda,) 182 | The path of lambda values as found by glmnet. This must be in 183 | decreasing order. 184 | 185 | coef_path : array, shape (n_features, n_lambda) or 186 | (n_classes, n_features, n_lambda) 187 | The path of coefficients as found by glmnet. 188 | 189 | intercept_path : array, shape (n_lambda,) or (n_classes, n_lambda) 190 | The path of intercepts as found by glmnet. 191 | 192 | Returns 193 | ------- 194 | coef : array, shape (n_features, n_lambda) or 195 | (n_classes, n_features, n_lambda) 196 | The interpolated path of coefficients. 197 | 198 | intercept : array, shape (n_lambda,) or (n_classes, n_lambda) 199 | The interpolated path of intercepts. 200 | """ 201 | if lambda_path.shape[0] == 1: 202 | warnings.warn("lambda_path has a single value, this may be an " 203 | "intercept-only model.", RuntimeWarning) 204 | coef = np.take(coef_path, 0, axis=-1) 205 | intercept = np.take(intercept_path, 0, axis=-1) 206 | else: 207 | coef = interp1d(lambda_path, coef_path)(lamb) 208 | intercept = interp1d(lambda_path, intercept_path)(lamb) 209 | 210 | return coef, intercept 211 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | 4 | # `Extension` from setuptools cannot compile Fortran code, so we have to use 5 | # the one from numpy. To do so, we also need to use the `setup` function 6 | # from numpy, not from setuptools. 7 | # Confusingly, we need to explicitly import setuptools *before* importing 8 | # from numpy, so that numpy can internally detect and use the `setup` function 9 | # from setuptools. 10 | # References: https://stackoverflow.com/a/51691203 11 | # and https://stackoverflow.com/a/55358607 12 | import setuptools # noqa: F401 13 | try: 14 | from numpy.distutils.core import Extension, setup 15 | except ImportError: 16 | sys.exit("install requires: 'numpy'." 17 | " use pip or easy_install." 18 | " \n $ pip install numpy") 19 | 20 | 21 | _VERSION = "2.2.1" 22 | 23 | f_compile_args = ['-ffixed-form', '-fdefault-real-8'] 24 | 25 | 26 | def read(fname): 27 | with open(os.path.join(os.path.dirname(__file__), fname)) as _in: 28 | return _in.read() 29 | 30 | 31 | def get_lib_dir(dylib): 32 | import subprocess 33 | from os.path import realpath, dirname 34 | 35 | p = subprocess.Popen("gfortran -print-file-name={}".format(dylib), 36 | stdout=subprocess.PIPE, stderr=subprocess.PIPE, 37 | shell=True) 38 | retcode = p.wait() 39 | if retcode != 0: 40 | raise Exception("Failed to find {}".format(dylib)) 41 | 42 | libdir = dirname(realpath(p.communicate()[0].strip().decode('ascii'))) 43 | 44 | return libdir 45 | 46 | 47 | if sys.platform == 'darwin': 48 | GFORTRAN_LIB = get_lib_dir('libgfortran.3.dylib') 49 | QUADMATH_LIB = get_lib_dir('libquadmath.0.dylib') 50 | ARGS = ["-Wl,-rpath,{}:{}".format(GFORTRAN_LIB, QUADMATH_LIB)] 51 | f_compile_args += ARGS 52 | library_dirs = [GFORTRAN_LIB, QUADMATH_LIB] 53 | else: 54 | library_dirs = None 55 | 56 | 57 | glmnet_lib = Extension(name='_glmnet', 58 | sources=['glmnet/_glmnet.pyf', 59 | 'glmnet/src/glmnet/glmnet5.f90'], 60 | extra_f90_compile_args=f_compile_args, 61 | library_dirs=library_dirs, 62 | ) 63 | 64 | if __name__ == "__main__": 65 | setup(name="glmnet", 66 | version=_VERSION, 67 | description="Python wrapper for glmnet", 68 | long_description=read('README.rst'), 69 | long_description_content_type="text/x-rst", 70 | author="Civis Analytics Inc", 71 | author_email="opensource@civisanalytics.com", 72 | url="https://github.com/civisanalytics/python-glmnet", 73 | install_requires=[ 74 | "numpy>=1.9.2", 75 | "scikit-learn>=0.18.0", 76 | "scipy>=0.14.1", 77 | "joblib>=0.14.1", 78 | ], 79 | python_requires=">=3.6.*", 80 | # We need pkg_resources, shipped with setuptools, 81 | # for version numbering. 82 | setup_requires=["setuptools"], 83 | ext_modules=[glmnet_lib], 84 | packages=['glmnet'], 85 | classifiers=[ 86 | 'Development Status :: 5 - Production/Stable', 87 | 'Environment :: Console', 88 | 'Programming Language :: Python', 89 | 'Programming Language :: Python :: 3', 90 | 'Programming Language :: Python :: 3.6', 91 | 'Programming Language :: Python :: 3.7', 92 | 'Programming Language :: Python :: 3.8', 93 | 'Programming Language :: Python :: 3 :: Only', 94 | 'Operating System :: OS Independent', 95 | 'Intended Audience :: Developers', 96 | 'Intended Audience :: Science/Research', 97 | 'License :: OSI Approved :: GNU General Public License v2 (GPLv2)', 98 | 'Topic :: Scientific/Engineering' 99 | ]) 100 | --------------------------------------------------------------------------------