├── .gitignore ├── LICENSE ├── README.md ├── __init__.py ├── cv ├── __init__.py ├── cv_glmnet.py ├── fit_and_scorers.py └── fold_generators.py ├── examples ├── __init__.py ├── example_enet.py ├── example_enet_cv.py ├── example_lognet.py └── example_lognet_cv.py ├── glmnet ├── __init__.py ├── build.sh ├── elastic_net.py ├── glmnet.f ├── glmnet.py ├── glmnet.pyf ├── glmnet_config.py └── logistic_net.py ├── images ├── oof_dev.png └── reg_paths.png ├── setup.py ├── tests ├── __init__.py ├── test_elastic_net.py └── test_logistic_net.py └── util ├── __init__.py └── importers.py /.gitignore: -------------------------------------------------------------------------------- 1 | build/ 2 | *.pyc 3 | 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Glmnet Wrappers for Python 2 | ========================== 3 | 4 | This package provides a convenient python interface to Jerome Friedman's 5 | Fortran implementation of glmnet. It provides Elastic and Logistic net models, 6 | cross validation, and plotting utilities. 7 | 8 | It is very much a work in progress. 9 | 10 | Authors 11 | ------- 12 | 13 | * David Warde-Farley (2010) 14 | * Matthew Drury (2014) 15 | 16 | Use 17 | --- 18 | 19 | To create an elastic net model: 20 | 21 | ``` 22 | from glmnet import ElasticNet 23 | enet = ElasticNet(alpha=.1) 24 | ``` 25 | 26 | (Here, `alpha` is the relative weighting between the `L1` and `L2` 27 | regularization terms). The model can then be fit using a design matrix `X` and 28 | a response vector `y`: 29 | 30 | ``` 31 | enet.fit(X, y) 32 | ``` 33 | 34 | glmnet also accepts sparse design matrices, using the compressed sparse 35 | column format: 36 | 37 | ``` 38 | from scipy.sparse import csc_matrix 39 | Xsp = csc_matrix(X) 40 | enet.fit(Xsp, y) 41 | ``` 42 | 43 | Fitting a glmnet automatically fits models for multiple values of `lambda`, 44 | the overall amount of regularization. 45 | 46 | After fitting, the model can be used to generate predictions on new data: 47 | 48 | ``` 49 | enet.predict(X') 50 | ``` 51 | 52 | note that this generates predictions for each lambda glmnet chose during 53 | fitting. 54 | 55 | or get a summary of the model: 56 | 57 | ``` 58 | enet.describe() 59 | ``` 60 | 61 | ``` 62 | A elastic net model with alpha = 0.1. 63 | The model was fit on 100 observations and 5 parameters. 64 | The model was fit in 169 passes over the data. 65 | There were 77 values of lambda resulting in non-zero models. 66 | There were 4 non-zero coefficients in the largest model. 67 | ``` 68 | 69 | Passing a lambda index into describe will display the fit coefficients: 70 | 71 | ``` 72 | enet.describe(lidx=20) 73 | ``` 74 | 75 | ``` 76 | Parameter Estiamtes for elastic net model with lambda = 7.12026e-02 77 | Varaible Name Coefficent Estiamte 78 | C(catagorical_var)[A] -2.17143e-01 79 | C(catagorical_var)[B] 2.13176e-01 80 | first_var 5.43638e-01 81 | another_var 0.00000e+00 82 | the_last_var 2.72075e-01 83 | ``` 84 | 85 | The parameter paths can also be visualized, that is, the values of the model 86 | parameters for each `lambda`: 87 | 88 | ``` 89 | enet.plot_paths() 90 | ``` 91 | 92 | ![Param-Plot](https://raw.githubusercontent.com/madrury/glmnet-python/master/images/reg_paths.png) 93 | 94 | To select a value of `lambda` cross-validation is provided: 95 | 96 | ``` 97 | from glmnet import ElasticNet, CVGlmNet 98 | enet = ElasticNet(alpha=.1) 99 | enet_cv = CVGlmNet(enet, folds=10, n_jobs=10) 100 | enet_cv.fit(X, y) 101 | ``` 102 | 103 | Glmnet then fits ten models for each value of `lambda` and chooses the best 104 | model by observing which optimizes the out of fold deviance. 105 | 106 | Once the cross validation is fit, the mean out of fold deviances for each value 107 | of `lambda` can be viewed, along with their standard deviations: 108 | 109 | ``` 110 | enet_cv.plot_oof_devs() 111 | ``` 112 | 113 | ![OOF-Dev-Plot](https://raw.githubusercontent.com/madrury/glmnet-python/master/images/oof_dev.png) 114 | The cross validation object can then be used to generate predictions at the 115 | optimal value of `lambda`: 116 | 117 | ``` 118 | enet_cv.predict(X') 119 | ``` 120 | 121 | **Note**: glmnet uses the joblib.Parallel function to parallelize its fitting 122 | across folds, there is a known bug in some versions of OSX where using this 123 | causes a race condition and the fitting will hang. Setting `n_jobs=1` will 124 | disable the cross validation, at the expense of fitting the models in series. 125 | If using the anaconda distribution of python, enabling the mkl optimizations 126 | will allow the models to be fit in parallel. The parallelization has also been 127 | tested on various linux boxes with no issues. See 128 | [this sklearn issue](https://github.com/scikit-learn/scikit-learn/issues/636) for more 129 | information. 130 | 131 | Building 132 | -------- 133 | 134 | A build script has been provided in the `glmnet/glmnet` directory, so to build 135 | the fortran extension: 136 | 137 | ``` 138 | cd glmnet/glmnet 139 | source build.sh 140 | ``` 141 | Once the build is complete, you should have a `_glmnet.so` file in the glmnet 142 | directory. Dropping into a python shell: 143 | 144 | ``` 145 | import _glmnet 146 | ``` 147 | 148 | Should work without error. 149 | 150 | AFAIK, this package requires `gfortran` to build. `g77` will not work as 151 | it does not support `-fdefault-real-8`. 152 | 153 | In order to get double precision working without modifying Friedman's code, 154 | some compiler trickery is required. The wrappers have been written such that 155 | everything returned is expected to be a `real*8` i.e. a double-precision 156 | floating point number, and unfortunately the code is written in a way 157 | Fortran is often written with simply `real` specified, letting the compiler 158 | decide on the appropriate width. `f2py` assumes `real` are always 4 byte/ 159 | single precision, hence the manual change in the wrappers to `real*8`, but 160 | that change requires the actual Fortran code to be compiled with 8-byte reals, 161 | otherwise bad things will happen (the stack will be blown, program will hang 162 | or segfault, etc.). 163 | 164 | Planned Enhancements 165 | -------------------- 166 | 167 | * Improved interface with cross validation. 168 | * Pretty graphs with better knobs and dials. 169 | * Wrapper classes for the Poisson and Cox models. 170 | 171 | License 172 | ------- 173 | 174 | Friedman's code in `glmnet.f` is released under the GPLv2, necessitating that 175 | any code that uses it (including my wrapper, and anyone using my wrapper) 176 | be released under the GPLv2 as well. See LICENSE for details. 177 | 178 | That said, to the extent that they are useful in the absence of the GPL Fortran 179 | code (i.e. not very), the other portions may be used under the 3-clause BSD 180 | license. 181 | 182 | Thanks 183 | ------ 184 | 185 | * To Jerome Friedman for the fantastically fast and efficient Fortran code. 186 | * To Pearu Peterson for writing `f2py` and answering my dumb questions. 187 | * To Dag Sverre Seljebotn for his help with `f2py` wrangling. 188 | * To Kevin Jacobs for showing me [his 189 | wrapper](http://code.google.com/p/glu-genetics/source/browse/trunk/glu/lib/glm/glmnet.pyf) 190 | which helped me side-step some problems with the auto-generated `.pyf`. 191 | 192 | References 193 | ---------- 194 | 195 | * J Friedman, T Hastie, R Tibshirani (2010). ["Regularization Paths for 196 | Generalized Linear Models via Coordinate 197 | Descent"](http://www.jstatsoft.org/v33/i01/paper). 198 | * J Friedman, T Hastie, H Hofling, R Tibshirani (2007). ["Pathwise Coordinate 199 | Optimization"](http://arxiv.org/pdf/0708.1485.pdf"). 200 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | from glmnet.elastic_net import ElasticNet 2 | from glmnet.logistic_net import LogisticNet 3 | from cv.cv_glmnet import CVGlmNet 4 | -------------------------------------------------------------------------------- /cv/__init__.py: -------------------------------------------------------------------------------- 1 | from cv_glmnet import CVGlmNet 2 | -------------------------------------------------------------------------------- /cv/cv_glmnet.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from fit_and_scorers import fit_and_score_switch 3 | from fold_generators import KFold 4 | 5 | from ..util.importers import import_joblib, import_pyplot 6 | jl = import_joblib() 7 | plt = import_pyplot() 8 | 9 | 10 | def _clone(glmnet): 11 | '''Make a copy of an unfit glmnet object.''' 12 | glmnet._check_if_unfit() 13 | return glmnet.__class__(**glmnet.__dict__) 14 | 15 | 16 | class CVGlmNet(object): 17 | '''Manage the optimization of the lambda parameter by utilizing cross 18 | validation. 19 | 20 | This class provides a management scheme for glmnets that abstracts 21 | away the process of determining an optimimal choice of the lambda 22 | parameter in a glmnet. It: 23 | 24 | * Provides a fit method that fits many glments over the same set of 25 | lambdas, but each on a random cross-fold of the training data 26 | provided. 27 | 28 | and after the CVGlmNet object has been fit: 29 | 30 | * Provides routines to introspect, score and visualize the resulting 31 | optimal model. 32 | * Calculate and visualize statistics pertaining to the cross validation 33 | process. 34 | 35 | Both weighted and unweighted training samples are supported, and the various 36 | glmnet models can be fit in parallel. 37 | ''' 38 | 39 | def __init__(self, glmnet, 40 | n_folds=3, n_jobs=1, include_full=True, 41 | shuffle=True, cv_folds=None, verbose=1 42 | ): 43 | '''Create a cross validation glmnet object. Accepts the following 44 | arguments: 45 | 46 | * glmnet: 47 | An object derived from the GlmNet class, currently either an 48 | ElasticNet or LogisticNet object. 49 | * n_folds: 50 | The number of cross validation folds to use when fitting. 51 | Ignored if the cv_folds argument is given. 52 | * include_full: 53 | Include the fitting of the full model during the cross 54 | validation, using a special 'all the data' fold. Allows the 55 | full model fit to participate in parallelism. 56 | * n_jobs: 57 | The number of cores to distribute the work to. 58 | * shuffle: 59 | Boolean, should the indicies of the cross validation 60 | folds be shuffled randomly. 61 | Ignored if the cv_folds argument is given. 62 | * cv_folds: 63 | A fold object. If not passed, will use n_folds and shuffle 64 | arguments to generate the folds. 65 | * verbose: 66 | Amount of talkyness. 67 | ''' 68 | if cv_folds: 69 | self.fold_generator = cv_folds 70 | n_folds = cv_folds.n_folds 71 | shuffle = cv_folds.shuffle 72 | else: 73 | self.fold_generator = None 74 | if n_jobs > 1 and not jl: 75 | raise ValueError( 76 | "joblib.Parallel not available, must set n_folds=1" 77 | ) 78 | self.base_estimator = glmnet 79 | self.n_folds = n_folds 80 | self.n_jobs = n_jobs 81 | self.include_full = include_full 82 | self.shuffle = shuffle 83 | self.verbose = verbose 84 | self._fit = False 85 | 86 | def fit(self, X, y, weights=None, lambdas=None, **kwargs): 87 | '''Determine the optimal value of lambda by cross validation. 88 | 89 | This method fits n_fold glmnet objects, each for the same sequence 90 | of lambdas, and each on a different subset of the training data. The 91 | resulting models are then scored on the held-out-from-fold data at each 92 | lambda, and the lambda that minimizes the mean out of fold deviance is 93 | found and memorized. Subsequently, the cross validation object can me 94 | scored and introspected, always using the optimal lambda we. 95 | ''' 96 | self._check_if_unfit() 97 | 98 | if not self.fold_generator: 99 | self.fold_generator = KFold( 100 | X.shape[0], 101 | n_folds=self.n_folds, 102 | shuffle=self.shuffle, 103 | include_full=self.include_full, 104 | weights=weights, 105 | ) 106 | # Get the appropriate fitter function. 107 | fitter = fit_and_score_switch[self.base_estimator.__class__.__name__] 108 | # Determine the sequence of lambda values to fit using cross validation. 109 | if lambdas is None: 110 | lmax = self.base_estimator._max_lambda(X, y, weights=weights) 111 | lmin = self.base_estimator.frac_lg_lambda * lmax 112 | lambdas = np.logspace(start = np.log10(lmin), 113 | stop = np.log10(lmax), 114 | num = self.base_estimator.n_lambdas, 115 | )[::-1] 116 | # Fit in-fold glmnets in parallel. For each such model, a tuple 117 | # containing the fitted model, and the models out-of-fold deviances 118 | # (one deviance per lambda). 119 | if jl: 120 | models_and_devs = ( 121 | jl.Parallel(n_jobs=self.n_jobs, verbose=self.verbose)( 122 | jl.delayed(fitter)(_clone(self.base_estimator), 123 | X, y, 124 | train_inds, valid_inds, 125 | weights, lambdas, 126 | **kwargs 127 | ) 128 | for train_inds, valid_inds in self.fold_generator 129 | ) 130 | ) 131 | else: 132 | models_and_devs = ([ 133 | fitter(_clone(self.base_estimator), 134 | X, y, 135 | train_inds, valid_inds, 136 | weights, lambdas, 137 | **kwargs 138 | ) 139 | for train_inds, valid_inds in self.fold_generator 140 | ]) 141 | # If the full model was fit by Parallel, then pull it off of the 142 | # object returned from Parallel. 143 | if self.include_full: 144 | self.base_estimator = models_and_devs[0][0] 145 | # If the full model has not been fit yet, do it now. 146 | if not self.base_estimator._is_fit(): 147 | self.base_estimator.fit(X, y, weights=weights, lambdas=lambdas, **kwargs) 148 | lambdas = self.base_estimator.out_lambdas 149 | # Unzip and finish up! 150 | self._models, self._all_deviances = zip(*models_and_devs[1:]) 151 | self.lambdas = lambdas 152 | self._fit = True 153 | 154 | @property 155 | def _oof_deviances(self): 156 | dev_stack = np.vstack(self._all_deviances) 157 | oof_deviances = np.mean(dev_stack, axis=0) 158 | return oof_deviances 159 | 160 | @property 161 | def _oof_stds(self): 162 | dev_stack = np.vstack(self._all_deviances) 163 | oof_stds = np.std(dev_stack, axis=0) 164 | return oof_stds 165 | 166 | @property 167 | def best_lambda_idx(self): 168 | self._check_if_fit() 169 | best_lambda_idx = np.argmin(self._oof_deviances) 170 | return best_lambda_idx 171 | 172 | @property 173 | def best_lambda(self): 174 | self._check_if_fit() 175 | return self.lambdas[self.best_lambda_idx] 176 | 177 | def predict(self, X): 178 | '''Score the optimal model given a model matrix.''' 179 | self._check_if_fit() 180 | return self.base_estimator.predict(X)[:, self.best_lambda_idx].ravel() 181 | 182 | def _describe_best_est(self): 183 | '''Describe the optimal model.''' 184 | self._check_if_fit() 185 | return self.base_estimator.describe(lidx=self.best_lambda_idx) 186 | 187 | def _describe_cv(self): 188 | s = ("A glmnet model with optimal lambda determined by cross " 189 | "validation.\n" 190 | "The model was fit on {0} folds of the training data.\n" 191 | "The model was fit on {1} lambdas, with the optimal value " 192 | "determined to be {2}.\n").format(self.n_folds, 193 | len(self.lambdas), 194 | self.best_lambda 195 | ) 196 | return s 197 | 198 | def describe(self): 199 | s = (self._describe_cv() + 200 | '-'*79 + '\n' + "Best Model:\n" + 201 | self._describe_best_est()) 202 | return s 203 | 204 | def plot_oof_devs(self): 205 | '''Produce a plot of the mean out of fold deviance for each lambda, 206 | with error bars showing the standard deviation of these devinaces 207 | across folds. 208 | ''' 209 | self._check_if_fit() 210 | if not plt: 211 | raise RuntimeError('pyplot unavailable.') 212 | 213 | plt.clf() 214 | fig, ax = plt.subplots() 215 | xvals = np.log(self.lambdas) 216 | ax.plot(xvals, self._oof_deviances, c='blue') 217 | ax.scatter(xvals, self._oof_deviances, 218 | s=3, c='blue', alpha=.5 219 | ) 220 | ax.errorbar(xvals, self._oof_deviances, yerr=self._oof_stds, 221 | c='grey', alpha=.5 222 | ) 223 | ax.axvline(np.log(self.best_lambda), alpha=.5) 224 | ax.set_title("Cross validation estimates of out of sample deviance.") 225 | ax.set_xlabel("log(lambda)") 226 | ax.set_ylabel("Deviance") 227 | plt.show() 228 | 229 | def _is_fit(self): 230 | return self._fit 231 | 232 | def _check_if_fit(self): 233 | if not self._is_fit(): 234 | raise RuntimeError("Operation is not supported with a fit model.") 235 | 236 | def _check_if_unfit(self): 237 | if self._is_fit(): 238 | raise RuntimeError("Operation is not supported with an unfit " 239 | "model." 240 | ) 241 | -------------------------------------------------------------------------------- /cv/fit_and_scorers.py: -------------------------------------------------------------------------------- 1 | '''A module containing utility routines to: 2 | 3 | * fit a glmnet model on a subset of a training data set. 4 | * score a glmnet model on a subset of a valation set. 5 | * return the deviances as estiamted using the validation set. 6 | 7 | these are indended to be mapped across a specification of cross validation 8 | folds in cv_glmnet. 9 | ''' 10 | 11 | def fit_and_score_elastic_net(elastic_net, X, y, 12 | train_inds, valid_inds, 13 | weights, lambdas, 14 | **kwargs): 15 | if weights is not None: 16 | train_weights = weights[train_inds] 17 | valid_weights = weights[valid_inds] 18 | else: 19 | train_weights, valid_weights = None, None 20 | elastic_net.fit(X[train_inds], y[train_inds], 21 | weights=train_weights, lambdas=lambdas, 22 | **kwargs 23 | ) 24 | if valid_inds.shape[0] != 0: 25 | valid_dev = elastic_net.deviance(X[valid_inds], y[valid_inds], 26 | weights=valid_weights 27 | ) 28 | else: 29 | valid_dev = None 30 | 31 | return (elastic_net, valid_dev) 32 | 33 | def fit_and_score_logistic_net(logistic_net, X, y, 34 | train_inds, valid_inds, 35 | weights, lambdas, 36 | **kwargs): 37 | logistic_net.fit(X[train_inds], y[train_inds], lambdas=lambdas, **kwargs) 38 | if valid_inds.shape[0] != 0: 39 | valid_dev = logistic_net.deviance(X[valid_inds], y[valid_inds]) 40 | else: 41 | valid_dev = None 42 | 43 | return (logistic_net, valid_dev) 44 | 45 | fit_and_score_switch = {'ElasticNet': fit_and_score_elastic_net, 46 | 'LogisticNet': fit_and_score_logistic_net 47 | } 48 | -------------------------------------------------------------------------------- /cv/fold_generators.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | class KFold(object): 4 | '''Manages cross validation fold indices. 5 | 6 | This calss exposes an interface for creating iteratiors that yield 7 | indices of cross validation folds. The canonical usage goes like this: 8 | 9 | kf = KFold(n=) 10 | for train_idxs, valid_idxs in kf: 11 | train_data_for_fold = data[train_idxs,:] 12 | validation_data_for_fold = data[valid_idxs,:] 13 | 14 | The class conforms to the api of sklearn.cross_validation.KFold, but also 15 | allows the user to supply sample weights, in this case the generated 16 | folds will be balanced by total weight instead of number of samples. 17 | 18 | The edge cases are balanced so that in the case where n_folds divides n 19 | exactly, each fold will contain exactly n / n_folds samples. 20 | ''' 21 | 22 | def __init__(self, n, n_folds, 23 | weights=None, shuffle=False, include_full=False): 24 | '''Create a CVfold object. The following arguments are accepted: 25 | 26 | * n: 27 | The number of samples in the data set being folded. 28 | * n_folds: 29 | The number of folds to create. 30 | * weights: 31 | Sample weights for the data being folded. If provides the 32 | generated folds are balanced by total weight. 33 | * shuffle: 34 | Randomize the sample indecies before creating folds. If not 35 | supplied the validation fold indicies generated will always be 36 | consecutive. 37 | * include_full: 38 | Include the full data set as a generated training fold, with 39 | the corresponding validation fold empty. 40 | ''' 41 | self.n = n 42 | self.n_folds = n_folds 43 | self.shuffle = shuffle 44 | self.weights = weights if weights != None else np.ones(shape=n) 45 | self.include_full = include_full 46 | 47 | def __iter__(self): 48 | '''Generator for producing indices into the training data correspoding 49 | to folds for cross validation. Yields pairs of arrays: 50 | 51 | (, 53 | 55 | ) 56 | 57 | The algorithm for producing folds of equal weight is the following. 58 | 59 | * Shuffle the weights vector if requested. 60 | * Compute an array representing the cumulative density function 61 | of the weights. 62 | * Compute the n_folds order statistics of the resulting cdf. 63 | That is, find the indicies into the cdf array that most closely 64 | partitions it into n_folds of equal weight. 65 | * For each order statistic os(k), yield the indicies corresponding 66 | to the weights between the statistics os(k-1) and os(k) as the 67 | validation indicies, and the complementary indicies as the train. 68 | 69 | Additionally, if the full data set was requested as a fold, we yield 70 | all indicies as a training fold and an empty validation fold as our 71 | first pair yielded to the caller. 72 | ''' 73 | weights = np.asarray(self.weights) 74 | n = self.n 75 | if weights.shape[0] != n: 76 | raise ValueError("Weights must have length n.") 77 | samples = np.arange(n) 78 | # Include the full data as a training fold if requested to do so 79 | if self.include_full: 80 | yield samples, np.array([]).astype(np.int64) 81 | # Randomize the order of the training indices if requested to do so 82 | if self.shuffle: 83 | np.random.shuffle(samples) 84 | pdf = weights[samples] 85 | else: 86 | pdf = weights 87 | # Compute and yield the folds 88 | cdf = np.cumsum(pdf) 89 | wsum = np.sum(weights) 90 | cutoffs = np.linspace(0, wsum, self.n_folds + 1) 91 | for i in range(1, self.n_folds + 1): 92 | valid_interval = (cdf > cutoffs[i-1])*(cdf <= cutoffs[i]) 93 | train_interval = 1 - valid_interval 94 | valid_inds = samples[np.nonzero(valid_interval)] 95 | train_inds = samples[np.nonzero(train_interval)] 96 | yield train_inds, valid_inds 97 | 98 | def _make_equal_weights(self): 99 | return np.ones(shape=self.n) 100 | -------------------------------------------------------------------------------- /examples/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madrury/glmnet-python/e22ad5c8ab50404dce19bb6fcaa7257f09e579a7/examples/__init__.py -------------------------------------------------------------------------------- /examples/example_enet.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from ..glmnet import ElasticNet 3 | from sklearn.datasets import make_regression 4 | 5 | display_bar = '-'*70 6 | 7 | X, y = make_regression( 8 | n_samples = 5000, 9 | n_features = 100, 10 | n_informative = 30, 11 | effective_rank = 40, 12 | noise = .1, 13 | ) 14 | 15 | print display_bar 16 | print "Fit an elastic net on some fake data" 17 | print display_bar 18 | 19 | enet = ElasticNet(alpha=.025) 20 | enet.fit(X, y) 21 | 22 | print enet 23 | 24 | print display_bar 25 | print "Predictions vs. actuals for the last elastic net model:" 26 | print display_bar 27 | 28 | preds = enet.predict(X) 29 | print y[:10] 30 | print preds[:10,np.shape(preds)[1]-1] 31 | 32 | enet.plot_paths() 33 | -------------------------------------------------------------------------------- /examples/example_enet_cv.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from ..glmnet import ElasticNet 3 | from sklearn.datasets import make_regression 4 | from ..cv.cv_glmnet import CVGlmNet 5 | 6 | display_bar = '-'*70 7 | 8 | X, y = make_regression( 9 | n_samples = 5000, 10 | n_features = 100, 11 | n_informative = 40, 12 | effective_rank = 30, 13 | noise = 8, 14 | ) 15 | w = np.random.uniform(size=5000) 16 | 17 | print display_bar 18 | print "Cross validate an elastic net on some fake data" 19 | print display_bar 20 | 21 | enet = ElasticNet(alpha=.1) 22 | enet_cv = CVGlmNet(enet, n_jobs=1) 23 | enet_cv.fit(X, y, weights=w) 24 | 25 | enet_cv.plot_oof_devs() 26 | -------------------------------------------------------------------------------- /examples/example_lognet.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from ..glmnet import LogisticNet 3 | from sklearn.datasets import make_classification 4 | 5 | display_bar = '-'*70 6 | 7 | X = np.random.randn(5000, 50) 8 | b = 3*np.random.randn(50) 9 | b[20:] = 0 10 | p = 1/(1 + np.exp(np.dot(X, b))) 11 | u = np.random.uniform(size=5000) 12 | y = np.int64(p > u) 13 | # Flip some random bits 14 | bits = np.random.binomial(1, .025, size=5000) 15 | y = (1 - bits)*y + bits*(1 - y) 16 | 17 | print display_bar 18 | print "Fit a logistic net on some fake data" 19 | print display_bar 20 | 21 | lognet = LogisticNet(alpha=1) 22 | lognet.fit(X, y) 23 | 24 | print lognet 25 | 26 | print display_bar 27 | print "Predictions vs. actuals for the last logistic net model:" 28 | print display_bar 29 | 30 | preds = lognet.predict(X) 31 | print preds[:10,np.shape(preds)[1]-1] 32 | print p[:10] 33 | 34 | lognet.plot_paths() 35 | -------------------------------------------------------------------------------- /examples/example_lognet_cv.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from ..glmnet import LogisticNet 3 | from sklearn.datasets import make_regression 4 | from ..cv.cv_glmnet import CVGlmNet 5 | 6 | display_bar = '-'*70 7 | 8 | X = np.random.randn(5000, 50) 9 | b = 3*np.random.randn(50) 10 | b[20:] = 0 11 | p = 1/(1 + np.exp(np.dot(X, b))) 12 | u = np.random.uniform(size=5000) 13 | y = np.int64(p > u) 14 | # Flip some random bits 15 | bits = np.random.binomial(1, .25, size=5000) 16 | y = (1 - bits)*y + bits*(1 - y) 17 | 18 | print display_bar 19 | print "Cross validate an elastic net on some fake data" 20 | print display_bar 21 | 22 | lognet = LogisticNet(alpha=.1) 23 | lognet_cv = CVGlmNet(lognet, n_jobs=1) 24 | lognet_cv.fit(X, y) 25 | 26 | print display_bar 27 | print lognet_cv.base_estimator 28 | 29 | lognet_cv.plot_oof_devs() 30 | -------------------------------------------------------------------------------- /glmnet/__init__.py: -------------------------------------------------------------------------------- 1 | from elastic_net import ElasticNet 2 | from logistic_net import LogisticNet 3 | -------------------------------------------------------------------------------- /glmnet/build.sh: -------------------------------------------------------------------------------- 1 | f2py -c --fcompiler=gnu95 --f77flags='-fdefault-real-8' --f90flags='-fdefault-real-8' glmnet.pyf glmnet.f 2 | -------------------------------------------------------------------------------- /glmnet/elastic_net.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from scipy.sparse import issparse 3 | import _glmnet 4 | from glmnet import GlmNet 5 | 6 | class ElasticNet(GlmNet): 7 | '''The elastic net: a multivariate linear model with both L1 and L2 8 | regularizers. 9 | 10 | This class implements the elastic net class of predictive models. These 11 | models combine the classical ridge and lasso regularizers into a combined 12 | penalty to the sum of squared residuals loss function. More specifically, 13 | the loss function minimized by this model is: 14 | 15 | L(\beta_0, \beta_1, ..., \beta_n) = 16 | RSS(\beta_0, \beta_1, ..., \beta_n; X, y) + 17 | \lambda * ((\alpha - 1)/2 * | \beta |_2 + \alpha * | \beta |_1) 18 | 19 | where RSS is the usual residual sum of squares: 20 | 21 | RSS(\beta_0, \beta_1, ..., \beta_n; X, y) = sum((\beta_i * X_i - y_i)^2) 22 | ''' 23 | 24 | # TODO: Implement offsets. 25 | def fit(self, X, y, col_names=None, 26 | lambdas=None, weights=None, rel_penalties=None, 27 | excl_preds=None, box_constraints=None): 28 | '''Fit an elastic net model. 29 | 30 | Arguments: 31 | 32 | * X: The model matrix. A n_obs * n_preds array. 33 | * y: The response. A n_obs array. 34 | 35 | Optional Arguments: 36 | 37 | * lambdas: 38 | A user supplied list of lambdas, an elastic net will be fit for 39 | each lambda supplied. If no array is passed, glmnet will generate 40 | its own array of lambdas equally spaced on a logaritmic scale 41 | between \lambda_max and \lambda_min. 42 | * weights: 43 | An n_obs array. Sample weights. 44 | * rel_penalties: 45 | An n_preds array. Relative panalty weights for the covariates. If 46 | none is passed, all covariates are penalized equally. If an array 47 | is passed, then a zero indicates an unpenalized parameter, and a 1 48 | a fully penalized parameter. Otherwise all covaraites recieve an 49 | equal penalty. 50 | * excl_preds: 51 | An n_preds array, used to exclude covaraites from the model. To 52 | exclude predictors, pass an array with a 1 in the first position, 53 | then a 1 in the i+1st position excludes the ith covaraite from 54 | model fitting. If no array is passed, all covaraites in X are 55 | included in the model. 56 | * box_constraints: 57 | An array with dimension 2 * n_obs. Interval constraints on the fit 58 | coefficients. The (0, i) entry is a lower bound on the ith 59 | covariate, and the (1, i) entry is an upper bound. These must 60 | satisfy lower_bound <= 0 <= upper_bound. If no array is passed, 61 | no box constraintes are allied to the parameters. 62 | 63 | After fitting, the following attributes are set: 64 | 65 | Private attributes: 66 | 67 | * _n_fit_obs: 68 | The number of rows in the model matrix X. 69 | * _n_fit_params: 70 | The number of columns in the model matrix X. 71 | * _col_names: 72 | Names for the columns in the model matrix. Used to display fit 73 | coefficients. 74 | * _out_n_lambdas: 75 | The number of lambdas associated with non-zero models (i.e. 76 | models with at least one none zero parameter estiamte) after 77 | fitting; for large enough lambda the models will become zero in 78 | the presense of an L1 regularizer. 79 | * _intecepts: 80 | A one dimensional array containing the intercept estiamtes for 81 | each value of lambda. See the intercepts (no underscore) 82 | property for a public version. 83 | * _comp_coef: 84 | The fit parameter estiamtes in a compressed form. This is a 85 | matrix with each row giving the estimates for a single 86 | coefficient for various values of \lambda. The order of the rows 87 | does not correspond to the order of the coefficents as given in 88 | the design matrix used to fit the model, this association is 89 | given by the _p_comp_coef attribute. Only estaimtes that are 90 | non-zero for some lambda are reported. 91 | * _p_comp_coef: 92 | A one dimensional integer array associating the coefficients in 93 | _comp_coef to columns in the model matrix. 94 | * _indices: 95 | The same information as _p_comp_coef, but zero indexed to be 96 | compatable with numpy arrays. 97 | * _n_comp_coef: 98 | The number of parameter estimates that are non-zero for some 99 | value of lambda. 100 | * _n_passes: 101 | The total number of passes over the data used to fit the model. 102 | * _error_flag: 103 | Error flag from the fortran code. 104 | 105 | Public Attributes: 106 | 107 | * r_sqs: 108 | An array of length _out_n_lambdas containing the r-squared 109 | statistic for each model. 110 | * out_lambdas: 111 | An array containing the lambda values associated with each fit 112 | model. 113 | ''' 114 | self._check_if_unfit() 115 | # Convert to arrays if native python objects 116 | try: 117 | if not issparse(X): 118 | X = np.asanyarray(X) 119 | y = np.asanyarray(y) 120 | except ValueError: 121 | raise ValueError("X and y must be either numpy arrays, or " 122 | "convertable to numpy arrays." 123 | ) 124 | # Grab the design info from patsy for later use, we are abbout to write 125 | # over this object in some cases. 126 | if hasattr(X, 'design_info'): 127 | design_info = X.design_info 128 | else: 129 | design_info = None 130 | # Make a copy if we are not able to overwrite X with its standardized 131 | # version. Note that if X is not fortran contiguous, then it will be 132 | # copied anyway. 133 | if not issparse(X) and np.isfortran(X) and not self.overwrite_pred_ok: 134 | X = X.copy(order='F') 135 | # Make a copy if we are not able to overwrite y with its standardized 136 | # version. 137 | if not self.overwrite_targ_ok: 138 | y = y.copy() 139 | # Validate all the inputs: 140 | self._validate_matrix(X) 141 | self._validate_inputs(X, y) 142 | self._validate_lambdas(X, y, lambdas) 143 | self._validate_weights(X, y, weights) 144 | self._validate_rel_penalties(X, y, rel_penalties) 145 | self._validate_excl_preds(X, y, excl_preds) 146 | self._validate_box_constraints(X, y, box_constraints) 147 | # Setup is complete, call into the extension module. 148 | if not issparse(X): 149 | (self._out_n_lambdas, 150 | self._intercepts, 151 | self._comp_coef, 152 | self._p_comp_coef, 153 | self._n_comp_coef, 154 | self.r_sqs, 155 | self.out_lambdas, 156 | self._n_passes, 157 | self._error_flag) = _glmnet.elnet( 158 | self.alpha, 159 | X, 160 | y, 161 | self.weights, 162 | self.excl_preds, 163 | self.rel_penalties, 164 | self.box_constraints, 165 | self.max_vars_all, 166 | self.frac_lg_lambda, 167 | self.lambdas, 168 | self.threshold, 169 | nlam=self.n_lambdas 170 | ) 171 | else: 172 | X.sort_indices() 173 | # Fortran arrays are 1 indexed. 174 | ind_ptrs = X.indptr + 1 175 | indices = X.indices + 1 176 | # Call 177 | (self._out_n_lambdas, 178 | self._intercepts, 179 | self._comp_coef, 180 | self._p_comp_coef, 181 | self._n_comp_coef, 182 | self.r_sqs, 183 | self.out_lambdas, 184 | self._n_passes, 185 | self._error_flag) = _glmnet.spelnet( 186 | self.alpha, 187 | X.shape[0], 188 | X.shape[1], 189 | X.data, 190 | ind_ptrs, 191 | indices, 192 | y, 193 | self.weights, 194 | self.excl_preds, 195 | self.rel_penalties, 196 | self.box_constraints, 197 | self.max_vars_all, 198 | self.frac_lg_lambda, 199 | self.lambdas, 200 | self.threshold, 201 | nlam=self.n_lambdas 202 | ) 203 | self._check_errors() 204 | # The indexes into the predictor array are off by one due to fortran 205 | # convention differing from numpys, this make them indexes into the the 206 | # numpy array. 207 | self._indices = np.trim_zeros(self._p_comp_coef, 'b') - 1 208 | # Keep some model metadata. 209 | self._n_fit_obs, self._n_fit_params = X.shape 210 | # Create a list of column names for the fit parameters, these can be 211 | # passed in, or attached to the matrix from patsy. If none are found 212 | # we crate our own stupid ones. 213 | if col_names != None: 214 | self._col_names = col_names 215 | elif design_info != None: 216 | self._col_names = design_info.column_names 217 | else: 218 | self._col_names = [ 219 | 'var_' + str(i) for i in range(self._n_fit_params) 220 | ] 221 | 222 | @property 223 | def _coefficients(self): 224 | '''The fit model coefficients for each lambda. 225 | 226 | A _n_comp_coef * _out_n_lambdas array containing the fit model 227 | coefficients for each value of lambda. 228 | ''' 229 | self._check_if_fit() 230 | return self._comp_coef[:np.max(self._n_comp_coef), 231 | :self._out_n_lambdas 232 | ] 233 | 234 | 235 | def _max_lambda(self, X, y, weights=None): 236 | '''Return the maximum value of lambda useful for fitting, i.e. that 237 | which first forces all coefficients to zero. 238 | 239 | The calculation is derived from the discussion in "Regularization 240 | Paths for Generalized Linear Models via Coordinate Descent". 241 | ''' 242 | if issparse(X): 243 | return self._max_lambda_sparse(X, y, weights) 244 | else: 245 | return self._max_lambda_dense(X, y, weights) 246 | 247 | def _max_lambda_dense(self, X, y, weights=None): 248 | dot = self._get_dot(X) 249 | if weights is None: 250 | # Standardize X and then find the maximum dot product. 251 | normfac = X.shape[0] 252 | mu = X.sum(axis=0) / normfac 253 | mu2 = (X*X).sum(axis=0) / normfac 254 | sigma = np.sqrt(mu2 - mu*mu) 255 | # Avoid divide by zero in constant case. 256 | sigma[sigma == 0] = 1 257 | X_scaled = (X - mu) / sigma 258 | dots = dot(y, X_scaled) / normfac 259 | else: 260 | # Standardize X using the sample weights and then find the 261 | # maximum weighted dot product. 262 | normfac = np.sum(weights) 263 | mu = dot(weights, X) / normfac 264 | mu2 = dot(weights, X*X) / normfac 265 | sigma = np.sqrt(mu2 - mu*mu) 266 | # Avoid divide by zero in constant case. 267 | sigma[sigma == 0] = 1 268 | X_scaled = (X - mu) / sigma 269 | y_wtd = y * weights 270 | dots = dot(y_wtd, X_scaled) / normfac 271 | # An alpha of zero (ridge) breaks the maximum lambda logic, the 272 | # coefficients are never all zero - so we readjust to a small 273 | # value. 274 | alpha = self.alpha if self.alpha > .0001 else .0001 275 | return np.max(np.abs(dots)) / alpha 276 | 277 | def _max_lambda_sparse(self, X, y, weights=None): 278 | '''To preserve the sparsity, we must avoid explicitly subtracting out 279 | the mean of the columns. 280 | ''' 281 | # Sparse dot 282 | dot = self._get_dot(X) 283 | # Calculate the dot product of y with X standardized, without 284 | # destorying the sparsity of X. The calculations themselves do not 285 | # differ from the dense case. 286 | if weights is None: 287 | normfac = X.shape[0] 288 | E = lambda M: np.asarray(M.sum(axis=0)).ravel() / normfac 289 | mu = E(X) 290 | mu_2 = E(X.multiply(X)) 291 | sigma = np.sqrt(mu_2 - mu*mu) 292 | sigma[sigma == 0] = 1.0 293 | dots = 1/sigma * (dot(y, X) - mu * np.sum(y)) / normfac 294 | else: 295 | normfac = np.sum(weights) 296 | E = lambda M: dot(weights, M) / normfac 297 | mu = E(X) 298 | mu_2 = E(X.multiply(X)) 299 | sigma = np.sqrt(mu_2 - mu*mu) 300 | sigma[sigma == 0] = 1.0 301 | y_wtd = y * weights 302 | dots = 1/sigma * (dot(y_wtd, X) - mu * np.sum(y_wtd)) / normfac 303 | # An alpha of zero (ridge) breaks the maximum lambda logic, the 304 | # coefficients are never all zero - so we readjust to a small 305 | # value. 306 | alpha = self.alpha if self.alpha > .0001 else .0001 307 | return np.max(np.abs(dots)) / alpha 308 | 309 | def deviance(self, X, y, weights = None): 310 | '''Calculate the normal deviance (i.e. sum of squared errors) for 311 | every lambda. The model must already be fit to call this method. 312 | ''' 313 | self._check_if_fit() 314 | if weights is not None and weights.shape[0] != X.shape[0]: 315 | raise ValueError("The weights vector must have the same length " 316 | "as X." 317 | ) 318 | y_hat = self.predict(X) 319 | # Take the response y, and repeat it as a column to produce a matrix 320 | # of the same dimensions as y_hat 321 | y_stacked = np.tile(np.array([y]).transpose(), y_hat.shape[1]) 322 | if weights is None: 323 | sq_residuals = (y_stacked - y_hat)**2 324 | normfac = X.shape[0] 325 | else: 326 | w_stacked = np.tile(np.array([weights]).transpose(), 327 | y_hat.shape[1] 328 | ) 329 | sq_residuals = w_stacked * (y_stacked - y_hat)**2 330 | normfac = np.sum(weights) 331 | return np.apply_along_axis(np.sum, 0, sq_residuals) / normfac 332 | 333 | def predict(self, X): 334 | '''Produce model predictions from new data.''' 335 | return self._predict_lp(X) 336 | 337 | def plot_paths(self): 338 | '''Plot parameter estiamte paths by log(lambda).''' 339 | self._plot_path('elastic') 340 | 341 | def __str__(self): 342 | return self._str('elastic') 343 | 344 | def describe(self, lidx=None): 345 | return self._describe(lidx, 'elastic') 346 | -------------------------------------------------------------------------------- /glmnet/glmnet.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from scipy.sparse import issparse, isspmatrix_csc 3 | from glmnet_config import (_DEFAULT_THRESH, 4 | _DEFAULT_FLMIN, 5 | _DEFAULT_NLAM) 6 | from warnings import warn 7 | 8 | from ..util.importers import import_pyplot 9 | plt = import_pyplot() 10 | 11 | 12 | class GlmNet(object): 13 | '''Parent class for glmnet model objects. 14 | 15 | Glmnet Info: 16 | 17 | Glmnets are a class of predictive models. They are a regularized version 18 | of generalized linear models that combines the ridge (L^2) and lasso (L^1) 19 | penalties. The general form of the loss function being optimized is: 20 | 21 | L(\beta_0, \beta_1, ..., \beta_n) = 22 | Dev(\beta_0, \beta_1, ..., \beta_n) + 23 | \lambda * ( (1 - \alpha)/2 * | \beta |_2 + \alpha * | \beta |_1 ) 24 | 25 | where Dev is the deviance of a classical glm, |x|_2 and |x|_1 are the L^2 26 | and L^1 norms, and \lambda and \alpha are tuning parameters: 27 | 28 | * \lambda controlls the overall ammount of regularization, and is usually 29 | tuned by cross validation. 30 | 31 | * \alpha controlls the balance between the L^1 and L^2 regularizers. In 32 | the extreme cases: \alpha = 0 : Ridge Regression 33 | \alpha = 1 : Lasso Regression 34 | 35 | All glmnet objects accept a value of \alpha at instantiation time. Glmnet 36 | defaults to fitting a full path of \lambda values, from \lambda_max (all 37 | parameters zero) to 0 (an unregularized model). The user may also choose to 38 | supply a list of \lambdas, in this case the default behavior is overriden and 39 | a glmnet is fit for each value of lambda the user supplies. 40 | 41 | The function Dev depends on the specific type of glmnet under 42 | consideration. Different choices of Dev determine various predictive 43 | models in the glmnet family. For details on the different types of 44 | glmnets, the reader should consult the various subclasses of GlmNet. 45 | 46 | This Class: 47 | 48 | This class is the parent of all glment models. As such, it only 49 | implements functionality in common to all glmnets, independednt of the 50 | specific loss function; this includes data needed to instantiate generic 51 | glmnets, checking inputs for validity, and checking error codes after 52 | fitting. 53 | ''' 54 | 55 | def __init__(self, 56 | alpha, 57 | standardize=True, 58 | max_vars_all=None, 59 | max_vars_largest=None, 60 | threshold=_DEFAULT_THRESH, 61 | frac_lg_lambda=_DEFAULT_FLMIN, 62 | n_lambdas=_DEFAULT_NLAM, 63 | overwrite_pred_ok=False, 64 | overwrite_targ_ok=False 65 | ): 66 | '''Create a glmnet object and implement configuration common to all 67 | subclasses. Accepts the following arguments: 68 | 69 | * alpha: 70 | Relative weighting between the L1 and L2 regularizers. 71 | * standardize: 72 | Boolean flag, do we standardize the predictor variables? Defaults 73 | to true, which is important for the regularizer to be fair. Note 74 | that the output parameters are allways reported on the scale of 75 | the origional predictors. 76 | * max_vars_largest: 77 | Maximum number of variables allowed in the largest model. This 78 | acts as a stopping criterion. 79 | * max_vars_all: 80 | Maximum number of non-zero variables allowed in any model. This 81 | controls memory alocation inside glmnet. 82 | * threshold: 83 | Convergence threshold for each lambda. For each lambda, 84 | iteration is stopped when imporvement is less than threashold. 85 | * frac_lg_lambda: 86 | Control parameter for range of lambda values to search: 87 | \lambda_min = frac_lg_lambda * (\lambda_max) 88 | where \lambda_max is calcualted based on the data and the 89 | model type. 90 | * n_lambdas: 91 | The number of lambdas to include in the grid search. 92 | * overwrite_pred_ok: 93 | Boolean, overwirte the memory holding the predictor when 94 | standardizing? 95 | * overwirte_targ_ok: 96 | Boolean, overwrite the memory holding the target when 97 | standardizing? 98 | ''' 99 | self.alpha = alpha 100 | self.standardize = standardize 101 | self.max_vars_all = max_vars_all 102 | self.max_vars_largest = max_vars_largest 103 | self.threshold = threshold 104 | self.frac_lg_lambda = _DEFAULT_FLMIN 105 | self.n_lambdas = _DEFAULT_NLAM 106 | self.overwrite_pred_ok = overwrite_pred_ok 107 | self.overwrite_targ_ok = overwrite_targ_ok 108 | 109 | def _validate_lambdas(self, X, y, lambdas): 110 | '''glmnet expects either a user supplied array of lambdas, or a signal 111 | to construct its own. 112 | ''' 113 | if lambdas is not None: 114 | self.lambdas = np.asarray(lambdas) 115 | self.n_lambdas = len(lambdas) 116 | # >1 indicates that the user passed in a list of lambdas 117 | self.frac_lg_lambda = 2 118 | else: 119 | self.lambdas = None 120 | 121 | def _validate_weights(self, X, y, weights): 122 | '''If no explicit sample weights are passed, each observation is given a 123 | unit weight. 124 | ''' 125 | self.weights = (np.ones(X.shape[0]) if weights is None 126 | else weights 127 | ) 128 | if self.weights.shape[0] != X.shape[0]: 129 | raise ValueError("The weights vector must have the same length " 130 | "as X." 131 | ) 132 | if np.any(self.weights < 0): 133 | raise ValueError("All sample weights must be non-negative.") 134 | 135 | def _validate_rel_penalties(self, X, y, rel_penalties): 136 | '''If no explicit penalty weights are passed, each varaible is given the 137 | same penalty weight. 138 | ''' 139 | self.rel_penalties = (np.ones(X.shape[1]) if rel_penalties is None 140 | else rel_penalties 141 | ) 142 | if self.rel_penalties.shape[0] != X.shape[1]: 143 | raise ValueError("The relative penalties vector must have the " 144 | "same length as the number of columns in X." 145 | ) 146 | if np.any(self.rel_penalties < 0): 147 | raise ValueError("All relative penalties must be non-negative.") 148 | if np.max(self.rel_penalties) <= 0: 149 | raise ValueError("Must have at least one positive relative " 150 | "penalty." 151 | ) 152 | 153 | def _validate_excl_preds(self, X, y, excl_preds): 154 | '''If no explicit exclusion is supplied, pass a zero to exclude nothing. 155 | ''' 156 | self.excl_preds = (np.zeros(1) if excl_preds is None 157 | else excl_preds 158 | ) 159 | if self.excl_preds.shape[0] != 1: 160 | if self.excl_preds.shape[0] > X.shape[1] + 1: 161 | raise ValueError("Non null excluded predictors array must " 162 | "have less entries than the number of " 163 | "columns in X." 164 | ) 165 | if np.any(self.excl_preds[1:] >= X.shape[1]): 166 | raise ValueError("Entries in non null excluded predictors " 167 | "array (except for the first entry) must " 168 | "enumerate columns in X." 169 | ) 170 | 171 | def _validate_box_constraints(self, X, y, box_constraints): 172 | '''Box constraints on parameter estimates.''' 173 | if box_constraints is None: 174 | bc = np.empty((2, X.shape[1]), order='F') 175 | bc[0,:] = float("-inf") 176 | bc[1,:] = float("inf") 177 | box_constraints = bc 178 | if (box_constraints.shape[1] != X.shape[1] or 179 | box_constraints.shape[0] != 2): 180 | raise ValueError("Box constraints must be a vector of shape 2, " 181 | "number of columns in X." 182 | ) 183 | if (np.any(box_constraints[0,:] > 0) or 184 | np.any(box_constraints[1,:] < 0)): 185 | raise ValueError("Box constraints must be intervals of the form " 186 | "[non-positive, non-negative] for each " 187 | "predictor." 188 | ) 189 | self.box_constraints = box_constraints.copy(order='F') 190 | 191 | def _validate_inputs(self, X, y): 192 | '''Validate and process the prectors and response for model fitting.''' 193 | # Check that the dimensions work out 194 | if X.shape[0] != y.shape[0]: 195 | raise ValueError("X and y must have the same length.") 196 | # Decide on the largest allowable models 197 | self.max_vars_all = ( 198 | X.shape[1] if self.max_vars_all is None else self.max_vars_all 199 | ) 200 | self.max_vars_largest = ( 201 | self.max_vars_all if self.max_vars_largest is None 202 | else self.max_vars_largest 203 | ) 204 | if self.max_vars_all < self.max_vars_largest: 205 | raise ValueError("Inconsistant parameters: need max_vars_largest " 206 | "< max_vars_all." 207 | ) 208 | 209 | @staticmethod 210 | def _validate_matrix(X): 211 | '''glmnet only accepts sparse matricies in compressed saprse column 212 | format. Note: while glment documentation says it wants sparse row 213 | format, it lies. 214 | ''' 215 | if issparse(X) and not isspmatrix_csc(X): 216 | raise ValueError("Sparse matrix detected, but not in compressed " 217 | "sparse row format." 218 | ) 219 | 220 | 221 | def _check_errors(self): 222 | '''Check for errors, documented in glmnet.f.''' 223 | # Fatal errors. 224 | if self._error_flag > 0: 225 | if self._error_flag == 10000: 226 | raise ValueError('Fatal: Cannot have max(vp) < 0.0.') 227 | elif self._error_flag == 7777: 228 | raise ValueError('Fatal: all used predictors have 0 variance.') 229 | elif self._error_flag < 7777: 230 | raise MemoryError('Fatal: Memory allocation error.') 231 | else: 232 | raise Exception('Fatal: Unknown error code: %d' 233 | % self._error_flag 234 | ) 235 | # Non-fatal errors. 236 | elif self._error_flag < 0: 237 | if self._error_flag > -10000: 238 | last_lambda = -self._error_flag 239 | w_msg = ("Convergence for {0:d}'th lambda value not reached " 240 | "after {1:d} iterations.") 241 | warn(w_msg.format(last_lambda, self._n_passes), RuntimeWarning) 242 | elif self._error_flag <= -10000: 243 | last_lambda = -(self._error_flag + 10000) 244 | w_msg = ("Number of non-zero coefficients exceeds {0:d} at " 245 | "{1:d}th lambda value.") 246 | warn(w_msg.format(self.max_vars_all, last_lambda), 247 | RuntimeWarning 248 | ) 249 | else: 250 | warn("Unknown warning %d" % self._error_flag) 251 | 252 | @property 253 | def intercepts(self): 254 | '''The fit model intercepts, one for each value of lambda.''' 255 | self._check_if_fit() 256 | return self._intercepts.ravel()[:self._out_n_lambdas] 257 | 258 | def get_coefficients_from_lambda_idx(self, lidx): 259 | '''Get the array of coefficient estimates given an index into the 260 | array of fit lambdas, that is, for a fixed lambda (index) return the 261 | estimates of the various coefficients given lambda. 262 | 263 | An array of length _n_fit_params (i.e. X.shape[1]). 264 | ''' 265 | coefs = np.zeros(shape=(self._n_fit_params,)) 266 | coefs[self._indices] = self._coefficients[:,lidx] 267 | return coefs 268 | 269 | def get_coefficients_from_col_idx(self, cidx): 270 | '''Get the array of coefficient estimates given the index of a 271 | column in the model matrix, that is, for a fixed column return all the 272 | various estimates of the associated coefficient across lambda. Returns 273 | the zero array if the coefficient is compressed out of _coefficients. 274 | 275 | An array of length _out_n_lambdas. 276 | ''' 277 | if cidx not in self._indices: 278 | return np.zeros(shape=(self._out_n_lambdas,)) 279 | else: 280 | ridx = np.where(self._indices == cidx) 281 | return self._coefficients[ridx,:].squeeze() 282 | 283 | def _describe(self, lidx=None, name='glmnet'): 284 | '''Display information about the model. Behaviour depends on 285 | state of the model when called, and whether a specific lambda index 286 | is passed. 287 | ''' 288 | display_str = self._str(name) 289 | if self._is_fit(): 290 | display_str = display_str + self._fit_str(name) 291 | if lidx != None: 292 | sep = '-'*79 + '\n' 293 | display_str = display_str + sep + self._coef_str(lidx, name) 294 | return display_str.strip() 295 | 296 | def _str(self, name): 297 | '''A generic message pertaining to all glmnets.''' 298 | return 'A {name} net model with alpha = {alpha}.\n'.format( 299 | name=name, alpha=self.alpha 300 | ) 301 | 302 | def _fit_str(self, name): 303 | '''A generic message pertaining to all fit glmnets.''' 304 | self._check_if_fit() 305 | s = ("The model was fit on {0} observations and {1} parameters. \n" 306 | "The model was fit in {2} passes over the data. \n" 307 | "There were {3} values of lambda resulting in non-zero models. \n" 308 | "There were {4} non-zero coefficients in the largest model. \n") 309 | return s.format(self._n_fit_obs, self._n_fit_params, 310 | self._n_passes, 311 | self._out_n_lambdas, 312 | np.max(self._n_comp_coef) 313 | ) 314 | 315 | def _coef_str(self, lidx, name): 316 | '''Create a string containing a table of coefficient estimates for 317 | a given value of lambda. 318 | ''' 319 | lam = self.out_lambdas[lidx] 320 | coefs = self.get_coefficients_from_lambda_idx(lidx) 321 | title = ("Parameter Estiamtes for {net} net model with " 322 | "lambda = {lam:1.5e}\n").format(net=name, lam=lam) 323 | header = "{nameh:<40}{valh:<10}\n".format( 324 | nameh="Varaible Name", valh="Coefficent Estiamte" 325 | ) 326 | line_template = "{name:<40}{val:1.5e}" 327 | body = "\n".join( 328 | line_template.format(name=self._col_names[i], val=coefs[i]) 329 | for i in range(self._n_fit_params) 330 | ) 331 | return title + header + body 332 | 333 | def _predict_lp(self, X): 334 | '''Model predictions on a linear predictor scale. 335 | 336 | Returns an n_obs * n_lambdas array, where n_obs is the number of rows 337 | in X. 338 | ''' 339 | self._check_if_fit() 340 | dot = self._get_dot(X) 341 | if np.max(self._n_comp_coef) > 0 : 342 | return self.intercepts + dot(X[:, self._indices], 343 | self._coefficients 344 | ) 345 | else : 346 | return np.tile(self.intercepts, (X.shape[0], 1)) 347 | 348 | def _plot_path(self, name): 349 | '''Plot the full regularization path of all the non-zero model 350 | coefficients. Creates an displays a plot of the parameter estimates 351 | at each value of log(\lambda). 352 | ''' 353 | self._check_if_fit() 354 | if not plt: 355 | raise RuntimeError('pyplot unavailable.') 356 | 357 | plt.clf() 358 | fig, ax = plt.subplots() 359 | xvals = np.log(self.out_lambdas[1:self._out_n_lambdas]) 360 | for coef_path in self._coefficients: 361 | ax.plot(xvals, coef_path[1:]) 362 | ax.set_title("Regularization paths for %s net with alpha = %s" % 363 | (name, self.alpha)) 364 | ax.set_xlabel("log(lambda)") 365 | ax.set_ylabel("Parameter Value") 366 | plt.show() 367 | 368 | def _is_fit(self): 369 | '''The model has been fit successfully if and only if the _n_fit_obs 370 | attribute exists. 371 | ''' 372 | return hasattr(self, '_n_fit_obs') 373 | 374 | def _check_if_fit(self, reverse=False): 375 | '''Raise exception if model is not fit.''' 376 | its_ok = (not self._is_fit()) if reverse else self._is_fit() 377 | word = 'already' if reverse else 'not' 378 | if its_ok: 379 | return 380 | else: 381 | raise RuntimeError('The operation cannot be performed on a model ' 382 | 'that has ' + word + ' been fit.' 383 | ) 384 | 385 | def _check_if_unfit(self): 386 | return self._check_if_fit(reverse=True) 387 | 388 | @staticmethod 389 | def _get_dot(X): 390 | '''Get the underlying function for a dot product of two matricies, 391 | independent of type. This allows us to write dot(X, Y) for both 392 | dense and sparse matricies. 393 | ''' 394 | if issparse(X): 395 | return X.dot.__func__ 396 | else: 397 | return np.dot 398 | -------------------------------------------------------------------------------- /glmnet/glmnet.pyf: -------------------------------------------------------------------------------- 1 | ! -*- f90 -*- 2 | ! Note: the context of this file is case sensitive. 3 | 4 | python module _glmnet ! in 5 | interface ! in :_glmnet 6 | 7 | subroutine get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx) ! in :glmnet5:glmnet5.f 8 | real*8 :: sml 9 | real*8 :: eps 10 | real*8 :: big 11 | integer :: mnlam 12 | real*8 :: rsqmax 13 | real*8 :: pmin 14 | real*8 :: exmx 15 | end subroutine get_int_parms 16 | 17 | subroutine chg_fract_dev(arg) ! in :glmnet5:glmnet5.f 18 | real*8 :: arg 19 | end subroutine chg_fract_dev 20 | 21 | subroutine chg_dev_max(arg) ! in :glmnet5:glmnet5.f 22 | real*8 :: arg 23 | end subroutine chg_dev_max 24 | 25 | subroutine chg_min_flmin(arg) ! in :glmnet5:glmnet5.f 26 | real*8 :: arg 27 | end subroutine chg_min_flmin 28 | 29 | subroutine chg_big(arg) ! in :glmnet5:glmnet5.f 30 | real*8 :: arg 31 | end subroutine chg_big 32 | 33 | subroutine chg_min_lambdas(irg) ! in :glmnet5:glmnet5.f 34 | integer :: irg 35 | end subroutine chg_min_lambdas 36 | 37 | subroutine chg_min_null_prob(arg) ! in :glmnet5:glmnet5.f 38 | real*8 :: arg 39 | end subroutine chg_min_null_prob 40 | 41 | subroutine chg_max_exp(arg) ! in :glmnet5:glmnet5.f 42 | real*8 :: arg 43 | end subroutine chg_max_exp 44 | 45 | subroutine elnet(ka,parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 46 | integer optional :: ka=1 ! Use covariance updates over naive by default 47 | real*8 :: parm 48 | integer intent(hide),check(shape(x,0)==no),depend(x) :: no=shape(x,0) 49 | integer intent(hide),check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 50 | real*8 dimension(no,ni) :: x 51 | real*8 dimension(no),depend(no) :: y 52 | real*8 dimension(no),depend(no) :: w 53 | integer dimension(*) :: jd 54 | real*8 dimension(ni),depend(ni) :: vp 55 | real*8 dimension(2,ni),depend(ni) :: cl 56 | integer optional,depend(x) :: ne=min(shape(x,1), nx) 57 | integer :: nx 58 | integer optional,check((flmin < 1.0 || len(ulam)==nlam)),depend(flmin,ulam) :: nlam=len(ulam) 59 | real*8 :: flmin 60 | real*8 dimension(nlam) :: ulam 61 | real*8 :: thr 62 | integer optional :: isd=1 ! Standardize predictors by default 63 | integer optional :: intr=1 ! Include intercept by default 64 | integer optional :: maxit=100000 ! Maximum iterations 65 | 66 | ! Outputs 67 | integer intent(out) :: lmu 68 | real*8 intent(out),dimension(nlam),depend(nlam) :: a0 69 | real*8 intent(out),dimension(nx,nlam),depend(nlam) :: ca 70 | integer intent(out),dimension(nx),depend(nx) :: ia 71 | integer intent(out),dimension(nlam),depend(nlam) :: nin 72 | real*8 intent(out),dimension(nlam),depend(nlam) :: rsq 73 | real*8 intent(out),dimension(nlam),depend(nlam) :: alm 74 | integer intent(out) :: nlp 75 | integer intent(out) :: jerr 76 | end subroutine elnet 77 | 78 | subroutine elnetu(parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 79 | real*8 :: parm 80 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 81 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 82 | real*8 dimension(no,ni) :: x 83 | real*8 dimension(no),depend(no) :: y 84 | real*8 dimension(no),depend(no) :: w 85 | integer dimension(*) :: jd 86 | real*8 dimension(ni),depend(ni) :: vp 87 | real*8 dimension(2,ni),depend(ni) :: cl 88 | integer :: ne 89 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 90 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 91 | real*8 :: flmin 92 | real*8 dimension(nlam) :: ulam 93 | real*8 :: thr 94 | integer :: isd 95 | integer :: intr 96 | integer :: maxit 97 | integer :: lmu 98 | real*8 dimension(nlam),depend(nlam) :: a0 99 | real*8 dimension(nx,nlam),depend(nlam) :: ca 100 | integer dimension(nx),depend(nx) :: ia 101 | integer dimension(nlam),depend(nlam) :: nin 102 | real*8 dimension(nlam),depend(nlam) :: rsq 103 | real*8 dimension(nlam),depend(nlam) :: alm 104 | integer :: nlp 105 | integer :: jerr 106 | end subroutine elnetu 107 | 108 | subroutine standard(no,ni,x,y,w,isd,intr,ju,g,xm,xs,ym,ys,xv,jerr) ! in :glmnet5:glmnet5.f 109 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 110 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 111 | real*8 dimension(no,ni) :: x 112 | real*8 dimension(no),depend(no) :: y 113 | real*8 dimension(no),depend(no) :: w 114 | integer :: isd 115 | integer :: intr 116 | integer dimension(ni),depend(ni) :: ju 117 | real*8 dimension(ni),depend(ni) :: g 118 | real*8 dimension(ni),depend(ni) :: xm 119 | real*8 dimension(ni),depend(ni) :: xs 120 | real*8 :: ym 121 | real*8 :: ys 122 | real*8 dimension(ni),depend(ni) :: xv 123 | integer :: jerr 124 | end subroutine standard 125 | 126 | subroutine elnet1(beta,ni,ju,vp,cl,g,no,ne,nx,x,nlam,flmin,ulam,thr,maxit,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 127 | real*8 :: beta 128 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 129 | integer dimension(ni) :: ju 130 | real*8 dimension(ni),depend(ni) :: vp 131 | real*8 dimension(2,ni),depend(ni) :: cl 132 | real*8 dimension(ni),depend(ni) :: g 133 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 134 | integer :: ne 135 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 136 | real*8 dimension(no,ni),depend(ni) :: x 137 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 138 | real*8 :: flmin 139 | real*8 dimension(nlam) :: ulam 140 | real*8 :: thr 141 | integer :: maxit 142 | real*8 dimension(ni),depend(ni) :: xv 143 | integer :: lmu 144 | real*8 dimension(nx,nlam),depend(nlam) :: ao 145 | integer dimension(nx),depend(nx) :: ia 146 | integer dimension(nlam),depend(nlam) :: kin 147 | real*8 dimension(nlam),depend(nlam) :: rsqo 148 | real*8 dimension(nlam),depend(nlam) :: almo 149 | integer :: nlp 150 | integer :: jerr 151 | end subroutine elnet1 152 | 153 | subroutine elnetn(parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 154 | real*8 :: parm 155 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 156 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 157 | real*8 dimension(no,ni) :: x 158 | real*8 dimension(no),depend(no) :: y 159 | real*8 dimension(no),depend(no) :: w 160 | integer dimension(*) :: jd 161 | real*8 dimension(ni),depend(ni) :: vp 162 | real*8 dimension(2,ni),depend(ni) :: cl 163 | integer :: ne 164 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 165 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 166 | real*8 :: flmin 167 | real*8 dimension(nlam) :: ulam 168 | real*8 :: thr 169 | integer :: isd 170 | integer :: intr 171 | integer :: maxit 172 | integer :: lmu 173 | real*8 dimension(nlam),depend(nlam) :: a0 174 | real*8 dimension(nx,nlam),depend(nlam) :: ca 175 | integer dimension(nx),depend(nx) :: ia 176 | integer dimension(nlam),depend(nlam) :: nin 177 | real*8 dimension(nlam),depend(nlam) :: rsq 178 | real*8 dimension(nlam),depend(nlam) :: alm 179 | integer :: nlp 180 | integer :: jerr 181 | end subroutine elnetn 182 | 183 | subroutine standard1(no,ni,x,y,w,isd,intr,ju,xm,xs,ym,ys,xv,jerr) ! in :glmnet5:glmnet5.f 184 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 185 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 186 | real*8 dimension(no,ni) :: x 187 | real*8 dimension(no),depend(no) :: y 188 | real*8 dimension(no),depend(no) :: w 189 | integer :: isd 190 | integer :: intr 191 | integer dimension(ni),depend(ni) :: ju 192 | real*8 dimension(ni),depend(ni) :: xm 193 | real*8 dimension(ni),depend(ni) :: xs 194 | real*8 :: ym 195 | real*8 :: ys 196 | real*8 dimension(ni),depend(ni) :: xv 197 | integer :: jerr 198 | end subroutine standard1 199 | 200 | subroutine elnet2(beta,ni,ju,vp,cl,y,no,ne,nx,x,nlam,flmin,ulam,thr,maxit,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 201 | real*8 :: beta 202 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 203 | integer dimension(ni) :: ju 204 | real*8 dimension(ni),depend(ni) :: vp 205 | real*8 dimension(2,ni),depend(ni) :: cl 206 | real*8 dimension(no) :: y 207 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 208 | integer :: ne 209 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 210 | real*8 dimension(no,ni),depend(no,ni) :: x 211 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 212 | real*8 :: flmin 213 | real*8 dimension(nlam) :: ulam 214 | real*8 :: thr 215 | integer :: maxit 216 | real*8 dimension(ni),depend(ni) :: xv 217 | integer :: lmu 218 | real*8 dimension(nx,nlam),depend(nlam) :: ao 219 | integer dimension(nx),depend(nx) :: ia 220 | integer dimension(nlam),depend(nlam) :: kin 221 | real*8 dimension(nlam),depend(nlam) :: rsqo 222 | real*8 dimension(nlam),depend(nlam) :: almo 223 | integer :: nlp 224 | integer :: jerr 225 | end subroutine elnet2 226 | 227 | subroutine chkvars(no,ni,x,ju) ! in :glmnet5:glmnet5.f 228 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 229 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 230 | real*8 dimension(no,ni) :: x 231 | integer dimension(ni),depend(ni) :: ju 232 | end subroutine chkvars 233 | 234 | subroutine uncomp(ni,ca,ia,nin,a) ! in :glmnet5:glmnet5.f 235 | integer, optional,check(len(a)>=ni),depend(a) :: ni=len(a) 236 | real*8 dimension(*) :: ca 237 | integer dimension(*) :: ia 238 | integer :: nin 239 | real*8 dimension(ni) :: a 240 | end subroutine uncomp 241 | 242 | subroutine modval(a0,ca,ia,nin,n,x,f) ! in :glmnet5:glmnet5.f 243 | real*8 :: a0 244 | real*8 dimension(nin) :: ca 245 | integer dimension(nin),depend(nin) :: ia 246 | integer, optional,check(len(ca)>=nin),depend(ca) :: nin=len(ca) 247 | integer, optional,check(shape(x,0)==n),depend(x) :: n=shape(x,0) 248 | real*8 dimension(n,*) :: x 249 | real*8 dimension(n),depend(n) :: f 250 | end subroutine modval 251 | 252 | subroutine spelnet(ka,parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 253 | integer optional :: ka=1 ! Use covariance updates over naive by default 254 | real*8 :: parm 255 | integer :: no 256 | integer :: ni 257 | real*8 dimension(*) :: x 258 | integer dimension(*) :: ix 259 | integer dimension(*) :: jx 260 | real*8 dimension(no),depend(no) :: y 261 | real*8 dimension(no),depend(no) :: w 262 | integer dimension(*) :: jd 263 | real*8 dimension(ni),depend(ni) :: vp 264 | real*8 dimension(2,ni),depend(ni) :: cl 265 | integer optional,depend(x) :: ne=min(ni, nx) 266 | integer :: nx 267 | integer optional,check((flmin < 1.0 || len(ulam)==nlam)),depend(flmin,ulam) :: nlam=len(ulam) 268 | real*8 :: flmin 269 | real*8 dimension(nlam) :: ulam 270 | real*8 :: thr 271 | integer optional :: isd=1 ! Standardize predictors by default 272 | integer optional :: intr=1 ! Include intercept by default 273 | integer optional :: maxit=100000 ! Maximum iterations 274 | 275 | ! Outputs 276 | integer intent(out) :: lmu 277 | real*8 intent(out),dimension(nlam),depend(nlam) :: a0 278 | real*8 intent(out),dimension(nx,nlam),depend(nlam) :: ca 279 | integer intent(out),dimension(nx),depend(nx) :: ia 280 | integer intent(out),dimension(nlam),depend(nlam) :: nin 281 | real*8 intent(out),dimension(nlam),depend(nlam) :: rsq 282 | real*8 intent(out),dimension(nlam),depend(nlam) :: alm 283 | integer intent(out) :: nlp 284 | integer intent(out) :: jerr 285 | end subroutine spelnet 286 | 287 | subroutine spelnetu(parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 288 | real*8 :: parm 289 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 290 | integer, optional,check(len(vp)>=ni),depend(vp) :: ni=len(vp) 291 | real*8 dimension(*) :: x 292 | integer dimension(*) :: ix 293 | integer dimension(*) :: jx 294 | real*8 dimension(no) :: y 295 | real*8 dimension(no),depend(no) :: w 296 | integer dimension(*) :: jd 297 | real*8 dimension(ni) :: vp 298 | real*8 dimension(2,ni),depend(ni) :: cl 299 | integer :: ne 300 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 301 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 302 | real*8 :: flmin 303 | real*8 dimension(nlam) :: ulam 304 | real*8 :: thr 305 | integer :: isd 306 | integer :: intr 307 | integer :: maxit 308 | integer :: lmu 309 | real*8 dimension(nlam),depend(nlam) :: a0 310 | real*8 dimension(nx,nlam),depend(nlam) :: ca 311 | integer dimension(nx),depend(nx) :: ia 312 | integer dimension(nlam),depend(nlam) :: nin 313 | real*8 dimension(nlam),depend(nlam) :: rsq 314 | real*8 dimension(nlam),depend(nlam) :: alm 315 | integer :: nlp 316 | integer :: jerr 317 | end subroutine spelnetu 318 | 319 | subroutine spstandard(no,ni,x,ix,jx,y,w,ju,isd,intr,g,xm,xs,ym,ys,xv,jerr) ! in :glmnet5:glmnet5.f 320 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 321 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 322 | real*8 dimension(*) :: x 323 | integer dimension(*) :: ix 324 | integer dimension(*) :: jx 325 | real*8 dimension(no) :: y 326 | real*8 dimension(no),depend(no) :: w 327 | integer dimension(ni) :: ju 328 | integer :: isd 329 | integer :: intr 330 | real*8 dimension(ni),depend(ni) :: g 331 | real*8 dimension(ni),depend(ni) :: xm 332 | real*8 dimension(ni),depend(ni) :: xs 333 | real*8 :: ym 334 | real*8 :: ys 335 | real*8 dimension(ni),depend(ni) :: xv 336 | integer :: jerr 337 | end subroutine spstandard 338 | 339 | subroutine spelnet1(beta,ni,g,no,w,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,ulam,thr,maxit,xm,xs,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 340 | real*8 :: beta 341 | integer, optional,check(len(g)>=ni),depend(g) :: ni=len(g) 342 | real*8 dimension(ni) :: g 343 | integer, optional,check(len(w)>=no),depend(w) :: no=len(w) 344 | real*8 dimension(no) :: w 345 | integer :: ne 346 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 347 | real*8 dimension(*) :: x 348 | integer dimension(*) :: ix 349 | integer dimension(*) :: jx 350 | integer dimension(ni),depend(ni) :: ju 351 | real*8 dimension(ni),depend(ni) :: vp 352 | real*8 dimension(2,ni),depend(ni) :: cl 353 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 354 | real*8 :: flmin 355 | real*8 dimension(nlam) :: ulam 356 | real*8 :: thr 357 | integer :: maxit 358 | real*8 dimension(ni),depend(ni) :: xm 359 | real*8 dimension(ni),depend(ni) :: xs 360 | real*8 dimension(ni),depend(ni) :: xv 361 | integer :: lmu 362 | real*8 dimension(nx,nlam),depend(nlam) :: ao 363 | integer dimension(nx),depend(nx) :: ia 364 | integer dimension(nlam),depend(nlam) :: kin 365 | real*8 dimension(nlam),depend(nlam) :: rsqo 366 | real*8 dimension(nlam),depend(nlam) :: almo 367 | integer :: nlp 368 | integer :: jerr 369 | end subroutine spelnet1 370 | 371 | subroutine spelnetn(parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 372 | real*8 :: parm 373 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 374 | integer, optional,check(len(vp)>=ni),depend(vp) :: ni=len(vp) 375 | real*8 dimension(*) :: x 376 | integer dimension(*) :: ix 377 | integer dimension(*) :: jx 378 | real*8 dimension(no) :: y 379 | real*8 dimension(no),depend(no) :: w 380 | integer dimension(*) :: jd 381 | real*8 dimension(ni) :: vp 382 | real*8 dimension(2,ni),depend(ni) :: cl 383 | integer :: ne 384 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 385 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 386 | real*8 :: flmin 387 | real*8 dimension(nlam) :: ulam 388 | real*8 :: thr 389 | integer :: isd 390 | integer :: intr 391 | integer :: maxit 392 | integer :: lmu 393 | real*8 dimension(nlam),depend(nlam) :: a0 394 | real*8 dimension(nx,nlam),depend(nlam) :: ca 395 | integer dimension(nx),depend(nx) :: ia 396 | integer dimension(nlam),depend(nlam) :: nin 397 | real*8 dimension(nlam),depend(nlam) :: rsq 398 | real*8 dimension(nlam),depend(nlam) :: alm 399 | integer :: nlp 400 | integer :: jerr 401 | end subroutine spelnetn 402 | 403 | subroutine spstandard1(no,ni,x,ix,jx,y,w,ju,isd,intr,xm,xs,ym,ys,xv,jerr) ! in :glmnet5:glmnet5.f 404 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 405 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 406 | real*8 dimension(*) :: x 407 | integer dimension(*) :: ix 408 | integer dimension(*) :: jx 409 | real*8 dimension(no) :: y 410 | real*8 dimension(no),depend(no) :: w 411 | integer dimension(ni) :: ju 412 | integer :: isd 413 | integer :: intr 414 | real*8 dimension(ni),depend(ni) :: xm 415 | real*8 dimension(ni),depend(ni) :: xs 416 | real*8 :: ym 417 | real*8 :: ys 418 | real*8 dimension(ni),depend(ni) :: xv 419 | integer :: jerr 420 | end subroutine spstandard1 421 | 422 | subroutine spelnet2(beta,ni,y,w,no,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,ulam,thr,maxit,xm,xs,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 423 | real*8 :: beta 424 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 425 | real*8 dimension(no) :: y 426 | real*8 dimension(no),depend(no) :: w 427 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 428 | integer :: ne 429 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 430 | real*8 dimension(*) :: x 431 | integer dimension(*) :: ix 432 | integer dimension(*) :: jx 433 | integer dimension(ni) :: ju 434 | real*8 dimension(ni),depend(ni) :: vp 435 | real*8 dimension(2,ni),depend(ni) :: cl 436 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 437 | real*8 :: flmin 438 | real*8 dimension(nlam) :: ulam 439 | real*8 :: thr 440 | integer :: maxit 441 | real*8 dimension(ni),depend(ni) :: xm 442 | real*8 dimension(ni),depend(ni) :: xs 443 | real*8 dimension(ni),depend(ni) :: xv 444 | integer :: lmu 445 | real*8 dimension(nx,nlam),depend(nlam) :: ao 446 | integer dimension(nx),depend(nx) :: ia 447 | integer dimension(nlam),depend(nlam) :: kin 448 | real*8 dimension(nlam),depend(nlam) :: rsqo 449 | real*8 dimension(nlam),depend(nlam) :: almo 450 | integer :: nlp 451 | integer :: jerr 452 | end subroutine spelnet2 453 | 454 | subroutine spchkvars(no,ni,x,ix,ju) ! in :glmnet5:glmnet5.f 455 | integer :: no 456 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 457 | real*8 dimension(*) :: x 458 | integer dimension(*) :: ix 459 | integer dimension(ni) :: ju 460 | end subroutine spchkvars 461 | 462 | subroutine cmodval(a0,ca,ia,nin,x,ix,jx,n,f) ! in :glmnet5:glmnet5.f 463 | real*8 :: a0 464 | real*8 dimension(*) :: ca 465 | integer dimension(*) :: ia 466 | integer :: nin 467 | real*8 dimension(*) :: x 468 | integer dimension(*) :: ix 469 | integer dimension(*) :: jx 470 | integer, optional,check(len(f)>=n),depend(f) :: n=len(f) 471 | real*8 dimension(n) :: f 472 | end subroutine cmodval 473 | 474 | function row_prod(i,j,ia,ja,ra,w) ! in :glmnet5:glmnet5.f 475 | integer :: i 476 | integer :: j 477 | integer dimension(*) :: ia 478 | integer dimension(*) :: ja 479 | real*8 dimension(*) :: ra 480 | real*8 dimension(*) :: w 481 | real*8 :: row_prod 482 | end function row_prod 483 | 484 | function dot(x,y,mx,my,nx,ny,w) ! in :glmnet5:glmnet5.f 485 | real*8 dimension(*) :: x 486 | real*8 dimension(*) :: y 487 | integer dimension(*) :: mx 488 | integer dimension(*) :: my 489 | integer :: nx 490 | integer :: ny 491 | real*8 dimension(*) :: w 492 | real*8 :: dot 493 | end function dot 494 | 495 | subroutine lognet(parm,no,ni,nc,x,y,g,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 496 | real*8 :: parm 497 | integer intent(hide),check(shape(x,0)==no),depend(x) :: no=shape(x,0) 498 | integer intent(hide),check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 499 | integer check(shape(y,1)==max(2,nc)),depend(y) :: nc 500 | real*8 dimension(no,ni) :: x 501 | real*8 dimension(no,max(2,nc)),depend(no) :: y 502 | real*8 dimension(no,shape(y,1)),depend(no) :: g 503 | integer dimension(*) :: jd 504 | real*8 dimension(ni),depend(ni) :: vp 505 | real*8 dimension(2,ni),depend(ni) :: cl 506 | integer optional,depend(x) :: ne=min(shape(x,1), nx) 507 | integer :: nx 508 | integer optional,check((flmin < 1.0 || len(ulam)<=nlam)),depend(flmin,ulam) :: nlam=len(ulam) 509 | real*8 :: flmin 510 | real*8 dimension(nlam) :: ulam 511 | real*8 :: thr 512 | integer optional :: isd=1 ! Standardize predictors by default 513 | integer optional :: intr=1 ! Include intercept by default 514 | integer optional :: maxit=100000 515 | integer optional :: kopt=0 ! Use Newton's by default 516 | 517 | ! Outputs 518 | integer intent(out) :: lmu 519 | real*8 intent(out),dimension(nc,nlam),depend(nlam,nc) :: a0 520 | real*8 intent(out),dimension(nx,nc,nlam),depend(nlam,nc) :: ca 521 | integer intent(out),dimension(nx),depend(nx) :: ia 522 | integer intent(out),dimension(nlam),depend(nlam) :: nin 523 | real*8 intent(out),dimension(nlam),depend(nlam) :: dev0 524 | real*8 intent(out),dimension(nlam),depend(nlam) :: dev 525 | real*8 intent(out),dimension(nlam),depend(nlam) :: alm 526 | integer intent(out) :: nlp 527 | integer intent(out) :: jerr 528 | end subroutine lognet 529 | 530 | subroutine lstandard1(no,ni,x,w,ju,isd,intr,xm,xs) ! in :glmnet5:glmnet5.f 531 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 532 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 533 | real*8 dimension(no,ni) :: x 534 | real*8 dimension(no),depend(no) :: w 535 | integer dimension(ni),depend(ni) :: ju 536 | integer :: isd 537 | integer :: intr 538 | real*8 dimension(ni),depend(ni) :: xm 539 | real*8 dimension(ni),depend(ni) :: xs 540 | end subroutine lstandard1 541 | subroutine multlstandard1(no,ni,x,w,ju,isd,intr,xm,xs,xv) ! in :glmnet5:glmnet5.f 542 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 543 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 544 | real*8 dimension(no,ni) :: x 545 | real*8 dimension(no),depend(no) :: w 546 | integer dimension(ni),depend(ni) :: ju 547 | integer :: isd 548 | integer :: intr 549 | real*8 dimension(ni),depend(ni) :: xm 550 | real*8 dimension(ni),depend(ni) :: xs 551 | real*8 dimension(ni),depend(ni) :: xv 552 | end subroutine multlstandard1 553 | subroutine lognet2n(parm,no,ni,x,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,kopt,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 554 | real*8 :: parm 555 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 556 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 557 | real*8 dimension(no,ni) :: x 558 | real*8 dimension(no),depend(no) :: y 559 | real*8 dimension(no),depend(no) :: g 560 | real*8 dimension(no),depend(no) :: w 561 | integer dimension(ni),depend(ni) :: ju 562 | real*8 dimension(ni),depend(ni) :: vp 563 | real*8 dimension(2,ni),depend(ni) :: cl 564 | integer :: ne 565 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 566 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 567 | real*8 :: flmin 568 | real*8 dimension(nlam) :: ulam 569 | real*8 :: shri 570 | integer :: isd 571 | integer :: intr 572 | integer :: maxit 573 | integer :: kopt 574 | integer :: lmu 575 | real*8 dimension(nlam),depend(nlam) :: a0 576 | real*8 dimension(nx,nlam),depend(nlam) :: a 577 | integer dimension(nx),depend(nx) :: m 578 | integer dimension(nlam),depend(nlam) :: kin 579 | real*8 :: dev0 580 | real*8 dimension(nlam),depend(nlam) :: dev 581 | real*8 dimension(nlam),depend(nlam) :: alm 582 | integer :: nlp 583 | integer :: jerr 584 | end subroutine lognet2n 585 | function dev2(n,w,y,p,pmin) ! in :glmnet5:glmnet5.f 586 | integer, optional,check(len(w)>=n),depend(w) :: n=len(w) 587 | real*8 dimension(n) :: w 588 | real*8 dimension(n),depend(n) :: y 589 | real*8 dimension(n),depend(n) :: p 590 | real*8 :: pmin 591 | real*8 :: dev2 592 | end function dev2 593 | function azero(n,y,g,q,jerr) ! in :glmnet5:glmnet5.f 594 | integer, optional,check(len(y)>=n),depend(y) :: n=len(y) 595 | real*8 dimension(n) :: y 596 | real*8 dimension(n),depend(n) :: g 597 | real*8 dimension(n),depend(n) :: q 598 | integer :: jerr 599 | real*8 :: azero 600 | end function azero 601 | subroutine lognetn(parm,no,ni,nc,x,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,kopt,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 602 | real*8 :: parm 603 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 604 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 605 | integer, optional,check(shape(y,1)==nc),depend(y) :: nc=shape(y,1) 606 | real*8 dimension(no,ni) :: x 607 | real*8 dimension(no,nc),depend(no) :: y 608 | real*8 dimension(no,nc),depend(no,nc) :: g 609 | real*8 dimension(no),depend(no) :: w 610 | integer dimension(ni),depend(ni) :: ju 611 | real*8 dimension(ni),depend(ni) :: vp 612 | real*8 dimension(2,ni),depend(ni) :: cl 613 | integer :: ne 614 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 615 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 616 | real*8 :: flmin 617 | real*8 dimension(nlam) :: ulam 618 | real*8 :: shri 619 | integer :: isd 620 | integer :: intr 621 | integer :: maxit 622 | integer :: kopt 623 | integer :: lmu 624 | real*8 dimension(nc,nlam),depend(nc,nlam) :: a0 625 | real*8 dimension(nx,nc,nlam),depend(nc,nlam) :: a 626 | integer dimension(nx),depend(nx) :: m 627 | integer dimension(nlam),depend(nlam) :: kin 628 | real*8 :: dev0 629 | real*8 dimension(nlam),depend(nlam) :: dev 630 | real*8 dimension(nlam),depend(nlam) :: alm 631 | integer :: nlp 632 | integer :: jerr 633 | end subroutine lognetn 634 | subroutine kazero(kk,n,y,g,q,az,jerr) ! in :glmnet5:glmnet5.f 635 | integer, optional,check(shape(y,1)==kk),depend(y) :: kk=shape(y,1) 636 | integer, optional,check(shape(y,0)==n),depend(y) :: n=shape(y,0) 637 | real*8 dimension(n,kk) :: y 638 | real*8 dimension(n,kk),depend(n,kk) :: g 639 | real*8 dimension(n),depend(n) :: q 640 | real*8 dimension(kk),depend(kk) :: az 641 | integer :: jerr 642 | end subroutine kazero 643 | function elc(parm,n,cl,a,m) ! in :glmnet5:glmnet5.f 644 | real*8 :: parm 645 | integer, optional,check(len(a)>=n),depend(a) :: n=len(a) 646 | real*8 dimension(2) :: cl 647 | real*8 dimension(n) :: a 648 | integer dimension(n),depend(n) :: m 649 | real*8 :: elc 650 | end function elc 651 | function nintot(ni,nx,nc,a,m,nin,is) ! in :glmnet5:glmnet5.f 652 | integer, optional,check(len(is)>=ni),depend(is) :: ni=len(is) 653 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 654 | integer, optional,check(shape(a,1)==nc),depend(a) :: nc=shape(a,1) 655 | real*8 dimension(nx,nc) :: a 656 | integer dimension(nx),depend(nx) :: m 657 | integer :: nin 658 | integer dimension(ni) :: is 659 | integer :: nintot 660 | end function nintot 661 | subroutine luncomp(ni,nx,nc,ca,ia,nin,a) ! in :glmnet5:glmnet5.f 662 | integer, optional,check(shape(a,0)==ni),depend(a) :: ni=shape(a,0) 663 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 664 | integer, optional,check(shape(ca,1)==nc),depend(ca) :: nc=shape(ca,1) 665 | real*8 dimension(nx,nc) :: ca 666 | integer dimension(nx),depend(nx) :: ia 667 | integer :: nin 668 | real*8 dimension(ni,nc),depend(nc) :: a 669 | end subroutine luncomp 670 | subroutine lmodval(nt,x,nc,nx,a0,ca,ia,nin,ans) ! in :glmnet5:glmnet5.f 671 | integer, optional,check(shape(x,0)==nt),depend(x) :: nt=shape(x,0) 672 | real*8 dimension(nt,*) :: x 673 | integer, optional,check(len(a0)>=nc),depend(a0) :: nc=len(a0) 674 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 675 | real*8 dimension(nc) :: a0 676 | real*8 dimension(nx,nc),depend(nc) :: ca 677 | integer dimension(nx),depend(nx) :: ia 678 | integer :: nin 679 | real*8 dimension(nc,nt),depend(nc,nt) :: ans 680 | end subroutine lmodval 681 | 682 | subroutine splognet(parm,no,ni,nc,x,ix,jx,y,g,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 683 | real*8 :: parm 684 | integer :: no 685 | integer :: ni 686 | integer :: nc 687 | real*8 dimension(*) :: x 688 | integer dimension(*) :: ix 689 | integer dimension(*) :: jx 690 | real*8 dimension(no,max(2,nc)),depend(no,nc) :: y 691 | real*8 dimension(no,shape(y,1)),depend(no,y) :: g 692 | integer dimension(*) :: jd 693 | real*8 dimension(ni),depend(ni) :: vp 694 | real*8 dimension(2,ni),depend(ni) :: cl 695 | integer optional,depend(ni,nx) :: ne=min(ni,nx) 696 | integer :: nx 697 | integer optional,check((flmin < 1.0 || len(ulam)<=nlam)),depend(flmin,ulam) :: nlam=len(ulam) 698 | real*8 :: flmin 699 | real*8 dimension(nlam) :: ulam 700 | real*8 :: thr 701 | integer optional :: isd=1 ! Standardize predictors by default 702 | integer optional :: intr=1 ! Include intercept by default 703 | integer optional :: maxit=100000 704 | integer optional :: kopt=0 ! Use Newton's by default 705 | 706 | ! Outputs 707 | integer intent(out) :: lmu 708 | real*8 intent(out),dimension(nc,nlam),depend(nlam,nc) :: a0 709 | real*8 intent(out),dimension(nx,nc,nlam),depend(nx,nc,nlam) :: ca 710 | integer intent(out),dimension(nx),depend(nx) :: ia 711 | integer intent(out),dimension(nlam),depend(nlam) :: nin 712 | real*8 intent(out),dimension(nlam),depend(nlam) :: dev0 713 | real*8 intent(out),dimension(nlam),depend(nlam) :: dev 714 | real*8 intent(out),dimension(nlam),depend(nlam) :: alm 715 | integer intent(out) :: nlp 716 | integer intent(out) :: jerr 717 | end subroutine splognet 718 | 719 | subroutine multsplstandard2(no,ni,x,ix,jx,w,ju,isd,intr,xm,xs,xv) ! in :glmnet5:glmnet5.f 720 | integer, optional,check(len(w)>=no),depend(w) :: no=len(w) 721 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 722 | real*8 dimension(*) :: x 723 | integer dimension(*) :: ix 724 | integer dimension(*) :: jx 725 | real*8 dimension(no) :: w 726 | integer dimension(ni) :: ju 727 | integer :: isd 728 | integer :: intr 729 | real*8 dimension(ni),depend(ni) :: xm 730 | real*8 dimension(ni),depend(ni) :: xs 731 | real*8 dimension(ni),depend(ni) :: xv 732 | end subroutine multsplstandard2 733 | subroutine splstandard2(no,ni,x,ix,jx,w,ju,isd,intr,xm,xs) ! in :glmnet5:glmnet5.f 734 | integer, optional,check(len(w)>=no),depend(w) :: no=len(w) 735 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 736 | real*8 dimension(*) :: x 737 | integer dimension(*) :: ix 738 | integer dimension(*) :: jx 739 | real*8 dimension(no) :: w 740 | integer dimension(ni) :: ju 741 | integer :: isd 742 | integer :: intr 743 | real*8 dimension(ni),depend(ni) :: xm 744 | real*8 dimension(ni),depend(ni) :: xs 745 | end subroutine splstandard2 746 | subroutine sprlognet2n(parm,no,ni,x,ix,jx,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,kopt,xb,xs,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 747 | real*8 :: parm 748 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 749 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 750 | real*8 dimension(*) :: x 751 | integer dimension(*) :: ix 752 | integer dimension(*) :: jx 753 | real*8 dimension(no) :: y 754 | real*8 dimension(no),depend(no) :: g 755 | real*8 dimension(no),depend(no) :: w 756 | integer dimension(ni) :: ju 757 | real*8 dimension(ni),depend(ni) :: vp 758 | real*8 dimension(2,ni),depend(ni) :: cl 759 | integer :: ne 760 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 761 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 762 | real*8 :: flmin 763 | real*8 dimension(nlam) :: ulam 764 | real*8 :: shri 765 | integer :: isd 766 | integer :: intr 767 | integer :: maxit 768 | integer :: kopt 769 | real*8 dimension(ni),depend(ni) :: xb 770 | real*8 dimension(ni),depend(ni) :: xs 771 | integer :: lmu 772 | real*8 dimension(nlam),depend(nlam) :: a0 773 | real*8 dimension(nx,nlam),depend(nlam) :: a 774 | integer dimension(nx),depend(nx) :: m 775 | integer dimension(nlam),depend(nlam) :: kin 776 | real*8 :: dev0 777 | real*8 dimension(nlam),depend(nlam) :: dev 778 | real*8 dimension(nlam),depend(nlam) :: alm 779 | integer :: nlp 780 | integer :: jerr 781 | end subroutine sprlognet2n 782 | subroutine sprlognetn(parm,no,ni,nc,x,ix,jx,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,kopt,xb,xs,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 783 | real*8 :: parm 784 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 785 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 786 | integer, optional,check(shape(y,1)==nc),depend(y) :: nc=shape(y,1) 787 | real*8 dimension(*) :: x 788 | integer dimension(*) :: ix 789 | integer dimension(*) :: jx 790 | real*8 dimension(no,nc) :: y 791 | real*8 dimension(no,nc),depend(no,nc) :: g 792 | real*8 dimension(no),depend(no) :: w 793 | integer dimension(ni) :: ju 794 | real*8 dimension(ni),depend(ni) :: vp 795 | real*8 dimension(2,ni),depend(ni) :: cl 796 | integer :: ne 797 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 798 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 799 | real*8 :: flmin 800 | real*8 dimension(nlam) :: ulam 801 | real*8 :: shri 802 | integer :: isd 803 | integer :: intr 804 | integer :: maxit 805 | integer :: kopt 806 | real*8 dimension(ni),depend(ni) :: xb 807 | real*8 dimension(ni),depend(ni) :: xs 808 | integer :: lmu 809 | real*8 dimension(nc,nlam),depend(nc,nlam) :: a0 810 | real*8 dimension(nx,nc,nlam),depend(nc,nlam) :: a 811 | integer dimension(nx),depend(nx) :: m 812 | integer dimension(nlam),depend(nlam) :: kin 813 | real*8 :: dev0 814 | real*8 dimension(nlam),depend(nlam) :: dev 815 | real*8 dimension(nlam),depend(nlam) :: alm 816 | integer :: nlp 817 | integer :: jerr 818 | end subroutine sprlognetn 819 | subroutine lcmodval(nc,nx,a0,ca,ia,nin,x,ix,jx,n,f) ! in :glmnet5:glmnet5.f 820 | integer, optional,check(len(a0)>=nc),depend(a0) :: nc=len(a0) 821 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 822 | real*8 dimension(nc) :: a0 823 | real*8 dimension(nx,nc),depend(nc) :: ca 824 | integer dimension(*) :: ia 825 | integer :: nin 826 | real*8 dimension(*) :: x 827 | integer dimension(*) :: ix 828 | integer dimension(*) :: jx 829 | integer, optional,check(shape(f,1)==n),depend(f) :: n=shape(f,1) 830 | real*8 dimension(nc,n),depend(nc) :: f 831 | end subroutine lcmodval 832 | subroutine coxnet(parm,no,ni,x,y,d,g,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,maxit,isd,lmu,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 833 | real*8 :: parm 834 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 835 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 836 | real*8 dimension(no,ni) :: x 837 | real*8 dimension(no),depend(no) :: y 838 | real*8 dimension(no),depend(no) :: d 839 | real*8 dimension(no),depend(no) :: g 840 | real*8 dimension(no),depend(no) :: w 841 | integer dimension(*) :: jd 842 | real*8 dimension(ni),depend(ni) :: vp 843 | real*8 dimension(2,ni),depend(ni) :: cl 844 | integer :: ne 845 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 846 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 847 | real*8 :: flmin 848 | real*8 dimension(nlam) :: ulam 849 | real*8 :: thr 850 | integer :: maxit 851 | integer :: isd 852 | integer :: lmu 853 | real*8 dimension(nx,nlam),depend(nlam) :: ca 854 | integer dimension(nx),depend(nx) :: ia 855 | integer dimension(nlam),depend(nlam) :: nin 856 | real*8 :: dev0 857 | real*8 dimension(nlam),depend(nlam) :: dev 858 | real*8 dimension(nlam),depend(nlam) :: alm 859 | integer :: nlp 860 | integer :: jerr 861 | end subroutine coxnet 862 | subroutine cstandard(no,ni,x,w,ju,isd,xs) ! in :glmnet5:glmnet5.f 863 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 864 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 865 | real*8 dimension(no,ni) :: x 866 | real*8 dimension(no),depend(no) :: w 867 | integer dimension(ni),depend(ni) :: ju 868 | integer :: isd 869 | real*8 dimension(ni),depend(ni) :: xs 870 | end subroutine cstandard 871 | subroutine coxnet1(parm,no,ni,x,y,d,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,cthri,isd,maxit,lmu,ao,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 872 | real*8 :: parm 873 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 874 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 875 | real*8 dimension(no,ni) :: x 876 | real*8 dimension(no),depend(no) :: y 877 | real*8 dimension(no),depend(no) :: d 878 | real*8 dimension(no),depend(no) :: g 879 | real*8 dimension(no),depend(no) :: q 880 | integer dimension(ni),depend(ni) :: ju 881 | real*8 dimension(ni),depend(ni) :: vp 882 | real*8 dimension(2,ni),depend(ni) :: cl 883 | integer :: ne 884 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 885 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 886 | real*8 :: flmin 887 | real*8 dimension(nlam) :: ulam 888 | real*8 :: cthri 889 | integer :: isd 890 | integer :: maxit 891 | integer :: lmu 892 | real*8 dimension(nx,nlam),depend(nlam) :: ao 893 | integer dimension(nx),depend(nx) :: m 894 | integer dimension(nlam),depend(nlam) :: kin 895 | real*8 :: dev0 896 | real*8 dimension(nlam),depend(nlam) :: dev 897 | real*8 dimension(nlam),depend(nlam) :: alm 898 | integer :: nlp 899 | integer :: jerr 900 | end subroutine coxnet1 901 | subroutine cxmodval(ca,ia,nin,n,x,f) ! in :glmnet5:glmnet5.f 902 | real*8 dimension(nin) :: ca 903 | integer dimension(nin),depend(nin) :: ia 904 | integer, optional,check(len(ca)>=nin),depend(ca) :: nin=len(ca) 905 | integer, optional,check(shape(x,0)==n),depend(x) :: n=shape(x,0) 906 | real*8 dimension(n,*) :: x 907 | real*8 dimension(n),depend(n) :: f 908 | end subroutine cxmodval 909 | subroutine groups(no,y,d,q,nk,kp,jp,t0,jerr) ! in :glmnet5:glmnet5.f 910 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 911 | real*8 dimension(no) :: y 912 | real*8 dimension(no),depend(no) :: d 913 | real*8 dimension(no),depend(no) :: q 914 | integer :: nk 915 | integer dimension(*) :: kp 916 | integer dimension(no),depend(no) :: jp 917 | real*8 :: t0 918 | integer :: jerr 919 | end subroutine groups 920 | subroutine outer(no,nk,d,dk,kp,jp,e,wr,w,jerr,u) ! in :glmnet5:glmnet5.f 921 | integer, optional,check(len(d)>=no),depend(d) :: no=len(d) 922 | integer, optional,check(len(dk)>=nk),depend(dk) :: nk=len(dk) 923 | real*8 dimension(no) :: d 924 | real*8 dimension(nk) :: dk 925 | integer dimension(nk),depend(nk) :: kp 926 | integer dimension(no),depend(no) :: jp 927 | real*8 dimension(no),depend(no) :: e 928 | real*8 dimension(no),depend(no) :: wr 929 | real*8 dimension(no),depend(no) :: w 930 | integer :: jerr 931 | real*8 dimension(no),depend(no) :: u 932 | end subroutine outer 933 | subroutine vars(no,ni,x,w,ixx,v) ! in :glmnet5:glmnet5.f 934 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 935 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 936 | real*8 dimension(no,ni) :: x 937 | real*8 dimension(no),depend(no) :: w 938 | integer dimension(ni),depend(ni) :: ixx 939 | real*8 dimension(ni),depend(ni) :: v 940 | end subroutine vars 941 | subroutine died(no,nk,d,kp,jp,dk) ! in :glmnet5:glmnet5.f 942 | integer, optional,check(len(d)>=no),depend(d) :: no=len(d) 943 | integer, optional,check(len(kp)>=nk),depend(kp) :: nk=len(kp) 944 | real*8 dimension(no) :: d 945 | integer dimension(nk) :: kp 946 | integer dimension(no),depend(no) :: jp 947 | real*8 dimension(nk),depend(nk) :: dk 948 | end subroutine died 949 | subroutine usk(no,nk,kp,jp,e,u) ! in :glmnet5:glmnet5.f 950 | integer, optional,check(len(jp)>=no),depend(jp) :: no=len(jp) 951 | integer, optional,check(len(kp)>=nk),depend(kp) :: nk=len(kp) 952 | integer dimension(nk) :: kp 953 | integer dimension(no) :: jp 954 | real*8 dimension(no),depend(no) :: e 955 | real*8 dimension(nk),depend(nk) :: u 956 | end subroutine usk 957 | function risk(no,ni,nk,d,dk,f,e,kp,jp,u) ! in :glmnet5:glmnet5.f 958 | integer, optional,check(len(d)>=no),depend(d) :: no=len(d) 959 | integer :: ni 960 | integer, optional,check(len(dk)>=nk),depend(dk) :: nk=len(dk) 961 | real*8 dimension(no) :: d 962 | real*8 dimension(nk) :: dk 963 | real*8 dimension(no),depend(no) :: f 964 | real*8 dimension(no),depend(no) :: e 965 | integer dimension(nk),depend(nk) :: kp 966 | integer dimension(no),depend(no) :: jp 967 | real*8 dimension(nk),depend(nk) :: u 968 | real*8 :: risk 969 | end function risk 970 | subroutine loglike(no,ni,x,y,d,g,w,nlam,a,flog,jerr) ! in :glmnet5:glmnet5.f 971 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 972 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 973 | real*8 dimension(no,ni) :: x 974 | real*8 dimension(no),depend(no) :: y 975 | real*8 dimension(no),depend(no) :: d 976 | real*8 dimension(no),depend(no) :: g 977 | real*8 dimension(no),depend(no) :: w 978 | integer, optional,check(shape(a,1)==nlam),depend(a) :: nlam=shape(a,1) 979 | real*8 dimension(ni,nlam),depend(ni) :: a 980 | real*8 dimension(nlam),depend(nlam) :: flog 981 | integer :: jerr 982 | end subroutine loglike 983 | subroutine fishnet(parm,no,ni,x,y,g,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 984 | real*8 :: parm 985 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 986 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 987 | real*8 dimension(no,ni) :: x 988 | real*8 dimension(no),depend(no) :: y 989 | real*8 dimension(no),depend(no) :: g 990 | real*8 dimension(no),depend(no) :: w 991 | integer dimension(*) :: jd 992 | real*8 dimension(ni),depend(ni) :: vp 993 | real*8 dimension(2,ni),depend(ni) :: cl 994 | integer :: ne 995 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 996 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 997 | real*8 :: flmin 998 | real*8 dimension(nlam) :: ulam 999 | real*8 :: thr 1000 | integer :: isd 1001 | integer :: intr 1002 | integer :: maxit 1003 | integer :: lmu 1004 | real*8 dimension(nlam),depend(nlam) :: a0 1005 | real*8 dimension(nx,nlam),depend(nlam) :: ca 1006 | integer dimension(nx),depend(nx) :: ia 1007 | integer dimension(nlam),depend(nlam) :: nin 1008 | real*8 :: dev0 1009 | real*8 dimension(nlam),depend(nlam) :: dev 1010 | real*8 dimension(nlam),depend(nlam) :: alm 1011 | integer :: nlp 1012 | integer :: jerr 1013 | end subroutine fishnet 1014 | subroutine fishnet1(parm,no,ni,x,y,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,lmu,a0,ca,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1015 | real*8 :: parm 1016 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1017 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1018 | real*8 dimension(no,ni) :: x 1019 | real*8 dimension(no),depend(no) :: y 1020 | real*8 dimension(no),depend(no) :: g 1021 | real*8 dimension(no),depend(no) :: q 1022 | integer dimension(ni),depend(ni) :: ju 1023 | real*8 dimension(ni),depend(ni) :: vp 1024 | real*8 dimension(2,ni),depend(ni) :: cl 1025 | integer :: ne 1026 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1027 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1028 | real*8 :: flmin 1029 | real*8 dimension(nlam) :: ulam 1030 | real*8 :: shri 1031 | integer :: isd 1032 | integer :: intr 1033 | integer :: maxit 1034 | integer :: lmu 1035 | real*8 dimension(nlam),depend(nlam) :: a0 1036 | real*8 dimension(nx,nlam),depend(nlam) :: ca 1037 | integer dimension(nx),depend(nx) :: m 1038 | integer dimension(nlam),depend(nlam) :: kin 1039 | real*8 :: dev0 1040 | real*8 dimension(nlam),depend(nlam) :: dev 1041 | real*8 dimension(nlam),depend(nlam) :: alm 1042 | integer :: nlp 1043 | integer :: jerr 1044 | end subroutine fishnet1 1045 | function nonzero(n,v) ! in :glmnet5:glmnet5.f 1046 | integer, optional,check(len(v)>=n),depend(v) :: n=len(v) 1047 | real*8 dimension(n) :: v 1048 | integer :: nonzero 1049 | end function nonzero 1050 | subroutine solns(ni,nx,lmu,a,ia,nin,b) ! in :glmnet5:glmnet5.f 1051 | integer, optional,check(shape(b,0)==ni),depend(b) :: ni=shape(b,0) 1052 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 1053 | integer, optional,check(shape(a,1)==lmu),depend(a) :: lmu=shape(a,1) 1054 | real*8 dimension(nx,lmu) :: a 1055 | integer dimension(nx),depend(nx) :: ia 1056 | integer dimension(lmu),depend(lmu) :: nin 1057 | real*8 dimension(ni,lmu),depend(lmu) :: b 1058 | end subroutine solns 1059 | subroutine lsolns(ni,nx,nc,lmu,a,ia,nin,b) ! in :glmnet5:glmnet5.f 1060 | integer, optional,check(shape(b,0)==ni),depend(b) :: ni=shape(b,0) 1061 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 1062 | integer, optional,check(shape(a,1)==nc),depend(a) :: nc=shape(a,1) 1063 | integer, optional,check(shape(a,2)==lmu),depend(a) :: lmu=shape(a,2) 1064 | real*8 dimension(nx,nc,lmu) :: a 1065 | integer dimension(nx),depend(nx) :: ia 1066 | integer dimension(lmu),depend(lmu) :: nin 1067 | real*8 dimension(ni,nc,lmu),depend(nc,lmu) :: b 1068 | end subroutine lsolns 1069 | subroutine deviance(no,ni,x,y,g,q,nlam,a0,a,flog,jerr) ! in :glmnet5:glmnet5.f 1070 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1071 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1072 | real*8 dimension(no,ni) :: x 1073 | real*8 dimension(no),depend(no) :: y 1074 | real*8 dimension(no),depend(no) :: g 1075 | real*8 dimension(no),depend(no) :: q 1076 | integer, optional,check(len(a0)>=nlam),depend(a0) :: nlam=len(a0) 1077 | real*8 dimension(nlam) :: a0 1078 | real*8 dimension(ni,nlam),depend(ni,nlam) :: a 1079 | real*8 dimension(nlam),depend(nlam) :: flog 1080 | integer :: jerr 1081 | end subroutine deviance 1082 | subroutine spfishnet(parm,no,ni,x,ix,jx,y,g,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1083 | real*8 :: parm 1084 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 1085 | integer, optional,check(len(vp)>=ni),depend(vp) :: ni=len(vp) 1086 | real*8 dimension(*) :: x 1087 | integer dimension(*) :: ix 1088 | integer dimension(*) :: jx 1089 | real*8 dimension(no) :: y 1090 | real*8 dimension(no),depend(no) :: g 1091 | real*8 dimension(no),depend(no) :: w 1092 | integer dimension(*) :: jd 1093 | real*8 dimension(ni) :: vp 1094 | real*8 dimension(2,ni),depend(ni) :: cl 1095 | integer :: ne 1096 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1097 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1098 | real*8 :: flmin 1099 | real*8 dimension(nlam) :: ulam 1100 | real*8 :: thr 1101 | integer :: isd 1102 | integer :: intr 1103 | integer :: maxit 1104 | integer :: lmu 1105 | real*8 dimension(nlam),depend(nlam) :: a0 1106 | real*8 dimension(nx,nlam),depend(nlam) :: ca 1107 | integer dimension(nx),depend(nx) :: ia 1108 | integer dimension(nlam),depend(nlam) :: nin 1109 | real*8 :: dev0 1110 | real*8 dimension(nlam),depend(nlam) :: dev 1111 | real*8 dimension(nlam),depend(nlam) :: alm 1112 | integer :: nlp 1113 | integer :: jerr 1114 | end subroutine spfishnet 1115 | subroutine spfishnet1(parm,no,ni,x,ix,jx,y,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,isd,intr,maxit,xb,xs,lmu,a0,ca,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1116 | real*8 :: parm 1117 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 1118 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 1119 | real*8 dimension(*) :: x 1120 | integer dimension(*) :: ix 1121 | integer dimension(*) :: jx 1122 | real*8 dimension(no) :: y 1123 | real*8 dimension(no),depend(no) :: g 1124 | real*8 dimension(no),depend(no) :: q 1125 | integer dimension(ni) :: ju 1126 | real*8 dimension(ni),depend(ni) :: vp 1127 | real*8 dimension(2,ni),depend(ni) :: cl 1128 | integer :: ne 1129 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1130 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1131 | real*8 :: flmin 1132 | real*8 dimension(nlam) :: ulam 1133 | real*8 :: shri 1134 | integer :: isd 1135 | integer :: intr 1136 | integer :: maxit 1137 | real*8 dimension(ni),depend(ni) :: xb 1138 | real*8 dimension(ni),depend(ni) :: xs 1139 | integer :: lmu 1140 | real*8 dimension(nlam),depend(nlam) :: a0 1141 | real*8 dimension(nx,nlam),depend(nlam) :: ca 1142 | integer dimension(nx),depend(nx) :: m 1143 | integer dimension(nlam),depend(nlam) :: kin 1144 | real*8 :: dev0 1145 | real*8 dimension(nlam),depend(nlam) :: dev 1146 | real*8 dimension(nlam),depend(nlam) :: alm 1147 | integer :: nlp 1148 | integer :: jerr 1149 | end subroutine spfishnet1 1150 | subroutine spdeviance(no,ni,x,ix,jx,y,g,q,nlam,a0,a,flog,jerr) ! in :glmnet5:glmnet5.f 1151 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 1152 | integer, optional,check(shape(a,0)==ni),depend(a) :: ni=shape(a,0) 1153 | real*8 dimension(*) :: x 1154 | integer dimension(*) :: ix 1155 | integer dimension(*) :: jx 1156 | real*8 dimension(no) :: y 1157 | real*8 dimension(no),depend(no) :: g 1158 | real*8 dimension(no),depend(no) :: q 1159 | integer, optional,check(len(a0)>=nlam),depend(a0) :: nlam=len(a0) 1160 | real*8 dimension(nlam) :: a0 1161 | real*8 dimension(ni,nlam),depend(nlam) :: a 1162 | real*8 dimension(nlam),depend(nlam) :: flog 1163 | integer :: jerr 1164 | end subroutine spdeviance 1165 | subroutine cspdeviance(no,x,ix,jx,y,g,q,nx,nlam,a0,ca,ia,nin,flog,jerr) ! in :glmnet5:glmnet5.f 1166 | integer, optional,check(len(y)>=no),depend(y) :: no=len(y) 1167 | real*8 dimension(*) :: x 1168 | integer dimension(*) :: ix 1169 | integer dimension(*) :: jx 1170 | real*8 dimension(no) :: y 1171 | real*8 dimension(no),depend(no) :: g 1172 | real*8 dimension(no),depend(no) :: q 1173 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1174 | integer, optional,check(len(a0)>=nlam),depend(a0) :: nlam=len(a0) 1175 | real*8 dimension(nlam) :: a0 1176 | real*8 dimension(nx,nlam),depend(nlam) :: ca 1177 | integer dimension(nx),depend(nx) :: ia 1178 | integer dimension(nlam),depend(nlam) :: nin 1179 | real*8 dimension(nlam),depend(nlam) :: flog 1180 | integer :: jerr 1181 | end subroutine cspdeviance 1182 | subroutine multelnet(parm,no,ni,nr,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1183 | real*8 :: parm 1184 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1185 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1186 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1187 | real*8 dimension(no,ni) :: x 1188 | real*8 dimension(no,nr),depend(no) :: y 1189 | real*8 dimension(no),depend(no) :: w 1190 | integer dimension(*) :: jd 1191 | real*8 dimension(ni),depend(ni) :: vp 1192 | real*8 dimension(2,ni),depend(ni) :: cl 1193 | integer :: ne 1194 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1195 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1196 | real*8 :: flmin 1197 | real*8 dimension(nlam) :: ulam 1198 | real*8 :: thr 1199 | integer :: isd 1200 | integer :: jsd 1201 | integer :: intr 1202 | integer :: maxit 1203 | integer :: lmu 1204 | real*8 dimension(nr,nlam),depend(nr,nlam) :: a0 1205 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ca 1206 | integer dimension(nx),depend(nx) :: ia 1207 | integer dimension(nlam),depend(nlam) :: nin 1208 | real*8 dimension(nlam),depend(nlam) :: rsq 1209 | real*8 dimension(nlam),depend(nlam) :: alm 1210 | integer :: nlp 1211 | integer :: jerr 1212 | end subroutine multelnet 1213 | subroutine multelnetn(parm,no,ni,nr,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1214 | real*8 :: parm 1215 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1216 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1217 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1218 | real*8 dimension(no,ni) :: x 1219 | real*8 dimension(no,nr),depend(no) :: y 1220 | real*8 dimension(no),depend(no) :: w 1221 | integer dimension(*) :: jd 1222 | real*8 dimension(ni),depend(ni) :: vp 1223 | real*8 dimension(2,ni),depend(ni) :: cl 1224 | integer :: ne 1225 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1226 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1227 | real*8 :: flmin 1228 | real*8 dimension(nlam) :: ulam 1229 | real*8 :: thr 1230 | integer :: isd 1231 | integer :: jsd 1232 | integer :: intr 1233 | integer :: maxit 1234 | integer :: lmu 1235 | real*8 dimension(nr,nlam),depend(nr,nlam) :: a0 1236 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ca 1237 | integer dimension(nx),depend(nx) :: ia 1238 | integer dimension(nlam),depend(nlam) :: nin 1239 | real*8 dimension(nlam),depend(nlam) :: rsq 1240 | real*8 dimension(nlam),depend(nlam) :: alm 1241 | integer :: nlp 1242 | integer :: jerr 1243 | end subroutine multelnetn 1244 | subroutine multstandard1(no,ni,nr,x,y,w,isd,jsd,intr,ju,xm,xs,ym,ys,xv,ys0,jerr) ! in :glmnet5:glmnet5.f 1245 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1246 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1247 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1248 | real*8 dimension(no,ni) :: x 1249 | real*8 dimension(no,nr),depend(no) :: y 1250 | real*8 dimension(no),depend(no) :: w 1251 | integer :: isd 1252 | integer :: jsd 1253 | integer :: intr 1254 | integer dimension(ni),depend(ni) :: ju 1255 | real*8 dimension(ni),depend(ni) :: xm 1256 | real*8 dimension(ni),depend(ni) :: xs 1257 | real*8 dimension(nr),depend(nr) :: ym 1258 | real*8 dimension(nr),depend(nr) :: ys 1259 | real*8 dimension(ni),depend(ni) :: xv 1260 | real*8 :: ys0 1261 | integer :: jerr 1262 | end subroutine multstandard1 1263 | subroutine multelnet2(beta,ni,nr,ju,vp,cl,y,no,ne,nx,x,nlam,flmin,ulam,thri,maxit,xv,ys0,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 1264 | real*8 :: beta 1265 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 1266 | integer, optional,check(shape(cl,1)==nr),depend(cl) :: nr=shape(cl,1) 1267 | integer dimension(ni) :: ju 1268 | real*8 dimension(ni),depend(ni) :: vp 1269 | real*8 dimension(2,nr,ni),depend(ni) :: cl 1270 | real*8 dimension(no,nr),depend(nr) :: y 1271 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1272 | integer :: ne 1273 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 1274 | real*8 dimension(no,ni),depend(no,ni) :: x 1275 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1276 | real*8 :: flmin 1277 | real*8 dimension(nlam) :: ulam 1278 | real*8 :: thri 1279 | integer :: maxit 1280 | real*8 dimension(ni),depend(ni) :: xv 1281 | real*8 :: ys0 1282 | integer :: lmu 1283 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ao 1284 | integer dimension(nx),depend(nx) :: ia 1285 | integer dimension(nlam),depend(nlam) :: kin 1286 | real*8 dimension(nlam),depend(nlam) :: rsqo 1287 | real*8 dimension(nlam),depend(nlam) :: almo 1288 | integer :: nlp 1289 | integer :: jerr 1290 | end subroutine multelnet2 1291 | subroutine chkbnds(nr,gk,gkn,xv,cl,al1,al2,a,isc,jerr) ! in :glmnet5:glmnet5.f 1292 | integer, optional,check(len(gk)>=nr),depend(gk) :: nr=len(gk) 1293 | real*8 dimension(nr) :: gk 1294 | real*8 :: gkn 1295 | real*8 :: xv 1296 | real*8 dimension(2,nr),depend(nr) :: cl 1297 | real*8 :: al1 1298 | real*8 :: al2 1299 | real*8 dimension(nr),depend(nr) :: a 1300 | integer dimension(nr),depend(nr) :: isc 1301 | integer :: jerr 1302 | end subroutine chkbnds 1303 | subroutine chkbnds1(nr,gk,gkn,xv,cl1,cl2,al1,al2,a,isc,jerr) ! in :glmnet5:glmnet5.f 1304 | integer, optional,check(len(gk)>=nr),depend(gk) :: nr=len(gk) 1305 | real*8 dimension(nr) :: gk 1306 | real*8 :: gkn 1307 | real*8 :: xv 1308 | real*8 :: cl1 1309 | real*8 :: cl2 1310 | real*8 :: al1 1311 | real*8 :: al2 1312 | real*8 dimension(nr),depend(nr) :: a 1313 | integer dimension(nr),depend(nr) :: isc 1314 | integer :: jerr 1315 | end subroutine chkbnds1 1316 | function bnorm(b0,al1p,al2p,g,usq,jerr) ! in :glmnet5:glmnet5.f 1317 | real*8 :: b0 1318 | real*8 :: al1p 1319 | real*8 :: al2p 1320 | real*8 :: g 1321 | real*8 :: usq 1322 | integer :: jerr 1323 | real*8 :: bnorm 1324 | end function bnorm 1325 | subroutine chg_bnorm(arg,irg) ! in :glmnet5:glmnet5.f 1326 | real*8 :: arg 1327 | integer :: irg 1328 | end subroutine chg_bnorm 1329 | subroutine get_bnorm(arg,irg) ! in :glmnet5:glmnet5.f 1330 | real*8 :: arg 1331 | integer :: irg 1332 | end subroutine get_bnorm 1333 | subroutine multsolns(ni,nx,nr,lmu,a,ia,nin,b) ! in :glmnet5:glmnet5.f 1334 | integer, optional,check(shape(b,0)==ni),depend(b) :: ni=shape(b,0) 1335 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 1336 | integer, optional,check(shape(a,1)==nr),depend(a) :: nr=shape(a,1) 1337 | integer, optional,check(shape(a,2)==lmu),depend(a) :: lmu=shape(a,2) 1338 | real*8 dimension(nx,nr,lmu) :: a 1339 | integer dimension(nx),depend(nx) :: ia 1340 | integer dimension(lmu),depend(lmu) :: nin 1341 | real*8 dimension(ni,nr,lmu),depend(nr,lmu) :: b 1342 | end subroutine multsolns 1343 | subroutine multuncomp(ni,nr,nx,ca,ia,nin,a) ! in :glmnet5:glmnet5.f 1344 | integer, optional,check(shape(a,0)==ni),depend(a) :: ni=shape(a,0) 1345 | integer, optional,check(shape(ca,1)==nr),depend(ca) :: nr=shape(ca,1) 1346 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1347 | real*8 dimension(nx,nr) :: ca 1348 | integer dimension(nx),depend(nx) :: ia 1349 | integer :: nin 1350 | real*8 dimension(ni,nr),depend(nr) :: a 1351 | end subroutine multuncomp 1352 | subroutine multmodval(nx,nr,a0,ca,ia,nin,n,x,f) ! in :glmnet5:glmnet5.f 1353 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1354 | integer, optional,check(len(a0)>=nr),depend(a0) :: nr=len(a0) 1355 | real*8 dimension(nr) :: a0 1356 | real*8 dimension(nx,nr),depend(nr) :: ca 1357 | integer dimension(nx),depend(nx) :: ia 1358 | integer :: nin 1359 | integer, optional,check(shape(x,0)==n),depend(x) :: n=shape(x,0) 1360 | real*8 dimension(n,*) :: x 1361 | real*8 dimension(nr,n),depend(nr,n) :: f 1362 | end subroutine multmodval 1363 | subroutine multspelnet(parm,no,ni,nr,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1364 | real*8 :: parm 1365 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1366 | integer, optional,check(len(vp)>=ni),depend(vp) :: ni=len(vp) 1367 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1368 | real*8 dimension(*) :: x 1369 | integer dimension(*) :: ix 1370 | integer dimension(*) :: jx 1371 | real*8 dimension(no,nr) :: y 1372 | real*8 dimension(no),depend(no) :: w 1373 | integer dimension(*) :: jd 1374 | real*8 dimension(ni) :: vp 1375 | real*8 dimension(2,ni),depend(ni) :: cl 1376 | integer :: ne 1377 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1378 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1379 | real*8 :: flmin 1380 | real*8 dimension(nlam) :: ulam 1381 | real*8 :: thr 1382 | integer :: isd 1383 | integer :: jsd 1384 | integer :: intr 1385 | integer :: maxit 1386 | integer :: lmu 1387 | real*8 dimension(nr,nlam),depend(nr,nlam) :: a0 1388 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ca 1389 | integer dimension(nx),depend(nx) :: ia 1390 | integer dimension(nlam),depend(nlam) :: nin 1391 | real*8 dimension(nlam),depend(nlam) :: rsq 1392 | real*8 dimension(nlam),depend(nlam) :: alm 1393 | integer :: nlp 1394 | integer :: jerr 1395 | end subroutine multspelnet 1396 | subroutine multspelnetn(parm,no,ni,nr,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1397 | real*8 :: parm 1398 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1399 | integer, optional,check(len(vp)>=ni),depend(vp) :: ni=len(vp) 1400 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1401 | real*8 dimension(*) :: x 1402 | integer dimension(*) :: ix 1403 | integer dimension(*) :: jx 1404 | real*8 dimension(no,nr) :: y 1405 | real*8 dimension(no),depend(no) :: w 1406 | integer dimension(*) :: jd 1407 | real*8 dimension(ni) :: vp 1408 | real*8 dimension(2,ni),depend(ni) :: cl 1409 | integer :: ne 1410 | integer, optional,check(shape(ca,0)==nx),depend(ca) :: nx=shape(ca,0) 1411 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1412 | real*8 :: flmin 1413 | real*8 dimension(nlam) :: ulam 1414 | real*8 :: thr 1415 | integer :: isd 1416 | integer :: jsd 1417 | integer :: intr 1418 | integer :: maxit 1419 | integer :: lmu 1420 | real*8 dimension(nr,nlam),depend(nr,nlam) :: a0 1421 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ca 1422 | integer dimension(nx),depend(nx) :: ia 1423 | integer dimension(nlam),depend(nlam) :: nin 1424 | real*8 dimension(nlam),depend(nlam) :: rsq 1425 | real*8 dimension(nlam),depend(nlam) :: alm 1426 | integer :: nlp 1427 | integer :: jerr 1428 | end subroutine multspelnetn 1429 | subroutine multspstandard1(no,ni,nr,x,ix,jx,y,w,ju,isd,jsd,intr,xm,xs,ym,ys,xv,ys0,jerr) ! in :glmnet5:glmnet5.f 1430 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1431 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 1432 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1433 | real*8 dimension(*) :: x 1434 | integer dimension(*) :: ix 1435 | integer dimension(*) :: jx 1436 | real*8 dimension(no,nr) :: y 1437 | real*8 dimension(no),depend(no) :: w 1438 | integer dimension(ni) :: ju 1439 | integer :: isd 1440 | integer :: jsd 1441 | integer :: intr 1442 | real*8 dimension(ni),depend(ni) :: xm 1443 | real*8 dimension(ni),depend(ni) :: xs 1444 | real*8 dimension(nr),depend(nr) :: ym 1445 | real*8 dimension(nr),depend(nr) :: ys 1446 | real*8 dimension(ni),depend(ni) :: xv 1447 | real*8 :: ys0 1448 | integer :: jerr 1449 | end subroutine multspstandard1 1450 | subroutine multspelnet2(beta,ni,nr,y,w,no,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,ulam,thri,maxit,xm,xs,xv,ys0,lmu,ao,ia,kin,rsqo,almo,nlp,jerr) ! in :glmnet5:glmnet5.f 1451 | real*8 :: beta 1452 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 1453 | integer, optional,check(shape(y,1)==nr),depend(y) :: nr=shape(y,1) 1454 | real*8 dimension(no,nr) :: y 1455 | real*8 dimension(no),depend(no) :: w 1456 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1457 | integer :: ne 1458 | integer, optional,check(shape(ao,0)==nx),depend(ao) :: nx=shape(ao,0) 1459 | real*8 dimension(*) :: x 1460 | integer dimension(*) :: ix 1461 | integer dimension(*) :: jx 1462 | integer dimension(ni) :: ju 1463 | real*8 dimension(ni),depend(ni) :: vp 1464 | real*8 dimension(2,nr,ni),depend(nr,ni) :: cl 1465 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1466 | real*8 :: flmin 1467 | real*8 dimension(nlam) :: ulam 1468 | real*8 :: thri 1469 | integer :: maxit 1470 | real*8 dimension(ni),depend(ni) :: xm 1471 | real*8 dimension(ni),depend(ni) :: xs 1472 | real*8 dimension(ni),depend(ni) :: xv 1473 | real*8 :: ys0 1474 | integer :: lmu 1475 | real*8 dimension(nx,nr,nlam),depend(nr,nlam) :: ao 1476 | integer dimension(nx),depend(nx) :: ia 1477 | integer dimension(nlam),depend(nlam) :: kin 1478 | real*8 dimension(nlam),depend(nlam) :: rsqo 1479 | real*8 dimension(nlam),depend(nlam) :: almo 1480 | integer :: nlp 1481 | integer :: jerr 1482 | end subroutine multspelnet2 1483 | subroutine multlognetn(parm,no,ni,nc,x,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,intr,maxit,xv,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1484 | real*8 :: parm 1485 | integer, optional,check(shape(x,0)==no),depend(x) :: no=shape(x,0) 1486 | integer, optional,check(shape(x,1)==ni),depend(x) :: ni=shape(x,1) 1487 | integer, optional,check(shape(y,1)==nc),depend(y) :: nc=shape(y,1) 1488 | real*8 dimension(no,ni) :: x 1489 | real*8 dimension(no,nc),depend(no) :: y 1490 | real*8 dimension(no,nc),depend(no,nc) :: g 1491 | real*8 dimension(no),depend(no) :: w 1492 | integer dimension(ni),depend(ni) :: ju 1493 | real*8 dimension(ni),depend(ni) :: vp 1494 | real*8 dimension(2,ni),depend(ni) :: cl 1495 | integer :: ne 1496 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 1497 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1498 | real*8 :: flmin 1499 | real*8 dimension(nlam) :: ulam 1500 | real*8 :: shri 1501 | integer :: intr 1502 | integer :: maxit 1503 | real*8 dimension(ni),depend(ni) :: xv 1504 | integer :: lmu 1505 | real*8 dimension(nc,nlam),depend(nc,nlam) :: a0 1506 | real*8 dimension(nx,nc,nlam),depend(nc,nlam) :: a 1507 | integer dimension(nx),depend(nx) :: m 1508 | integer dimension(nlam),depend(nlam) :: kin 1509 | real*8 :: dev0 1510 | real*8 dimension(nlam),depend(nlam) :: dev 1511 | real*8 dimension(nlam),depend(nlam) :: alm 1512 | integer :: nlp 1513 | integer :: jerr 1514 | end subroutine multlognetn 1515 | subroutine multsprlognetn(parm,no,ni,nc,x,ix,jx,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,intr,maxit,xv,xb,xs,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr) ! in :glmnet5:glmnet5.f 1516 | real*8 :: parm 1517 | integer, optional,check(shape(y,0)==no),depend(y) :: no=shape(y,0) 1518 | integer, optional,check(len(ju)>=ni),depend(ju) :: ni=len(ju) 1519 | integer, optional,check(shape(y,1)==nc),depend(y) :: nc=shape(y,1) 1520 | real*8 dimension(*) :: x 1521 | integer dimension(*) :: ix 1522 | integer dimension(*) :: jx 1523 | real*8 dimension(no,nc) :: y 1524 | real*8 dimension(no,nc),depend(no,nc) :: g 1525 | real*8 dimension(no),depend(no) :: w 1526 | integer dimension(ni) :: ju 1527 | real*8 dimension(ni),depend(ni) :: vp 1528 | real*8 dimension(2,ni),depend(ni) :: cl 1529 | integer :: ne 1530 | integer, optional,check(shape(a,0)==nx),depend(a) :: nx=shape(a,0) 1531 | integer, optional,check(len(ulam)>=nlam),depend(ulam) :: nlam=len(ulam) 1532 | real*8 :: flmin 1533 | real*8 dimension(nlam) :: ulam 1534 | real*8 :: shri 1535 | integer :: intr 1536 | integer :: maxit 1537 | real*8 dimension(ni),depend(ni) :: xv 1538 | real*8 dimension(ni),depend(ni) :: xb 1539 | real*8 dimension(ni),depend(ni) :: xs 1540 | integer :: lmu 1541 | real*8 dimension(nc,nlam),depend(nc,nlam) :: a0 1542 | real*8 dimension(nx,nc,nlam),depend(nc,nlam) :: a 1543 | integer dimension(nx),depend(nx) :: m 1544 | integer dimension(nlam),depend(nlam) :: kin 1545 | real*8 :: dev0 1546 | real*8 dimension(nlam),depend(nlam) :: dev 1547 | real*8 dimension(nlam),depend(nlam) :: alm 1548 | integer :: nlp 1549 | integer :: jerr 1550 | end subroutine multsprlognetn 1551 | subroutine psort7(v,a,ii,jj) ! in :glmnet5:glmnet5.f 1552 | real*8 dimension(jj) :: v 1553 | integer dimension(jj),depend(jj) :: a 1554 | integer :: ii 1555 | integer, optional,check(len(v)>=jj),depend(v) :: jj=len(v) 1556 | end subroutine psort7 1557 | end interface 1558 | end python module glmnet5 1559 | 1560 | ! This file was auto-generated with f2py (version:2). 1561 | ! See http://cens.ioc.ee/projects/f2py2e/ 1562 | -------------------------------------------------------------------------------- /glmnet/glmnet_config.py: -------------------------------------------------------------------------------- 1 | _DEFAULT_THRESH = 1.0e-4 2 | _DEFAULT_FLMIN = 0.001 3 | _DEFAULT_NLAM = 100 4 | -------------------------------------------------------------------------------- /glmnet/logistic_net.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | from scipy.sparse import issparse 3 | import _glmnet 4 | from glmnet import GlmNet 5 | 6 | class LogisticNet(GlmNet): 7 | '''The logistic net: a multivariate logistic model with both L1 and L2 8 | regularizers. 9 | 10 | This class implements the logistic net class of predictive models. These 11 | models combine the classical ridge and lasso regularizers into a combined 12 | penalty to the binomial deviance loss function. More specifically, the 13 | loss function minimized by this model is: 14 | 15 | L(\beta_0, \beta_1, ..., \beta_n) = 16 | BinDev(\beta_0, \beta_1, ..., \beta_n; X, y) + 17 | \lambda * ((\alpha - 1)/2 * | \beta |_2 + \alpha * | \beta |_1) 18 | 19 | where BinDev is the usual binomial deviance: 20 | 21 | BinDev(\beta_0, \beta_1, ..., \beta_n; X, y) = 22 | -2*sum( y*log(1 - p) + (1 - y)*log(1 - p) ) 23 | 24 | in which p is formed by applying the usual logistic function to a linear 25 | predictor. 26 | ''' 27 | 28 | def fit(self, X, y, col_names=None, 29 | lambdas=None, weights=None, rel_penalties=None, 30 | excl_preds=None, box_constraints=None, offsets=None): 31 | '''Fit a logistic or multinomial net model. 32 | 33 | Arguments: 34 | 35 | * X: The model matrix. A n_obs * n_preds array. 36 | * y: The response. This method accepts the response in two 37 | differnt configurations: 38 | 39 | - An n_obs * n_classes array. In this case, each column in y must 40 | be of boolean (0, 1) type indicating whether the observation is 41 | or is not of a given class. 42 | - An n_obs array. In this case the array must contain a discrete 43 | number of values, and is converted into the previous form before 44 | being passed to the model. 45 | 46 | Optional Arguments: 47 | 48 | * lambdas: 49 | A user supplied list of lambdas, an elastic net will be fit for 50 | each lambda supplied. If no array is passed, glmnet will generate 51 | its own array of lambdas equally spaced on a logaritmic scale 52 | between \lambda_max and \lambda_min. 53 | * weights: 54 | An n_obs array. Sample weights. It is an error to pass a weights 55 | array to a logistic model. 56 | * rel_penalties: 57 | An n_preds array. Relative panalty weights for the covariates. If 58 | none is passed, all covariates are penalized equally. If an array 59 | is passed, then a zero indicates an unpenalized parameter, and a 1 60 | a fully penalized parameter. Otherwise all covaraites recieve an 61 | equal penalty. 62 | * excl_preds: 63 | An n_preds array, used to exclude covaraites from the model. To 64 | exclude predictors, pass an array with a 1 in the first position, 65 | then a 1 in the i+1st position excludes the ith covaraite from 66 | model fitting. If no array is passed, all covaraites in X are 67 | included in the model. 68 | * box_constraints: 69 | An array with dimension 2 * n_obs. Interval constraints on the fit 70 | coefficients. The (0, i) entry is a lower bound on the ith 71 | covariate, and the (1, i) entry is an upper bound. These must 72 | satisfy lower_bound <= 0 <= upper_bound. If no array is passed, 73 | no box constraintes are allied to the parameters. 74 | * offsets: 75 | A n_preds * n_classes array. Used as initial offsets for the 76 | model fitting. 77 | 78 | After fitting, the following attributes are set: 79 | 80 | Private attributes: 81 | 82 | * _n_fit_obs: 83 | The number of rows in the model matrix X. 84 | * _n_fit_params: 85 | The number of columns in the model matrix X. 86 | * _out_n_lambdas: 87 | The number of lambdas associated with non-zero models (i.e. 88 | models with at least one none zero parameter estiamte) after 89 | fitting; for large enough lambda the models will become zero in 90 | the presense of an L1 regularizer. 91 | * _intecepts: 92 | A one dimensional array containing the intercept estiamtes for 93 | each value of lambda. See the intercepts (no underscore) 94 | property for a public version. 95 | * _comp_coef: 96 | The fit parameter estiamtes in a compressed form. This is a 97 | matrix with each row giving the estimates for a single 98 | coefficient for various values of \lambda. The order of the rows 99 | does not correspond to the order of the coefficents as given in 100 | the design matrix used to fit the model, this association is 101 | given by the _p_comp_coef attribute. Only estaimtes that are 102 | non-zero for some lambda are reported. 103 | * _p_comp_coef: 104 | A one dimensional integer array associating the coefficients in 105 | _comp_coef to columns in the model matrix. 106 | * _indices: 107 | The same information as _p_comp_coef, but zero indexed to be 108 | compatable with numpy arrays. 109 | * _n_comp_coef: 110 | The number of parameter estimates that are non-zero for some 111 | value of lambda. 112 | * _n_passes: 113 | The total number of passes over the data used to fit the model. 114 | * _error_flag: 115 | Error flag from the fortran code. 116 | 117 | Public Attributes: 118 | 119 | * null_dev: 120 | The devaince of the null (mean) model. 121 | * exp_dev: 122 | The devaince explained by the model. 123 | * out_lambdas: 124 | An array containing the lambda values associated with each fit 125 | model. 126 | ''' 127 | self._check_if_unfit() 128 | self._check_y(y) 129 | self._check_weights(weights) 130 | # Convert to arrays is native python objects 131 | try: 132 | if not issparse(X): 133 | X = np.asanyarray(X) 134 | y = np.asanyarray(y) 135 | except ValueError: 136 | raise ValueError("X and y must be wither numpy arrays, or " 137 | "convertable to numpy arrays." 138 | ) 139 | # Fortran expects an n_obs * n_classes array for y. If a one 140 | # dimensional array is passed, we construct an appropriate widening. 141 | # TODO: Save class names as attribute. 142 | y = np.asanyarray(y) 143 | if len(y.shape) == 1: 144 | y_classes = np.unique(y) 145 | y = np.float64(np.column_stack(y == c for c in y_classes)) 146 | self._n_classes = y.shape[1] 147 | # Two class predictions are handled as a special case, as is usual 148 | # with logistic models, this is signaled to fortran by passing in a 149 | # 1 for nc (num classes). 150 | if self._n_classes == 2: 151 | f_n_classes = np.array([1]) 152 | else: 153 | f_n_classes = np.array([self._n_classes]) 154 | # Grab the design info from patsy for later use, we are abbout to write 155 | # over this object in some cases. 156 | if hasattr(X, 'design_info'): 157 | design_info = X.design_info 158 | else: 159 | design_info = None 160 | # Make a copy if we are not able to overwrite X with its standardized 161 | # version. Note that if X is not fortran contiguous, then it will be 162 | # copied anyway. 163 | if not issparse(X) and np.isfortran(X) and not self.overwrite_pred_ok: 164 | X = X.copy(order='F') 165 | # Make a copy if we are not able to overwrite y with its standardized 166 | # version. 167 | if np.isfortran(y) and not self.overwrite_targ_ok: 168 | y = y.copy(order='F') 169 | # Validate all the inputs: 170 | self._validate_matrix(X) 171 | self._validate_inputs(X, y) 172 | self._validate_lambdas(X, y, lambdas) 173 | self._validate_rel_penalties(X, y, rel_penalties) 174 | self._validate_excl_preds(X, y, excl_preds) 175 | self._validate_box_constraints(X, y, box_constraints) 176 | self._validate_offsets(X, y, offsets) 177 | # Setup is complete, call the wrapper 178 | if not issparse(X): 179 | (self._out_n_lambdas, 180 | self._intercepts, 181 | self._comp_coef, 182 | self._p_comp_coef, 183 | self._n_comp_coef, 184 | self.null_dev, 185 | self.exp_dev, 186 | self.out_lambdas, 187 | self._n_passes, 188 | self._error_flag) = _glmnet.lognet( 189 | self.alpha, 190 | f_n_classes, 191 | X, 192 | y, 193 | self.offsets, 194 | self.excl_preds, 195 | self.rel_penalties, 196 | self.box_constraints, 197 | self.max_vars_all, 198 | self.frac_lg_lambda, 199 | self.lambdas, 200 | self.threshold, 201 | nlam=self.n_lambdas 202 | ) 203 | else: 204 | X.sort_indices() 205 | # Fortran arrays are 1 indexed. 206 | ind_ptrs = X.indptr + 1 207 | indices = X.indices + 1 208 | # Call 209 | (self._out_n_lambdas, 210 | self._intercepts, 211 | self._comp_coef, 212 | self._p_comp_coef, 213 | self._n_comp_coef, 214 | self.null_dev, 215 | self.exp_dev, 216 | self.out_lambdas, 217 | self._n_passes, 218 | self._error_flag) = _glmnet.splognet( 219 | self.alpha, 220 | X.shape[0], 221 | X.shape[1], 222 | f_n_classes, 223 | X.data, 224 | ind_ptrs, 225 | indices, 226 | y, 227 | self.offsets, 228 | self.excl_preds, 229 | self.rel_penalties, 230 | self.box_constraints, 231 | self.max_vars_all, 232 | self.frac_lg_lambda, 233 | self.lambdas, 234 | self.threshold, 235 | nlam=self.n_lambdas 236 | ) 237 | self._check_errors() 238 | # Keep some model metadata 239 | self._n_fit_obs, self._n_fit_params = X.shape 240 | # The indexes into the predictor array are off by one due to fortran 241 | # convention differing from numpys, this make them indexes into the the 242 | # numpy array. 243 | self._indices = np.trim_zeros(self._p_comp_coef, 'b') - 1 244 | # Create a list of column names for the fit parameters, these can be 245 | # passed in, or attached to the matrix from patsy. If none are found 246 | # we crate our own stupid ones. 247 | if col_names != None: 248 | self._col_names = col_names 249 | elif design_info != None: 250 | self._col_names = design_info.column_names 251 | else: 252 | self._col_names = [ 253 | 'var_' + str(i) for i in range(self._n_fit_params) 254 | ] 255 | 256 | def _validate_offsets(self, X, y, offsets): 257 | '''If no explicit offset was setup, we assume a zero offset.''' 258 | # TODO: Allow for passing a one-dim array of offsets in the two 259 | # class case. 260 | if offsets is None: 261 | self.offsets = np.zeros((X.shape[0], y.shape[1]), order='F') 262 | else: 263 | self.offsets = offsets 264 | if (self.offsets.shape[0] != X.shape[0] or 265 | self.offsets.shape[1] != y.shape[1]): 266 | raise ValueError("Offsets must share its first dimenesion with X " 267 | "and its second dimension must be the number of " 268 | "response classes." 269 | ) 270 | 271 | @property 272 | def _coefficients(self): 273 | '''The fit model coefficients for each lambda. 274 | 275 | The dimensions of this array depend on whether of not a two class 276 | response was used (i.e. was logistic regression performed): 277 | 278 | * If so: 279 | A _n_comp_coef * _out_n_lambdas array containing the fit model 280 | coefficients for each value of lambda. 281 | 282 | * If not: 283 | A _n_comp_coef * n_classes * _out_n_lambdas array containing the 284 | model coefficients for each level of the response, for each 285 | lambda. 286 | ''' 287 | self._check_if_fit() 288 | return self._comp_coef[:np.max(self._n_comp_coef), 289 | :, 290 | :self._out_n_lambdas 291 | ].squeeze() 292 | 293 | def _max_lambda(self, X, y, weights=None): 294 | '''Return the maximum value of lambda useful for fitting, i.e. that 295 | which first forces all coefficients to zero. 296 | 297 | This calculation is derived from the discussion in "Regularization 298 | Paths for Generalized Linear Models via Coordinate Descent" in the 299 | section "Regularized Logistic Regression", using an analogy with the 300 | linear case with weighted samples. 301 | 302 | We apply the reasoning from the linear case to a quadratic 303 | approximation to the binomial likelihood evaluated at an initial 304 | value of the parameters. The initial parameters are taken to be: 305 | 306 | \beta_0 = log(p/(1-p)) 307 | \beta_i = 0 308 | 309 | I.e., our initial model is the intercept only model. Comapring the 310 | approximation to the elastic net case gives us a working response and 311 | working weights. These are used in the formula giving the maximum 312 | lambda in the elastic net. 313 | 314 | There is one complication: the working weights in the quadratic 315 | approximation to the logistic loss are not normalized, while the 316 | discussion in the linear case makes this assumption. To compensate 317 | for this, we must adjust the value of lambda to live on the same 318 | scale as the weights, which causes the normalization factor to drop 319 | out of the equation. 320 | ''' 321 | # TODO: These calculations assume the y is a one dimensional vector 322 | # of 0s and 1s, they do not cover the case of a multi column y. 323 | self._check_y(y) 324 | self._check_weights(weights) 325 | if issparse(X): 326 | return self._max_lambda_sparse(X, y, weights) 327 | else: 328 | return self._max_lambda_dense(X, y, weights) 329 | 330 | def _max_lambda_dense(self, X, y, weights=None): 331 | '''Calculation based on least squares case with a working response 332 | and working weights. 333 | ''' 334 | if weights is not None: 335 | raise ValueError("LogisticNet cannot be fit with weights.") 336 | p = np.mean(y) 337 | working_resp = np.log(p/(1-p)) + (y - p) / (p*(1 - p)) 338 | working_weights = p*(1 - p) / (X.shape[0]) 339 | # Standardize the model matrix. 340 | normfac = X.shape[0] 341 | mu = X.sum(axis=0) / normfac 342 | mu2 = (X*X).sum(axis=0) / normfac 343 | sigma = np.sqrt(mu2 - mu*mu) 344 | # Avoid divide by zero in constant case 345 | sigma[sigma == 0] = 1 346 | X_scaled = (X - mu) / sigma 347 | # Now mimic the calculation for the quadratic case 348 | y_wtd = working_resp * working_weights 349 | dots = y_wtd.dot(X_scaled) 350 | # An alpha of zero (ridge) breaks the maximum lambda logic, the 351 | # coefficients are never all zero - so we readjust to a small 352 | # value. 353 | alpha = self.alpha if self.alpha > .0001 else .0001 354 | return np.max(np.abs(dots)) / alpha 355 | 356 | def _max_lambda_sparse(self, X, y, weights=None): 357 | '''Calculation based on least squares case with a working response 358 | and working weights. 359 | 360 | To preserve the sparsity, we must avoid explicitly subtracting out 361 | the mean of the columns. 362 | ''' 363 | if weights is not None: 364 | raise ValueError("LogisticNet cannot be fit with weights.") 365 | # Sparse dot product 366 | dot = self._get_dot(X) 367 | p = np.mean(y) 368 | working_resp = np.log(p/(1-p)) + (y - p) / (p*(1 - p)) 369 | working_weights = p*(1 - p) / (X.shape[0]) 370 | # Sample expectataion value of the columns in a matrix 371 | E = lambda M: np.asarray(M.sum(axis=0)).ravel() / M.shape[0] 372 | mu = E(X) 373 | mu_2 = E(X.multiply(X)) 374 | sigma = np.sqrt(mu_2 - mu*mu) 375 | # Avoid divide by zero in case of constant column. 376 | sigma[sigma == 0] = 1 377 | # Calculating the dot product of y with X standardized, without 378 | # destorying the sparsity of X 379 | y_wtd = working_resp * working_weights 380 | dots = 1/sigma * (dot(y_wtd, X) - mu * np.sum(y_wtd)) 381 | # An alpha of zero (ridge) breaks the maximum lambda logic, the 382 | # coefficients are never all zero - so we readjust to a small 383 | # value. 384 | alpha = self.alpha if self.alpha > .0001 else .0001 385 | return np.max(np.abs(dots)) / alpha 386 | 387 | def deviance(self, X, y): 388 | '''Calculate the binomial deviance for every lambda. The model must 389 | already be fit to call this method. 390 | ''' 391 | # TODO: This only covers the case of a one dimensional y, need to 392 | # support multiple columns for y. 393 | y_hat = self.predict(X) 394 | # Take the response y, and repeat it as a column to produce a matrix 395 | # of the same dimensions as y_hat 396 | y_stacked = np.tile(np.array([y]).transpose(), y_hat.shape[1]) 397 | #print y_stacked 398 | bin_dev = y_stacked*np.log(y_hat) + (1 - y_stacked)*np.log(1 - y_hat) 399 | normfac = X.shape[0] 400 | return np.apply_along_axis(np.sum, 0, -2*bin_dev) / normfac 401 | 402 | def predict(self, X): 403 | '''Return model predictions on the probability scale.''' 404 | # TODO This only support a one dimensional y. 405 | return 1 / ( 1 + np.exp(self._predict_lp(X)) ) 406 | 407 | def plot_paths(self): 408 | self._plot_path('logistic') 409 | 410 | def _check_y(self, y): 411 | '''Temporary safety net. glmnet has the ability to fit a mutinomial 412 | model, but this class currently cannot handle that case. 413 | ''' 414 | if len(y.shape) != 1: 415 | raise NotImplementedError( 416 | "This method does not yet support non-one dimensional y " 417 | "arrays." 418 | ) 419 | 420 | def _check_weights(self, weights): 421 | '''Logistic models do not support sample weights.''' 422 | if weights is not None: 423 | raise ValueError("LogisticNet cannot be fit with weights.") 424 | 425 | def __str__(self): 426 | return self._str('logistic') 427 | 428 | def describe(self, lidx=None): 429 | return self._describe('logistic') 430 | -------------------------------------------------------------------------------- /images/oof_dev.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madrury/glmnet-python/e22ad5c8ab50404dce19bb6fcaa7257f09e579a7/images/oof_dev.png -------------------------------------------------------------------------------- /images/reg_paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madrury/glmnet-python/e22ad5c8ab50404dce19bb6fcaa7257f09e579a7/images/reg_paths.png -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from numpy.distutils.misc_util import Configuration 2 | from numpy.distutils.system_info import get_info 3 | import os, sys 4 | 5 | config = Configuration( 6 | 'glmnet', 7 | parent_package=None, 8 | top_path=None 9 | ) 10 | 11 | f_sources = ['glmnet/glmnet.pyf','glmnet/glmnet.f'] 12 | fflags= ['-fdefault-real-8', '-ffixed-form'] 13 | 14 | config.add_extension(name='_glmnet', 15 | sources=f_sources, 16 | extra_f77_compile_args=fflags, 17 | extra_f90_compile_args=fflags 18 | ) 19 | 20 | config_dict = config.todict() 21 | 22 | if __name__ == '__main__': 23 | from numpy.distutils.core import setup 24 | setup(version='0.9', 25 | description='Python wrappers for the GLMNET package', 26 | author='Matthew Drury', 27 | author_email='matthew.drury.83@gmail.com', 28 | url='github.com/madrury/glmnet-python', 29 | license='GPL2', 30 | requires=['NumPy (>= 1.3)'], 31 | **config_dict 32 | ) 33 | 34 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madrury/glmnet-python/e22ad5c8ab50404dce19bb6fcaa7257f09e579a7/tests/__init__.py -------------------------------------------------------------------------------- /tests/test_elastic_net.py: -------------------------------------------------------------------------------- 1 | from glmnet import ElasticNet 2 | import numpy as np 3 | from scipy.sparse import csc_matrix, csr_matrix 4 | import unittest 5 | 6 | np.random.seed(123) 7 | 8 | class TestElasticNet(unittest.TestCase): 9 | '''Test suite for ElasticNet models.''' 10 | 11 | def test_unregularized_models(self): 12 | '''Test that fitting an unregularized model (lambda=0) gives 13 | expected results for both dense and sparse model matricies. 14 | 15 | We test that an unregularized model captures a perfect linear 16 | relationship without error. That is, the fit parameters equals the 17 | true coefficients. 18 | ''' 19 | Xdn = np.random.random(size=(5000,10)) 20 | Xsp = csc_matrix(Xdn) 21 | w = np.random.random(size=(10,)) 22 | y = np.dot(Xdn, w) 23 | for alpha in [0, .5, 1]: 24 | for X in (Xdn, Xsp): 25 | enet = ElasticNet(alpha=alpha) 26 | enet.fit(X, y, lambdas=[0]) 27 | test_preds = np.allclose(enet.predict(X).ravel(), y, atol=.01) 28 | self.assertTrue(test_preds) 29 | test_coefs = np.allclose(enet._coefficients.ravel(), w, atol=.02) 30 | self.assertTrue(test_coefs) 31 | 32 | def test_lasso_models(self): 33 | '''Test that a pure lasso (alpha=1) model gives expected results 34 | for both dense and sparse design matricies. 35 | 36 | We test that the lasso model has the ability to pick out zero 37 | parameters from a linear relationship. To see this, we generate 38 | linearly related data were some number of the coefficients are 39 | exactly zero, and make sure the lasso model can pick these out. 40 | ''' 41 | Xdn = np.random.random(size=(25000,10)) 42 | Xsp = csc_matrix(Xdn) 43 | w = np.random.random(size=(10,)) 44 | for w_mask in range(1, 10): 45 | for X in (Xdn, Xsp): 46 | w_masked = w.copy() 47 | w_masked[w_mask:] = 0 48 | y = np.dot(Xdn, w_masked) 49 | enet = ElasticNet(alpha=1) 50 | enet.fit(X, y, lambdas=[.01]) 51 | test = (len(enet._coefficients.ravel() == w_mask)) 52 | self.assertTrue(test) 53 | 54 | def test_ridge_models(self): 55 | '''Test that a pure ridge (alpha=0) model gives expected results 56 | for both dense and sparse matricies. 57 | 58 | We test that the ridge model, when fit on uncorrelated predictors, 59 | shrinks the parameter estiamtes uniformly. To see this, we generate 60 | linearly related data with a correlation free model matrix, then test 61 | that the array of ratios of fit parameters to true coefficients is 62 | a constant array. 63 | 64 | This test generates more samples than the others to guarentee that the 65 | data is sufficiently correlation free, otherwise the effect to be 66 | measured does not occur. 67 | ''' 68 | Xdn = np.random.random(size=(50000,3)) 69 | Xsp = csc_matrix(Xdn) 70 | w = np.random.random(size=(3,)) 71 | for X in (Xdn, Xsp): 72 | for lam in np.linspace(0, 1, 10): 73 | y = np.dot(Xdn, w) 74 | enet = ElasticNet(alpha=0) 75 | enet.fit(X, y, lambdas=[lam]) 76 | ratios = enet._coefficients.ravel() / w 77 | norm_ratios = ratios / np.max(ratios) 78 | test = np.allclose( 79 | norm_ratios, 1, atol=.05 80 | ) 81 | self.assertTrue(test) 82 | 83 | def test_unregularized_with_weights(self): 84 | '''Test that fitting an unregularized model (lambda=0) gives expected 85 | results when sample weights are used. 86 | ''' 87 | Xdn = np.random.random(size=(5000,10)) 88 | Xsp = csc_matrix(Xdn) 89 | w = np.random.random(size=(10,)) 90 | y = np.dot(Xdn, w) 91 | sw = np.random.uniform(size=(5000,)) 92 | for alpha in [0, .5, 1]: 93 | for X in (Xdn, Xsp): 94 | enet = ElasticNet(alpha=alpha) 95 | enet.fit(X, y, lambdas=[0], weights=sw) 96 | test_preds = np.allclose(enet.predict(X).ravel(), y, atol=.01) 97 | self.assertTrue(test_preds) 98 | test_coefs = np.allclose(enet._coefficients.ravel(), w, atol=.02) 99 | self.assertTrue(test_coefs) 100 | 101 | def test_lasso_with_weights(self): 102 | '''Test that a pure lasso (alpha=1) model gives expected results when 103 | sample weights are used. 104 | ''' 105 | Xdn = np.random.random(size=(25000,10)) 106 | Xsp = csc_matrix(Xdn) 107 | w = np.random.random(size=(10,)) 108 | sw = np.random.uniform(size=(25000,)) 109 | sw = sw / np.sum(sw) 110 | for w_mask in range(1, 10): 111 | for X in (Xdn, Xsp): 112 | w_masked = w.copy() 113 | w_masked[w_mask:] = 0 114 | y = np.dot(Xdn, w_masked) 115 | enet = ElasticNet(alpha=1) 116 | enet.fit(X, y, lambdas=[.01], weights=sw) 117 | test = (len(enet._coefficients.ravel() == w_mask)) 118 | self.assertTrue(test) 119 | #raise RuntimeError 120 | 121 | def test_ridge_with_weights(self): 122 | '''Test that a pure ridge (alpha=0) model gives expected results 123 | for both dense and sparse matricies. 124 | ''' 125 | Xdn = np.random.random(size=(50000,3)) 126 | Xsp = csc_matrix(Xdn) 127 | w = np.random.random(size=(3,)) 128 | sw = np.random.uniform(size=(50000,)) 129 | sw = sw / np.sum(sw) 130 | for X in (Xdn, Xsp): 131 | for lam in np.linspace(0, 1, 10): 132 | y = np.dot(Xdn, w) 133 | enet = ElasticNet(alpha=0) 134 | enet.fit(X, y, lambdas=[lam], weights=sw) 135 | ratios = enet._coefficients.ravel() / w 136 | norm_ratios = ratios / np.max(ratios) 137 | test = np.allclose( 138 | norm_ratios, 1, atol=.05 139 | ) 140 | self.assertTrue(test) 141 | 142 | def test_max_lambda(self): 143 | '''Test that the calculations of max_lambda inside the fortran code and 144 | inside the python code give the same result on both dense and sparse 145 | matricies. 146 | 147 | Note that the implementation of max_lambda for alpha=0 in 148 | the fortran code is unknown, so we currently do not test against it. 149 | ''' 150 | Xdn = np.random.random(size=(50,10)) 151 | Xsp = csc_matrix(Xdn) 152 | w = np.random.random(size=(10,)) 153 | y = np.dot(Xdn, w) 154 | for alpha in [.01, .5, 1]: 155 | for X in (Xdn, Xsp): 156 | enet = ElasticNet(alpha=alpha) 157 | enet.fit(X, y) 158 | ol = enet.out_lambdas 159 | max_lambda_from_fortran = ol[1] * (ol[1]/ol[2]) 160 | max_lambda_from_python = enet._max_lambda(X, y) 161 | self.assertAlmostEqual( 162 | max_lambda_from_fortran, max_lambda_from_python, 4 163 | ) 164 | 165 | def test_max_lambda_with_weights(self): 166 | '''Test that the calculations of max_lambda inside the fortran code and 167 | inside the python code give the same result on both dense and sparse 168 | matricies, even when sample weights come into play. 169 | ''' 170 | Xdn = np.random.random(size=(50,10)) 171 | Xsp = csc_matrix(Xdn) 172 | w = np.random.random(size=(10,)) 173 | y = np.dot(Xdn, w) 174 | sw = np.random.uniform(size=(50,)) 175 | for alpha in [.01, .5, 1]: 176 | for X in (Xdn, Xsp): 177 | enet = ElasticNet(alpha=alpha) 178 | enet.fit(X, y, weights=sw) 179 | ol = enet.out_lambdas 180 | max_lambda_from_fortran = ol[1] * (ol[1]/ol[2]) 181 | max_lambda_from_python = enet._max_lambda(X, y, weights=sw) 182 | self.assertAlmostEqual( 183 | max_lambda_from_fortran, max_lambda_from_python, 4 184 | ) 185 | 186 | def test_validate_matrix(self): 187 | '''Test the _validate_matrix method.''' 188 | Xdn = np.random.random(size=(50,10)) 189 | enet = ElasticNet(alpha=.5) 190 | # Invalid use: 191 | # Passing in a sparse matrix in the incorrect format. 192 | with self.assertRaises(ValueError): 193 | Xsp = csr_matrix(Xdn) 194 | enet._validate_matrix(Xsp) 195 | # Valid use: 196 | # Passing in a matrix in compressed sparse column format. 197 | Xsp = csc_matrix(Xdn) 198 | enet._validate_matrix(Xsp) 199 | 200 | def test_validate_inputs(self): 201 | X = np.random.random(size=(50,10)) 202 | w = np.random.random(size=(10,)) 203 | enet = ElasticNet(alpha=.5) 204 | # Invalid Use: 205 | # Passing in a y that is too short. 206 | with self.assertRaises(ValueError): 207 | yprime = np.random.random(size=(49,)) 208 | enet._validate_inputs(X, yprime) 209 | # Invalid use: 210 | # Passing in a y that matches the wrong dimenstion of X. 211 | with self.assertRaises(ValueError): 212 | yprime = np.random.random(size=(10,)) 213 | enet._validate_inputs(X, yprime) 214 | # Valid Use: 215 | # Passing in a y of the correct dimension. 216 | yprime = np.random.random(size=(50,)) 217 | enet._validate_inputs(X, yprime) 218 | 219 | def test_validate_weights(self): 220 | X = np.random.random(size=(50,10)) 221 | w = np.random.random(size=(10,)) 222 | y = np.dot(X, w) 223 | enet = ElasticNet(alpha=.5) 224 | # Invalid use 225 | # Passing in a sample weight vector that is too short. 226 | with self.assertRaises(ValueError): 227 | sw = np.ones(shape=(49,)) 228 | enet._validate_weights(X, y, weights=sw) 229 | # Invalid use: 230 | # Passing in a weight vector that matches the wrong dimenstion of X. 231 | with self.assertRaises(ValueError): 232 | sw = np.ones(shape=(10,)) 233 | enet._validate_weights(X, y, weights=sw) 234 | # Invalid use: 235 | # Passing in a weight vector containing a negative entry. 236 | with self.assertRaises(ValueError): 237 | sw = np.ones(shape=(50,)) 238 | sw[25] = -1 239 | enet._validate_weights(X, y, weights=sw) 240 | # Valid Use: 241 | # Weight vector of the correct dimension with all non-negative 242 | # entries. 243 | sw = np.ones(shape=(50,)) 244 | enet._validate_weights(X, y, weights=sw) 245 | 246 | def test_validate_rel_penalties(self): 247 | X = np.random.random(size=(50,10)) 248 | w = np.random.random(size=(10,)) 249 | y = np.dot(X, w) 250 | enet = ElasticNet(alpha=.5) 251 | # Invalid use: 252 | # Passing in a rel_penalties vector that is too short. 253 | with self.assertRaises(ValueError): 254 | rel_pens = np.ones(shape=(9,)) 255 | enet._validate_rel_penalties(X, y, rel_penalties=rel_pens) 256 | # Invalid use: 257 | # Passing in a rel_penalties that matches the wrong dimenstion of X. 258 | with self.assertRaises(ValueError): 259 | rel_pens = np.ones(shape=(50,)) 260 | enet._validate_rel_penalties(X, y, rel_penalties=rel_pens) 261 | # Invalid use: 262 | # Passing in a rel_penalties containing a negative entry. 263 | with self.assertRaises(ValueError): 264 | rel_pens = np.ones(shape=(10,)) 265 | rel_pens[5] = -1 266 | enet._validate_rel_penalties(X, y, rel_penalties=rel_pens) 267 | # Invalid use: 268 | # Passing in a rel_panalties with no positive entries. 269 | with self.assertRaises(ValueError): 270 | rel_pens = np.zeros(shape=(10,)) 271 | enet._validate_rel_penalties(X, y, rel_penalties=rel_pens) 272 | # Valid use: 273 | # Rel_penalties has the correct dimenstion with all non-negative 274 | # entries. 275 | rel_pens = np.ones(shape=(10,)) 276 | rel_pens[5] = 0 277 | enet._validate_rel_penalties(X, y, rel_penalties=rel_pens) 278 | 279 | def test_validate_excl_preds(self): 280 | X = np.random.random(size=(50,10)) 281 | w = np.random.random(size=(10,)) 282 | y = np.dot(X, w) 283 | enet = ElasticNet(alpha=.5) 284 | # Invalid use 285 | # Passing in a excl_preds array that is to long. 286 | with self.assertRaises(ValueError): 287 | excl_preds = np.ones(shape=(12,)) 288 | enet._validate_excl_preds(X, y, excl_preds=excl_preds) 289 | # Invalid use 290 | # Alltempt to exclude a predictor out of range, i.e. that does 291 | # not exist. 292 | with self.assertRaises(ValueError): 293 | excl_preds = np.ones(shape=(11,)) 294 | excl_preds[0] = 1 295 | excl_preds[5] = 10 296 | enet._validate_excl_preds(X, y, excl_preds=excl_preds) 297 | # Valid use 298 | # Exclude some in range predictors. 299 | excl_preds = np.array([1, 2, 4, 6, 8]) 300 | enet._validate_excl_preds(X, y, excl_preds=excl_preds) 301 | 302 | def test_validate_box_constraints(self): 303 | X = np.random.random(size=(50,10)) 304 | w = np.random.random(size=(10,)) 305 | y = np.dot(X, w) 306 | enet = ElasticNet(alpha=.5) 307 | # Invalid use 308 | # Second dimension matches incorrect dimension of X. 309 | with self.assertRaises(ValueError): 310 | box_const = np.empty(shape=(2,50)) 311 | enet._validate_box_constraints(X, y, box_constraints=box_const) 312 | # Invalid use 313 | # Incorrect first dimension. 314 | with self.assertRaises(ValueError): 315 | box_const = np.empty(shape=(1,10)) 316 | enet._validate_box_constraints(X, y, box_constraints=box_const) 317 | # Invalid use 318 | # Transpose of a correct use. 319 | with self.assertRaises(ValueError): 320 | box_const = np.empty(shape=(10,2)) 321 | enet._validate_box_constraints(X, y, box_constraints=box_const) 322 | # Invalid use 323 | # Positive lower bound in constraint. 324 | with self.assertRaises(ValueError): 325 | box_const = np.empty(shape=(2,10)) 326 | box_const[0,:] = -1; box_const[1,:] = 1 327 | box_const[0,5] = 1 328 | enet._validate_box_constraints(X, y, box_constraints=box_const) 329 | # Invalid use 330 | # Negative upper bound in constraint. 331 | with self.assertRaises(ValueError): 332 | box_const = np.empty(shape=(2,10)) 333 | box_const[0,:] = -1; box_const[1,:] = 1 334 | box_const[1,5] = -1 335 | enet._validate_box_constraints(X, y, box_constraints=box_const) 336 | # Valid use 337 | # Impose some box constraints. 338 | box_const = np.empty(shape=(2,10)) 339 | box_const[0,:] = -1; box_const[1,:] = 1 340 | enet._validate_box_constraints(X, y, box_constraints=box_const) 341 | 342 | def test_edge_cases(self): 343 | '''Edge cases in model specification.''' 344 | X = np.random.random(size=(50,10)) 345 | w = np.random.random(size=(10,)) 346 | y = np.dot(X, w) 347 | # Edge case 348 | # A single lambda is so big that it sets all estimated coefficients 349 | # to zero. This used to break the predict method. 350 | enet = ElasticNet(alpha=1) 351 | enet.fit(X, y, lambdas=[10**5]) 352 | _ = enet.predict(X) 353 | # Edge case 354 | # Multiple lambdas are so big as to set all estiamted coefficients 355 | # to zero. This used to break the predict method. 356 | enet = ElasticNet(alpha=1) 357 | enet.fit(X, y, lambdas=[10**5, 2*10**5]) 358 | _ = enet.predict(X) 359 | # Edge case: 360 | # Some predictors have zero varaince. This used to break lambda 361 | # max. 362 | X = np.random.random(size=(50,10)) 363 | X[:,2] = 0; X[:,8] = 1 364 | y = np.dot(X, w) 365 | enet = ElasticNet(alpha=.1) 366 | enet.fit(X, y) 367 | ol = enet.out_lambdas 368 | max_lambda_from_fortran = ol[1] * (ol[1]/ol[2]) 369 | max_lambda_from_python = enet._max_lambda(X, y) 370 | self.assertAlmostEqual( 371 | max_lambda_from_fortran, max_lambda_from_python, 4 372 | ) 373 | # Edge case. 374 | # All predictors have zero variance. This is an error in 375 | # sepcification. 376 | with self.assertRaises(ValueError): 377 | X = np.ones(shape=(50,10)) 378 | enet = ElasticNet(alpha=.1) 379 | enet.fit(X, y) 380 | -------------------------------------------------------------------------------- /tests/test_logistic_net.py: -------------------------------------------------------------------------------- 1 | from glmnet import LogisticNet 2 | import numpy as np 3 | from scipy.sparse import csc_matrix, csr_matrix 4 | import unittest 5 | 6 | np.random.seed(123) 7 | 8 | class TestLogisticNet(unittest.TestCase): 9 | '''Test suite for LogisticNet models.''' 10 | 11 | def test_unregularized_models(self): 12 | Xdn = np.random.uniform(-1, 1, size=(50,10)) 13 | Xsp = csc_matrix(Xdn) 14 | w = np.random.uniform(-1, 1, size=(10,)) 15 | y = (np.dot(Xdn, w) >= 0).astype(int) 16 | for alpha in [0, .5, 1]: 17 | for X in (Xdn, Xsp): 18 | lnet = LogisticNet(alpha=alpha) 19 | lnet.fit(X, y, lambdas=[0]) 20 | preds = (lnet.predict(X) >= .5).astype(int) 21 | self.assertTrue(np.all(y == preds)) 22 | 23 | def test_lasso_models(self): 24 | Xdn = np.random.uniform(-1, 1, size=(15000,10)) 25 | Xsp = csc_matrix(Xdn) 26 | w = (np.random.uniform(-1, 1, size=(10,)) >= 0).astype(int) - .5 27 | for w_mask in range(1, 10): 28 | for X in (Xdn, Xsp): 29 | w_masked = w.copy() 30 | w_masked[w_mask:] = 0 31 | y = (np.dot(Xdn, w_masked) >= 0).astype(int) 32 | lnet = LogisticNet(alpha=1) 33 | lnet.fit(X, y, lambdas=[.01]) 34 | lc = lnet._coefficients 35 | self.assertTrue( 36 | np.sum(np.abs(lc / np.max(np.abs(lc))) > .05) == w_mask 37 | ) 38 | 39 | def test_ridge_models(self): 40 | Xdn = np.random.uniform(-1, 1, size=(50000,3)) 41 | Xsp = csc_matrix(Xdn) 42 | w = (np.random.uniform(-1, 1, size=(3,)) >= 0).astype(int) - .5 43 | for X in (Xdn, Xsp): 44 | for lam in np.linspace(.1, 1, 10): 45 | y = (np.dot(Xdn, w) >= 0).astype(int) 46 | lnet = LogisticNet(alpha=0) 47 | lnet.fit(X, y, lambdas=[lam]) 48 | ratios = lnet._coefficients.ravel() / w 49 | norm_ratios = ratios / np.max(ratios) 50 | test = np.allclose( 51 | norm_ratios, 1, atol=.05 52 | ) 53 | self.assertTrue(test) 54 | 55 | def test_max_lambda(self): 56 | Xdn = np.random.uniform(-1, 1, size=(50,10)) 57 | Xsp = csc_matrix(Xdn) 58 | w = np.random.uniform(-1, 1, size=(10,)) 59 | y = (np.dot(Xdn, w) >= 0).astype(int) 60 | for X in (Xdn, Xsp): 61 | for alpha in [.01, .5, 1]: 62 | lnet = LogisticNet(alpha=alpha) 63 | lnet.fit(X, y) 64 | ol = lnet.out_lambdas 65 | max_lambda_from_fortran = ol[1] * (ol[1]/ol[2]) 66 | max_lambda_from_python = lnet._max_lambda(X, y) 67 | self.assertAlmostEqual( 68 | max_lambda_from_fortran, max_lambda_from_python, 4 69 | ) 70 | 71 | def test_edge_cases(self): 72 | '''Edge cases in model specification.''' 73 | X = np.random.uniform(-1, 1, size=(50,10)) 74 | w = np.random.uniform(-1, 1, size=(10,)) 75 | y = (np.dot(X, w) >= 0).astype(int) 76 | # Edge case 77 | # A single lambda is so big that it sets all estimated coefficients 78 | # to zero. This used to break the predict method. 79 | lnet = LogisticNet(alpha=1) 80 | lnet.fit(X, y, lambdas=[10**5]) 81 | _ = lnet.predict(X) 82 | # Edge case 83 | # Multiple lambdas are so big as to set all estiamted coefficients 84 | # to zero. This used to break the predict method. 85 | lnet = LogisticNet(alpha=1) 86 | lnet.fit(X, y, lambdas=[10**5, 2*10**5]) 87 | _ = lnet.predict(X) 88 | # Edge case: 89 | # Some predictors have zero varaince. This used to break lambda 90 | # max. 91 | X = np.random.uniform(-1, 1, size=(50,10)) 92 | X[2,:] = 0; X[8,:] = 0 93 | y = (np.dot(X, w) >= 0).astype(int) 94 | lnet = LogisticNet(alpha=.1) 95 | lnet.fit(X, y) 96 | ol = lnet.out_lambdas 97 | max_lambda_from_fortran = ol[1] * (ol[1]/ol[2]) 98 | max_lambda_from_python = lnet._max_lambda(X, y) 99 | self.assertAlmostEqual( 100 | max_lambda_from_fortran, max_lambda_from_python, 4 101 | ) 102 | # Edge case. 103 | # All predictors have zero variance. This is an error in 104 | # sepcification. 105 | with self.assertRaises(ValueError): 106 | X = np.ones(shape=(50,10)) 107 | lnet = LogisticNet(alpha=.1) 108 | lnet.fit(X, y) 109 | -------------------------------------------------------------------------------- /util/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madrury/glmnet-python/e22ad5c8ab50404dce19bb6fcaa7257f09e579a7/util/__init__.py -------------------------------------------------------------------------------- /util/importers.py: -------------------------------------------------------------------------------- 1 | '''Handles optional dependencies.''' 2 | from warnings import warn 3 | 4 | def import_joblib(verbose=1): 5 | 6 | try: 7 | import sklearn.externals.joblib as jl 8 | except ImportError: 9 | jl = None 10 | 11 | if not jl: 12 | try: 13 | import joblib as jl 14 | except ImportError: 15 | jl = None 16 | 17 | if not jl and verbose > 0: 18 | warn('joblib not found, parallel fitting of models will be ' 19 | 'unavailable.', ImportWarning) 20 | 21 | return jl 22 | 23 | 24 | def import_pyplot(verbose=1): 25 | try: 26 | import matplotlib.pyplot as plt 27 | except ImportError: 28 | plt = None 29 | 30 | if not plt and verbose > 0: 31 | warn('pyplot unavailabe, plotting functionality will be unavaliabe.', 32 | ImportWarning) 33 | 34 | return plt 35 | --------------------------------------------------------------------------------