├── .gitignore ├── .travis.yml ├── CHANGELOG ├── LICENSE ├── README.rst ├── build_and_test.sh ├── lazy_import ├── VERSION ├── __init__.py └── test_lazy.py ├── setup.cfg └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # Editor stuff 29 | *.swp 30 | 31 | # PyInstaller 32 | # Usually these files are written by a python script from a template 33 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 34 | *.manifest 35 | *.spec 36 | 37 | # Installer logs 38 | pip-log.txt 39 | pip-delete-this-directory.txt 40 | 41 | # Unit test / coverage reports 42 | htmlcov/ 43 | .tox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | .hypothesis/ 51 | 52 | # Translations 53 | *.mo 54 | *.pot 55 | 56 | # Django stuff: 57 | *.log 58 | local_settings.py 59 | 60 | # Flask stuff: 61 | instance/ 62 | .webassets-cache 63 | 64 | # Scrapy stuff: 65 | .scrapy 66 | 67 | # Sphinx documentation 68 | docs/_build/ 69 | 70 | # PyBuilder 71 | target/ 72 | 73 | # Jupyter Notebook 74 | .ipynb_checkpoints 75 | 76 | # pyenv 77 | .python-version 78 | 79 | # celery beat schedule file 80 | celerybeat-schedule 81 | 82 | # SageMath parsed files 83 | *.sage.py 84 | 85 | # dotenv 86 | .env 87 | 88 | # virtualenv 89 | .venv 90 | venv/ 91 | ENV/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | - "3.4" 5 | - "3.5" 6 | - "3.6" 7 | # command to install dependencies 8 | install: 9 | - pip install -U pip 10 | - pip install -U setuptools 11 | - pip install -U .[test] 12 | 13 | # command to run tests 14 | script: 15 | - py.test -v -n2 --boxed --pyargs lazy_import 16 | -------------------------------------------------------------------------------- /CHANGELOG: -------------------------------------------------------------------------------- 1 | ??/??/?? v0.2.3 2 | - More informative error when inadvertently trying to use a lazy callable a 3 | baseclass (Issue #2). 4 | - Extra level of debugging (trace debugging), toggled via lazy_import.logger. 5 | 22/01/2018 v0.2.2 6 | - fixed a serious bug when lazy-loading a submodule of a fully-loaded base. 7 | 17/01/2018 v0.2.1 8 | - annoying release to fix install notice on PyPi (because README.rst was 9 | incorrect). 10 | 17/01/2018 v0.2 11 | - stabilized a python 3 bug when importing modules using 'import modulename' 12 | syntax. 13 | - added a test shortcut. 14 | - published to PyPi (yay!) 15 | 16 | 25/08/2017 v0.1 17 | - along with python 2.7, the code is now Python 3 compatible 18 | (at least versions 3.4 through 3.6). 19 | - a replacement of lazyModule (lazy_module, which defers most work to 20 | _lazy_module) was implemented that uses a modified LazyModule class; 21 | - a different LazyModule class is now created per instance, so that 22 | reverting the __getattribute__ behavior can be done safely; 23 | - a function to lazily import module callables was added; 24 | - customization is fully supported, either as passable custom error messages 25 | or as passable LazyModule subclass alternatives. 26 | - module reloading is now more stable, especially in the case of submodules. 27 | - added testing. 28 | 29 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | ############################################################################### 2 | 3 | lazy_import is distributed under the GNU General Public License, Version 3: 4 | 5 | GNU GENERAL PUBLIC LICENSE 6 | Version 3, 29 June 2007 7 | 8 | Copyright (C) 2007 Free Software Foundation, Inc. 9 | Everyone is permitted to copy and distribute verbatim copies 10 | of this license document, but changing it is not allowed. 11 | 12 | Preamble 13 | 14 | The GNU General Public License is a free, copyleft license for 15 | software and other kinds of works. 16 | 17 | The licenses for most software and other practical works are designed 18 | to take away your freedom to share and change the works. By contrast, 19 | the GNU General Public License is intended to guarantee your freedom to 20 | share and change all versions of a program--to make sure it remains free 21 | software for all its users. We, the Free Software Foundation, use the 22 | GNU General Public License for most of our software; it applies also to 23 | any other work released this way by its authors. You can apply it to 24 | your programs, too. 25 | 26 | When we speak of free software, we are referring to freedom, not 27 | price. Our General Public Licenses are designed to make sure that you 28 | have the freedom to distribute copies of free software (and charge for 29 | them if you wish), that you receive source code or can get it if you 30 | want it, that you can change the software or use pieces of it in new 31 | free programs, and that you know you can do these things. 32 | 33 | To protect your rights, we need to prevent others from denying you 34 | these rights or asking you to surrender the rights. Therefore, you have 35 | certain responsibilities if you distribute copies of the software, or if 36 | you modify it: responsibilities to respect the freedom of others. 37 | 38 | For example, if you distribute copies of such a program, whether 39 | gratis or for a fee, you must pass on to the recipients the same 40 | freedoms that you received. You must make sure that they, too, receive 41 | or can get the source code. And you must show them these terms so they 42 | know their rights. 43 | 44 | Developers that use the GNU GPL protect your rights with two steps: 45 | (1) assert copyright on the software, and (2) offer you this License 46 | giving you legal permission to copy, distribute and/or modify it. 47 | 48 | For the developers' and authors' protection, the GPL clearly explains 49 | that there is no warranty for this free software. For both users' and 50 | authors' sake, the GPL requires that modified versions be marked as 51 | changed, so that their problems will not be attributed erroneously to 52 | authors of previous versions. 53 | 54 | Some devices are designed to deny users access to install or run 55 | modified versions of the software inside them, although the manufacturer 56 | can do so. This is fundamentally incompatible with the aim of 57 | protecting users' freedom to change the software. The systematic 58 | pattern of such abuse occurs in the area of products for individuals to 59 | use, which is precisely where it is most unacceptable. Therefore, we 60 | have designed this version of the GPL to prohibit the practice for those 61 | products. If such problems arise substantially in other domains, we 62 | stand ready to extend this provision to those domains in future versions 63 | of the GPL, as needed to protect the freedom of users. 64 | 65 | Finally, every program is threatened constantly by software patents. 66 | States should not allow patents to restrict development and use of 67 | software on general-purpose computers, but in those that do, we wish to 68 | avoid the special danger that patents applied to a free program could 69 | make it effectively proprietary. To prevent this, the GPL assures that 70 | patents cannot be used to render the program non-free. 71 | 72 | The precise terms and conditions for copying, distribution and 73 | modification follow. 74 | 75 | TERMS AND CONDITIONS 76 | 77 | 0. Definitions. 78 | 79 | "This License" refers to version 3 of the GNU General Public License. 80 | 81 | "Copyright" also means copyright-like laws that apply to other kinds of 82 | works, such as semiconductor masks. 83 | 84 | "The Program" refers to any copyrightable work licensed under this 85 | License. Each licensee is addressed as "you". "Licensees" and 86 | "recipients" may be individuals or organizations. 87 | 88 | To "modify" a work means to copy from or adapt all or part of the work 89 | in a fashion requiring copyright permission, other than the making of an 90 | exact copy. The resulting work is called a "modified version" of the 91 | earlier work or a work "based on" the earlier work. 92 | 93 | A "covered work" means either the unmodified Program or a work based 94 | on the Program. 95 | 96 | To "propagate" a work means to do anything with it that, without 97 | permission, would make you directly or secondarily liable for 98 | infringement under applicable copyright law, except executing it on a 99 | computer or modifying a private copy. Propagation includes copying, 100 | distribution (with or without modification), making available to the 101 | public, and in some countries other activities as well. 102 | 103 | To "convey" a work means any kind of propagation that enables other 104 | parties to make or receive copies. Mere interaction with a user through 105 | a computer network, with no transfer of a copy, is not conveying. 106 | 107 | An interactive user interface displays "Appropriate Legal Notices" 108 | to the extent that it includes a convenient and prominently visible 109 | feature that (1) displays an appropriate copyright notice, and (2) 110 | tells the user that there is no warranty for the work (except to the 111 | extent that warranties are provided), that licensees may convey the 112 | work under this License, and how to view a copy of this License. If 113 | the interface presents a list of user commands or options, such as a 114 | menu, a prominent item in the list meets this criterion. 115 | 116 | 1. Source Code. 117 | 118 | The "source code" for a work means the preferred form of the work 119 | for making modifications to it. "Object code" means any non-source 120 | form of a work. 121 | 122 | A "Standard Interface" means an interface that either is an official 123 | standard defined by a recognized standards body, or, in the case of 124 | interfaces specified for a particular programming language, one that 125 | is widely used among developers working in that language. 126 | 127 | The "System Libraries" of an executable work include anything, other 128 | than the work as a whole, that (a) is included in the normal form of 129 | packaging a Major Component, but which is not part of that Major 130 | Component, and (b) serves only to enable use of the work with that 131 | Major Component, or to implement a Standard Interface for which an 132 | implementation is available to the public in source code form. A 133 | "Major Component", in this context, means a major essential component 134 | (kernel, window system, and so on) of the specific operating system 135 | (if any) on which the executable work runs, or a compiler used to 136 | produce the work, or an object code interpreter used to run it. 137 | 138 | The "Corresponding Source" for a work in object code form means all 139 | the source code needed to generate, install, and (for an executable 140 | work) run the object code and to modify the work, including scripts to 141 | control those activities. However, it does not include the work's 142 | System Libraries, or general-purpose tools or generally available free 143 | programs which are used unmodified in performing those activities but 144 | which are not part of the work. For example, Corresponding Source 145 | includes interface definition files associated with source files for 146 | the work, and the source code for shared libraries and dynamically 147 | linked subprograms that the work is specifically designed to require, 148 | such as by intimate data communication or control flow between those 149 | subprograms and other parts of the work. 150 | 151 | The Corresponding Source need not include anything that users 152 | can regenerate automatically from other parts of the Corresponding 153 | Source. 154 | 155 | The Corresponding Source for a work in source code form is that 156 | same work. 157 | 158 | 2. Basic Permissions. 159 | 160 | All rights granted under this License are granted for the term of 161 | copyright on the Program, and are irrevocable provided the stated 162 | conditions are met. This License explicitly affirms your unlimited 163 | permission to run the unmodified Program. The output from running a 164 | covered work is covered by this License only if the output, given its 165 | content, constitutes a covered work. This License acknowledges your 166 | rights of fair use or other equivalent, as provided by copyright law. 167 | 168 | You may make, run and propagate covered works that you do not 169 | convey, without conditions so long as your license otherwise remains 170 | in force. You may convey covered works to others for the sole purpose 171 | of having them make modifications exclusively for you, or provide you 172 | with facilities for running those works, provided that you comply with 173 | the terms of this License in conveying all material for which you do 174 | not control copyright. Those thus making or running the covered works 175 | for you must do so exclusively on your behalf, under your direction 176 | and control, on terms that prohibit them from making any copies of 177 | your copyrighted material outside their relationship with you. 178 | 179 | Conveying under any other circumstances is permitted solely under 180 | the conditions stated below. Sublicensing is not allowed; section 10 181 | makes it unnecessary. 182 | 183 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 184 | 185 | No covered work shall be deemed part of an effective technological 186 | measure under any applicable law fulfilling obligations under article 187 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 188 | similar laws prohibiting or restricting circumvention of such 189 | measures. 190 | 191 | When you convey a covered work, you waive any legal power to forbid 192 | circumvention of technological measures to the extent such circumvention 193 | is effected by exercising rights under this License with respect to 194 | the covered work, and you disclaim any intention to limit operation or 195 | modification of the work as a means of enforcing, against the work's 196 | users, your or third parties' legal rights to forbid circumvention of 197 | technological measures. 198 | 199 | 4. Conveying Verbatim Copies. 200 | 201 | You may convey verbatim copies of the Program's source code as you 202 | receive it, in any medium, provided that you conspicuously and 203 | appropriately publish on each copy an appropriate copyright notice; 204 | keep intact all notices stating that this License and any 205 | non-permissive terms added in accord with section 7 apply to the code; 206 | keep intact all notices of the absence of any warranty; and give all 207 | recipients a copy of this License along with the Program. 208 | 209 | You may charge any price or no price for each copy that you convey, 210 | and you may offer support or warranty protection for a fee. 211 | 212 | 5. Conveying Modified Source Versions. 213 | 214 | You may convey a work based on the Program, or the modifications to 215 | produce it from the Program, in the form of source code under the 216 | terms of section 4, provided that you also meet all of these conditions: 217 | 218 | a) The work must carry prominent notices stating that you modified 219 | it, and giving a relevant date. 220 | 221 | b) The work must carry prominent notices stating that it is 222 | released under this License and any conditions added under section 223 | 7. This requirement modifies the requirement in section 4 to 224 | "keep intact all notices". 225 | 226 | c) You must license the entire work, as a whole, under this 227 | License to anyone who comes into possession of a copy. This 228 | License will therefore apply, along with any applicable section 7 229 | additional terms, to the whole of the work, and all its parts, 230 | regardless of how they are packaged. This License gives no 231 | permission to license the work in any other way, but it does not 232 | invalidate such permission if you have separately received it. 233 | 234 | d) If the work has interactive user interfaces, each must display 235 | Appropriate Legal Notices; however, if the Program has interactive 236 | interfaces that do not display Appropriate Legal Notices, your 237 | work need not make them do so. 238 | 239 | A compilation of a covered work with other separate and independent 240 | works, which are not by their nature extensions of the covered work, 241 | and which are not combined with it such as to form a larger program, 242 | in or on a volume of a storage or distribution medium, is called an 243 | "aggregate" if the compilation and its resulting copyright are not 244 | used to limit the access or legal rights of the compilation's users 245 | beyond what the individual works permit. Inclusion of a covered work 246 | in an aggregate does not cause this License to apply to the other 247 | parts of the aggregate. 248 | 249 | 6. Conveying Non-Source Forms. 250 | 251 | You may convey a covered work in object code form under the terms 252 | of sections 4 and 5, provided that you also convey the 253 | machine-readable Corresponding Source under the terms of this License, 254 | in one of these ways: 255 | 256 | a) Convey the object code in, or embodied in, a physical product 257 | (including a physical distribution medium), accompanied by the 258 | Corresponding Source fixed on a durable physical medium 259 | customarily used for software interchange. 260 | 261 | b) Convey the object code in, or embodied in, a physical product 262 | (including a physical distribution medium), accompanied by a 263 | written offer, valid for at least three years and valid for as 264 | long as you offer spare parts or customer support for that product 265 | model, to give anyone who possesses the object code either (1) a 266 | copy of the Corresponding Source for all the software in the 267 | product that is covered by this License, on a durable physical 268 | medium customarily used for software interchange, for a price no 269 | more than your reasonable cost of physically performing this 270 | conveying of source, or (2) access to copy the 271 | Corresponding Source from a network server at no charge. 272 | 273 | c) Convey individual copies of the object code with a copy of the 274 | written offer to provide the Corresponding Source. This 275 | alternative is allowed only occasionally and noncommercially, and 276 | only if you received the object code with such an offer, in accord 277 | with subsection 6b. 278 | 279 | d) Convey the object code by offering access from a designated 280 | place (gratis or for a charge), and offer equivalent access to the 281 | Corresponding Source in the same way through the same place at no 282 | further charge. You need not require recipients to copy the 283 | Corresponding Source along with the object code. If the place to 284 | copy the object code is a network server, the Corresponding Source 285 | may be on a different server (operated by you or a third party) 286 | that supports equivalent copying facilities, provided you maintain 287 | clear directions next to the object code saying where to find the 288 | Corresponding Source. Regardless of what server hosts the 289 | Corresponding Source, you remain obligated to ensure that it is 290 | available for as long as needed to satisfy these requirements. 291 | 292 | e) Convey the object code using peer-to-peer transmission, provided 293 | you inform other peers where the object code and Corresponding 294 | Source of the work are being offered to the general public at no 295 | charge under subsection 6d. 296 | 297 | A separable portion of the object code, whose source code is excluded 298 | from the Corresponding Source as a System Library, need not be 299 | included in conveying the object code work. 300 | 301 | A "User Product" is either (1) a "consumer product", which means any 302 | tangible personal property which is normally used for personal, family, 303 | or household purposes, or (2) anything designed or sold for incorporation 304 | into a dwelling. In determining whether a product is a consumer product, 305 | doubtful cases shall be resolved in favor of coverage. For a particular 306 | product received by a particular user, "normally used" refers to a 307 | typical or common use of that class of product, regardless of the status 308 | of the particular user or of the way in which the particular user 309 | actually uses, or expects or is expected to use, the product. A product 310 | is a consumer product regardless of whether the product has substantial 311 | commercial, industrial or non-consumer uses, unless such uses represent 312 | the only significant mode of use of the product. 313 | 314 | "Installation Information" for a User Product means any methods, 315 | procedures, authorization keys, or other information required to install 316 | and execute modified versions of a covered work in that User Product from 317 | a modified version of its Corresponding Source. The information must 318 | suffice to ensure that the continued functioning of the modified object 319 | code is in no case prevented or interfered with solely because 320 | modification has been made. 321 | 322 | If you convey an object code work under this section in, or with, or 323 | specifically for use in, a User Product, and the conveying occurs as 324 | part of a transaction in which the right of possession and use of the 325 | User Product is transferred to the recipient in perpetuity or for a 326 | fixed term (regardless of how the transaction is characterized), the 327 | Corresponding Source conveyed under this section must be accompanied 328 | by the Installation Information. But this requirement does not apply 329 | if neither you nor any third party retains the ability to install 330 | modified object code on the User Product (for example, the work has 331 | been installed in ROM). 332 | 333 | The requirement to provide Installation Information does not include a 334 | requirement to continue to provide support service, warranty, or updates 335 | for a work that has been modified or installed by the recipient, or for 336 | the User Product in which it has been modified or installed. Access to a 337 | network may be denied when the modification itself materially and 338 | adversely affects the operation of the network or violates the rules and 339 | protocols for communication across the network. 340 | 341 | Corresponding Source conveyed, and Installation Information provided, 342 | in accord with this section must be in a format that is publicly 343 | documented (and with an implementation available to the public in 344 | source code form), and must require no special password or key for 345 | unpacking, reading or copying. 346 | 347 | 7. Additional Terms. 348 | 349 | "Additional permissions" are terms that supplement the terms of this 350 | License by making exceptions from one or more of its conditions. 351 | Additional permissions that are applicable to the entire Program shall 352 | be treated as though they were included in this License, to the extent 353 | that they are valid under applicable law. If additional permissions 354 | apply only to part of the Program, that part may be used separately 355 | under those permissions, but the entire Program remains governed by 356 | this License without regard to the additional permissions. 357 | 358 | When you convey a copy of a covered work, you may at your option 359 | remove any additional permissions from that copy, or from any part of 360 | it. (Additional permissions may be written to require their own 361 | removal in certain cases when you modify the work.) You may place 362 | additional permissions on material, added by you to a covered work, 363 | for which you have or can give appropriate copyright permission. 364 | 365 | Notwithstanding any other provision of this License, for material you 366 | add to a covered work, you may (if authorized by the copyright holders of 367 | that material) supplement the terms of this License with terms: 368 | 369 | a) Disclaiming warranty or limiting liability differently from the 370 | terms of sections 15 and 16 of this License; or 371 | 372 | b) Requiring preservation of specified reasonable legal notices or 373 | author attributions in that material or in the Appropriate Legal 374 | Notices displayed by works containing it; or 375 | 376 | c) Prohibiting misrepresentation of the origin of that material, or 377 | requiring that modified versions of such material be marked in 378 | reasonable ways as different from the original version; or 379 | 380 | d) Limiting the use for publicity purposes of names of licensors or 381 | authors of the material; or 382 | 383 | e) Declining to grant rights under trademark law for use of some 384 | trade names, trademarks, or service marks; or 385 | 386 | f) Requiring indemnification of licensors and authors of that 387 | material by anyone who conveys the material (or modified versions of 388 | it) with contractual assumptions of liability to the recipient, for 389 | any liability that these contractual assumptions directly impose on 390 | those licensors and authors. 391 | 392 | All other non-permissive additional terms are considered "further 393 | restrictions" within the meaning of section 10. If the Program as you 394 | received it, or any part of it, contains a notice stating that it is 395 | governed by this License along with a term that is a further 396 | restriction, you may remove that term. If a license document contains 397 | a further restriction but permits relicensing or conveying under this 398 | License, you may add to a covered work material governed by the terms 399 | of that license document, provided that the further restriction does 400 | not survive such relicensing or conveying. 401 | 402 | If you add terms to a covered work in accord with this section, you 403 | must place, in the relevant source files, a statement of the 404 | additional terms that apply to those files, or a notice indicating 405 | where to find the applicable terms. 406 | 407 | Additional terms, permissive or non-permissive, may be stated in the 408 | form of a separately written license, or stated as exceptions; 409 | the above requirements apply either way. 410 | 411 | 8. Termination. 412 | 413 | You may not propagate or modify a covered work except as expressly 414 | provided under this License. Any attempt otherwise to propagate or 415 | modify it is void, and will automatically terminate your rights under 416 | this License (including any patent licenses granted under the third 417 | paragraph of section 11). 418 | 419 | However, if you cease all violation of this License, then your 420 | license from a particular copyright holder is reinstated (a) 421 | provisionally, unless and until the copyright holder explicitly and 422 | finally terminates your license, and (b) permanently, if the copyright 423 | holder fails to notify you of the violation by some reasonable means 424 | prior to 60 days after the cessation. 425 | 426 | Moreover, your license from a particular copyright holder is 427 | reinstated permanently if the copyright holder notifies you of the 428 | violation by some reasonable means, this is the first time you have 429 | received notice of violation of this License (for any work) from that 430 | copyright holder, and you cure the violation prior to 30 days after 431 | your receipt of the notice. 432 | 433 | Termination of your rights under this section does not terminate the 434 | licenses of parties who have received copies or rights from you under 435 | this License. If your rights have been terminated and not permanently 436 | reinstated, you do not qualify to receive new licenses for the same 437 | material under section 10. 438 | 439 | 9. Acceptance Not Required for Having Copies. 440 | 441 | You are not required to accept this License in order to receive or 442 | run a copy of the Program. Ancillary propagation of a covered work 443 | occurring solely as a consequence of using peer-to-peer transmission 444 | to receive a copy likewise does not require acceptance. However, 445 | nothing other than this License grants you permission to propagate or 446 | modify any covered work. These actions infringe copyright if you do 447 | not accept this License. Therefore, by modifying or propagating a 448 | covered work, you indicate your acceptance of this License to do so. 449 | 450 | 10. Automatic Licensing of Downstream Recipients. 451 | 452 | Each time you convey a covered work, the recipient automatically 453 | receives a license from the original licensors, to run, modify and 454 | propagate that work, subject to this License. You are not responsible 455 | for enforcing compliance by third parties with this License. 456 | 457 | An "entity transaction" is a transaction transferring control of an 458 | organization, or substantially all assets of one, or subdividing an 459 | organization, or merging organizations. If propagation of a covered 460 | work results from an entity transaction, each party to that 461 | transaction who receives a copy of the work also receives whatever 462 | licenses to the work the party's predecessor in interest had or could 463 | give under the previous paragraph, plus a right to possession of the 464 | Corresponding Source of the work from the predecessor in interest, if 465 | the predecessor has it or can get it with reasonable efforts. 466 | 467 | You may not impose any further restrictions on the exercise of the 468 | rights granted or affirmed under this License. For example, you may 469 | not impose a license fee, royalty, or other charge for exercise of 470 | rights granted under this License, and you may not initiate litigation 471 | (including a cross-claim or counterclaim in a lawsuit) alleging that 472 | any patent claim is infringed by making, using, selling, offering for 473 | sale, or importing the Program or any portion of it. 474 | 475 | 11. Patents. 476 | 477 | A "contributor" is a copyright holder who authorizes use under this 478 | License of the Program or a work on which the Program is based. The 479 | work thus licensed is called the contributor's "contributor version". 480 | 481 | A contributor's "essential patent claims" are all patent claims 482 | owned or controlled by the contributor, whether already acquired or 483 | hereafter acquired, that would be infringed by some manner, permitted 484 | by this License, of making, using, or selling its contributor version, 485 | but do not include claims that would be infringed only as a 486 | consequence of further modification of the contributor version. For 487 | purposes of this definition, "control" includes the right to grant 488 | patent sublicenses in a manner consistent with the requirements of 489 | this License. 490 | 491 | Each contributor grants you a non-exclusive, worldwide, royalty-free 492 | patent license under the contributor's essential patent claims, to 493 | make, use, sell, offer for sale, import and otherwise run, modify and 494 | propagate the contents of its contributor version. 495 | 496 | In the following three paragraphs, a "patent license" is any express 497 | agreement or commitment, however denominated, not to enforce a patent 498 | (such as an express permission to practice a patent or covenant not to 499 | sue for patent infringement). To "grant" such a patent license to a 500 | party means to make such an agreement or commitment not to enforce a 501 | patent against the party. 502 | 503 | If you convey a covered work, knowingly relying on a patent license, 504 | and the Corresponding Source of the work is not available for anyone 505 | to copy, free of charge and under the terms of this License, through a 506 | publicly available network server or other readily accessible means, 507 | then you must either (1) cause the Corresponding Source to be so 508 | available, or (2) arrange to deprive yourself of the benefit of the 509 | patent license for this particular work, or (3) arrange, in a manner 510 | consistent with the requirements of this License, to extend the patent 511 | license to downstream recipients. "Knowingly relying" means you have 512 | actual knowledge that, but for the patent license, your conveying the 513 | covered work in a country, or your recipient's use of the covered work 514 | in a country, would infringe one or more identifiable patents in that 515 | country that you have reason to believe are valid. 516 | 517 | If, pursuant to or in connection with a single transaction or 518 | arrangement, you convey, or propagate by procuring conveyance of, a 519 | covered work, and grant a patent license to some of the parties 520 | receiving the covered work authorizing them to use, propagate, modify 521 | or convey a specific copy of the covered work, then the patent license 522 | you grant is automatically extended to all recipients of the covered 523 | work and works based on it. 524 | 525 | A patent license is "discriminatory" if it does not include within 526 | the scope of its coverage, prohibits the exercise of, or is 527 | conditioned on the non-exercise of one or more of the rights that are 528 | specifically granted under this License. You may not convey a covered 529 | work if you are a party to an arrangement with a third party that is 530 | in the business of distributing software, under which you make payment 531 | to the third party based on the extent of your activity of conveying 532 | the work, and under which the third party grants, to any of the 533 | parties who would receive the covered work from you, a discriminatory 534 | patent license (a) in connection with copies of the covered work 535 | conveyed by you (or copies made from those copies), or (b) primarily 536 | for and in connection with specific products or compilations that 537 | contain the covered work, unless you entered into that arrangement, 538 | or that patent license was granted, prior to 28 March 2007. 539 | 540 | Nothing in this License shall be construed as excluding or limiting 541 | any implied license or other defenses to infringement that may 542 | otherwise be available to you under applicable patent law. 543 | 544 | 12. No Surrender of Others' Freedom. 545 | 546 | If conditions are imposed on you (whether by court order, agreement or 547 | otherwise) that contradict the conditions of this License, they do not 548 | excuse you from the conditions of this License. If you cannot convey a 549 | covered work so as to satisfy simultaneously your obligations under this 550 | License and any other pertinent obligations, then as a consequence you may 551 | not convey it at all. For example, if you agree to terms that obligate you 552 | to collect a royalty for further conveying from those to whom you convey 553 | the Program, the only way you could satisfy both those terms and this 554 | License would be to refrain entirely from conveying the Program. 555 | 556 | 13. Use with the GNU Affero General Public License. 557 | 558 | Notwithstanding any other provision of this License, you have 559 | permission to link or combine any covered work with a work licensed 560 | under version 3 of the GNU Affero General Public License into a single 561 | combined work, and to convey the resulting work. The terms of this 562 | License will continue to apply to the part which is the covered work, 563 | but the special requirements of the GNU Affero General Public License, 564 | section 13, concerning interaction through a network will apply to the 565 | combination as such. 566 | 567 | 14. Revised Versions of this License. 568 | 569 | The Free Software Foundation may publish revised and/or new versions of 570 | the GNU General Public License from time to time. Such new versions will 571 | be similar in spirit to the present version, but may differ in detail to 572 | address new problems or concerns. 573 | 574 | Each version is given a distinguishing version number. If the 575 | Program specifies that a certain numbered version of the GNU General 576 | Public License "or any later version" applies to it, you have the 577 | option of following the terms and conditions either of that numbered 578 | version or of any later version published by the Free Software 579 | Foundation. If the Program does not specify a version number of the 580 | GNU General Public License, you may choose any version ever published 581 | by the Free Software Foundation. 582 | 583 | If the Program specifies that a proxy can decide which future 584 | versions of the GNU General Public License can be used, that proxy's 585 | public statement of acceptance of a version permanently authorizes you 586 | to choose that version for the Program. 587 | 588 | Later license versions may give you additional or different 589 | permissions. However, no additional obligations are imposed on any 590 | author or copyright holder as a result of your choosing to follow a 591 | later version. 592 | 593 | 15. Disclaimer of Warranty. 594 | 595 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 596 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 597 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 598 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 599 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 600 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 601 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 602 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 603 | 604 | 16. Limitation of Liability. 605 | 606 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 607 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 608 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 609 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 610 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 611 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 612 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 613 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 614 | SUCH DAMAGES. 615 | 616 | 17. Interpretation of Sections 15 and 16. 617 | 618 | If the disclaimer of warranty and limitation of liability provided 619 | above cannot be given local legal effect according to their terms, 620 | reviewing courts shall apply local law that most closely approximates 621 | an absolute waiver of all civil liability in connection with the 622 | Program, unless a warranty or assumption of liability accompanies a 623 | copy of the Program in return for a fee. 624 | 625 | END OF TERMS AND CONDITIONS 626 | 627 | How to Apply These Terms to Your New Programs 628 | 629 | If you develop a new program, and you want it to be of the greatest 630 | possible use to the public, the best way to achieve this is to make it 631 | free software which everyone can redistribute and change under these terms. 632 | 633 | To do so, attach the following notices to the program. It is safest 634 | to attach them to the start of each source file to most effectively 635 | state the exclusion of warranty; and each file should have at least 636 | the "copyright" line and a pointer to where the full notice is found. 637 | 638 | {one line to give the program's name and a brief idea of what it does.} 639 | Copyright (C) {year} {name of author} 640 | 641 | This program is free software: you can redistribute it and/or modify 642 | it under the terms of the GNU General Public License as published by 643 | the Free Software Foundation, either version 3 of the License, or 644 | (at your option) any later version. 645 | 646 | This program is distributed in the hope that it will be useful, 647 | but WITHOUT ANY WARRANTY; without even the implied warranty of 648 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 649 | GNU General Public License for more details. 650 | 651 | You should have received a copy of the GNU General Public License 652 | along with this program. If not, see . 653 | 654 | Also add information on how to contact you by electronic and paper mail. 655 | 656 | If the program does terminal interaction, make it output a short 657 | notice like this when it starts in an interactive mode: 658 | 659 | {project} Copyright (C) {year} {fullname} 660 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 661 | This is free software, and you are welcome to redistribute it 662 | under certain conditions; type `show c' for details. 663 | 664 | The hypothetical commands `show w' and `show c' should show the appropriate 665 | parts of the General Public License. Of course, your program's commands 666 | might be different; for a GUI interface, you would use an "about box". 667 | 668 | You should also get your employer (if you work as a programmer) or school, 669 | if any, to sign a "copyright disclaimer" for the program, if necessary. 670 | For more information on this, and how to apply and follow the GNU GPL, see 671 | . 672 | 673 | The GNU General Public License does not permit incorporating your program 674 | into proprietary programs. If your program is a subroutine library, you 675 | may consider it more useful to permit linking proprietary applications with 676 | the library. If this is what you want to do, use the GNU Lesser General 677 | Public License instead of this License. But first, please read 678 | . 679 | 680 | 681 | ############################################################################### 682 | 683 | PEAK Release 0.5 alpha 3 is distributed under the Zope Public License, 684 | Version 2.0: 685 | 686 | Zope Public License (ZPL) Version 2.0 687 | ----------------------------------------------- 688 | 689 | This software is Copyright (c) Zope Corporation (tm) and 690 | Contributors. All rights reserved. 691 | 692 | This license has been certified as open source. It has also 693 | been designated as GPL compatible by the Free Software 694 | Foundation (FSF). 695 | 696 | Redistribution and use in source and binary forms, with or 697 | without modification, are permitted provided that the 698 | following conditions are met: 699 | 700 | 1. Redistributions in source code must retain the above 701 | copyright notice, this list of conditions, and the following 702 | disclaimer. 703 | 704 | 2. Redistributions in binary form must reproduce the above 705 | copyright notice, this list of conditions, and the following 706 | disclaimer in the documentation and/or other materials 707 | provided with the distribution. 708 | 709 | 3. The name Zope Corporation (tm) must not be used to 710 | endorse or promote products derived from this software 711 | without prior written permission from Zope Corporation. 712 | 713 | 4. The right to distribute this software or to use it for 714 | any purpose does not give you the right to use Servicemarks 715 | (sm) or Trademarks (tm) of Zope Corporation. Use of them is 716 | covered in a separate agreement (see 717 | http://www.zope.com/Marks). 718 | 719 | 5. If any files are modified, you must cause the modified 720 | files to carry prominent notices stating that you changed 721 | the files and the date of any change. 722 | 723 | Disclaimer 724 | 725 | THIS SOFTWARE IS PROVIDED BY ZOPE CORPORATION ``AS IS'' 726 | AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT 727 | NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY 728 | AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN 729 | NO EVENT SHALL ZOPE CORPORATION OR ITS CONTRIBUTORS BE 730 | LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 731 | EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 732 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 733 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 734 | HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 735 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 736 | OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 737 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH 738 | DAMAGE. 739 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | lazy_import 2 | =========== 3 | 4 | |Build Status| 5 | 6 | ``lazy_import`` provides a set of functions that load modules, and related 7 | attributes, in a lazy fashion. This allows deferring of ``ImportErrors`` to 8 | actual module use-time. Likewise, actual module initialization only takes place 9 | at use-time. This is useful when using optional dependencies with heavy loading 10 | times and/or footprints, since that cost is only paid if the module is actually 11 | used. 12 | 13 | For minimal impact to other code running in the same session ``lazy_import`` 14 | functionality is implemented without the use of import hooks. 15 | 16 | ``lazy_import`` is compatible with Python ≥ 2.7 or ≥ 3.4. 17 | 18 | Examples: lazy module loading 19 | ----------------------------- 20 | 21 | .. code:: python 22 | 23 | import lazy_import 24 | np = lazy_import.lazy_module("numpy") 25 | # np is now available in the namespace and is listed in sys.modules under 26 | # the 'numpy' key: 27 | import sys 28 | sys.modules['numpy'] 29 | # The module is present as "Lazily-loaded module numpy" 30 | 31 | # Subsequent imports of the same module return the lazy version present 32 | # in sys.modules 33 | import numpy # At this point numpy and np point to the same lazy module. 34 | # This is true for any import of 'numpy', even if from other modules! 35 | 36 | # Accessing attributes causes the full loading of the module ... 37 | np.pi 38 | # ... and the module is changed in place. np and numpy are now 39 | # "" 40 | 41 | # Lazy-importing a module that's already fully loaded returns the full 42 | # module instead (even if it was loaded elsewhere in the current session) 43 | # because there's no point in being lazy in this case: 44 | os = lazy_import.lazy_module("os") 45 | # "" 46 | 47 | In the above code it can be seen that issuing 48 | ``lazy_import.lazy_module("numpy")`` registers the lazy module in the 49 | session-wide ``sys.modules`` registry. This means that *any* subsequent import 50 | of ``numpy`` in the same session, while the module is still not fully loaded, 51 | will get served a lazy version of the ``numpy`` module. This will happen also 52 | outside the code that calls ``lazy_module``: 53 | 54 | .. code:: python 55 | 56 | import lazy_import 57 | np = lazy_import.lazy_module("numpy") 58 | import module_that_uses_numpy # This module will get a lazy module upon 59 | # 'import numpy' 60 | 61 | Normally this is ok because the lazy module will behave pretty much as the real 62 | thing once fully-loaded. Still, it might be a good practice to document that 63 | you're lazily importing modules so-and-so, so that users are warned. 64 | 65 | Further uses are to delay ``ImportErrors``: 66 | 67 | .. code:: python 68 | 69 | import lazy_import 70 | # The following succeeds even when asking for a module that's not available 71 | missing = lazy_import.lazy_module("missing_module") 72 | 73 | missing.some_attr # This causes the full loading of the module, which now fails. 74 | "ImportError: __main__ attempted to use a functionality that requires module 75 | missing_module, but it couldn't be loaded. Please install missing_module and retry." 76 | 77 | 78 | Submodules work too: 79 | 80 | .. code:: python 81 | 82 | import lazy_import 83 | mod = lazy_import.lazy_module("some.sub.module") 84 | # mod now points to the some.sub.module lazy module 85 | # equivalent to "from some.sub import module as mod" 86 | 87 | # Alternatively the returned reference can be made to point to the 88 | # base module: 89 | some = lazy_import.lazy_module("some.sub.module", level="base") 90 | 91 | # This is equivalent to "import some.sub.module" in that only the base 92 | # module's name is added to the namespace. All submodules must be accessed 93 | # via that: 94 | some.sub # Returns lazy module 'some.sub' without triggering full loading. 95 | some.sub.attr # Triggers full loading of 'some' and 'some.sub'. 96 | some.sub.module.function() # Triggers loading also of 'some.sub.module'. 97 | 98 | 99 | Finally, if you want to mark some modules and submodules your package imports 100 | as always being lazy, it is as simple as lazily importing them at the root 101 | `__init__.py` level. Other files can then import all modules normally, and 102 | those that have already been loaded as lazy in `__init__.py` will remain so: 103 | 104 | .. code:: python 105 | 106 | # in __init__.py: 107 | 108 | import lazy_import 109 | lazy_import.lazy_module("numpy") 110 | lazy_import.lazy_module("scipy.stats") 111 | 112 | 113 | # then, in any other file in the package just use the imports normally: 114 | 115 | import requests # This one is not lazy. 116 | import numpy # This one is lazy, as long as no other code caused its 117 | # loading in the meantime. 118 | import scipy # This one is also lazy. It was lazily loaded as part of the 119 | # lazy loading of scipy.stats. 120 | import scipy.stats # Also lazy. 121 | import scipy.linalg # Uh-oh, we didn't lazily import the 'linalg' submodule 122 | # earlier, and importing it like this here will cause 123 | # both scipy and scipy.linalg (but not scipy.stats) to 124 | # immediately become fully loaded. 125 | 126 | 127 | Examples: lazy callable loading 128 | ------------------------------- 129 | 130 | To emulate the ``from some.module import function`` syntax ``lazy_module`` 131 | provides ``lazy_callable``. It returns a wrapper function. Only upon being 132 | called will it trigger the loading of the target module and the calling of the 133 | target callable (function, class, etc.). 134 | 135 | .. code:: python 136 | 137 | import lazy_import 138 | fn = lazy_import.lazy_callable("numpy.arange") 139 | # 'numpy' is now in sys.modules and is 'Lazily-loaded module numpy' 140 | 141 | fn(10) 142 | # array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) 143 | 144 | ``lazy_callable`` is only useful when the target callable is going to be called: 145 | 146 | .. code:: python 147 | 148 | import lazy_import 149 | cl = lazy_import.lazy_callable("numpy.ndarray") # a class 150 | 151 | obj = cl([1, 2]) # This works OK (and also triggers the loading of numpy) 152 | 153 | class MySubclass(cl): # This fails because cl is just a wrapper, 154 | pass # not an actual class. 155 | 156 | 157 | Installation 158 | ------------ 159 | 160 | .. code:: bash 161 | 162 | pip install lazy_import 163 | 164 | Or, to include dependencies needed to run regression tests: 165 | 166 | .. code:: bash 167 | 168 | pip install lazy_import[test] 169 | 170 | Tests 171 | ----- 172 | 173 | The ``lazy_module`` module comes with a series of tests. If you install with 174 | test dependencies (see above), just run 175 | 176 | .. code:: python 177 | 178 | import lazy_import.test_lazy 179 | lazy_import.test_lazy.run() 180 | # This will automatically parallelize over the available number of cores 181 | 182 | Alternatively, tests can be run from the command line: 183 | 184 | .. code:: bash 185 | 186 | pytest -n 4 --boxed -v --pyargs lazy_import 187 | # (replace '4' with the number of cores in your machine, or set to 1 if 188 | # you'd rather test in serial) 189 | 190 | Tests depend only on |pytest|_ and |pytest-xdist|_, so if you didn't install 191 | them along ``lazy_import`` (as described under `Installation`_) just run 192 | 193 | .. code:: bash 194 | 195 | pip install pytest pytest-xdist 196 | 197 | Note that ``pytest-xdist`` is required even for serial testing because of its 198 | ``--boxed`` functionality. 199 | 200 | License 201 | ------- 202 | 203 | ``lazy_import`` is released under GPL v3. It was based on code from the 204 | |importing|_ module from the PEAK_ package. The licenses for both 205 | ``lazy_import`` and the PEAK package are included in the ``LICENSE`` file. The 206 | respective license notices are reproduced here: 207 | 208 | lazy_import — a module to allow lazy importing of python modules 209 | 210 | Copyright (C) 2017-2018 Manuel Nuno Melo 211 | 212 | lazy_import is free software: you can redistribute it and/or modify 213 | it under the terms of the GNU General Public License as published by 214 | the Free Software Foundation, either version 3 of the License, or 215 | (at your option) any later version. 216 | 217 | lazy_import is distributed in the hope that it will be useful, 218 | but WITHOUT ANY WARRANTY; without even the implied warranty of 219 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 220 | GNU General Public License for more details. 221 | 222 | You should have received a copy of the GNU General Public License 223 | along with lazy_import. If not, see . 224 | 225 | 226 | The PEAK ``importing`` code is 227 | 228 | Copyright (C) 1996-2004 by Phillip J. Eby and Tyler C. Sarna. 229 | All rights reserved. This software may be used under the same terms 230 | as Zope or Python. THERE ARE ABSOLUTELY NO WARRANTIES OF ANY KIND. 231 | Code quality varies between modules, from "beta" to "experimental 232 | pre-alpha". :) 233 | 234 | Code pertaining to lazy loading from PEAK ``importing`` was included in 235 | ``lazy_import``, modified in a number of ways. These are detailed in the 236 | ``CHANGELOG`` file of ``lazy_import``. Changes mainly involved Python 3 237 | compatibility, extension to allow customizable behavior, and added 238 | functionality (lazy importing of callable objects). 239 | 240 | 241 | .. |Build Status| image:: https://api.travis-ci.org/mnmelo/lazy_import.svg 242 | :target: https://travis-ci.org/mnmelo/lazy_import 243 | 244 | .. |importing| replace:: ``importing`` 245 | .. |pytest| replace:: ``pytest`` 246 | .. |pytest-xdist| replace:: ``pytest-xdist`` 247 | 248 | .. _importing: http://peak.telecommunity.com/DevCenter/Importing 249 | .. _PEAK: http://peak.telecommunity.com/DevCenter/FrontPage 250 | .. _pytest: https://docs.pytest.org/en/latest/ 251 | .. _pytest-xdist: https://pypi.python.org/pypi/pytest-xdist 252 | -------------------------------------------------------------------------------- /build_and_test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | basedir=$(pwd) 3 | cd 4 | pip3 install --force-reinstall --user $basedir 5 | pip2 install --force-reinstall --user $basedir 6 | python3 -m pytest --boxed -n 4 -v --pyargs lazy_import 7 | python2 -m pytest --boxed -n 4 -v --pyargs lazy_import 8 | cd $basedir 9 | -------------------------------------------------------------------------------- /lazy_import/VERSION: -------------------------------------------------------------------------------- 1 | 0.2.3-dev 2 | -------------------------------------------------------------------------------- /lazy_import/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- Mode: python; tab-width: 4; indent-tabs-mode:nil; coding:utf-8 -*- 2 | # vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4 3 | # 4 | # lazy_import --- https://github.com/mnmelo/lazy_import 5 | # Copyright (C) 2017-2018 Manuel Nuno Melo 6 | # 7 | # This file is part of lazy_import. 8 | # 9 | # lazy_import is free software: you can redistribute it and/or modify 10 | # it under the terms of the GNU General Public License as published by 11 | # the Free Software Foundation, either version 3 of the License, or 12 | # (at your option) any later version. 13 | # 14 | # lazy_import is distributed in the hope that it will be useful, 15 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 16 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 17 | # GNU General Public License for more details. 18 | # 19 | # You should have received a copy of the GNU General Public License 20 | # along with lazy_import. If not, see . 21 | # 22 | # lazy_import was based on code from the importing module from the PEAK 23 | # package (see ). The PEAK 24 | # package is released under the following license, reproduced here: 25 | # 26 | # Copyright (C) 1996-2004 by Phillip J. Eby and Tyler C. Sarna. 27 | # All rights reserved. This software may be used under the same terms 28 | # as Zope or Python. THERE ARE ABSOLUTELY NO WARRANTIES OF ANY KIND. 29 | # Code quality varies between modules, from "beta" to "experimental 30 | # pre-alpha". :) 31 | # 32 | # Code pertaining to lazy loading from PEAK importing was included in 33 | # lazy_import, modified in a number of ways. These are detailed in the 34 | # CHANGELOG file of lazy_import. Changes mainly involved Python 3 35 | # compatibility, extension to allow customizable behavior, and added 36 | # functionality (lazy importing of callable objects). 37 | # 38 | 39 | """ 40 | Lazy module loading 41 | =================== 42 | 43 | Functions and classes for lazy module loading that also delay import errors. 44 | Heavily borrowed from the `importing`_ module. 45 | 46 | .. _`importing`: http://peak.telecommunity.com/DevCenter/Importing 47 | 48 | Files and directories 49 | --------------------- 50 | 51 | .. autofunction:: module 52 | .. autofunction:: callable 53 | 54 | """ 55 | 56 | __all__ = ['lazy_module', 'lazy_callable', 'lazy_function', 'lazy_class', 57 | 'LazyModule', 'LazyCallable', 'module_basename', '_MSG', 58 | '_MSG_CALLABLE'] 59 | 60 | from types import ModuleType 61 | import sys 62 | try: 63 | from importlib._bootstrap import _ImportLockContext 64 | except ImportError: 65 | # Python 2 doesn't have the context manager. Roll it ourselves (copied from 66 | # Python 3's importlib/_bootstrap.py) 67 | import imp 68 | class _ImportLockContext: 69 | """Context manager for the import lock.""" 70 | def __enter__(self): 71 | imp.acquire_lock() 72 | def __exit__(self, exc_type, exc_value, exc_traceback): 73 | imp.release_lock() 74 | 75 | 76 | # Adding a __spec__ doesn't really help. I'll leave the code here in case 77 | # future python implementations start relying on it. 78 | #try: 79 | # from importlib.machinery import ModuleSpec 80 | #except ImportError: 81 | # ModuleSpec = None 82 | 83 | import six 84 | from six import raise_from 85 | from six.moves import reload_module 86 | # It is sometime useful to have access to the version number of a library. 87 | # This is usually done through the __version__ special attribute. 88 | # To make sure the version number is consistent between setup.py and the 89 | # library, we read the version number from the file called VERSION that stays 90 | # in the module directory. 91 | import os 92 | VERSION_FILE = os.path.join(os.path.dirname(__file__), 'VERSION') 93 | with open(VERSION_FILE) as infile: 94 | __version__ = infile.read().strip() 95 | 96 | # Logging 97 | import logging 98 | # adding a TRACE level for stack debugging 99 | _LAZY_TRACE = 1 100 | logging.addLevelName(1, "LAZY_TRACE") 101 | logging.basicConfig(level=logging.WARNING) 102 | # Logs a formatted stack (takes no message or args/kwargs) 103 | def _lazy_trace(self): 104 | if self.isEnabledFor(_LAZY_TRACE): 105 | import traceback 106 | self._log(_LAZY_TRACE, " ### STACK TRACE ###", ()) 107 | for line in traceback.format_stack(sys._getframe(2)): 108 | for subline in line.split("\n"): 109 | self._log(_LAZY_TRACE, subline.rstrip(), ()) 110 | logging.Logger.lazy_trace = _lazy_trace 111 | logger = logging.getLogger(__name__) 112 | 113 | ################################ 114 | # Module/function registration # 115 | ################################ 116 | 117 | #### Lazy classes #### 118 | 119 | class LazyModule(ModuleType): 120 | """Class for lazily-loaded modules that triggers proper loading on access. 121 | 122 | Instantiation should be made from a subclass of :class:`LazyModule`, with 123 | one subclass per instantiated module. Regular attribute set/access can then 124 | be recovered by setting the subclass's :meth:`__getattribute__` and 125 | :meth:`__setattribute__` to those of :class:`types.ModuleType`. 126 | """ 127 | # peak.util.imports sets __slots__ to (), but it seems pointless because 128 | # the base ModuleType doesn't itself set __slots__. 129 | def __getattribute__(self, attr): 130 | logger.debug("Getting attr {} of LazyModule instance of {}" 131 | .format(attr, super(LazyModule, self) 132 | .__getattribute__("__name__"))) 133 | logger.lazy_trace() 134 | # IPython tries to be too clever and constantly inspects, asking for 135 | # modules' attrs, which causes premature module loading and unesthetic 136 | # internal errors if the lazily-loaded module doesn't exist. 137 | if (run_from_ipython() 138 | and (attr.startswith(("__", "_ipython")) 139 | or attr == "_repr_mimebundle_") 140 | and module_basename(_caller_name()) in ('inspect', 'IPython')): 141 | logger.debug("Ignoring request for {}, deemed from IPython's " 142 | "inspection.".format(super(LazyModule, self) 143 | .__getattribute__("__name__"), attr)) 144 | raise AttributeError 145 | if not attr in ('__name__','__class__','__spec__'): 146 | # __name__ and __class__ yield their values from the LazyModule; 147 | # __spec__ causes an AttributeError. Maybe in the future it will be 148 | # necessary to return an actual ModuleSpec object, but it works as 149 | # it is without that now. 150 | 151 | # If it's an already-loaded submodule, we return it without 152 | # triggering a full loading 153 | try: 154 | return sys.modules[self.__name__+"."+attr] 155 | except KeyError: 156 | pass 157 | # Check if it's one of the lazy callables 158 | try: 159 | _callable = type(self)._lazy_import_callables[attr] 160 | logger.debug("Returning lazy-callable '{}'.".format(attr)) 161 | return _callable 162 | except (AttributeError, KeyError) as err: 163 | logger.debug("Proceeding to load module {}, " 164 | "from requested value {}" 165 | .format(super(LazyModule, self) 166 | .__getattribute__("__name__"), attr)) 167 | _load_module(self) 168 | logger.debug("Returning value '{}'.".format(super(LazyModule, self) 169 | .__getattribute__(attr))) 170 | return super(LazyModule, self).__getattribute__(attr) 171 | 172 | def __setattr__(self, attr, value): 173 | logger.debug("Setting attr {} to value {}, in LazyModule instance " 174 | "of {}".format(attr, value, super(LazyModule, self) 175 | .__getattribute__("__name__"))) 176 | _load_module(self) 177 | return super(LazyModule, self).__setattr__(attr, value) 178 | 179 | 180 | class LazyCallable(object): 181 | """Class for lazily-loaded callables that triggers module loading on access 182 | 183 | """ 184 | def __init__(self, *args): 185 | if len(args) != 2: 186 | # Maybe the user tried to base a class off this lazy callable? 187 | try: 188 | logger.debug("Got wrong number of args when init'ing " 189 | "LazyCallable. args is '{}'".format(args)) 190 | base = args[1][0] 191 | if isinstance(base, LazyCallable) and len(args) == 3: 192 | raise NotImplementedError("It seems you are trying to use " 193 | "a lazy callable as a class " 194 | "base. This is not supported.") 195 | except (IndexError, TypeError): 196 | raise_from(TypeError("LazyCallable takes exactly 2 arguments: " 197 | "a module/lazy module object and the name of " 198 | "a callable to be lazily loaded."), None) 199 | self.module, self.cname = args 200 | self.modclass = type(self.module) 201 | self.callable = None 202 | # Need to save these, since the module-loading gets rid of them 203 | self.error_msgs = self.modclass._lazy_import_error_msgs 204 | self.error_strings = self.modclass._lazy_import_error_strings 205 | 206 | def __call__(self, *args, **kwargs): 207 | # No need to go through all the reloading more than once. 208 | if self.callable: 209 | return self.callable(*args, **kwargs) 210 | try: 211 | del self.modclass._lazy_import_callables[self.cname] 212 | except (AttributeError, KeyError): 213 | pass 214 | try: 215 | self.callable = getattr(self.module, self.cname) 216 | except AttributeError: 217 | msg = self.error_msgs['msg_callable'] 218 | raise_from(AttributeError( 219 | msg.format(callable=self.cname, **self.error_strings)), None) 220 | except ImportError as err: 221 | # Import failed. We reset the dict and re-raise the ImportError. 222 | try: 223 | self.modclass._lazy_import_callables[self.cname] = self 224 | except AttributeError: 225 | self.modclass._lazy_import_callables = {self.cname: self} 226 | raise_from(err, None) 227 | else: 228 | return self.callable(*args, **kwargs) 229 | 230 | 231 | ### Functions ### 232 | 233 | def lazy_module(modname, error_strings=None, lazy_mod_class=LazyModule, 234 | level='leaf'): 235 | """Function allowing lazy importing of a module into the namespace. 236 | 237 | A lazy module object is created, registered in `sys.modules`, and 238 | returned. This is a hollow module; actual loading, and `ImportErrors` if 239 | not found, are delayed until an attempt is made to access attributes of the 240 | lazy module. 241 | 242 | A handy application is to use :func:`lazy_module` early in your own code 243 | (say, in `__init__.py`) to register all modulenames you want to be lazy. 244 | Because of registration in `sys.modules` later invocations of 245 | `import modulename` will also return the lazy object. This means that after 246 | initial registration the rest of your code can use regular pyhon import 247 | statements and retain the lazyness of the modules. 248 | 249 | Parameters 250 | ---------- 251 | modname : str 252 | The module to import. 253 | error_strings : dict, optional 254 | A dictionary of strings to use when module-loading fails. Key 'msg' 255 | sets the message to use (defaults to :attr:`lazy_import._MSG`). The 256 | message is formatted using the remaining dictionary keys. The default 257 | message informs the user of which module is missing (key 'module'), 258 | what code loaded the module as lazy (key 'caller'), and which package 259 | should be installed to solve the dependency (key 'install_name'). 260 | None of the keys is mandatory and all are given smart names by default. 261 | lazy_mod_class: type, optional 262 | Which class to use when instantiating the lazy module, to allow 263 | deep customization. The default is :class:`LazyModule` and custom 264 | alternatives **must** be a subclass thereof. 265 | level : str, optional 266 | Which submodule reference to return. Either a reference to the 'leaf' 267 | module (the default) or to the 'base' module. This is useful if you'll 268 | be using the module functionality in the same place you're calling 269 | :func:`lazy_module` from, since then you don't need to run `import` 270 | again. Setting *level* does not affect which names/modules get 271 | registered in `sys.modules`. 272 | For *level* set to 'base' and *modulename* 'aaa.bbb.ccc':: 273 | 274 | aaa = lazy_import.lazy_module("aaa.bbb.ccc", level='base') 275 | # 'aaa' becomes defined in the current namespace, with 276 | # (sub)attributes 'aaa.bbb' and 'aaa.bbb.ccc'. 277 | # It's the lazy equivalent to: 278 | import aaa.bbb.ccc 279 | 280 | For *level* set to 'leaf':: 281 | 282 | ccc = lazy_import.lazy_module("aaa.bbb.ccc", level='leaf') 283 | # Only 'ccc' becomes set in the current namespace. 284 | # Lazy equivalent to: 285 | from aaa.bbb import ccc 286 | 287 | Returns 288 | ------- 289 | module 290 | The module specified by *modname*, or its base, depending on *level*. 291 | The module isn't immediately imported. Instead, an instance of 292 | *lazy_mod_class* is returned. Upon access to any of its attributes, the 293 | module is finally loaded. 294 | 295 | Examples 296 | -------- 297 | >>> import lazy_import, sys 298 | >>> np = lazy_import.lazy_module("numpy") 299 | >>> np 300 | Lazily-loaded module numpy 301 | >>> np is sys.modules['numpy'] 302 | True 303 | >>> np.pi # This causes the full loading of the module ... 304 | 3.141592653589793 305 | >>> np # ... and the module is changed in place. 306 | 307 | 308 | >>> import lazy_import, sys 309 | >>> # The following succeeds even when asking for a module that's not available 310 | >>> missing = lazy_import.lazy_module("missing_module") 311 | >>> missing 312 | Lazily-loaded module missing_module 313 | >>> missing is sys.modules['missing_module'] 314 | True 315 | >>> missing.some_attr # This causes the full loading of the module, which now fails. 316 | ImportError: __main__ attempted to use a functionality that requires module missing_module, but it couldn't be loaded. Please install missing_module and retry. 317 | 318 | See Also 319 | -------- 320 | :func:`lazy_callable` 321 | :class:`LazyModule` 322 | 323 | """ 324 | if error_strings is None: 325 | error_strings = {} 326 | _set_default_errornames(modname, error_strings) 327 | 328 | mod = _lazy_module(modname, error_strings, lazy_mod_class) 329 | if level == 'base': 330 | return sys.modules[module_basename(modname)] 331 | elif level == 'leaf': 332 | return mod 333 | else: 334 | raise ValueError("Parameter 'level' must be one of ('base', 'leaf')") 335 | 336 | 337 | def _lazy_module(modname, error_strings, lazy_mod_class): 338 | with _ImportLockContext(): 339 | fullmodname = modname 340 | fullsubmodname = None 341 | # ensure parent module/package is in sys.modules 342 | # and parent.modname=module, as soon as the parent is imported 343 | while modname: 344 | try: 345 | mod = sys.modules[modname] 346 | # We reached a (base) module that's already loaded. Let's stop 347 | # the cycle. Can't use 'break' because we still want to go 348 | # through the fullsubmodname check below. 349 | modname = '' 350 | except KeyError: 351 | err_s = error_strings.copy() 352 | err_s.setdefault('module', modname) 353 | 354 | class _LazyModule(lazy_mod_class): 355 | _lazy_import_error_msgs = {'msg': err_s.pop('msg')} 356 | try: 357 | _lazy_import_error_msgs['msg_callable'] = \ 358 | err_s.pop('msg_callable') 359 | except KeyError: 360 | pass 361 | _lazy_import_error_strings = err_s 362 | _lazy_import_callables = {} 363 | _lazy_import_submodules = {} 364 | 365 | def __repr__(self): 366 | return "Lazily-loaded module {}".format(self.__name__) 367 | # A bit of cosmetic, to make AttributeErrors read more natural 368 | _LazyModule.__name__ = 'module' 369 | # Actual module instantiation 370 | mod = sys.modules[modname] = _LazyModule(modname) 371 | # No need for __spec__. Maybe in the future. 372 | #if ModuleSpec: 373 | # ModuleType.__setattr__(mod, '__spec__', 374 | # ModuleSpec(modname, None)) 375 | if fullsubmodname: 376 | submod = sys.modules[fullsubmodname] 377 | ModuleType.__setattr__(mod, submodname, submod) 378 | _LazyModule._lazy_import_submodules[submodname] = submod 379 | fullsubmodname = modname 380 | modname, _, submodname = modname.rpartition('.') 381 | return sys.modules[fullmodname] 382 | 383 | 384 | def lazy_callable(modname, *names, **kwargs): 385 | """Performs lazy importing of one or more callables. 386 | 387 | :func:`lazy_callable` creates functions that are thin wrappers that pass 388 | any and all arguments straight to the target module's callables. These can 389 | be functions or classes. The full loading of that module is only actually 390 | triggered when the returned lazy function itself is called. This lazy 391 | import of the target module uses the same mechanism as 392 | :func:`lazy_module`. 393 | 394 | If, however, the target module has already been fully imported prior 395 | to invocation of :func:`lazy_callable`, then the target callables 396 | themselves are returned and no lazy imports are made. 397 | 398 | :func:`lazy_function` and :func:`lazy_function` are aliases of 399 | :func:`lazy_callable`. 400 | 401 | Parameters 402 | ---------- 403 | modname : str 404 | The base module from where to import the callable(s) in *names*, 405 | or a full 'module_name.callable_name' string. 406 | names : str (optional) 407 | The callable name(s) to import from the module specified by *modname*. 408 | If left empty, *modname* is assumed to also include the callable name 409 | to import. 410 | error_strings : dict, optional 411 | A dictionary of strings to use when reporting loading errors (either a 412 | missing module, or a missing callable name in the loaded module). 413 | *error_string* follows the same usage as described under 414 | :func:`lazy_module`, with the exceptions that 1) a further key, 415 | 'msg_callable', can be supplied to be used as the error when a module 416 | is successfully loaded but the target callable can't be found therein 417 | (defaulting to :attr:`lazy_import._MSG_CALLABLE`); 2) a key 'callable' 418 | is always added with the callable name being loaded. 419 | lazy_mod_class : type, optional 420 | See definition under :func:`lazy_module`. 421 | lazy_call_class : type, optional 422 | Analogously to *lazy_mod_class*, allows setting a custom class to 423 | handle lazy callables, other than the default :class:`LazyCallable`. 424 | 425 | Returns 426 | ------- 427 | wrapper function or tuple of wrapper functions 428 | If *names* is passed, returns a tuple of wrapper functions, one for 429 | each element in *names*. 430 | If only *modname* is passed it is assumed to be a full 431 | 'module_name.callable_name' string, in which case the wrapper for the 432 | imported callable is returned directly, and not in a tuple. 433 | 434 | Notes 435 | ----- 436 | Unlike :func:`lazy_module`, which returns a lazy module that eventually 437 | mutates into the fully-functional version, :func:`lazy_callable` only 438 | returns thin wrappers that never change. This means that the returned 439 | wrapper object never truly becomes the one under the module's namespace, 440 | even after successful loading of the module in *modname*. This is fine for 441 | most practical use cases, but may break code that relies on the usage of 442 | the returned objects oter than calling them. One such example is the lazy 443 | import of a class: it's fine to use the returned wrapper to instantiate an 444 | object, but it can't be used, for instance, to subclass from. 445 | 446 | Examples 447 | -------- 448 | >>> import lazy_import, sys 449 | >>> fn = lazy_import.lazy_callable("numpy.arange") 450 | >>> sys.modules['numpy'] 451 | Lazily-loaded module numpy 452 | >>> fn(10) 453 | array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) 454 | >>> sys.modules['numpy'] 455 | 456 | 457 | >>> import lazy_import, sys 458 | >>> cl = lazy_import.lazy_callable("numpy.ndarray") # a class 459 | >>> obj = cl([1, 2]) # This works OK (and also triggers the loading of numpy) 460 | >>> class MySubclass(cl): # This fails because cls is just a wrapper, 461 | >>> pass # not an actual class. 462 | 463 | See Also 464 | -------- 465 | :func:`lazy_module` 466 | :class:`LazyCallable` 467 | :class:`LazyModule` 468 | 469 | """ 470 | if not names: 471 | modname, _, name = modname.rpartition(".") 472 | lazy_mod_class = _setdef(kwargs, 'lazy_mod_class', LazyModule) 473 | lazy_call_class = _setdef(kwargs, 'lazy_call_class', LazyCallable) 474 | error_strings = _setdef(kwargs, 'error_strings', {}) 475 | _set_default_errornames(modname, error_strings, call=True) 476 | 477 | if not names: 478 | # We allow passing a single string as 'modname.callable_name', 479 | # in which case the wrapper is returned directly and not as a list. 480 | return _lazy_callable(modname, name, error_strings.copy(), 481 | lazy_mod_class, lazy_call_class) 482 | return tuple(_lazy_callable(modname, cname, error_strings.copy(), 483 | lazy_mod_class, lazy_call_class) for cname in names) 484 | 485 | lazy_function = lazy_class = lazy_callable 486 | 487 | def _lazy_callable(modname, cname, error_strings, 488 | lazy_mod_class, lazy_call_class): 489 | # We could do most of this in the LazyCallable __init__, but here we can 490 | # pre-check whether to actually be lazy or not. 491 | module = _lazy_module(modname, error_strings, lazy_mod_class) 492 | modclass = type(module) 493 | if (issubclass(modclass, LazyModule) and 494 | hasattr(modclass, '_lazy_import_callables')): 495 | modclass._lazy_import_callables.setdefault( 496 | cname, lazy_call_class(module, cname)) 497 | return getattr(module, cname) 498 | 499 | 500 | ####################### 501 | # Real module loading # 502 | ####################### 503 | 504 | def _load_module(module): 505 | """Ensures that a module, and its parents, are properly loaded 506 | 507 | """ 508 | modclass = type(module) 509 | # We only take care of our own LazyModule instances 510 | if not issubclass(modclass, LazyModule): 511 | raise TypeError("Passed module is not a LazyModule instance.") 512 | with _ImportLockContext(): 513 | parent, _, modname = module.__name__.rpartition('.') 514 | logger.debug("loading module {}".format(modname)) 515 | # We first identify whether this is a loadable LazyModule, then we 516 | # strip as much of lazy_import behavior as possible (keeping it cached, 517 | # in case loading fails and we need to reset the lazy state). 518 | if not hasattr(modclass, '_lazy_import_error_msgs'): 519 | # Alreay loaded (no _lazy_import_error_msgs attr). Not reloading. 520 | return 521 | # First, ensure the parent is loaded (using recursion; *very* unlikely 522 | # we'll ever hit a stack limit in this case). 523 | modclass._LOADING = True 524 | try: 525 | if parent: 526 | logger.debug("first loading parent module {}".format(parent)) 527 | setattr(sys.modules[parent], modname, module) 528 | if not hasattr(modclass, '_LOADING'): 529 | logger.debug("Module {} already loaded by the parent" 530 | .format(modname)) 531 | # We've been loaded by the parent. Let's bail. 532 | return 533 | cached_data = _clean_lazymodule(module) 534 | try: 535 | # Get Python to do the real import! 536 | reload_module(module) 537 | except: 538 | # Loading failed. We reset our lazy state. 539 | logger.debug("Failed to load module {}. Resetting..." 540 | .format(modname)) 541 | _reset_lazymodule(module, cached_data) 542 | raise 543 | else: 544 | # Successful load 545 | logger.debug("Successfully loaded module {}".format(modname)) 546 | delattr(modclass, '_LOADING') 547 | _reset_lazy_submod_refs(module) 548 | 549 | except (AttributeError, ImportError) as err: 550 | logger.debug("Failed to load {}.\n{}: {}" 551 | .format(modname, err.__class__.__name__, err)) 552 | logger.lazy_trace() 553 | # Under Python 3 reloading our dummy LazyModule instances causes an 554 | # AttributeError if the module can't be found. Would be preferrable 555 | # if we could always rely on an ImportError. As it is we vet the 556 | # AttributeError as thoroughly as possible. 557 | if ((six.PY3 and isinstance(err, AttributeError)) and not 558 | err.args[0] == "'NoneType' object has no attribute 'name'"): 559 | # Not the AttributeError we were looking for. 560 | raise 561 | msg = modclass._lazy_import_error_msgs['msg'] 562 | raise_from(ImportError( 563 | msg.format(**modclass._lazy_import_error_strings)), None) 564 | 565 | 566 | ############################## 567 | # Helper functions/constants # 568 | ############################## 569 | 570 | _MSG = ("{caller} attempted to use a functionality that requires module " 571 | "{module}, but it couldn't be loaded. Please install {install_name} " 572 | "and retry.") 573 | 574 | _MSG_CALLABLE = ("{caller} attempted to use a functionality that requires " 575 | "{callable}, of module {module}, but it couldn't be found in that " 576 | "module. Please install a version of {install_name} that has " 577 | "{module}.{callable} and retry.") 578 | 579 | _CLS_ATTRS = ("_lazy_import_error_strings", "_lazy_import_error_msgs", 580 | "_lazy_import_callables", "_lazy_import_submodules", "__repr__") 581 | 582 | _DELETION_DICT = ("_lazy_import_submodules",) 583 | 584 | def _setdef(argdict, name, defaultvalue): 585 | """Like dict.setdefault but sets the default value also if None is present. 586 | 587 | """ 588 | if not name in argdict or argdict[name] is None: 589 | argdict[name] = defaultvalue 590 | return argdict[name] 591 | 592 | 593 | def module_basename(modname): 594 | return modname.partition('.')[0] 595 | 596 | 597 | def _set_default_errornames(modname, error_strings, call=False): 598 | # We don't set the modulename default here because it will change for 599 | # parents of lazily imported submodules. 600 | error_strings.setdefault('caller', _caller_name(3, default='Python')) 601 | error_strings.setdefault('install_name', module_basename(modname)) 602 | error_strings.setdefault('msg', _MSG) 603 | if call: 604 | error_strings.setdefault('msg_callable', _MSG_CALLABLE) 605 | 606 | 607 | def _caller_name(depth=2, default=''): 608 | """Returns the name of the calling namespace. 609 | 610 | """ 611 | # the presence of sys._getframe might be implementation-dependent. 612 | # It isn't that serious if we can't get the caller's name. 613 | try: 614 | return sys._getframe(depth).f_globals['__name__'] 615 | except AttributeError: 616 | return default 617 | 618 | 619 | def _clean_lazymodule(module): 620 | """Removes all lazy behavior from a module's class, for loading. 621 | 622 | Also removes all module attributes listed under the module's class deletion 623 | dictionaries. Deletion dictionaries are class attributes with names 624 | specified in `_DELETION_DICT`. 625 | 626 | Parameters 627 | ---------- 628 | module: LazyModule 629 | 630 | Returns 631 | ------- 632 | dict 633 | A dictionary of deleted class attributes, that can be used to reset the 634 | lazy state using :func:`_reset_lazymodule`. 635 | """ 636 | modclass = type(module) 637 | _clean_lazy_submod_refs(module) 638 | 639 | modclass.__getattribute__ = ModuleType.__getattribute__ 640 | modclass.__setattr__ = ModuleType.__setattr__ 641 | cls_attrs = {} 642 | for cls_attr in _CLS_ATTRS: 643 | try: 644 | cls_attrs[cls_attr] = getattr(modclass, cls_attr) 645 | delattr(modclass, cls_attr) 646 | except AttributeError: 647 | pass 648 | return cls_attrs 649 | 650 | 651 | def _clean_lazy_submod_refs(module): 652 | modclass = type(module) 653 | for deldict in _DELETION_DICT: 654 | try: 655 | delnames = getattr(modclass, deldict) 656 | except AttributeError: 657 | continue 658 | for delname in delnames: 659 | try: 660 | super(LazyModule, module).__delattr__(delname) 661 | except AttributeError: 662 | # Maybe raise a warning? 663 | pass 664 | 665 | 666 | def _reset_lazymodule(module, cls_attrs): 667 | """Resets a module's lazy state from cached data. 668 | 669 | """ 670 | modclass = type(module) 671 | del modclass.__getattribute__ 672 | del modclass.__setattr__ 673 | try: 674 | del modclass._LOADING 675 | except AttributeError: 676 | pass 677 | for cls_attr in _CLS_ATTRS: 678 | try: 679 | setattr(modclass, cls_attr, cls_attrs[cls_attr]) 680 | except KeyError: 681 | pass 682 | _reset_lazy_submod_refs(module) 683 | 684 | 685 | def _reset_lazy_submod_refs(module): 686 | modclass = type(module) 687 | for deldict in _DELETION_DICT: 688 | try: 689 | resetnames = getattr(modclass, deldict) 690 | except AttributeError: 691 | continue 692 | for name, submod in resetnames.items(): 693 | super(LazyModule, module).__setattr__(name, submod) 694 | 695 | 696 | def run_from_ipython(): 697 | # Taken from https://stackoverflow.com/questions/5376837 698 | try: 699 | __IPYTHON__ 700 | return True 701 | except NameError: 702 | return False 703 | 704 | -------------------------------------------------------------------------------- /lazy_import/test_lazy.py: -------------------------------------------------------------------------------- 1 | # -*- Mode: python; tab-width: 4; indent-tabs-mode:nil; coding:utf-8 -*- 2 | # vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4 3 | # 4 | # Testing for lazy_import --- https://github.com/mnmelo/lazy_import 5 | # Copyright (C) 2017-2018 Manuel Nuno Melo 6 | # 7 | # This file is part of lazy_import. 8 | # 9 | # lazy_import is free software: you can redistribute it and/or modify 10 | # it under the terms of the GNU General Public License as published by 11 | # the Free Software Foundation, either version 3 of the License, or 12 | # (at your option) any later version. 13 | # 14 | # lazy_import is distributed in the hope that it will be useful, 15 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 16 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 17 | # GNU General Public License for more details. 18 | # 19 | # You should have received a copy of the GNU General Public License 20 | # along with lazy_import. If not, see . 21 | # 22 | 23 | import pytest 24 | import importlib 25 | import sys 26 | import six 27 | from six.moves import range 28 | import itertools 29 | import string 30 | import random 31 | random.seed(42) # For consistency when parallel-testing 32 | 33 | import lazy_import 34 | 35 | # Shortcut to allow direct testing from within python 36 | 37 | def run(np=None): 38 | if np is None: 39 | import multiprocessing 40 | np = multiprocessing.cpu_count() 41 | import os 42 | path = os.path.dirname(__file__) 43 | if np > 1: 44 | pytest.main(['-n', str(np), '--boxed', path]) 45 | else: 46 | pytest.main(['--boxed', path]) 47 | ############################################################################### 48 | # Constants and util functions 49 | 50 | RANDOM_PREFIX = "_lazy_test_" 51 | RANDOM_ALPHABET = string.ascii_letters + string.digits 52 | def random_str(size=5): 53 | # We make it start with a letter 54 | return "m" + ''.join(random.choice(RANDOM_ALPHABET) for _ in range(size)) 55 | 56 | def random_modname(submodules=0): 57 | while True: 58 | subname = ".".join(random_str() for _ in range(submodules + 1)) 59 | modname = RANDOM_PREFIX + subname 60 | if modname in _GENERATED_NAMES or modname in sys.modules: 61 | continue 62 | _GENERATED_NAMES.append(modname) 63 | return modname 64 | 65 | class _TestLazyModule(lazy_import.LazyModule): 66 | pass 67 | 68 | _GENERATED_NAMES = [] 69 | # Modules not usually loaded on startup. Must include at least one with 70 | # submodule 71 | NAMES_EXISTING = ("sched", "distutils.core") 72 | 73 | LEVELS = ("leaf", "base") 74 | CLASSES = (_TestLazyModule, lazy_import.LazyModule) 75 | CALLABLE_NAMES =("fn1", ("fn1"), ("fn1", "fn2"), ("fn1", "fn2", "fn3")) 76 | ERROR_MSGS = (None, {"msg":"Module: {module}\n" 77 | "Caller: {caller}\n" 78 | "Install Name: {install_name}\n", 79 | "msg_callable":"Module: {module}\n" 80 | "Caller: {caller}\n" 81 | "Install Name: {install_name}\n" 82 | "Callable: {callable}\n", 83 | "module":"error_modname", 84 | "caller":"error_callername", 85 | "install_name":"error_installname"}) 86 | CALLABLE_ALIASES = (lazy_import.lazy_callable, 87 | lazy_import.lazy_function, lazy_import.lazy_class) 88 | 89 | ############################################################################### 90 | # Fixtures 91 | 92 | @pytest.fixture(params=itertools.product(LEVELS, CLASSES, ERROR_MSGS)) 93 | def lazy_opts(request): 94 | return request.param 95 | 96 | ############################################################################### 97 | # Re-usable tests 98 | 99 | def _check_reuse(modname): 100 | curr_module_id = id(sys.modules[modname]) 101 | newmod = importlib.import_module(modname) 102 | assert id(newmod) == curr_module_id 103 | 104 | def _check_lazy_loading(modname): 105 | names = modname.split(".") 106 | _check_not_loaded(modname) 107 | basename = lazy_import.module_basename(modname) 108 | mod = lazy_import.lazy_module(modname, level="leaf") 109 | assert sys.modules[modname] is mod 110 | assert str(mod) == "Lazily-loaded module " + modname 111 | base = lazy_import.lazy_module(modname, level="base") 112 | assert sys.modules[basename] is base 113 | assert str(base) == "Lazily-loaded module " + basename 114 | 115 | # A subtle bug only rears its head if an actual import statement is done. 116 | # Can only test that for the module names we know of 117 | # (meaning, no random ones) 118 | if modname in NAMES_EXISTING: 119 | if modname == 'sched': 120 | import sched as newmod 121 | elif modname == 'distutils.core': 122 | import distutils.core as newmod 123 | assert str(newmod) == "Lazily-loaded module " + modname 124 | 125 | # Check that all submodules are in and that submodule access works 126 | curr_name = basename 127 | curr_mod = base 128 | for submod in names[1:]: 129 | curr_name += "." + submod 130 | curr_mod = getattr(curr_mod, submod) 131 | assert curr_name in sys.modules 132 | assert str(curr_mod) == "Lazily-loaded module " + curr_name 133 | assert isinstance(curr_mod, lazy_import.LazyModule) 134 | _check_reuse(curr_name) 135 | # Check that missing modules raise errors 136 | if modname in _GENERATED_NAMES: 137 | _check_module_missing(curr_mod) 138 | 139 | def _check_module_missing(obj, msg=None, call=False): 140 | with pytest.raises(ImportError) as excinfo: 141 | if call: 142 | obj() 143 | else: 144 | obj.modattr 145 | if msg is not None: 146 | assert str(excinfo.value) == msg 147 | 148 | def _check_callable_missing(obj, msg=None): 149 | with pytest.raises(AttributeError) as excinfo: 150 | obj() 151 | if msg is not None: 152 | assert str(excinfo.value) == msg 153 | 154 | def _check_not_loaded(modname): 155 | assert modname not in sys.modules, \ 156 | modname + " already loaded. Maybe use with pytest's xdist's '--boxed'?" 157 | 158 | ############################################################################### 159 | # TESTS TESTS TESTS # 160 | ############################################################################### 161 | 162 | @pytest.mark.parametrize("modname", tuple(random_modname(i) for i in range(3)) 163 | + NAMES_EXISTING) 164 | def test_lazyload(modname): 165 | _check_lazy_loading(modname) 166 | 167 | def test_presentload(): 168 | import os.path 169 | mod = lazy_import.lazy_module("os") 170 | assert mod is os 171 | mod = lazy_import.lazy_module("os.path") 172 | assert mod is os.path 173 | mod = lazy_import.lazy_module("os.path", level="base") 174 | assert mod is os 175 | assert not isinstance(mod, lazy_import.LazyModule) 176 | 177 | @pytest.mark.parametrize("nsub", range(3)) 178 | def test_opts(nsub, lazy_opts): 179 | modname = random_modname(nsub) 180 | level, modclass, errors = lazy_opts 181 | mod = lazy_import.lazy_module(modname, error_strings=errors, 182 | lazy_mod_class=modclass, level=level) 183 | names = modname.split(".") 184 | basename = lazy_import.module_basename(modname) 185 | if level == "leaf": 186 | assert sys.modules[modname] is mod 187 | err_modname = modname 188 | elif level == "base": 189 | assert sys.modules[basename] is mod 190 | err_modname = basename 191 | else: 192 | raise ValueError("Unexpected value {} for 'level'".format(level)) 193 | # Test the exception err msg 194 | assert isinstance(mod, modclass) 195 | if errors is None: 196 | expected_err = lazy_import._MSG.format(module=err_modname, 197 | caller=__name__, 198 | install_name=basename) 199 | else: 200 | expected_err = errors["msg"].format(**errors) 201 | _check_module_missing(mod, msg=expected_err) 202 | 203 | 204 | @pytest.mark.parametrize("nsub, errors, cnames, fn", 205 | itertools.product(range(3), ERROR_MSGS, CALLABLE_NAMES, 206 | CALLABLE_ALIASES)) 207 | def test_callable_missing_module(nsub, errors, cnames, fn): 208 | modname = random_modname(nsub) 209 | basename = lazy_import.module_basename(modname) 210 | if isinstance(cnames, six.string_types): 211 | lazys = (fn(modname+"."+cnames, error_strings=errors),) 212 | cnames = (cnames, ) 213 | else: 214 | lazys = fn(modname, *cnames, error_strings=errors) 215 | for lazy in lazys: 216 | if errors is None: 217 | expected_err = lazy_import._MSG.format(module=modname, 218 | caller=__name__, 219 | install_name=basename) 220 | else: 221 | expected_err = errors["msg"].format(**errors) 222 | _check_module_missing(lazy, msg=expected_err, call=True) 223 | 224 | @pytest.mark.parametrize("modname, errors, cnames, fn", 225 | itertools.product(NAMES_EXISTING, ERROR_MSGS, CALLABLE_NAMES, 226 | CALLABLE_ALIASES)) 227 | def test_callable_missing(modname, errors, cnames, fn): 228 | _check_not_loaded(modname) 229 | basename = lazy_import.module_basename(modname) 230 | if isinstance(cnames, six.string_types): 231 | lazys = (fn(modname+"."+cnames, error_strings=errors),) 232 | cnames = (cnames, ) 233 | else: 234 | lazys = fn(modname, *cnames, error_strings=errors) 235 | for lazy, cname in zip(lazys, cnames): 236 | if errors is None: 237 | expected_err = lazy_import._MSG_CALLABLE.format(module=modname, 238 | caller=__name__, 239 | install_name=basename, 240 | callable=cname) 241 | else: 242 | errors['callable'] = cname 243 | expected_err = errors["msg_callable"].format(**errors) 244 | _check_callable_missing(lazy, msg=expected_err) 245 | 246 | @pytest.mark.parametrize("modname, errors, cnames, fn", 247 | itertools.product(NAMES_EXISTING, ERROR_MSGS, CALLABLE_NAMES, 248 | CALLABLE_ALIASES)) 249 | def test_error_callable_as_baseclass(modname, errors, cnames, fn): 250 | _check_not_loaded(modname) 251 | if isinstance(cnames, six.string_types): 252 | lazys = (fn(modname+"."+cnames, error_strings=errors),) 253 | else: 254 | lazys = fn(modname, *cnames, error_strings=errors) 255 | for lazy in lazys: 256 | with pytest.raises(NotImplementedError) as excinfo: 257 | class TestClass(lazy): 258 | pass 259 | 260 | @pytest.mark.parametrize("modname", 261 | [modname for modname in NAMES_EXISTING if '.' in modname]) 262 | def test_load_lazysub_on_fullmodule(modname, lazy_opts): 263 | level, modclass, errors = lazy_opts 264 | base = modname.split('.')[0] 265 | # A fully loaded base... 266 | importlib.import_module(base) 267 | # and a lazy sub... 268 | mod = lazy_import.lazy_module(modname, error_strings=errors, 269 | lazy_mod_class=modclass, level=level) 270 | if level == 'base': 271 | assert not isinstance(mod, modclass) 272 | else: 273 | assert isinstance(mod, modclass) 274 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [bdist_wheel] 2 | universal=1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from setuptools import setup, find_packages 4 | 5 | with open('README.rst') as infile: 6 | readme = infile.read() 7 | 8 | with open('lazy_import/VERSION') as infile: 9 | version = infile.read().strip() 10 | 11 | tests_require = ['pytest', 'pytest-xdist'] 12 | 13 | setup(name='lazy_import', 14 | version=version, 15 | description='A module for lazy loading of Python modules', 16 | long_description=readme, 17 | url='https://github.com/mnmelo/lazy_import', 18 | author='Manuel Nuno Melo', 19 | author_email='manuel.nuno.melo@gmail.com', 20 | license='GPL', 21 | platforms = ["any"], 22 | classifiers=['Development Status :: 4 - Beta', 23 | # Indicate who your project is intended for 24 | 'Intended Audience :: Developers', 25 | 'Topic :: Software Development :: Libraries :: ' 26 | 'Python Modules', 27 | 28 | 'License :: OSI Approved :: ' 29 | 'GNU General Public License v3 or later (GPLv3+)', 30 | 31 | 'Programming Language :: Python :: 2', 32 | 'Programming Language :: Python :: 2.7', 33 | 'Programming Language :: Python :: 3', 34 | 'Programming Language :: Python :: 3.3', 35 | 'Programming Language :: Python :: 3.4', 36 | 'Programming Language :: Python :: 3.5', 37 | 'Programming Language :: Python :: 3.6', 38 | 39 | 'Operating System :: OS Independent', 40 | ], 41 | packages=find_packages(), 42 | install_requires=['six'], 43 | test_suite='lazy_import.test_lazy', 44 | tests_require=tests_require, 45 | extras_require={'test': tests_require}, 46 | package_data={'lazy_import': ['VERSION']} 47 | ) 48 | --------------------------------------------------------------------------------